Electronic device, audio processing system and method for respiratory state recognition

Abstract

An electronic device, an audio processing system and a method for respiratory tract state recognition, the device comprising: a sensor configured to collect user sound to obtain audio data, the user sound being respiratory tract related sound; a processor configured to input at least part of the audio data and / or feature information of the audio data into a pre-trained recognition model to obtain a plurality of probability values output by the recognition model, the plurality of probability values being used to represent a respiratory tract state of a user. Through the scheme provided in the present application, respiratory tract related index parameters can be conveniently and accurately obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an electronic device for respiratory tract state recognition, an audio processing system and method. BACKGROUND

[0002] At present, respiratory tract related index parameters need to be obtained by chest radiography and the like. The convenience of this scheme is poor. SUMMARY

[0003] The technical purpose of the embodiments of the present application is to provide an electronic device for respiratory tract state recognition, an audio processing system and method, which can be beneficial to conveniently and accurately obtain respiratory tract related index parameters.

[0004] In a first aspect, the embodiments of the present application provide an electronic device, which comprises: a sensor configured to collect user sound to obtain audio data, the user sound being respiratory tract related sound; and a processor configured to input at least part of the audio data and / or feature information of the audio data into a pre-trained recognition model to obtain a plurality of probability values output by the recognition model, the plurality of probability values being used to represent a respiratory tract state of the user.

[0005] Optionally, the audio data is obtained by collecting the user sound in response to a collection instruction.

[0006] Optionally, the processor is further configured to perform the following steps: identifying a candidate audio segment from the audio data, wherein the audio amplitude in the candidate audio segment is greater than or equal to a first threshold; extracting feature information of the candidate audio segment; and inputting the feature information of the candidate audio segment into a pre-trained discrimination model to obtain a discrimination result output by the discrimination model, wherein if the discrimination result is yes, the candidate audio segment is valid.

[0007] Optionally, the processor is further configured to input the valid candidate audio segment into the recognition model to obtain the plurality of probability values.

[0008] Optionally, the processor is further configured to train a first preset model using first training data until a training stop condition is met to obtain the discrimination model, wherein the first training data comprises feature information of a sample candidate audio segment and label information, the label information being used to indicate whether the sample candidate audio segment is valid.

[0009] Optionally, the processor is further configured to calculate a similarity between the feature information of the candidate audio segment and historical feature information, the historical feature information being based on feature information of historical valid audio segments of the user; and if the similarity is greater than or equal to a similarity threshold, the third processing unit is configured to input the feature information of the candidate audio segment into the discrimination model.

[0010] Optionally, the processor is further configured to identify one or more user audio segments from the audio data according to the first threshold; and for each user audio segment, if a maximum value of audio amplitudes in the user audio segment is greater than or equal to a second threshold, determine the user audio segment as the candidate audio segment; wherein the audio amplitudes of a start time point and an end time point of the user audio segment are the first threshold, and the audio amplitudes of time points between the start time point and the end time point are all greater than the first threshold.

[0011] Optionally, the processor is further configured to divide the audio data into a plurality of audio units according to a preset time length; calculate an average audio amplitude of each audio unit; and determine the first threshold according to average audio amplitudes of N audio units, wherein N is a positive integer, and the N audio units are N audio units with the smallest average audio amplitudes in the plurality of audio units.

[0012] Optionally, the processor is further configured to output a recording prompt information if the number of valid candidate audio segments is less than a third threshold, the recording prompt information comprising an instruction of re-recording and / or a tutorial video.

[0013] In a second aspect, the embodiments of the present application further provide an audio processing system, comprising: a terminal device configured to collect audio data, the audio data being collected from user sound, the user sound being sound related to a respiratory tract; and a cloud device configured to receive the audio data from the terminal device, and input at least part of the audio data and / or feature information of the audio data into a pre-trained identification model to obtain a plurality of probability values output by the identification model, the plurality of probability values being used to represent a respiratory tract state of a user.

[0014] Optionally, the audio data is collected from the user sound in response to a collection instruction.

[0015] Optionally, the cloud device is configured to identify a candidate audio segment from the audio data, an audio amplitude in the candidate audio segment being greater than or equal to a first threshold; the cloud device is further configured to extract feature information of the candidate audio segment, and input the feature information of the candidate audio segment into a pre-trained discrimination model to obtain a discrimination result output by the discrimination model, wherein if the discrimination result is yes, the candidate audio segment is valid.

[0016] Optionally, the cloud device is configured to input the valid candidate audio segment into the recognition model to obtain the plurality of probability values.

[0017] Optionally, the cloud device is further configured to train a first preset model using first training data until a training stop condition is met to obtain the discrimination model; wherein the first training data comprises feature information of a sample candidate audio segment and label information, the label information being used to indicate whether the sample candidate audio segment is valid.

[0018] Optionally, the cloud device is further configured to calculate a similarity between the feature information of the candidate audio segment and historical feature information, the historical feature information being obtained based on feature information of historical valid audio segments of the user; if the similarity is greater than or equal to a similarity threshold, the cloud device is configured to input the feature information of the candidate audio segment into the discrimination model.

[0019] Optionally, the cloud device is further configured to identify one or more user audio segments from the audio data according to the first threshold, wherein an audio amplitude at a start time point and an end time point of the user audio segment is the first threshold, and an audio amplitude at a time point between the start time point and the end time point is greater than the first threshold; and for each user audio segment, if a maximum value of the audio amplitude in the user audio segment is greater than or equal to a second threshold, the user audio segment is determined as the candidate audio segment.

[0020] Optionally, the cloud device is further configured to divide the audio data according to a preset time length to obtain a plurality of audio units; calculate an average audio amplitude of each audio unit; and determine the first threshold according to average audio amplitudes of N audio units, wherein N is a positive integer, and the N audio units are N audio units with the smallest average audio amplitudes in the plurality of audio units.

[0021] Optionally, the cloud device is further configured to send recording prompt information to the terminal device if a number of valid candidate audio segments is less than a third threshold, the recording prompt information comprising an instruction for re-recording and / or a tutorial video.

[0022] In a third aspect, an embodiment of the present application provides an audio processing method, which comprises: obtaining audio data, the audio data being obtained by collecting user sound, the user sound being sound related to a respiratory tract; inputting at least part of the audio data and / or feature information of the audio data into a pre-trained identification model to obtain a plurality of probability values output by the identification model, the plurality of probability values being used to represent a respiratory tract state of a user.

[0023] Optionally, the audio data is obtained by collecting the user sound in response to a collection instruction.

[0024] Optionally, before inputting at least part of the audio data and / or the feature information of the audio data into the pre-trained identification model, the method further comprises: identifying a candidate audio segment from the audio data, wherein an audio amplitude in the candidate audio segment is greater than or equal to a first threshold; extracting feature information of the candidate audio segment; inputting the feature information of the candidate audio segment into a pre-trained discrimination model to obtain a discrimination result output by the discrimination model, wherein if the discrimination result is yes, the candidate audio segment is valid.

[0025] Optionally, inputting at least part of the audio data and / or the feature information of the audio data into the pre-trained identification model comprises: inputting the valid candidate audio segment into the identification model to obtain the plurality of probability values.

[0026] Optionally, before obtaining the audio data, the method further comprises: training a first preset model by using first training data until a training stop condition is met to obtain the discrimination model; wherein the first training data comprises feature information of a sample candidate audio segment and label information, the label information being used to indicate whether the sample candidate audio segment is valid.

[0027] Optionally, before inputting the feature information of the candidate audio segment into the pre-trained discrimination model, the method further comprises: calculating a similarity between the feature information of the candidate audio segment and historical feature information, the historical feature information being obtained based on feature information of a historical valid audio segment of the user; inputting the feature information of the candidate audio segment into the pre-trained discrimination model comprises: if the similarity is greater than or equal to a similarity threshold, inputting the feature information of the candidate audio segment into the discrimination model.

[0028] Optionally, identifying the candidate audio segment from the audio data comprises: identifying one or more user audio segments from the audio data according to the first threshold, wherein the audio amplitude of the start time point and the end time point of the user audio segment is the first threshold, and the audio amplitude of the time points between the start time point and the end time point is greater than the first threshold; for each user audio segment, if the maximum value of the audio amplitude in the user audio segment is greater than or equal to a second threshold, determining the user audio segment as the candidate audio segment.

[0029] Optionally, before identifying the at least one candidate audio segment from the audio data, the method further comprises: dividing the audio data into a plurality of audio units according to a preset time length; calculating the average audio amplitude of each audio unit; and determining the first threshold according to the average audio amplitudes of N audio units, wherein N is a positive integer, and the N audio units are N audio units with the smallest average audio amplitude in the plurality of audio units.

[0030] Optionally, the method further comprises: if the number of valid candidate audio segments is less than a third threshold, outputting a recording prompt information, wherein the recording prompt information comprises an instruction of re-recording and / or a tutorial video.

[0031] In a fourth aspect, an audio processing method is provided, comprising: obtaining audio data, wherein the audio data is obtained by recording a sound of a user's voluntary cough in response to a user-issued collection instruction; identifying a candidate audio segment from the audio data, wherein the audio amplitude in the candidate audio segment is greater than or equal to a first threshold; extracting feature information of the candidate audio segment; and inputting the feature information of the candidate audio segment into a pre-trained discrimination model to obtain a discrimination result output by the discrimination model, wherein if the discrimination result is yes, the candidate audio segment is valid.

[0032] Optionally, before inputting the feature information of the candidate audio segment into the pre-trained discrimination model, the method further comprises: calculating a similarity between the feature information of the candidate audio segment and historical feature information, wherein the historical feature information is obtained based on feature information of historical valid audio segments of the user; and inputting the feature information of the candidate audio segment into the discrimination model comprises: if the similarity is greater than or equal to a similarity threshold, inputting the feature information of the candidate audio segment into the discrimination model.

[0033] Optionally, identifying the candidate audio segment from the audio data comprises: identifying one or more user audio segments from the audio data according to the first threshold, wherein the audio amplitude of the start time point and the end time point of the user audio segment is the first threshold, and the audio amplitude of the time points between the start time point and the end time point is greater than the first threshold; for each user audio segment, if the maximum value of the audio amplitude in the user audio segment is greater than or equal to a second threshold, the user audio segment is determined as the candidate audio segment.

[0034] Optionally, before identifying the candidate audio segment from the audio data, the method further comprises: dividing the audio data into a plurality of audio units according to a preset time length; calculating the average audio amplitude of each audio unit; and determining the first threshold according to the average audio amplitudes of N audio units, wherein N is a positive integer, and the N audio units are N audio units with the smallest average audio amplitude in the plurality of audio units.

[0035] Optionally, the method further comprises: if the number of valid candidate audio segments is less than a third threshold, outputting a recording prompt information, wherein the recording prompt information comprises an instruction for re-recording and / or a tutorial video.

[0036] In a fifth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is run by a processor to execute the steps of the audio processing method.

[0037] Compared with the prior art, the technical scheme of the embodiment of the present application has the following beneficial effects:

[0038] In the scheme of the embodiment of the present application, the electronic device comprises a sensor and a processor, wherein the sensor is configured to collect user sound to obtain audio data, and the user sound is related to the respiratory tract; the processor is configured to input at least part of the audio data and / or feature information of the audio data into a pre-trained identification model to obtain a plurality of probability values output by the identification model, and the plurality of probability values are used to represent the respiratory tract state of the user. In the above scheme, the electronic device obtains a plurality of probability values used to represent the respiratory tract state based on the collected audio data related to the respiratory tract. On the one hand, in the above scheme, only the audio data needs to be collected by the electronic device, and professional medical images do not need to be collected, so that the convenience of the scheme is better, and the user can use the electronic device to obtain the index parameters related to the respiratory tract at any time. On the other hand, since the above probability values are calculated based on the pre-trained identification model, the accuracy of the probability values can be ensured. Therefore, the scheme provided in the embodiment of the present application can conveniently and accurately obtain the index parameters related to the respiratory tract.

[0039] Further, in the scheme of the embodiment of the present application, in response to the received collection instruction, the sound of the active cough of the user is collected to obtain audio data, then the candidate audio segment is identified from the audio data, and feature extraction is performed on the candidate audio segment, and further the feature information of the extracted candidate audio segment is input to the pre-trained discrimination model to obtain the discrimination result of the candidate audio segment, and whether the candidate audio segment is valid is determined based on the discrimination result. Thus, the above scheme can identify the valid audio segment from the audio data of the active cough through feature extraction and model identification, that is, the invalid audio segment can be excluded, which is conducive to subsequent identification of the respiratory tract state based on the active cough sound, and ensures the accuracy of the identification.

[0040] Further, in the scheme of the embodiment of the present application, the similarity between the feature information of the candidate audio segment and the historical feature information is calculated, and whether the candidate audio segment is valid is determined on the premise that the similarity is greater than or equal to a similarity threshold. By adopting such a scheme, considering that the respiratory tract states of the same user are usually common, and by the similarity between the candidate audio segment and the historical feature information, the candidate audio segment obviously not belonging to the valid audio segment is eliminated, so as to improve the accuracy of the validity determination.

[0041] Further, in the scheme of the embodiment of the present application, the first threshold value for identifying the candidate audio segment is calculated based on the audio data, and compared with the scheme of adopting the pre-set first threshold value, the above scheme can improve the accuracy of the first threshold value, so that the first threshold value more accurately represents the environmental background sound in the audio data. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a flowchart of an audio processing method in an embodiment of the present application;

[0043] Figure 2 is an architectural diagram of an audio processing system in an embodiment of the present application;

[0044] Figure 3 is a flowchart of another audio processing method in an embodiment of the present application;

[0045] Figure 4 is a schematic diagram of an amplitude curve of audio data in an embodiment of the present application;

[0046] Figure 5 is a structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0048] Example 1

[0049] Refer to Figure 1 , Figure 1 FIG. 1 is a flowchart of an audio data processing method according to an embodiment of the present application. Figure 1 The method shown can be implemented or performed by an electronic device. The electronic device can be various devices with data processing capability. For example, the electronic device can be a mobile phone computer, a tablet computer, a portable device, a wearable device, a voice recorder, etc. For another example, the electronic device can be a sensor integrated with a processor, that is, the electronic device can be a sensor with data processing function.

[0050] It should be noted that the electronic device described above is only an example, and the electronic device in the embodiment can be various devices in future as long as they have data processing function.

[0051] It should be further noted that the electronic device can also include a memory, and the processor can execute the method steps in the following manner: reading a computer program from the memory and executing the computer program to execute the steps described below.

[0052] Figure 1 The method shown includes steps S11 and S12.

[0053] S11: obtaining audio data, the audio data being obtained by collecting user voice, the user voice being respiratory tract related voice;

[0054] S12: inputting at least part of the audio data and / or feature information of the audio data to a pre-trained recognition model to obtain a plurality of probability values output by the recognition model, the plurality of probability values being used to represent a respiratory tract state of the user.

[0055] In S11, the audio data can be obtained by the electronic device collecting user voice. The user voice is respiratory tract related voice. The user voice can include at least one of the following sounds: respiratory sound, cough sound, lung sound, etc., but is not limited thereto.

[0056] In a specific implementation, the electronic device can receive a collection instruction, the collection instruction being used to instruct the electronic device to collect user voice. In response to the received collection instruction, the sensor records the user voice. For example, the collection instruction can be issued by the user whose voice is collected. That is, the electronic device can receive the collection instruction issued by the user, and in response to the collection instruction issued by the user, the electronic device records the user voice of the user, thereby obtaining the audio data.

[0057] For example, the collection instruction can be issued by a third party user, that is, the person who issues the collection instruction and the person whose voice is recorded can not be the same person.

[0058] It should be noted that in the embodiments of the present application, the collected data are all anonymized data, and based on the collected data, it cannot be known which person's data is. That is, when collecting audio data, information related to the real identity of the user will not be obtained.

[0059] In step S12, at least part of the audio data and / or the feature information of the audio data can be input to the pre-trained recognition model to obtain a plurality of probability values output by the recognition model.

[0060] Specifically, the recognition model can be deployed on the processor, and the recognition model can be obtained by training a neural network based on a large amount of training data. The training data of the recognition model can include sample audio data, and the sample audio data is labeled with a label, and the label can be an index parameter corresponding to the sample audio data.

[0061] In specific implementation, the recognition model can be trained based on a gradient descent method or the like. That is, in the training process of the recognition model, the model parameters can be automatically iteratively updated. Alternatively, the training process of the recognition model can be artificially parameterized, and the present embodiment does not limit this.

[0062] When the training data is used for training, if the training stopping condition is reached, the training can be stopped, thereby obtaining the recognition model. The training stopping condition can be that the evaluation index of the model reaches a set index threshold. For example, the evaluation index of the model can include recall rate, precision rate, accuracy rate, etc., but is not limited thereto.

[0063] In addition, the processor can also be deployed with a discrimination model, and the specific content of the discrimination model can be referred to the related description below, which will not be repeated here.

[0064] It should be noted that the names of the "recognition model" and the "discrimination model" do not limit the models mentioned in the embodiments of the present application. In other embodiments, the "recognition model" and the "discrimination model" mentioned above can also be referred to as "first model", "second model" and the like.

[0065] The plurality of probability values can correspond to a plurality of evaluation indexes respectively, that is, the probability values and the evaluation indexes can be one-to-one correspondence. For example, the evaluation indexes can be obstructive indexes and / or restrictive indexes. The obstructive indexes can be used to describe airway outflow obstruction, and the corresponding probability values can represent the severity of the outflow obstruction. The restrictive indexes can be used to describe airway inflow obstruction, and the corresponding probability values can represent the severity of the inflow obstruction.

[0066] In a specific implementation, if the user sound collected by the audio data includes multiple sounds (such as respiratory sound, cough sound, and lung sound), the probability values can be obtained based on the audio parts corresponding to the multiple sounds, or the probability values can be obtained based on the audio part corresponding to one of the multiple sounds.

[0067] As a possible implementation, in S12, feature extraction can be performed on the audio data to obtain feature information of the audio data, and then the feature information of the audio data is input into the recognition model to obtain the plurality of probability values output by the recognition model.

[0068] For example, the electronic device can extract feature information of the audio parts corresponding to the multiple user sounds in the audio data, then fuse the feature information of the audio parts corresponding to the multiple user sounds to obtain a fused feature vector, and then input the fused feature vector into the recognition model to obtain the plurality of probability values.

[0069] As another possible implementation, in S12, the audio data can be directly input into the recognition model to obtain the plurality of probability values output by the recognition model.

[0070] As another possible implementation, in S12, the respiratory sound audio part and the cough sound audio part can be identified from the audio data. The identification can be based on the mel spectrum of the audio data.

[0071] Further, if the proportion of the respiratory sound audio part in the audio data is less than a pre-set proportion threshold, the respiratory sound audio part in the audio data can be ignored. Further, feature information of the cough sound audio part can be extracted, and then the feature information of the cough sound audio part is input into the pre-trained recognition model as the feature information of the audio data. For example, in the case where the user records the cough sound, a part of the respiratory sound will inevitably be introduced, and in the case where the subsequent processing is based on the cough sound, the respiratory sound can be ignored.

[0072] For example, if the proportion of the respiratory sound audio part in the audio data is less than a pre-set proportion threshold, the respiratory sound audio part can be ignored, and the cough sound audio part can be executed Figure 3 Some steps in the method shown.

[0073] If the proportion of the respiratory sound audio part in the audio data and the proportion of the cough sound audio part in the audio data are both greater than the proportion threshold, the feature information of the respiratory sound audio part and the cough sound audio part can be extracted respectively, and then the feature information of the respiratory sound audio part and the feature information of the cough sound audio part are fused to obtain fused feature information, and then the fused feature information is input into the recognition model to obtain a plurality of probability values.

[0074] If the proportion of the respiratory sound audio part in the audio data and the proportion of the cough sound audio part in the audio data are both less than the proportion threshold, the user can be reminded to re-record the user sound.

[0075] It should be particularly pointed out that the probability values obtained above are related to the evaluation indexes, but these evaluation indexes themselves are not related to the health status of the respiratory tract. In other words, the probability values obtained above are related to the respiratory tract state, but the "respiratory tract state" referred to in the present application is not the "health status of the respiratory tract". In order to determine the health status of the respiratory tract, one or more other indexes need to be combined, such as blood test results, chest X-ray recognition results, etc. This is because even a person in a healthy state can cause these evaluation indexes to be in a very regular probability value interval due to various factors such as exercise, temporary muscle disorders, etc.

[0076] In the above scheme, the electronic device obtains a plurality of probability values for representing the respiratory tract state based on the collected respiratory tract related audio. Since only audio data needs to be collected in the above scheme, without the need to collect professional medical images, the convenience of the scheme is better, and the user can use the electronic device to obtain the above objective data at any time. On the other hand, since the above probability values are calculated based on the pre-trained recognition model, the accuracy of the probability values can be guaranteed. Therefore, the scheme provided by the embodiments of the present application can conveniently and accurately obtain the index parameters related to the respiratory tract.

[0077] Example 2

[0078] The embodiments of the present application also provide an audio processing system, in which the audio processing method of the present application can be executed by the interaction of the terminal device and the cloud device. By using such a scheme, the computing power of the cloud device can be used to improve the execution efficiency of the audio processing method.

[0079] Reference Figure 2 , Figure 2 is a schematic diagram of the architecture of an audio processing system in the embodiments of the present application. The following will be described in combination with Figure 2 The application scenarios of the audio processing system and the audio processing method provided in the embodiments of the present application will be specifically described.

[0080] As Figure 2As shown, the audio processing system 10 can include a terminal device 11 and a cloud device 12, the cloud device 12 being coupled with the terminal device 11 so that the cloud device 12 can interact with the terminal device 11. In a specific implementation, the cloud device 12 can be a server or other electronic device having data receiving and processing capabilities. Exemplarily, the cloud device 12 and the terminal device 11 can be connected through a peer-to-peer network.

[0081] The number of terminal devices 11 coupled with the cloud device 12 is not limited in the embodiments of the present application. The terminal device 11 can be an electronic device used by a user, for example, a mobile phone, a computer, a tablet computer, a wearable device, a voice recorder, etc. More details about the terminal device 11 can be referred to the specific description of the electronic device above, which will not be repeated here.

[0082] In the application scenario of the embodiments of the present application, the terminal device 11 can receive a collection instruction issued by a user, and in response to the collection instruction issued by the user, the terminal device 11 records a sound to obtain audio data, and then the terminal device 11 sends the audio data to the cloud device 12.

[0083] Exemplarily, the user sound can include a sound of active coughing of the user. Specifically, the terminal device 11 can send a detection request to the cloud device 12, and in response to the received detection request, the cloud device 12 can send response information of the detection request to the terminal device 11. The response information of the detection request can include a tutorial video, which can be used to guide the user to actively cough, and the user can record the sound of active coughing using the terminal device 11 after watching the tutorial video.

[0084] Exemplarily, after the cloud device 12 receives the detection request sent by the terminal device, the cloud device 12 can first send a basic information questionnaire to the terminal device 11, and the terminal device 11 receives the basic information questionnaire and fills it out by the user. Further, the terminal device 11 sends the filled basic information questionnaire to the cloud device 12. Further, in response to the received filled basic information questionnaire, the cloud device 12 can send response information of the detection request to the terminal device 11. Through the filling of the basic information questionnaire, the historical state information of the user, such as the past medical history, etc. can be collected. It should be noted that the user can authorize the cloud device, and the cloud device can collect or record the historical state information of the user on the premise of the authorization of the user. It should be noted that the above-mentioned basic information questionnaire does not involve the real identity of the user, and the basic information questionnaire can be saved or processed in the cloud device 12 based on the virtual identifier of the user. The virtual identifier can be used to uniquely identify the user and does not involve the real identity information of the user.

[0085] In a specific implementation, for the tutorial video, the user can choose to watch the tutorial video or skip the tutorial video, and after the tutorial video is watched, the user can also choose to replay the tutorial video.

[0086] Further, after the tutorial video is played or skipped, the terminal device 11 can display a recording screen, and the user can click a "record" control to issue a collection instruction to cause the terminal device 11 to record the user voice. In addition, the instruction to end recording can also be issued by the user, that is, in response to the instruction to end recording issued by the user, the terminal device 11 stops recording.

[0087] In addition, the response information of the detection request can also include a microphone access permission request, and the user can authorize the cloud device 12 first, and then issue the collection instruction after authorization.

[0088] From the above, through the above interaction process, the terminal device 11 can collect audio data and send the collected audio data to the cloud device 12.

[0089] Further, after the cloud device 12 receives the audio data sent by the terminal device 11, the cloud device 12 can perform the audio processing method provided by the embodiments of the present application on the audio data.

[0090] For more information about the above audio processing system 10 and the audio processing method, please refer to the detailed description of other embodiments, which will not be repeated here.

[0091] In other embodiments of the present application (such as the above-mentioned embodiment 1), the terminal device 11 receives the collection instruction issued by the user, and in response to the collection instruction issued by the user, the terminal device 11 records the user voice to obtain audio data, and then the terminal device 11 does not send the audio data to the cloud device, but processes it locally, that is, the terminal device 11 performs the audio processing method provided by the embodiments of the present application.

[0092] Example 3

[0093] Considering that cough can well reflect the state of the respiratory tract, for example, the respiratory tract is in different states, the cough is different in severity, duration, and frequency. Therefore, the probability value can be calculated based on the cough sound in the user voice. For example, the user's cough sound can be continuously monitored to capture the user's cough sound.

[0094] Specifically, the cough sound collected through continuous monitoring is the sound of passive cough of the user. Passive cough can refer to cough caused by respiratory tract discomfort that is not under the control of the user. Passive cough has a high incidence, and therefore, continuous monitoring is required to capture the cough sound of passive cough.

[0095] In order to identify the respiratory tract state based on the cough sound of the user without continuous monitoring, the inventors of the present application consider detecting the respiratory tract state based on the cough sound of the active cough of the user. Compared with passive cough, the active cough has high autonomy and is not limited by time and space. The user can choose the time at any time and the required detection time is shorter, which greatly reduces the user's concerns about privacy.

[0096] However, the active cough is a cough performed by the user autonomously. Unlike passive cough, the user may not exert force properly or exert force incorrectly when actively coughing. For example, the user may easily confuse clearing the throat with coughing. Therefore, when the user actively coughs, some cough sounds may not accurately restore the state of passive cough, which may not be suitable for identifying the respiratory tract state.

[0097] Therefore, in the scheme of the present embodiment, before inputting the collected audio data into the identification model, candidate audio segments are identified from the audio data, and features of the candidate audio segments are extracted. The extracted feature information of the candidate audio segments is further input into the pre-trained discrimination model to obtain a discrimination result of the candidate audio segments, and whether the candidate audio segments are valid is determined based on the discrimination result. Thus, the above scheme can identify valid audio segments from audio data through feature extraction and model identification, i.e., can exclude invalid audio segments, which is conducive to ensuring the accuracy of identification when identifying the respiratory tract state based on active cough sounds. For example, it can be conducive to ensuring that the plurality of probability values obtained above can accurately represent the respiratory tract state.

[0098] It should be noted that, Figure 3 The method shown can be used for the effectiveness determination of cough sounds, that is, by Figure 3 The scheme shown can identify valid active cough sounds from audio data or the cough sound audio part mentioned above. In other embodiments, Figure 3 The method shown can also be used for the effectiveness determination of respiratory sounds. That is, by Figure 3 The scheme shown can identify valid respiratory sounds from audio data. Specifically, if the audio data is recorded outdoors, wind noise is easily introduced during recording due to weather and other reasons. The audio characteristics of wind noise and the audio characteristics of respiratory sounds have a certain degree of similarity. Therefore, the respiratory sound audio part obtained above needs to be determined for effectiveness to identify valid respiratory sounds.

[0099] Referring to Figure 3 , Figure 3is a flowchart of another audio processing method in the embodiments of the present application. The method can be executed by the electronic device described above or can be executed by the audio processing system described above. The following will be described taking the audio processing system as an execution subject. Figure 3 The illustrated audio processing method can include:

[0100] Step S21: Obtain audio data, which is obtained by recording user sound in response to a collection instruction issued by a user;

[0101] Step S22: Identify a candidate audio segment from the audio data, wherein the audio amplitude in the candidate audio segment is greater than or equal to a first threshold value;

[0102] Step S23: Extract feature information of the candidate audio segment;

[0103] Step S24: Input the feature information of the candidate audio segment into a pre-trained discrimination model to obtain a discrimination result output by the discrimination model, wherein if the discrimination result is yes, the candidate audio segment is valid.

[0104] In step S21, after the terminal device records the user sound, the audio data is obtained, and then the terminal device sends the audio data to the cloud device. Alternatively, the terminal device can also not send the audio data to the cloud device, but perform subsequent processing on the audio data locally. The user sound can include user voluntary coughing sound, and can also include breathing sound, etc.

[0105] Specifically, the original audio data collected by the terminal device can be a pulse code modulation (PCM) data stream. Before sending the audio data, the terminal device can sample the PCM data stream to obtain the audio data. The sampling rate for sampling the PCM data stream can be set according to actual application requirements, for example, the sampling rate can be 16000 Hz.

[0106] In a specific implementation, before sending the audio data to the cloud device, the terminal device can first calculate a background noise value of the audio data, and determine whether the background noise value is greater than or equal to a noise threshold value. If the background noise value is greater than or equal to the noise threshold value, the terminal device can issue a prompt to the user to re-record, and if the background noise value is less than the noise threshold value, the terminal device can send the audio data to the cloud device.

[0107] In one specific example, the terminal device can divide the audio data into a plurality of first time windows according to a first time length, and then calculate the sound pressure level of the audio data in each first time window. Further, the terminal device can select a part of the sound pressure levels from the sound pressure levels of the plurality of first time windows to calculate the mean and standard deviation of the sound pressure levels of the plurality of first time windows. For example, the terminal device can sort the sound pressure levels of the first time windows in ascending order, and select the top 10% of the sound pressure levels with the smallest values to calculate the mean and standard deviation of the sound pressure levels of all the first time windows. Then the sum of the mean and standard deviation is taken as the background noise value of the audio data. The first time length can be 0.025 seconds, but is not limited thereto.

[0108] Alternatively, in other embodiments, the terminal device can take the result of subtracting the standard deviation from the mean as the background noise value of the audio data. Alternatively, in other embodiments, only the mean of the sound pressure levels of the plurality of first time windows can be calculated, and the mean is taken as the background noise value of the audio data.

[0109] In step S22, the cloud device can identify candidate audio segments from the audio data uploaded by the terminal device. In the case of cough sound effectiveness judgment, the candidate audio segment can refer to an active cough sound segment. The following is mainly described by way of example with cough sound. In the case of respiratory sound effectiveness judgment, the candidate audio segment can refer to a respiratory sound segment, for example, the candidate audio segment can be determined from the audio data based on a preset threshold corresponding to the respiratory sound, and then input into the discriminant model for effectiveness judgment, so as to identify the effective respiratory sound in the audio data.

[0110] Specifically, the cloud device can first identify a user audio segment from the audio data, and the user audio segment contains the sound information emitted by the user. In a specific implementation, the user audio segment can be identified from the audio data based on a first threshold, and the first threshold can be used to represent the environmental background sound, and a second threshold can be used to represent the cough sound.

[0111] Further, for each user audio segment, if the maximum value of the audio amplitude in the user audio segment is greater than the second threshold, the cloud device can determine that the user audio segment is a candidate audio segment, and if the maximum value is less than the second threshold, the cloud device can determine that the user audio segment is not a candidate audio segment. The audio amplitude can refer to the amplitude of the audio signal.

[0112] In a specific implementation, the first threshold value and / or the second threshold value can be preset, or the first threshold value and / or the second threshold value can also be calculated based on the audio data. For example, the first threshold value and / or the second threshold value can be calculated based on an audio signal envelope.

[0113] As an example, before identifying the candidate audio segment, the cloud device can divide the audio data according to a preset time length to obtain a plurality of audio units; then calculate the average audio amplitude of each audio unit; and determine the first threshold value based on the average audio amplitudes of N audio units, where N is a positive integer, and the N audio units are N audio units with the smallest average audio amplitude in the plurality of audio units. Wherein, N can be a fixed value, or N can also be determined according to the number of audio units. For example, N = 10% x number of audio units.

[0114] In an embodiment of the present application, before identifying the candidate audio segment, the cloud device can process the audio data to obtain an audio signal envelope, and then determine the start time point and the end time point based on the audio signal envelope, and then determine the candidate audio segment in the audio data based on the start time point and the end time point.

[0115] Specifically, the cloud device can perform Hilbert transform on the audio data, then take the absolute value of the transformed audio data, and then perform low-pass filtering on the data obtained after taking the absolute value, thereby obtaining the audio signal envelope. Wherein, the points on the audio signal envelope can be used to represent the audio amplitude at a certain time.

[0116] Further, the cloud device can determine the zero-crossing point on the audio signal envelope relative to the first threshold value based on the first threshold value and the audio signal envelope, wherein the zero-crossing point relative to the first threshold value can refer to the point on the audio signal envelope with a value of the first threshold value. In other words, the zero-crossing point relative to the first threshold value can refer to the intersection of the audio signal envelope and the first threshold line.

[0117] Further, if at least a part of the audio signal envelope between the two adjacent zero-crossing points is greater than the second threshold value, it can be determined that the two adjacent zero-crossing points are the start time point and the end time point, respectively. Specifically, the former zero-crossing point in the two adjacent zero-crossing points is the start time point, and the latter zero-crossing point is the end time point.

[0118] Further, based on the start time point and the end time point, the candidate audio segment can be intercepted in the audio data in step S22.

[0119] In another embodiment, the cloud device can divide the audio signal envelope according to a second time length to obtain a plurality of second time windows. For example, the second time length can be 0.05 seconds. Further, based on the audio envelope in each time window, the mean value of the audio amplitude in the second time window can be calculated, and the obtained mean value is taken as the audio amplitude corresponding to the second time window. Thus, the audio amplitude corresponding to each second time window can be obtained.

[0120] Further, the cloud device can select a part of the audio amplitudes corresponding to all the second time windows to calculate the mean value and the standard deviation. For example, the audio amplitudes corresponding to each second time window are sorted in ascending order, and the first 10% of the audio amplitudes with the smallest values are selected to calculate the mean value and the standard deviation of the audio amplitudes corresponding to all the second time windows.

[0121] More specifically, if the sum of the calculated mean value and the standard deviation is less than or equal to a set threshold value, the sum of the mean value and the standard deviation is taken as the first threshold value, and if the sum of the calculated mean value and the standard deviation is greater than the set threshold value, the set threshold value is taken as the first threshold value.

[0122] In addition, based on the audio amplitudes corresponding to each second time window, an amplitude curve of the audio signal can be drawn. Then, the starting time point and the ending time point are determined according to the amplitude curve and the first threshold value and the second threshold value.

[0123] Reference Figure 4 , Figure 4 An amplitude curve of an audio signal in an embodiment of the present application is shown. Figure 4 The physical quantity of the abscissa is time, and the physical quantity of the ordinate can be used to represent the audio amplitude. Specifically, Figure 4 The amplitude curve of the audio data shown can be obtained based on the audio signal envelope. In addition, Figure 4 The first threshold line and the second threshold line are also shown. The focus of the first threshold line and the amplitude curve is the zero-crossing point of the amplitude curve relative to the first threshold value, and if at least a part of the amplitude curve between two adjacent zero-crossing points is greater than the second threshold value, the two adjacent zero-crossing points are the starting time point and the ending time point of a candidate audio segment, respectively. Figure 4 As shown, Figure 4 T0 in the above represents the starting time point, and T1 represents the ending time point.

[0124] It should be noted that before performing S22, the cough audio part can be extracted from the audio data, and then the candidate audio segment can be identified from the cough audio part. The specific content of extracting the cough audio part from the audio data can refer to the related description of the above Figure 1 , which will not be described here in detail.

[0125] Continuing to refer toFigure 3 In step S23, for each candidate audio segment, feature information of the candidate audio segment can be extracted. Exemplarily, there can be multiple types of feature information, and for each candidate audio segment, multiple types of feature information of the candidate audio segment can be concatenated to obtain the feature information of the candidate audio segment.

[0126] It should be noted that the type and extraction method of the feature information are not limited in this embodiment. Exemplarily, the energy feature information and the spectral feature information of the candidate audio segment can be extracted, for example, the energy feature information can include one or more of the following: mean energy, standard root mean square error, and the spectral feature information can include: spectral slope, spectral centroid, maximum frequency.

[0127] In step S24, the cloud device can determine whether the candidate audio segment is valid based on the feature information of the candidate audio segment, that is, determine whether the candidate audio segment is an effective audio segment. Wherein, the effective audio segment can refer to an effective active cough sound, that is, the effective audio segment can reflect the respiratory tract state of passive cough and can be used for subsequent respiratory tract state recognition and detection. Specifically, the effective audio segment can be used for subsequent probability value calculation.

[0128] Specifically, the cloud device can input the feature information of each candidate audio segment into a pre-trained discriminant model to obtain a discriminant result of the candidate audio segment. If the discriminant result is yes, the candidate audio segment is valid, that is, the candidate audio segment is an effective audio segment, otherwise, if the discriminant result is no, the candidate audio segment is invalid, that is, the candidate audio segment is an invalid audio segment. In other words, the effective audio segment is an effective active cough sound, and the invalid audio segment is not an effective active cough sound.

[0129] That is, the input of the discriminant model can be the feature information of the candidate audio segment, and the output of the discriminant model can be the discriminant result of the candidate audio segment.

[0130] Specifically, the discriminant model can be a pre-trained binary classification model. More specifically, the first preset model can be trained using first training data until a training stop condition is reached, thereby obtaining the discriminant model. Wherein, the first training data can include: feature information of a sample candidate audio segment and label information of the sample candidate audio segment, and the label information can be used to indicate whether the sample candidate audio segment is valid.

[0131] It should be noted that the structure of the first preset model can be the structure of various existing appropriate binary classification models, and the present embodiment does not limit this. In addition, the training method for training the first preset model using the first training data can be an existing appropriate neural network model training method, such as gradient descent method, and the present embodiment does not limit this.

[0132] From the above, in the above scheme, the cloud device first extracts the active cough sound (i.e., the candidate audio segment) from the audio data, and then determines whether the active cough sound is valid through feature extraction and model recognition, so as to be able to identify the valid audio segment from the active cough audio data, that is, to be able to exclude the invalid audio segment, which is beneficial to subsequent identification of the respiratory tract state based on the active cough sound, so as to ensure the accuracy of the identification.

[0133] In a specific implementation, if the number of valid audio segments in the audio data uploaded by the user is less than the third threshold, the cloud device can send recording prompt information to the terminal device, and the recording prompt information can include instructions and / or tutorial videos for re-recording.

[0134] In one non-limiting example, before step S24 is performed, the cloud device can first calculate the similarity between the feature information of the candidate audio segment and the historical feature information, wherein the historical feature information can be obtained based on the feature information of the historical valid audio segment of the user. That is, if the result of step S24 is yes, the cloud device can save the feature information input to the discrimination model. Further, if the same user uploads audio data multiple times, the cloud device can update the historical feature information corresponding to the user based on the feature information of the valid audio segment in each uploaded audio data, so that the historical feature information saved at the cloud device can reflect the respiratory tract state of the user in the recent period of time. It should be noted that the saving and use of the feature information can be performed based on the authorization of the user, and in addition, the saving and use of the feature information can be based on the virtual identifier of the user.

[0135] Further, if the similarity is greater than or equal to the similarity threshold, the cloud device can continue to perform step S24, and if the above similarity is less than the similarity threshold, it can be considered that the candidate audio segment is obviously not matched with the respiratory tract state of the user in the recent period of time, and the cloud device can send the above recording prompt information to the terminal device to prompt the user to re-record the active cough sound.

[0136] Therefore, in the above scheme, the respiratory tract state of the same user is generally common, and for this reason, the present scheme eliminates the candidate audio segment that is obviously not a valid audio segment through the similarity between the candidate audio segment and the historical feature information, so as to improve the accuracy of the validity determination.

[0137] In another non-limiting example, considering that cough sounds are usually regular and there is usually a certain interval between two adjacent cough sounds, then the environment where the user is during the recording process can have sudden loud noises, such as construction noise, etc.

[0138] To this end, before performing step S23, for two adjacent candidate audio segments identified in step S22, if the interval between the two adjacent candidate audio segments is greater than or equal to the interval threshold, the cloud device can continue to perform step S23. If the interval between the two adjacent candidate audio segments is less than the interval threshold, the cloud device can discard the two candidate audio segments. That is, if the interval between the two adjacent candidate audio segments is less than the interval threshold, it means that one or both of the candidate audio segments are likely not cough sounds, for example, they can be sudden loud noises, in which case the cloud device can discard the two candidate audio segments, where "discard" means not performing subsequent feature extraction and effective judgment steps.

[0139] With such a scheme, it is beneficial to exclude candidate audio segments that are obviously not cough sounds, and it is beneficial to improve the accuracy of effective audio segment judgment.

[0140] Further, for the effective audio segment, the cloud device can further calculate the plurality of probability values based on the effective audio segment. Specifically, the cloud device can be further deployed with a pre-trained identification model, which can be used to determine the plurality of probability values.

[0141] More specifically, the cloud device can input the effective audio segment into the identification model to obtain the plurality of probability values output by the identification model.

[0142] In the following, only one identification model in the application embodiment is described by way of example.

[0143] In a specific implementation, if the length of the effective audio segment input into the identification model can be fixed. For example, it can be 0.65 seconds. If the length of the effective audio segment is greater than the fixed length corresponding to the identification model, the cloud device can cut a fixed-length audio segment from the effective audio segment, and then input the cut audio segment into the identification model. If the length of the effective audio segment is less than the fixed length, the cloud device can perform a zero padding operation to make the length of the effective audio segment reach the fixed length.

[0144] Specifically, the recognition model can include a pre-processing module, a feature extraction module, and a prediction module. The input of the feature extraction module is connected with the output of the pre-processing module, and the output of the feature extraction module is connected with the output of the classification module. Exemplarily, the feature module and the classification module can be connected through a residual network (ResNet). The pre-processing module can be used to pre-process the valid audio segments input to the recognition model. Specifically, the pre-processing can include converting the valid audio segments of a fixed time length into a spectrum, and normalizing the converted spectrum.

[0145] Further, the feature extraction module can be used to extract feature information of the valid audio segments. Specifically, the feature extraction module can perform feature extraction on the output data of the pre-processing module to obtain the feature information of the valid audio segments. Exemplarily, the feature extraction module can include a Bi-directional Long Short-Term Memory (BiLSTM) layer, for example, the feature extraction module can include two layers of BiLSTM layers.

[0146] Further, the prediction module can calculate a plurality of probability values based on the feature information of the valid audio segments.

[0147] For the above-mentioned audio data, the cloud device can respectively input each valid audio data in the audio data to the recognition model to obtain a probability value of each valid audio data, and then calculate a plurality of probability values corresponding to the audio data based on the probability values of each valid audio data.

[0148] Exemplarily, the recognition model can also include a linear fitting layer, the input of the linear fitting layer is connected with the output of the recognition model, and the output of the linear fitting layer is recorded as the plurality of probability values corresponding to the audio data. The linear fitting layer can be used to perform weighted average processing on the plurality of probability values of each valid audio data in the same audio data.

[0149] Alternatively, for the above-mentioned audio data, the cloud device can respectively fuse the feature information of each valid audio data, and then input the fused feature information to the prediction module to obtain a plurality of probability values corresponding to the audio data.

[0150] It should be noted that the recognition model can be a multi-label model obtained through pre-training. More specifically, the second pre-set model can be trained using second training data until a training stop condition is reached, thereby obtaining the recognition model. The second training data can include sample valid audio segments and label information of the sample valid audio segments, and the label information of the sample valid audio segments can refer to the grades of the plurality of evaluation indexes corresponding to the sample valid audio segments.

[0151] It should be further noted that the training method for training the second preset model by using the second training data can be a training method of an existing appropriate neural network model, such as a gradient descent method, but is not limited thereto.

[0152] In a case where a terminal device executes the audio processing method provided in the embodiments of the present application, the discrimination model and the recognition model described above can be deployed on the terminal device.

[0153] It can be understood that in a specific implementation, the above method can be implemented in the form of a software program running in a processor integrated in a chip or a chip module; or the method can be implemented in a hardware manner or a combination of software and hardware, for example, implemented by a dedicated chip or chip module or implemented by a dedicated chip or chip module in combination with a software program.

[0154] It should be understood that each of the above embodiments can be used alone or in combination to achieve different technical effects.

[0155] Reference Figure 5 , Figure 5 is a structural schematic diagram of an electronic device in the embodiments of the present application. As shown in Figure 5 , Figure 5 The electronic device shown can include:

[0156] The sensor 41 is configured to collect user sound to obtain audio data, the user sound being sound related to a respiratory tract;

[0157] The processor 42 is configured to input at least part of the audio data and / or feature information of the audio data to a pre-trained recognition model to obtain a plurality of probability values output by the recognition model, the plurality of probability values being used to represent a respiratory tract state of the user.

[0158] For more details about the working principle, working method and beneficial effects of the electronic device in the embodiments of the present application, reference can be made to the related description of the audio processing method above, which will not be repeated here.

[0159] The embodiments of the present application further provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program is run by a processor to perform the steps of the audio processing method described above. The storage medium can include ROM, RAM, magnetic disk or optical disk, etc. The storage medium can further include a non-volatile memory or a non-transitory memory, etc.

[0160] The electronic device provided by the embodiments of the present application can include a memory and a processor, the memory stores a computer program which can run on the processor, and the processor executes the steps of the audio processing method when running the computer program. The electronic device includes, but is not limited to, a mobile phone, a computer, a tablet computer, and other terminal devices.

[0161] The embodiments of the present application also provide an audio processing system, which can include a terminal device and a cloud device. The terminal device is configured to collect audio data, the audio data is obtained by collecting user sound, and the user sound is related to a respiratory tract. The cloud device is configured to receive the audio data from the terminal device, and input at least part of the audio data and / or feature information of the audio data into a pre-trained identification model to obtain a plurality of probability values output by the identification model, the plurality of probability values are used to represent a respiratory tract state of a user.

[0162] For more information about the terminal device and the cloud device in the audio processing system, please refer to the relevant description above, which will not be repeated here.

[0163] It should be understood that, in the embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0164] It should also be understood that the memory in the embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM) used as an external cache. By way of example, but not by way of limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM) and direct rambus random access memory (DR RAM)

[0165] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer program can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer program can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired or wireless means.

[0166] In several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other manners. For example, the described device embodiments are merely illustrative; the division of the units is merely logical function division; and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. The units described as separated components can or can not be physical separated, and the components displayed as units can or can not be physical units; that is, they can be located in one place, or distributed to a plurality of network units. In actual implementation, a part or all of the units can be selected according to the actual needs to achieve the purposes of the embodiments.

[0167] In addition, each functional unit in the various embodiments of the present application can be integrated into one processing unit, or each unit can be a separate physical unit, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of hardware plus software function units. For example, for each device or product applied to or integrated into a chip, each module / unit contained therein can be implemented in the form of a circuit or other hardware, or at least part of the modules / units can be implemented in the form of a software program running on a processor integrated in the chip, and the remaining (if any) part of the modules / units can be implemented in the form of a circuit or other hardware; for each device or product applied to or integrated into a chip module, each module / unit contained therein can be implemented in the form of a circuit or other hardware, and different modules / units can be located in the same component (for example, a chip, a circuit module, etc.) or different components of the chip module, or at least part of the modules / units can be implemented in the form of a software program running on a processor integrated in the chip module, and the remaining (if any) part of the modules / units can be implemented in the form of a circuit or other hardware; for each device or product applied to or integrated into a terminal, each module / unit contained therein can be implemented in the form of a circuit or other hardware, and different modules / units can be located in the same component (for example, a chip, a circuit module, etc.) or different components of the terminal, or at least part of the modules / units can be implemented in the form of a software program running on a processor integrated in the terminal, and the remaining (if any) part of the modules / units can be implemented in the form of a circuit or other hardware.

[0168] It should be understood that the term "and / or" herein merely describes an associated relationship with the associated objects, and can represent three relationships, for example, A and / or B, which can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " herein represents an "or" relationship between the front and rear associated objects.

[0169] The "multiple" appearing in the embodiments of the present application refers to two or more than two. The first, second, and the like appearing in the embodiments of the present application are only for illustrative and distinguishing purposes, and do not have any order, nor represent a special limitation on the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application.

[0170] Although the present application is disclosed as above, the present application is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and therefore the scope of protection of the present application should be subject to the scope defined by the claims.

Claims

1. An electronic device, comprising: The device comprises: a sensor configured to collect user sound to obtain audio data, the user sound being airway-related sound; a processor configured to input at least part of the audio data and / or feature information of the audio data into a pre-trained identification model to obtain a plurality of probability values output by the identification model, the plurality of probability values being used to represent a respiratory tract state of a user; the plurality of probability values respectively correspond to different evaluation indexes, the evaluation indexes comprising an obstructive index and / or a restrictive index, the probability value corresponding to the obstructive index representing a severity of an expiratory disorder, and the probability value corresponding to the restrictive index representing a severity of an inspiratory disorder; the inputting of at least part of the audio data and / or the feature information of the audio data into the pre-trained identification model to obtain the plurality of probability values output by the identification model comprises: identifying a candidate audio segment from the audio data, wherein an audio amplitude in the candidate audio segment is greater than or equal to a first threshold value; extracting feature information of the candidate audio segment; inputting the feature information of the candidate audio segment into a pre-trained discrimination model to obtain a discrimination result output by the discrimination model, wherein if the discrimination result is yes, the candidate audio segment is valid, and the valid candidate audio segment is input into the identification model to obtain the plurality of probability values; the processor is further configured to divide the audio data according to a preset time length to obtain a plurality of audio units, calculate an average audio amplitude of each audio unit, and determine the first threshold value according to average audio amplitudes of N audio units, wherein N is a positive integer, and the N audio units are N audio units with the smallest average audio amplitudes among the plurality of audio units.

2. The electronic device of claim 1, wherein, The audio data is obtained by collecting the user sound in response to a collection instruction.

3. The electronic device of claim 1, wherein, The processor is further configured to identify one or more user audio segments from the audio data according to the first threshold value, and determine, for each user audio segment, that the user audio segment is the candidate audio segment if a maximum value of audio amplitudes in the user audio segment is greater than or equal to a second threshold value. The audio amplitudes of a starting time point and an ending time point of the user audio segment are the first threshold value, and the audio amplitudes of time points between the starting time point and the ending time point are all greater than the first threshold value.

4. An audio processing system, characterized by The system comprises: a terminal device configured to collect audio data, the audio data being obtained by collecting user sound, the user sound being airway-related sound; The cloud device is configured to receive the audio data from the terminal device, and input at least part of the audio data and / or feature information of the audio data into a pre-trained identification model to obtain a plurality of probability values output by the identification model, the plurality of probability values being used to represent a respiratory tract state of a user; the plurality of probability values respectively correspond to different evaluation indexes one by one, the evaluation indexes including an obstructive index and / or a restrictive index, the probability value corresponding to the obstructive index representing a severity of an expiratory disorder, and the probability value corresponding to the restrictive index representing a severity of an inspiratory disorder. The cloud device is configured to receive the audio data from the terminal device, and input at least part of the audio data and / or feature information of the audio data into a pre-trained identification model to obtain a plurality of probability values output by the identification model, the plurality of probability values being used to represent a respiratory tract state of a user; the plurality of probability values respectively correspond to different evaluation indexes one by one, the evaluation indexes including an obstructive index and / or a restrictive index, the probability value corresponding to the obstructive index representing a severity of an expiratory disorder, and the probability value corresponding to the restrictive index representing a severity of an inspiratory disorder. The cloud device is further configured to divide the audio data according to a preset time length to obtain a plurality of audio units, calculate an average audio amplitude of each audio unit, and determine the first threshold value according to average audio amplitudes of N audio units, where N is a positive integer, and the N audio units are N audio units with the smallest average audio amplitudes in the plurality of audio units.

5. The audio processing system of claim 4, wherein, The audio data is obtained by collecting the user sound in response to a collection instruction.

6. The audio processing system of claim 4, wherein, The cloud device is further configured to identify one or more user audio segments from the audio data according to the first threshold value, where an audio amplitude of a starting time point and an ending time point of the user audio segment is the first threshold value, and an audio amplitude of a time point between the starting time point and the ending time point is greater than the first threshold value, and for each user audio segment, if a maximum value of the audio amplitude in the user audio segment is greater than or equal to a second threshold value, the user audio segment is determined as the candidate audio segment.

7. An audio processing method, characterized by, The method comprises: obtaining audio data, the audio data being obtained by collecting user sound, the user sound being a sound related to a respiratory tract; inputting at least part of the audio data and / or feature information of the audio data into a pre-trained identification model to obtain a plurality of probability values output by the identification model, the plurality of probability values being used to represent a respiratory tract state of a user; the plurality of probability values respectively correspond to different evaluation indexes one by one, the evaluation indexes including an obstructive index and / or a restrictive index, the probability value corresponding to the obstructive index representing a severity of an expiratory disorder, and the probability value corresponding to the restrictive index representing a severity of an inspiratory disorder; The inputting of the at least part of the audio data and / or the feature information of the audio data into the pre-trained identification model to obtain a plurality of probability values output by the identification model comprises: identifying a candidate audio segment from the audio data, wherein an audio amplitude in the candidate audio segment is greater than or equal to a first threshold; extracting feature information of the candidate audio segment; inputting the feature information of the candidate audio segment into a pre-trained discrimination model to obtain a discrimination result output by the discrimination model, wherein if the discrimination result is yes, the candidate audio segment is valid, and the valid candidate audio segment is input into the identification model to obtain the plurality of probability values. The first threshold is obtained by the following steps: dividing the audio data according to a preset time length to obtain a plurality of audio units; calculating an average audio amplitude of each audio unit; and determining the first threshold according to average audio amplitudes of N audio units, wherein N is a positive integer, and the N audio units are N audio units with the smallest average audio amplitudes in the plurality of audio units.

8. The audio processing method of claim 7, wherein, The audio data is obtained by collecting the user voice in response to a collection instruction issued by the user.

9. The audio processing method of claim 7, wherein, The identifying of the candidate audio segment from the audio data comprises: According to the first threshold, one or more user audio segments are identified from the audio data, wherein an audio amplitude at a starting time point and an ending time point of the user audio segment is the first threshold, and an audio amplitude at a time point between the starting time point and the ending time point is greater than the first threshold; For each user audio segment, if a maximum value of the audio amplitude in the user audio segment is greater than or equal to a second threshold, the user audio segment is determined as the candidate audio segment.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, performs the steps of the audio processing method of any one of claims 7 to 9.

Images