Method of audio detection and electronic device

Abstract

The application provides a method for audio detection and an electronic device. The method is applied to the electronic device and includes: collecting a first audio and identifying a type of the first audio; collecting a first signal through one or more sensors, wherein the electronic device includes the one or more sensors; when it is identified that the first audio is a set type, determining a first time interval according to the first audio, the first time interval including a start time and an end time; determining a second signal according to the first time interval and the first signal; and displaying first information if the second signal matches the first audio. The scheme provided in the application embodiment can detect a matching relationship between an audio collected by the electronic device and other signals, and then subsequent processing can be performed according to the matching relationship.

Description

Technical Field

[0001] This application relates to the field of electronic equipment technology, and in particular to an audio detection method and electronic equipment. Background Technology

[0002] Respiratory diseases are common and frequently occurring, and some of them are even contagious. Therefore, it is essential to conduct simple and efficient tests for some respiratory diseases.

[0003] Current routine methods for detecting respiratory diseases mainly include doctor consultations, chest imaging, and sputum culture. These methods generally require manual intervention, are costly, and time-consuming, resulting in low detection efficiency. This has spurred more research into detecting respiratory diseases (such as asthma, COPD, and COVID-19) based on cough and breath sounds. Since most respiratory diseases share similar symptoms such as cough, fever, and shortness of breath, intelligent algorithms can be used to analyze these symptom characteristics to detect whether a respiratory disease has occurred.

[0004] Current methods for detecting respiratory diseases based on cough sounds are mainly applied in wearable devices. By detecting a user's cough sounds in real time, these devices can quickly analyze the user's risk of developing respiratory diseases. However, in this method, when multiple users are present in the environment (e.g., in a public setting), the wearable device may mistakenly identify other users' cough sounds as those of the user wearing the device during the risk analysis process. This can lead to false positives and inaccurate results for the user wearing the device. Summary of the Invention

[0005] This application provides an audio detection method and an electronic device for detecting the matching relationship between audio collected by the electronic device and other signals, and then performing subsequent processing based on the matching relationship.

[0006] In a first aspect, this application provides an audio detection method applied to an electronic device. The method includes: acquiring a first audio and identifying the type of the first audio; acquiring a first signal through one or more sensors, wherein the electronic device includes one or more sensors; when the first audio is identified as a set type, determining a first time interval based on the first audio, the first time interval including a start time and an end time; determining a second signal based on the first time interval and the first signal; and if the second signal matches the first audio, displaying first information.

[0007] In this method, an electronic device can match a set type of audio with other signals generated during the audio's production period to determine the matching relationship between the audio and other signals. Appropriate subsequent processing can then be performed when the audio matches other signals. Based on this, the method can achieve matching between specific audio and other signals, and then perform subsequent processing based on the matching relationship, such as further matching the audio with the objects corresponding to the other signals. For example, when the other signals are signals measured against a user, this method can match the audio with other signals to achieve a match between the audio and the user, thus clarifying the correspondence between the audio and the user. Subsequent processing can then be performed based on this correspondence, thereby improving the accuracy of the subsequent processing. For example, after determining that the audio matches the user, the user's disease risk can be detected based on the audio, thereby improving the accuracy of the detection and reducing errors.

[0008] In one possible design, the method further includes: displaying second information if the second signal does not match the first audio.

[0009] In this method, the processing performed by the electronic device differs depending on whether the audio and signal match or not. Therefore, by matching the audio and signal, the electronic device can perform corresponding processing based on a defined matching relationship, ensuring the smooth execution of subsequent processing and improving the accuracy and flexibility of subsequent processing to a certain extent.

[0010] In one possible design, before determining the first time interval based on the first audio, the method further includes: determining a first risk parameter based on the first audio, wherein the first risk parameter is greater than or equal to a set threshold, and wherein the first risk parameter is used to indicate the disease risk corresponding to the first audio.

[0011] In this method, the electronic device can determine the corresponding risk parameters based on the audio. When the risk parameter is large, risk response processing is necessary, and the electronic device can continue with the subsequent matching process. When the risk parameter is small, risk response processing is not required, and the electronic device can skip the subsequent matching process and begin the next audio acquisition and processing cycle. Based on this, the electronic device can flexibly switch processing methods according to the value of the risk parameter, thereby improving the accuracy of subsequent processing.

[0012] In one possible design, the first information is used to indicate that: the first audio source is a first user, and / or, the first user is at risk of infection, wherein the first user is a user wearing an electronic device; the second information is used to indicate that: the first audio source is not a first user, and / or, the environment in which the first user is located is at risk of infection.

[0013] In this method, the second signal can be a signal collected from the first user. When the first audio signal matches the second signal, it can be determined that the first audio signal also matches the first user; when the first audio signal does not match the second signal, it can be determined that the first audio signal also does not match the first user. Furthermore, if the first risk parameter is high, and the first audio signal matches the second signal, it can be determined that the first user has a high risk of illness; if the first audio signal does not match the second signal, it can be determined that there is an infection risk in the environment where the first user is located. Based on this, the electronic device can provide more comprehensive and accurate risk warnings to the first user, enabling the first user to proactively take appropriate measures to reduce the risk, thereby improving the user experience.

[0014] In one possible design, determining a first risk parameter based on a first audio audio includes: extracting feature data from the first audio audio; wherein the feature data is used to characterize the time-domain and / or frequency-domain features of the first audio audio; and determining the first risk parameter based on a set disease screening model and the feature data; wherein the disease screening model is used to represent the relationship between the time-domain and / or frequency-domain features of the audio audio and the risk parameter corresponding to the audio audio.

[0015] In this method, a more comprehensive audio analysis can be performed based on the time-domain features, frequency-domain features, and other features of the audio. The use of models can improve the accuracy of audio analysis, thereby more accurately determining the risk parameters corresponding to the first audio and improving the accuracy of audio analysis.

[0016] In one possible design, before displaying the first information if the second signal matches the first audio, the method further includes: determining a first action corresponding to the first audio based on the first audio, and determining a second action corresponding to the second signal based on the second signal; the matching of the second signal with the first audio includes: the type of the second action corresponding to the second signal being the same as the type of the first action corresponding to the first audio.

[0017] In this method, audio and other signals are different types of information. However, if the audio and other signals originate from the same object, they can all reflect some characteristics of that object. Therefore, based on whether the audio and other signals correspond to the same characteristics, it can be determined whether the audio and other signals match. In this method, the same characteristic can be the type of action. By comparing whether the types of actions corresponding to the audio and signals are the same, matching between the audio and signals can be achieved. For example, when both the audio and signals originate from a user, the action can be an action performed by the user. Therefore, when the types of actions corresponding to the audio and signals are the same, it can be determined that the audio and signals originate from the same user.

[0018] In one possible design, determining the second action corresponding to the second signal based on the second signal includes: determining the second action and the confidence level of the second action based on a set action recognition model and the second signal, wherein the action recognition model is used to represent the relationship between the second signal, the second action, and the confidence level, and the confidence level is used to characterize the recognition accuracy of the action recognition model; determining that the confidence level is greater than or equal to a set first threshold; or determining that the confidence level is greater than or equal to a set second threshold and less than or equal to a set third threshold; displaying first indication information, the first indication information being used to indicate: whether to confirm the execution of the second action; and receiving second indication information, the second indication information being used to indicate: confirm the execution of the second action.

[0019] In this method, on the one hand, the model can improve the accuracy of action recognition, thereby more accurately determining the action and type corresponding to the second signal, and thus improving the accuracy of matching. On the other hand, the first indication information can be sent to the user, and the second indication information can be from the user. The electronic device can flexibly switch subsequent processing procedures based on the accuracy parameter, i.e., the confidence level, of the action recognition model to ensure the accuracy of the final matching result. When the confidence level is low, it can be considered that the first audio and the second signal do not match, so the electronic device can skip the subsequent matching process and start the next audio acquisition and processing process. When the confidence level is within an intermediate range, additional confirmation can be performed, and the decision on whether to proceed with the subsequent matching process can be made based on the confirmation result, ensuring the accuracy of the processing. Based on this, the electronic device can flexibly switch processing methods according to the confidence level, thereby improving the accuracy of the subsequent matching process.

[0020] In one possible design, the second signal includes at least one of the following: a signal acquired by an accelerometer; a signal acquired by a gyroscope sensor; or a signal acquired by a photoelectric sensor.

[0021] In this method, the signals measured by the accelerometer or gyroscope can characterize the user's motion and posture characteristics, and the signals measured by the photoelectric sensor can characterize the user's physiological characteristics. These signals can all reflect the relevant characteristics when the user performs an action. Therefore, based on these signals, the action and type corresponding to the second signal can be accurately determined, thereby improving the accuracy of matching with the audio.

[0022] In one possible design, the setting type includes at least one of the following: cough sound, breathing sound, sneezing sound, joint popping sound.

[0023] This method uses audio recordings such as coughs and breathing sounds emitted by the user to predict their risk of illness. Therefore, by matching this type of audio with other signals, the matching results and the predicted risk can be combined to more accurately identify individuals at risk of illness or infection, thereby providing further response prompts and improving the user experience.

[0024] Secondly, this application provides an audio detection method applied to a first electronic device. The method includes: acquiring a first audio signal and identifying the type of the first audio signal; when the first audio signal is identified as a set type, determining a first time interval based on the first audio signal, the first time interval including a start time and an end time; sending a request message to a second electronic device, the request message being used to request the acquisition of a signal corresponding to the first time interval; receiving a first signal from the second electronic device, the first signal being acquired by the second electronic device through one or more sensors, wherein the second electronic device includes one or more sensors; and displaying first information if the first signal matches the first audio signal.

[0025] In one possible design, the method further includes: displaying second information if the first signal does not match the first audio.

[0026] In one possible design, before determining the first time interval based on the first audio, the method further includes: determining a first risk parameter based on the first audio, wherein the first risk parameter is greater than or equal to a set threshold, and wherein the first risk parameter is used to indicate the disease risk corresponding to the first audio.

[0027] In one possible design, the first signal is the signal acquired by the second electronic device within the first time interval.

[0028] In one possible design, the first information is used to indicate that: the first audio source is a first user, and / or, the first user is at risk of infection, wherein the first user is a user wearing a second electronic device; the second information is used to indicate that: the first audio source is not a first user, and / or, the environment in which the first user is located is at risk of infection.

[0029] In one possible design, determining a first risk parameter based on a first audio audio includes: extracting feature data from the first audio audio; wherein the feature data is used to characterize the time-domain and / or frequency-domain features of the first audio audio; and determining the first risk parameter based on a set disease screening model and the feature data; wherein the disease screening model is used to represent the relationship between the time-domain and / or frequency-domain features of the audio audio and the risk parameter corresponding to the audio audio.

[0030] In one possible design, before displaying the first information if the first signal matches the first audio, the method further includes: determining a first action corresponding to the first audio based on the first audio, and determining a second action corresponding to the first signal based on the first signal; the first signal matching the first audio includes: the type of the second action corresponding to the first signal is the same as the type of the first action corresponding to the first audio.

[0031] In one possible design, determining the second action corresponding to the first signal based on the first signal includes: determining the second action and its confidence level based on a set action recognition model and the first signal, wherein the action recognition model is used to represent the relationship between the first signal, the second action, and the confidence level, and the confidence level is used to characterize the recognition accuracy of the action recognition model; determining that the confidence level is greater than or equal to a set first threshold; or determining that the confidence level is greater than or equal to a set second threshold and less than or equal to a set third threshold; displaying first indication information, which indicates whether to confirm the execution of the second action; and receiving second indication information, which indicates whether to confirm the execution of the second action.

[0032] In one possible design, the first signal includes at least one of the following: a signal acquired by an accelerometer; a signal acquired by a gyroscope sensor; or a signal acquired by a photoelectric sensor.

[0033] In one possible design, the setting type includes at least one of the following: cough sound, breathing sound, sneezing sound, joint popping sound.

[0034] Thirdly, this application provides an audio detection method applied to a system composed of a first electronic device and a second electronic device. The method includes: the first electronic device acquiring a first audio signal and identifying the type of the first audio signal; and the second electronic device acquiring a first signal through one or more sensors, wherein the second electronic device includes one or more sensors; when the first electronic device identifies the first audio signal as a set type, it determines a first time interval based on the first audio signal, the first time interval including a start time and an end time; the first electronic device sending a request message to the second electronic device, the request message being used to request the acquisition of a signal corresponding to the first time interval; the second electronic device determining the first time interval based on the received request message, and determining a second signal based on the first time interval and the first signal; the second electronic device sending the second signal to the first electronic device; and if the first electronic device determines that the received second signal matches the first audio signal, it displays first information.

[0035] In one possible design, the method further includes: if the first electronic device determines that the second signal does not match the first audio, it displays second information.

[0036] In one possible design, before the first electronic device determines the first time interval based on the first audio, the method further includes: the first electronic device determining a first risk parameter based on the first audio, the first risk parameter being greater than or equal to a set threshold, wherein the first risk parameter is used to indicate the disease risk corresponding to the first audio.

[0037] In one possible design, the first information is used to indicate that: the first audio source is a first user, and / or, the first user is at risk of infection, wherein the first user is a user wearing a second electronic device; the second information is used to indicate that: the first audio source is not a first user, and / or, the environment in which the first user is located is at risk of infection.

[0038] In one possible design, the first electronic device determines a first risk parameter based on a first audio signal, including: the first electronic device extracting feature data from the first audio signal; wherein the feature data is used to characterize the time-domain and / or frequency-domain features of the first audio signal; and the first electronic device determining the first risk parameter based on a set disease screening model and the feature data; wherein the disease screening model is used to represent the relationship between the time-domain and / or frequency-domain features of the audio signal and the risk parameter corresponding to the audio signal.

[0039] In one possible design, before the first electronic device displays the first information if it determines that the second signal matches the first audio, the method further includes: the first electronic device determining a first action corresponding to the first audio based on the first audio, and determining a second action corresponding to the second signal based on the second signal; the matching of the second signal with the first audio includes: the type of the second action corresponding to the second signal being the same as the type of the first action corresponding to the first audio.

[0040] In one possible design, the first electronic device determines a second action corresponding to the second signal based on the second signal, including: the first electronic device determining the second action and the confidence level of the second action based on a set action recognition model and the second signal, wherein the action recognition model is used to represent the relationship between the second signal, the second action, and the confidence level, and the confidence level is used to characterize the recognition accuracy of the action recognition model; the first electronic device determines that the confidence level is greater than or equal to a set first threshold; or, the first electronic device determines that the confidence level is greater than or equal to a set second threshold and less than or equal to a set third threshold; displays first indication information, which indicates whether to confirm the execution of the second action; and receives second indication information, which indicates whether to confirm the execution of the second action.

[0041] In one possible design, the second signal includes at least one of the following: a signal acquired by an accelerometer; a signal acquired by a gyroscope sensor; or a signal acquired by a photoelectric sensor.

[0042] In one possible design, the setting type includes at least one of the following: cough sound, breathing sound, sneezing sound, joint popping sound.

[0043] Fourthly, this application provides a system comprising the first electronic device and the second electronic device described in the third aspect above.

[0044] Fifthly, this application provides an electronic device including a display screen, a memory, and one or more processors; wherein the memory is used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed by one or more processors, the electronic device performs the method described in the first aspect or any possible design of the first aspect, or performs the method described in the second aspect or any possible design of the second aspect, or performs the method performed by the first electronic device or the second electronic device in the third aspect or any possible design of the third aspect.

[0045] Sixthly, this application provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method described in the first aspect or any possible design of the first aspect, or to perform the method described in the second aspect or any possible design of the second aspect, or to perform the method performed by the first electronic device or the second electronic device in the third aspect or any possible design of the third aspect.

[0046] In a seventh aspect, this application provides a computer program product comprising a computer program or instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect or any possible design of the first aspect, or to perform the method described in the second aspect or any possible design of the second aspect, or to perform the method performed by a first electronic device or a second electronic device in the third aspect or any possible design of the third aspect.

[0047] For the beneficial effects described in aspects two through seven above, please refer to the description of the beneficial effects in aspect one above, which will not be repeated here. Attached Figure Description

[0048] Figure 1 A schematic diagram of the hardware architecture of an electronic device provided in an embodiment of this application;

[0049] Figure 2 A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0050] Figure 3 A schematic diagram illustrating an audio detection method provided in an embodiment of this application;

[0051] Figure 4 This application provides a schematic diagram of a mobile phone interface for displaying prompt information.

[0052] Figure 5 This is a schematic diagram of another mobile phone interface for displaying prompt information, provided in an embodiment of this application.

[0053] Figure 6 This is a schematic diagram of another mobile phone interface for displaying prompt information, provided in an embodiment of this application.

[0054] Figure 7 This is a schematic diagram of another mobile phone interface for displaying prompt information, provided in an embodiment of this application.

[0055] Figure 8 A flowchart illustrating a method for detecting a user's respiratory infection risk, provided as an embodiment of this application;

[0056] Figure 9 A schematic diagram of the function control interface of a smartwatch provided in an embodiment of this application;

[0057] Figure 10 This application provides a schematic diagram of the interface for displaying disease risk warning information on a smartwatch.

[0058] Figure 11 This is a schematic diagram of another smartwatch interface for displaying disease risk warning information, provided in an embodiment of this application.

[0059] Figure 12 A schematic diagram of the prompt interface of a smartwatch provided in an embodiment of this application;

[0060] Figure 13 A schematic diagram illustrating an audio detection method provided in an embodiment of this application;

[0061] Figure 14 A schematic diagram illustrating an audio detection method provided in an embodiment of this application;

[0062] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings. In the description of the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0064] For ease of understanding, exemplary descriptions of concepts related to this application are provided for reference.

[0065] The electronic device may be a device with wireless connectivity. In some embodiments of this application, the electronic device may also have audio detection (sound detection) and / or sensing functions. In the embodiments of this application, audio may also be referred to as sound.

[0066] In some embodiments of this application, the electronic device may be a portable device, such as a mobile phone, tablet computer, wearable device with wireless communication function (e.g., watch, bracelet, helmet, earphone, etc.), vehicle terminal device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), smart home device (e.g., smart TV, smart speaker, etc.), smart robot, workshop equipment, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, or wireless terminal in smart home, flying equipment (e.g., smart robot, hot air balloon, drone, airplane), etc.

[0067] Wearable devices are portable devices that users can wear directly on their bodies or integrate into their clothing or accessories. In this application, the wearable device can be a portable device with sensing and audio detection capabilities.

[0068] In some embodiments of this application, the electronic device may also be a portable terminal device that includes other functions such as a personal digital assistant and / or a music player. Exemplary embodiments of the portable terminal device include, but are not limited to, devices equipped with... Alternatively, it could be a portable terminal device with another operating system. The aforementioned portable terminal device could also be other portable terminal devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of this application, the aforementioned electronic device may not be a portable terminal device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0069] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0070] To address the problem that current wearable devices using cough sounds to detect respiratory diseases have difficulty identifying the source of the cough sound, this application provides an audio detection method and electronic device. This solution can more accurately identify the object emitting the audio and improve the accuracy of identifying the audio source.

[0071] For example, the method provided in this application embodiment can be used in wearable devices. When the method provided in this application embodiment is applied to a wearable device, the wearable device can accurately identify whether the detected cough sound belongs to the user wearing the wearable device, and then perform a more accurate disease risk detection for the user based on the cough sound belonging to the user.

[0072] See below. Figure 1 The structure of the electronic device to which the method provided in the embodiments of this application is applicable will be described.

[0073] like Figure 1As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a USB interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a SIM card interface 195, etc.

[0074] The sensor module 180 may include a gyroscope sensor, an accelerometer, a proximity sensor, a fingerprint sensor, a touch sensor, a temperature sensor, a pressure sensor, a distance sensor, a magnetic sensor, an ambient light sensor, a barometric pressure sensor, a bone conduction sensor, etc.

[0075] Understandable, Figure 1 The electronic device 100 shown is merely an example and does not constitute a limitation on the electronic device. The electronic device may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. Figure 1 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0076] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, memory, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the central nervous system and command center of the electronic device 100. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0077] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0078] The audio detection method provided in this application embodiment can be executed by the processor 110, which can control or call other components. For example, it can call the processing program of this application embodiment stored in the internal memory 121, or call the processing program of this application embodiment stored in a third-party device through the external memory interface 120 to control the wireless communication module 160 to communicate with other devices, thereby improving the intelligence and convenience of the electronic device 100 and enhancing the user experience. The processor 110 may include different devices. For example, when integrating a CPU and a GPU, the CPU and GPU can cooperate to execute the audio detection method provided in this application embodiment. For example, some algorithms in the audio detection method can be executed by the CPU, and other algorithms can be executed by the GPU to achieve faster processing efficiency.

[0079] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays screens 194, where N is a positive integer greater than 1. Display screen 194 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces (GUIs). For example, display screen 194 can display photos, videos, web pages, or documents, etc.

[0080] In this embodiment of the application, the display screen 194 can be a single flexible display screen, or it can be a splicing display screen composed of two rigid screens and a flexible screen located between the two rigid screens.

[0081] Camera 193 (a front-facing camera or a rear-facing camera, or a single camera that can function as both) is used to capture still images or videos. Typically, camera 193 may include a light-sensing element such as a lens assembly and an image sensor. The lens assembly includes multiple lenses (convex or concave lenses) for collecting light signals reflected from the object being photographed and transmitting these signals to the image sensor. The image sensor then generates a raw image of the object being photographed based on the light signals.

[0082] Internal memory 121 can be used to store executable program code, including instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area can store the operating system, application code (such as audio detection functions), etc. The data storage area can store data created during the use of electronic device 100.

[0083] The internal memory 121 may also store one or more computer programs corresponding to the audio detection algorithm provided in the embodiments of this application. The one or more computer programs are stored in the internal memory 121 and configured to be executed by one or more processors 110. The one or more computer programs include instructions that can be used to perform the various steps in the following embodiments.

[0084] In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0085] Of course, the code for the audio detection algorithm provided in this embodiment can also be stored in external memory. In this case, the processor 110 can run the code for the audio detection algorithm stored in external memory through the external memory interface 120.

[0086] The sensor module 180 may include a gyroscope sensor, an accelerometer sensor, a proximity sensor, a fingerprint sensor, a touch sensor, etc.

[0087] A touch sensor, also known as a "touch panel," can be located on the display screen 194. The touch sensor and display screen 194 together form a touch display screen, also called a "touch screen." The touch sensor detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor may also be located on the surface of the electronic device 100, in a different position than the display screen 194.

[0088] For example, the display screen 194 of the electronic device 100 displays a main interface, which includes icons for multiple applications (such as a camera application, a WeChat application, etc.). The user taps the camera application icon on the main interface using a touch sensor, triggering the processor 110 to launch the camera application and open the camera 193. The display screen 194 then displays the camera application's interface, such as a viewfinder.

[0089] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0090] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0091] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device. In this embodiment, the mobile communication module 150 can also be used for information interaction with other devices.

[0092] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0093] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2. In this embodiment, the wireless communication module 160 is used to establish connections with other electronic devices for data interaction. Alternatively, the wireless communication module 160 can be used to access access point devices, send control commands to other electronic devices, or receive data from other electronic devices.

[0094] In addition, the electronic device 100 can implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, and an application processor, such as music playback and recording. The electronic device 100 can receive input from buttons 190, generating key signal inputs related to user settings and function control. The electronic device 100 can use a motor 191 to generate vibration alerts (such as vibration alerts for incoming calls). The indicator 192 in the electronic device 100 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. The SIM card interface 195 in the electronic device 100 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device 100.

[0095] It should be understood that, in practical applications, electronic device 100 may include more than Figure 1 The number of more or fewer components shown is not limited in the embodiments of this application. The illustrated electronic device 100 is merely an example, and the electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0096] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of the electronic device.

[0097] Layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example... Figure 2 As shown, the software architecture can be divided into four layers, from top to bottom: the application layer, the application framework layer (framework, FWK), the Android runtime and system libraries, and the Linux kernel layer.

[0098] The application layer is the top layer of the operating system and includes native operating system applications such as camera, gallery, calendar, Bluetooth, music, video, and messaging. The applications discussed in this application are referred to as apps (APPs), which are software programs capable of performing one or more specific functions. Typically, multiple apps can be installed on an electronic device. Examples include camera apps, email apps, and smart home control apps. The apps mentioned below can be system apps pre-installed at the factory or third-party apps downloaded by the user from the network or obtained from other electronic devices during use.

[0099] Of course, for developers, they can write applications and install them into this layer. In one possible implementation, the application can be developed using the Java language, by calling the application programming interface (API) provided by the application framework layer. Developers can then interact with the underlying operating system (such as the kernel layer) through the application framework to develop their own applications.

[0100] The application framework layer provides APIs and a programming framework for applications within the application layer. The application framework layer can include predefined functions. It may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0101] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0102] Content providers are used to store and retrieve data, making that data accessible to applications. Data can include files (such as documents, videos, images, and audio), text, and other information.

[0103] A view system includes visual controls, such as controls that display text, images, documents, and other content. View systems can be used to build applications. An interface in a display window can consist of one or more views. For example, a display interface including a text message notification icon could include a view that displays text and a view that displays images.

[0104] The phone manager provides communication functionality for electronic devices. The notification manager allows applications to display notification information in the status bar; it can be used to convey informative messages and can disappear automatically after a short pause without user interaction.

[0105] The Android runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0106] The core libraries of the Android system consist of two parts: one part contains the functionalities that Java calls, and the other part is the core libraries of the Android system. The application layer and application framework layer run in a virtual machine. Taking Java as an example, the virtual machine executes the Java files in the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0107] The system library can include multiple functional modules. For example: a surface manager, a media library, a 3D graphics processing library (e.g., OpenGL ES), and a 2D graphics engine (e.g., SGL). The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of various common audio and video formats, as well as still image files. The media library supports various audio and video encoding formats, such as MPEG4, H.564, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D graphics engine is the drawing engine for 2D graphics.

[0108] The kernel layer provides the core system services of the operating system, such as security, memory management, process management, network protocol stack, and driver models, all of which are implemented based on the kernel layer. The kernel layer also serves as an abstraction layer between the hardware and software stacks. This layer contains many drivers related to electronic devices, including: display drivers; keyboard drivers as input devices; Flash drivers for memory-based devices; camera drivers; audio drivers; Bluetooth drivers; and WiFi drivers.

[0109] It is important to understand that the above functional services are just an example. In practical applications, electronic devices can also be divided into more or fewer functional services according to other factors, or the functions of each service can be divided in other ways, or they can work as a whole without dividing into functional services.

[0110] The solution provided in this application will be described in detail below with reference to specific embodiments.

[0111] See Figure 3 The audio detection method provided in this application includes:

[0112] S301: The electronic device determines that the acquired audio contains target audio, and the type of the target audio is a set type.

[0113] In some embodiments of this application, the target user can be a user who wears or carries an electronic device, or the target user can be a pre-defined user associated with an electronic device.

[0114] For example, the target audio (or set type) mentioned above can be a cough sound, a breathing sound, a sneeze sound, a joint popping sound (i.e., the sound made by the joints of the human body when they move), etc.

[0115] In some embodiments of this application, the electronic device can be a device with audio detection capabilities, enabling it to collect and detect audio (or sound) in its environment in real time, thereby obtaining the target audio signal (i.e., the audio signal of the target audio). The electronic device can also be a device with wireless connectivity, allowing it to receive target audio signals collected by other audio detection devices. Of course, the electronic device can also possess both audio detection and wireless connectivity capabilities, allowing it to acquire the target audio signal using any of the aforementioned methods.

[0116] For example, the target audio can be the sound emitted by any user in the environment where the electronic device is located. For instance, when the electronic device and the target user are in the same environment, the electronic device can receive and identify the audio appearing in the environment in real time. When the electronic device recognizes that the received audio is of a set type, such as a cough sound, it can determine that there is a user coughing in the environment. Therefore, the electronic device can obtain the audio signal during the user's cough and obtain the corresponding cough sound signal.

[0117] In some embodiments of this application, after the electronic device acquires the target audio, it can identify the target action corresponding to the target audio. Specifically, this identification can be achieved using a trained network model.

[0118] In some embodiments of this application, after acquiring a target audio signal, the electronic device can first detect whether the user who emitted the target audio has a risk of illness based on the target audio signal. Specifically, after acquiring the target audio signal, the electronic device can first preprocess the target audio signal, such as noise reduction, pre-emphasis, framing, windowing, etc. Then, feature data of the target audio can be extracted from the preprocessed target audio signal, and the extracted feature data can be used to detect whether the user to whom the target audio belongs has a risk of illness.

[0119] The feature data of the target audio may include, but is not limited to, at least one of the following:

[0120] 1) Temporal characteristics.

[0121] For example, a time-domain feature could be the zero-crossing rate (ZCR) of an audio signal. The zero-crossing rate refers to the number of times the signal crosses a zero point (changing from positive to negative or from negative to positive) in each frame of the audio signal. Other time-domain features may include onset time, autocorrelation parameters, or waveform characteristics of the audio signal. Onset time refers to the duration of the rise in audio energy. The autocorrelation parameter indicates the similarity between the audio signal and its time-shifted counterpart.

[0122] 2) Frequency domain characteristics.

[0123] For example, frequency domain features can include Mei-freguency ceptracoefficients (MFCC), power spectral density, spectral centroid, spectral flatness, and spectral flux. MFCC is the cepstral coefficient extracted from the Mei-scale frequency domain; the Mei-scale describes the nonlinear characteristics of human ear frequency perception. Power spectral density refers to the power (mean square value) per unit frequency band of the signal. The spectral centroid is the point of energy concentration in the signal spectrum, used to describe the brightness of the signal's timbre. Spectral flatness refers to the similarity between the quantized signal and noise. Spectral flux refers to the degree of variation between adjacent frames of the quantized signal.

[0124] 3) Energy characteristics.

[0125] For example, energy characteristics can be root-mean-square energy, etc. Root-mean-square energy refers to the average energy of a signal over a certain time range.

[0126] 4) Music theory characteristics.

[0127] For example, music theory features can include fundamental frequency, detuning, etc. The fundamental frequency refers to the frequency of the sound's pitch. Detuning refers to the degree of deviation between the overtone frequency of the signal and an integer multiple of the fundamental frequency.

[0128] 5) Perceptual features.

[0129] For example, perceptual features can be sound loudness (intensity), sharpness, etc. Loudness refers to the strength of a signal perceived by the human ear (e.g., the volume of the sound). Sharpness represents the energy level of the high-frequency components in an audio signal; the higher the energy of the high-frequency components, the higher the sharpness, and the sharper the sound perceived by the human ear.

[0130] In some embodiments of this application, the electronic device can employ model recognition to detect the presence of disease risk based on the extracted feature data of the target audio. Specifically, after extracting the feature data of the target audio, the electronic device can input the extracted feature data into a trained disease screening model to obtain a disease risk parameter output by the disease screening model. This parameter can be used to indicate the level of disease risk (e.g., high risk, medium risk, low risk, etc.), or it can be used to indicate a specific value representing the magnitude of the disease risk. Optionally, the disease risk parameter can also be used to indicate the disease type.

[0131] The disease screening model used in electronic devices can be obtained by training a network model using algorithms such as logistic regression (LR) and extreme gradient boosting (XGBoost).

[0132] In some embodiments of this application, when the electronic device detects that the user emitting the target audio has a disease risk or a high disease risk (e.g., the disease risk parameter output by the disease screening model is greater than or equal to a set risk threshold) based on the target audio signal, it performs the following step S302: acquiring the physiological data of the target user, and determining whether the target audio was emitted by the target user based on the physiological data of the target user. Optionally, when the electronic device detects that the user emitting the target audio does not have a disease risk or has a low disease risk (e.g., the disease risk parameter output by the disease screening model is less than a set risk threshold) based on the target audio signal, it may not perform additional processing; for example, it may simply continue to perform the original audio monitoring.

[0133] S302: The electronic device acquires physiological data of the target user, which is used to characterize the physiological characteristics of the target user.

[0134] In real-world scenarios, users typically perform actions while emitting sounds. For example, a cough sound is accompanied by the action of coughing, and a breathing sound is accompanied by the action of breathing. Therefore, there is a correspondence between the sounds emitted and the actions performed by the user; that is, the type of sound emitted corresponds to the type of action performed. It can be considered that the sounds emitted by the user are generated during the execution of an action. Therefore, in this embodiment, the target audio can be the sound emitted by the user during the execution of a target action. Here, the target audio is a set type, and the type of the target action corresponds to the set type. For example, when the target audio is the aforementioned cough sound, breathing sound, sneezing sound, or joint popping sound, the corresponding actions are coughing, breathing, sneezing, and joint movement, respectively.

[0135] On the other hand, a user's physiological characteristics change differently when performing different actions. Therefore, a user's physiological characteristics are generally different when performing different actions, and the corresponding actions performed by the user can be identified based on these physiological characteristics. Furthermore, the user's voice can be associated with the user's physiological data based on the user's actions, thereby associating the user's voice with the user. Therefore, in this embodiment of the application, the electronic device can identify the actions performed by the target user based on the target user's physiological characteristics, and then determine the relationship between the target audio detected by the electronic device and the target user based on the relationship between the target user's actions and the actions corresponding to the audio detected by the electronic device.

[0136] In some embodiments of this application, the electronic device can be a device equipped with sensing functions or sensors, enabling it to detect the user's physiological characteristics in real time and collect physiological parameters characterizing those characteristics. The electronic device can also be a device with wireless connectivity, allowing it to receive physiological data collected by other physiological monitoring devices. Of course, the electronic device can also possess both sensing and wireless connectivity functions, allowing it to acquire the user's physiological data using any of the aforementioned methods.

[0137] In some embodiments of this application, during the process of an electronic device collecting physiological data from a user or receiving physiological data from a physiological monitoring device, physiological data collected within the most recent time period can be saved. The duration of this time period can be a preset duration.

[0138] For example, the electronic device can be a smartwatch or a mobile phone. When the electronic device is a smartwatch, the target user is the user wearing the smartwatch. The smartwatch itself can monitor the target user's physiological characteristics and collect and save the user's physiological data in real time. When the electronic device is a mobile phone, the target user can be the user holding the phone or the registered owner of the phone. The target user can wear a wearable device for physiological characteristic monitoring, such as a wristband. The wristband can monitor the target user's physiological characteristics in real time and send the collected physiological data to the mobile phone, which can save the physiological data from the wristband. Optionally, after collecting physiological data, the wristband can temporarily not report it to the electronic device. Instead, the wristband saves the collected physiological data and reports it according to the instructions of the electronic device when needed.

[0139] Based on the above method, when the electronic device performs step S302, it can obtain the required physiological data from its own stored physiological data, or it can instruct the physiological monitoring device to report the required physiological data.

[0140] In some embodiments of this application, the physiological data of the target user acquired by the electronic device is the physiological data of that user collected within a target time period. The target time period is the time period during which the electronic device or audio detection device collects the target audio signal, and it is also the time period during which the target audio is generated. Based on this, it can be ensured that the generation time of the physiological data acquired by the electronic device is consistent with the generation time of the target audio, so that matching can be performed based on the audio signal and physiological data within the same time period to ensure the accuracy of the matching.

[0141] Various sensors can be configured in electronic devices or physiological monitoring devices to detect physiological data. In some embodiments of this application, the physiological data acquired by the electronic device includes at least motion posture parameters and / or heart rate parameters. Motion posture parameters, used to indicate the motion or posture characteristics of the user being tested, can be measured by devices such as an acceleration transducer (ACC) and a gyroscope. Heart rate parameters, used to indicate the heart rate and pulse characteristics of the user being tested, can be measured by a photoelectric sensor using photoplethysmography (PPG). Optionally, the physiological data may also include, but is not limited to, at least one of the following: respiratory rate parameters, blood oxygen parameters, blood pressure parameters, pulse parameters, etc., which can be obtained by analyzing and calculating data measured by sensors such as the ACC, gyroscope, and photoelectric sensor.

[0142] The aforementioned physiological data can be used to detect the actions performed by the corresponding user, and then determine whether the target audio belongs to that user based on whether the user's actions match the actions corresponding to the target audio. For example, when the motion posture parameters measured by ACC show large peaks or significant fluctuations over time, and the heart rate parameters show a significant increase, it can be determined that the user has performed a coughing action. If the target audio information signal is a coughing sound signal, then it can be determined that the target audio belongs to that user.

[0143] S303: The electronic device determines whether the target audio originates from the target user based on this physiological data.

[0144] After acquiring the physiological data of the target user, the electronic device can determine whether the target audio originates from (belongs to) the target user based on the physiological data. Specifically, the electronic device can utilize a pre-trained action recognition model to identify whether the target user performs the target action corresponding to the target audio based on the target user's physiological data. Then, it determines whether the target audio and the target user match based on whether the target user performs the target action. The type of the target action corresponds to the type of the target audio, and the target action can be determined based on the target audio. In some embodiments of this application, the action recognition model can be obtained by training a network model using algorithms such as Support Vector Machines (SVM) and Logistic Regression (LR).

[0145] As an optional implementation, physiological sample parameters of users performing various actions can be collected during the model training phase to form a model database of various physiological parameters. During the matching phase, the action recognition model can match the physiological data input to the model with the physiological parameters in the model database. If the physiological parameters of a certain type of action have the highest matching degree with the physiological data input to the action recognition model, or if the matching degree is greater than a set matching degree threshold, then that type of action is determined to be the action corresponding to the physiological data input to the action recognition model. If this type of action is the same as the target action, then it can be determined that the user to whom the physiological data belongs, i.e., the target user, performed the target action, and thus the target audio can be determined to be the sound emitted by the target user during the execution of the target action. Conversely, if this type of action is different from the target action, then the result of judgment can be obtained that the target audio is not the sound emitted by the target user during the execution of the target action.

[0146] As an alternative implementation, when performing source detection for specific types of audio, such as the audio of a target action being performed, the model training phase can also involve acquiring only the physiological sample parameters of different users performing the target action, thus constructing a model database corresponding to that type of physiological parameter. During the matching phase, the action recognition model can match the physiological parameters of the input model with the physiological parameters in the model database. If the matching degree between the input physiological data and more than a set number of physiological parameters in the model database is greater than a set matching degree threshold, then it can be determined that the target audio is the sound emitted by the target user during the performance of the target action; otherwise, it is determined that the target audio is not the sound emitted by the target user during the performance of the target action.

[0147] As another optional implementation, the action recognition model can be used to identify the user state or user action posture corresponding to the input physiological data. When it is determined that the action performed by the user corresponding to the physiological data is the target action, it can also be determined that the target user to which the physiological data belongs has performed the target action. Therefore, it can be considered that the physiological data matches the target audio signal, and thus it can be determined that the target audio is the sound emitted by the target user during the execution of the target action. In this method, during the model training phase, the collected physiological data of the user performing different actions and the corresponding action indicator labels can be used to form data pairs and used as training data to train the network model.

[0148] In some embodiments of this application, the input data of the action recognition model can be physiological data acquired by the electronic device, and the output data of the action recognition model can be a judgment result on whether the target user has performed a target action, or the confidence score of the action recognition model, which represents the probability that the user will perform the target action. The higher the confidence score, the greater the probability that the user will perform the target action. By inputting the acquired physiological data into the action recognition model, the electronic device can obtain the recognition result or the corresponding confidence score of whether the user has performed the target action, and thus determine whether the target audio is the sound emitted by the target user.

[0149] Specifically, when the output indicates that the target user performed the target action or the model's confidence level is greater than or equal to a set first confidence threshold, the electronic device can determine that the target audio originates from the target user. When the output indicates that the target user did not perform the target action or the model's confidence level is less than or equal to a set second confidence threshold, the electronic device can determine that the target audio does not originate from the target user. When the output indicates that the model's confidence level is less than the first confidence threshold but greater than the second confidence threshold, the electronic device can further determine the relationship between the target audio and the target user by prompting the target user for confirmation. Here, the first confidence threshold is greater than the second confidence threshold.

[0150] Optionally, the electronic device may determine that the target audio originates from the target user when the confidence level of the above model is greater than or equal to the set third confidence threshold, and determine that the target audio does not originate from the target user when the confidence level of the model is less than the third confidence threshold.

[0151] When prompting the target user for confirmation, the electronic device can display a prompt message to ask the user whether the target action has been performed, and determine whether the user has performed the target action based on the user's feedback. If so, it is finally determined that the target audio comes from the target user; otherwise, it is determined that the target audio does not come from the target user.

[0152] For example, when the electronic device is a smartwatch and the target action is coughing, the device can prompt the user to check its screen via vibration or other means. The screen will display a prompt message such as "Did you just cough?", along with user-selectable control options like "Yes" and "No." The user can then select the appropriate option based on whether they coughed. The electronic device can then determine the user's status after receiving their input.

[0153] When the aforementioned electronic device determines that the target audio does not originate from the target user, it determines that the target audio originates from other users present in the target user's environment.

[0154] In some embodiments of this application, when the electronic device determines that the target audio originates from the target user, it can further determine the target user's risk of illness (including the type of illness and the corresponding risk level); when the electronic device determines that the target audio originates from other users in the target user's environment, it can also further determine the risk of illness of other users. When it is determined that other users may have infectious diseases, relevant prompts can be given to the target user so that the target user can respond according to the prompts, thereby avoiding or reducing the possibility of contracting diseases from the environment.

[0155] The following explanation uses electronic devices to determine the disease risk of a target user as an example. The methods used by electronic devices to determine the disease risk of other users are the same as those used to determine the disease risk of a target user, and will not be repeated below.

[0156] As an optional implementation, when the electronic device determines the disease risk of the target user, it can use the disease risk parameter detected based on the feature data of the target audio in step S301 as the disease risk parameter of the target user, thereby obtaining the disease risk of the target user.

[0157] As an alternative implementation, the electronic device can combine more information to identify and analyze the target user's disease risk. For example, the electronic device can use a trained disease screening model to identify corresponding disease risk parameters based on at least one of the following: characteristic data of the target audio, personal information of the target user, and physiological data of the target user, thereby obtaining the target user's disease risk.

[0158] For example, the feature data of the target audio may include at least one of the features provided in the above embodiments. The personal information of the target user may include information such as the target user's gender, age, height, and weight, which may be pre-recorded by the user on an electronic device.

[0159] As another alternative implementation method, the electronic device can select the corresponding risk level from the preset correspondence between different features and risk levels based on at least one feature data, and use the selected risk level as the risk level of the target user's disease, thereby obtaining the target user's disease risk.

[0160] For example, the feature data may include the various feature data mentioned above, and may also include at least one of the following feature parameters: a parameter indicating the intensity of the target audio signal; the number of times the electronic device or audio detection device detects the target audio signal within a preset duration; the risk level output by the disease screening model; and the duration of the longest-lasting target audio signal among the multiple target audio signals detected by the electronic device or audio detection device. Of course, at least one parameter can also serve as feature data for audio, used in the process of identifying disease risk using the disease screening model.

[0161] In one possible scenario, when the aforementioned electronic device determines that the target user has a risk of illness or a high risk of illness, it can alert the target user to the risk of illness so that the target user can seek medical attention in a timely manner.

[0162] For example, when the electronic device is a mobile phone and the target action is coughing, the electronic device can alert the target user to the risk of illness through notification messages or other means. For instance, when the target user's risk of illness is low, the electronic device can display... Figure 4 The interface shown is shown in the image. When the target user has a high risk of developing the disease, the electronic device can display... Figure 5 The interface shown. In Figure 4 or Figure 5 In the interface shown, the electronic device can display risk warning information (such as...). Figure 4 The phrase "potential risk of respiratory infection" or... Figure 5 The "High Risk of Respiratory Infection (Suspected Pneumonia)" indicator shown can also display the date, time, and corresponding recommendations. For example, if it is determined that a user may be at risk of respiratory infection, the electronic device can also display the following recommendations: Based on your recent testing data, you may be at risk of respiratory infection. Please take proactive measurements or seek medical attention. If it is determined that a user is at high risk of respiratory infection, the electronic device can also display the following recommendations: Based on your recent testing data, you may be at a high risk of respiratory infection. Please seek medical attention promptly. Optionally, the electronic device can also display a prompt such as "This test is not intended as a professional clinical diagnosis of respiratory health."

[0163] Optionally, when the electronic device detects, based on the target audio signal, that the user emitting the target audio has no or very low risk of illness, the electronic device can also periodically display notification information to alert the user to their health status. For example, the electronic device can periodically display... Figure 6 The interface shown indicates the user's current risk of illness and can also display relevant suggestions and prompts such as "Based on the analysis of your recent measurement data, your risk of respiratory infection is low and within the healthy range. Continuing to maintain good lifestyle habits will bring you continued health" and "This test is not intended as a professional clinical diagnostic basis for respiratory health."

[0164] In another possible scenario, when the aforementioned electronic device determines that other users are at risk of contracting a disease or have a high risk of contracting a disease, and the type of disease that other users may be contracting is an infectious disease, it can alert the target user that there may be a risk of disease infection in the surrounding environment and prompt the target user to take protective measures.

[0165] For example, when the electronic device is a mobile phone and the target action is coughing, the electronic device can alert the target user to the potential risk of respiratory infection in the current environment through notification messages. For instance, the electronic device can display... Figure 7 The interface shown may include information indicating the risk level of respiratory infection in the environment. Optionally, the interface may also display the target user's current location, relevant detected characteristic parameters (such as the number of cough sounds detected in the environment), and corresponding suggestions. For example, the interface may also include prompts such as "You are currently located at X location. The number of coughs in the surrounding environment within XX time period is XXX. It is recommended that you leave this location or take protective measures" and "This test is not intended as a professional clinical diagnosis of respiratory health."

[0166] Optionally, the above-mentioned electronic device may also display information such as the waveform of the detected cough sound on the display interface, which is not specifically limited in this embodiment.

[0167] In the above embodiments, the electronic device can determine the relationship between the target audio and the target user based on the target user's physiological parameters when the target audio is detected, thereby distinguishing the target user's audio from the audio of other users. Furthermore, in subsequent processing, the target audio can also be processed based on its source. For example, if the source of the target audio is determined to be the target user, then the target audio has high reference value for detecting the target user's disease risk, and the results detected based on the target audio signal also have high reliability; therefore, the target user's own disease status can be detected based on the target audio. If the source of the target audio is determined not to be the target user, then the target audio has low reference value for detecting the target user's disease risk, thus avoiding the situation where the target audio is mistakenly used as the target user's audio for disease risk detection, thereby improving the accuracy of related detections.

[0168] The following example illustrates the application of the solution provided in this application to detect the risk of respiratory infection in users.

[0169] The following explanation uses an electronic device as an example, where the target user wears a wearable device, and the target user uses the wearable device to detect the risk of respiratory infection. The target audio is a cough sound. The wearable device has an audio detection function, which can detect cough sounds present in the environment and determine the risk of respiratory infection based on the detected cough sounds. Furthermore, the wearable device has a physiological monitoring function, which can monitor the target user's physiological characteristics in real time.

[0170] Reference Figure 8 The process of the method for detecting the risk of respiratory infection in users provided in this application embodiment includes:

[0171] S801: Wearable devices can collect and detect audio generated in the environment in real time, and detect and save the physiological data of the target user in real time.

[0172] The function of wearable devices in detecting users' respiratory infection risk can be controlled and implemented through a health monitoring application installed on the wearable device. When the respiratory monitoring function in the health monitoring application of the wearable device is turned on, the wearable device can perform the respiratory infection risk detection and other processing according to the method provided in this example. When the respiratory monitoring function in the health monitoring application of the wearable device is turned off, the wearable device can stop performing the respiratory infection risk detection and other processing according to the method provided in this example.

[0173] For example, such as Figure 9As shown, the wearable device's health monitoring application can provide a control switch for respiratory monitoring functionality. Users can operate this control switch to turn the respiratory health monitoring function on or off. When the respiratory health monitoring function is activated, the wearable device can execute the method provided in this application embodiment to monitor the respiratory health of the user wearing the wearable device.

[0174] Optionally, after a user activates the respiratory health monitoring function, the wearable device can run the function in the background, and the interface displayed in the foreground can be updated as the user operates.

[0175] S802: When a wearable device identifies the presence of a cough sound by recognizing audio in the environment, it determines the corresponding cough sound signal.

[0176] Optionally, wearable devices can use a trained audio recognition model to identify whether the captured audio is a cough sound.

[0177] S803: Feature data for extracting cough sound signals from wearable devices.

[0178] S804: Wearable devices input characteristic data of cough sound signals into a trained disease screening model.

[0179] S805: The wearable device uses a disease screening model to determine whether the cough sound corresponds to a risk of respiratory infection; if yes, proceed to step S806; otherwise, proceed to step S801.

[0180] The specific implementation of the above steps can refer to the method of using a disease screening model to detect the presence of disease risk in step S301 above, and will not be repeated here.

[0181] S806: Wearable devices determine the time period during which a cough sound occurs.

[0182] This time period includes at least the start and stop times of the cough sound, and may also include the duration of the cough sound.

[0183] S807: Wearable devices acquire stored physiological data detected during the specified time period.

[0184] The specific implementation method of this step can be referred to the relevant method in step S302 above, and will not be repeated here.

[0185] S808: Wearable devices input physiological data into a trained motion recognition model.

[0186] S809: Wearable devices determine whether physiological data corresponds to the execution of a target action based on a motion recognition model, and determine the corresponding confidence level.

[0187] The specific implementation method of this step can be referred to the method of recognizing physiological data using an action recognition model described in step S303 above, and will not be repeated here.

[0188] S810: The wearable device determines whether the confidence level is greater than or equal to the set first confidence level threshold; if yes, proceed to step S811, otherwise proceed to step S812.

[0189] S811: Wearable devices determine that the cough sound belongs to the target user and display a prompt message to alert the target user to the risk of respiratory infection.

[0190] For example, wearable devices can display Figure 10 The interface shown contains a notification message indicating that the target user is at risk of respiratory infection.

[0191] S812: The wearable device determines whether the confidence level is less than or equal to the set second confidence level threshold; if yes, proceed to step S813, otherwise proceed to step S814.

[0192] The second confidence threshold is less than the first confidence threshold.

[0193] S813: Wearable devices determine that the cough sound does not belong to the target user and display a prompt message to alert the target user that there is a risk of respiratory infection in their environment.

[0194] For example, wearable devices can display Figure 11 The interface shown contains a notification message indicating to the target user that there is a risk of respiratory infection in their environment.

[0195] S814: The wearable device displays a prompt message to ask the target user if they have a cough. Then, step S815 is executed.

[0196] For example, wearable devices can display Figure 12 The interface shown contains a prompt asking the user if they have a cough.

[0197] S815: The wearable device determines whether the target user is coughing based on the information provided by the user; if yes, proceed to step S811; otherwise, proceed to step S813.

[0198] The specific execution of the above steps can be referred to the relevant descriptions in the above embodiments, and will not be repeated in this example.

[0199] It should be noted that the specific implementation process provided in the above examples is only an example of the method process applicable to the embodiments of this application. The execution order of each step can be adjusted according to actual needs, and other steps can be added or some steps can be reduced.

[0200] In the above embodiments, the wearable device, by detecting the source of the cough sound, ensures that the cough sound used for respiratory disease risk prediction originates from the target user, reducing the interference of erroneously collected audio signals on the prediction process and thus improving prediction accuracy. Furthermore, by distinguishing whether the cough sound originates from the target user wearing the wearable device or other users in the environment, predictions can be made separately for the target user's own respiratory infection risk and the respiratory infection risk in the environment. This improves both the collection and prediction effectiveness of the target user's own cough sound and, by identifying infection sources and risks in the environment, promptly reminds the target user to take personal protective measures to reduce the risk of infection. In summary, this solution can quickly identify the source of the cough sound and further predict and warn of respiratory infection risks, with low real-time performance and implementation costs, and significantly improves the user experience. Moreover, this solution has significant application value in indoor and other public scenarios and is highly feasible.

[0201] Based on the above embodiments and the same concept, this application also provides an audio detection method, such as... Figure 13 As shown, the method includes:

[0202] S1301: The electronic device acquires the first audio and identifies the type of the first audio.

[0203] For example, the electronic device can be one of the embodiments described above ( Figure 3 The electronic device described in the above embodiments can also be the one described in the above embodiments. Figure 8 The wearable devices described in ().

[0204] S1302: An electronic device acquires a first signal through one or more sensors, wherein the electronic device includes one or more sensors.

[0205] The first signal is a signal collected from a first user. For example, the first user can be the target user described in the above embodiments. The first signal can be a signal corresponding to the physiological data described in the above embodiments, and the electronic device can determine the physiological data described in the above embodiments based on the collected first signal.

[0206] S1303: When the electronic device recognizes that the first audio is of a set type, it determines a first time interval based on the first audio, and the first time interval includes the start time and the end time.

[0207] For example, the first audio of the set type can be the target audio described in the above embodiments, and the first time interval is the time interval in which the first audio is generated, which can be the target time period described in the above embodiments.

[0208] S1304: The electronic device determines the second signal based on the first time interval and the first signal.

[0209] For example, the second signal can be a signal corresponding to physiological data within the target time period described in the above embodiments, and the electronic device can determine the physiological data based on the collected second signal.

[0210] S1305: If the second signal matches the first audio, the electronic device displays the first information.

[0211] Optionally, if the second signal does not match the first audio, the electronic device displays the second information.

[0212] For example, the first information can be the notification information used to notify the target user of the risk of illness, as described in the above embodiments, for example... Figure 6 or Figure 10 The information shown. The first information can be the notification information described in the above embodiments used to notify the target user that there is a risk of infection in their environment, for example. Figure 7 or Figure 11 The information shown.

[0213] Specifically, the specific steps performed by the electronic device in this method can be found in the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0214] Based on the above embodiments and the same concept, this application also provides an audio detection method, applied to a system including a first electronic device and a second electronic device, such as... Figure 14 As shown, the method includes:

[0215] S1401: The first electronic device acquires the first audio and identifies the type of the first audio.

[0216] For example, the first electronic device may be one of the embodiments described above ( Figure 3 The electronic device described in the document. For example, the first electronic device may be a mobile terminal device such as a mobile phone.

[0217] S1402: The second electronic device acquires the first signal through one or more sensors, wherein the second electronic device includes one or more sensors.

[0218] For example, the second electronic device can be the physiological monitoring device described in the above embodiments. For instance, the second electronic device can be a wearable device such as a watch or a bracelet.

[0219] The first signal is a signal collected from a first user. For example, the first user can be the target user described in the above embodiments. The first signal can be a signal corresponding to the physiological data described in the above embodiments.

[0220] It should be noted that there is no strict time limit for the execution of the above steps S1401 and S1402. For example, step S1401 can be executed earlier than step S1402, later than step S1402, or simultaneously (or synchronously) with step S1402.

[0221] S1403: When the first electronic device recognizes that the first audio is of a set type, it determines a first time interval based on the first audio, and the first time interval includes the start time and the end time.

[0222] For example, the first audio of the set type can be the target audio described in the above embodiments, and the first time interval is the time interval in which the first audio is generated, which can be the target time period described in the above embodiments.

[0223] S1404: The first electronic device sends a request message to the second electronic device. The request message is used to request the acquisition of the signal corresponding to the first time interval.

[0224] S1405: The second electronic device determines a first time interval based on the received request information, and determines a second signal based on the first time interval and the first signal.

[0225] For example, the second signal can be the signal corresponding to the physiological data within the target time period described in the above embodiments.

[0226] S1406: The second electronic device sends a second signal to the first electronic device.

[0227] S1407: If the first electronic device determines that the received second signal matches the first audio, it displays the first information.

[0228] Optionally, if the second signal does not match the first audio, the electronic device displays the second information.

[0229] For example, the first information can be the notification information used to notify the target user of the risk of illness, as described in the above embodiments, for example... Figure 6 or Figure 10 The information shown. The first information can be the notification information described in the above embodiments used to notify the target user that there is a risk of infection in their environment, for example. Figure 7 or Figure 11 The information shown.

[0230] Specifically, the specific steps performed by the first or second electronic device in this method can be found in the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0231] Based on the above embodiments and the same concept, this application also provides an electronic device for implementing the audio detection method provided in this application. Figure 15 As shown, electronic device 1500 may include: display screen 1501, memory 1502, one or more processors 1503, and one or more computer programs (not shown). These devices may be coupled via one or more communication buses 1504.

[0232] The display screen 1501 is used to display images, videos, application interfaces, and other related user interfaces. The memory 1502 stores one or more computer programs (code), and the one or more computer programs include computer instructions; one or more processors 1503 call the computer instructions stored in the memory 1502, causing the electronic device 1500 to execute the audio detection method provided in the embodiments of this application.

[0233] In a specific implementation, memory 1502 may include high-speed random access memory and may also include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 1502 may store an operating system (hereinafter referred to as the system), such as embedded operating systems like Android, iOS, Windows, or Linux. Memory 1502 can be used to store implementation programs of the embodiments of this application. Memory 1502 may also store network communication programs, which can be used to communicate with one or more additional devices, one or more user devices, or one or more network devices. One or more processors 1503 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs in the scheme of this application.

[0234] It should be noted that, Figure 15 This is merely one implementation of the electronic device 1500 provided in this application embodiment. In actual applications, the electronic device 1500 may include more or fewer components, which is not limited here.

[0235] Based on the above embodiments and the same concept, this application also provides a system, which includes the first electronic device and the second electronic device described above.

[0236] Based on the above embodiments and the same concept, this application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method provided in the above embodiments.

[0237] Based on the above embodiments and the same concept, this application also provides a computer program product, which includes a computer program or instructions that, when run on a computer, cause the computer to perform the methods provided in the above embodiments.

[0238] The methods provided in this application can be implemented, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented, in whole or in part, in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs), or semiconductor media (e.g., SSDs), etc.

[0239] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method of audio detection applied to an electronic device, the method comprising: The method comprises: collecting a first audio and identifying a type of the first audio; collecting a first signal through one or more sensors, wherein the electronic device comprises the one or more sensors; when it is identified that the first audio is of a set type, determining a first time interval according to the first audio, the first time interval comprising a start time and an end time; determining a second signal according to the first time interval and the first signal; if the second signal matches the first audio, displaying first information; wherein the first information is used to indicate that the first audio is from a first user, and / or that the first user is at risk of illness; the first user is a user wearing the electronic device; if the second signal does not match the first audio, displaying second information; wherein the second information is used to indicate that the first audio is not from the first user, and / or that the environment in which the first user is located is at risk of infection; wherein the second signal matches the first audio, comprising that a type of a second action corresponding to the second signal is the same as a type of a first action corresponding to the first audio.

2. The method of claim 1, wherein, Before determining the first time interval according to the first audio, the method further comprises: determining a first risk parameter according to the first audio, the first risk parameter being greater than or equal to a set threshold, wherein the first risk parameter is used to indicate a disease risk corresponding to the first audio.

3. The method of claim 2, wherein, The determination of the first risk parameter according to the first audio comprises: extracting feature data from the first audio; wherein the feature data is used to represent time domain features and / or frequency domain features of the first audio; determining the first risk parameter according to a set disease screening model and the feature data; wherein the disease screening model is used to represent a relationship between time domain features and / or frequency domain features of an audio and a risk parameter corresponding to the audio.

4. The method according to any one of claims 1 to 3, characterized in that, Before displaying the first information if the second signal matches the first audio, the method further comprises: determining the first action corresponding to the first audio according to the first audio, and determining the second action corresponding to the second signal according to the second signal.

5. The method of claim 4, wherein, Determining the second action corresponding to the second signal according to the second signal comprises: determining the second action and a confidence degree of the second action according to a set action recognition model and the second signal, wherein the action recognition model is used to represent a relationship between the second signal and the second action and the confidence degree, and the confidence degree is used to represent an identification accuracy of the action recognition model; determining that the confidence degree is greater than or equal to a set first threshold; or determining that the confidence degree is greater than or equal to a set second threshold and less than or equal to a set third threshold; displaying first indication information, the first indication information being used to indicate whether to confirm execution of the second action; receiving second indication information, the second indication information being used to indicate confirmation of execution of the second action.

6. The method according to any one of claims 1 to 5, wherein The second signal comprises at least one of: a signal collected through an acceleration sensor; a signal collected through a gyroscope sensor; The signal collected by the photoelectric sensor.

7. The method according to any one of claims 1 to 6, wherein The setting type includes at least one of the following: Cough, breath, sneeze, joint click. 8.A method of audio detection applied to a first electronic device, comprising: The method comprises: Collecting a first audio and identifying a type of the first audio; When the first audio is identified as a setting type, determining a first time interval according to the first audio, the first time interval comprising a start time and an end time; Sending request information to a second electronic device, the request information being used to request to obtain a signal corresponding to the first time interval; Receiving a first signal from the second electronic device, the first signal being collected by one or more sensors of the second electronic device, wherein the second electronic device comprises the one or more sensors; If the first signal matches the first audio, displaying first information; wherein the first information is used to indicate that the first audio is from a first user, and / or the first user has a risk of disease; the first user is a user wearing the second electronic device; If the first signal does not match the first audio, displaying second information; wherein the second information is used to indicate that the first audio is not from the first user, and / or the environment where the first user is located has an infection risk; Wherein the first signal matches the first audio, comprising: the type of a second action corresponding to the first signal is the same as the type of a first action corresponding to the first audio.

9. The method of claim 8, wherein, Before determining the first time interval according to the first audio, the method further comprises: Determining a first risk parameter according to the first audio, the first risk parameter being greater than or equal to a setting threshold, wherein the first risk parameter is used to indicate a disease risk corresponding to the first audio.

10. The method of claim 9, wherein, Determining a first risk parameter according to the first audio comprises: Extracting feature data from the first audio; wherein the feature data is used to represent time domain features and / or frequency domain features of the first audio; According to a setting disease screening model and the feature data, determining the first risk parameter; wherein the disease screening model is used to represent the relationship between the time domain features and / or frequency domain features of the audio and the risk parameter corresponding to the audio.

11. The method of any one of claims 8-10, wherein, Before displaying the first information if the first signal matches the first audio, the method further comprises: According to the first audio, determining the first action corresponding to the first audio, and according to the first signal, determining the second action corresponding to the first signal.

12. The method of claim 11, wherein, According to the first signal, determining the second action corresponding to the first signal comprises: According to a setting action recognition model and the first signal, determining the second action and a confidence degree of the second action, wherein the action recognition model is used to represent the relationship between the first signal and the second action and the confidence degree, and the confidence degree is used to represent the recognition accuracy of the action recognition model; Determining that the confidence degree is greater than or equal to a setting first threshold; or determining that the confidence is greater than or equal to a set second threshold and less than or equal to a set third threshold; displaying first indication information, the first indication information being used to indicate whether to confirm to perform the second action; and receiving second indication information, the second indication information being used to indicate to confirm to perform the second action.

13. The method of any one of claims 8-12, wherein, The first signal includes at least one of: a signal collected by an acceleration sensor; a signal collected by a gyroscope sensor; a signal collected by a photoelectric sensor.

14. The method of any one of claims 8-13, wherein, The set type includes at least one of: cough sound, breathing sound, sneeze sound, joint click.

15. An electronic device, comprising: The electronic device includes a display screen, a memory and one or more processors; The memory is configured to store computer program code including computer instructions, and when the computer instructions are executed by the one or more processors, the electronic device is caused to perform the method of any one of claims 1-7 or the method of any one of claims 8-14.

16. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the computer program runs on a computer, the computer is caused to perform the method of any one of claims 1-7 or the method of any one of claims 8-14.

Images