Hearing loss risk assessment method and system, intelligent device and storage medium

Abstract

The invention is suitable for the technical field of intelligent equipment, and provides a hearing loss risk assessment method and system, intelligent equipment and a storage medium, and the method comprises the steps: a near-ear wearable device collects an audio signal through a configured acoustic sensor, carries out the sound source classification marking of the audio signal based on a current working mode, and carries out the sound source classification marking of the audio signal; and according to a set time interval or a trigger condition, counting a signal intensity mean value of each sound source type audio signal, sending a sound source classification mark and the counted signal intensity mean value to a cloud server corresponding to the user, and indicating the cloud server to carry out hearing loss risk assessment on the user according to the sound source classification mark and the counted signal intensity mean value. The hearing loss risk assessment method and device can effectively improve the convenience, accuracy and universality of hearing loss risk assessment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent devices, and in particular to a hearing loss risk assessment method and system, an intelligent device, and a storage medium. BACKGROUND

[0002] Hearing loss is an increasingly serious public health problem, which not only leads to communication barriers, but also causes a series of negative effects such as cognitive decline and social isolation, greatly reducing the quality of life of users and causing heavy burdens on individuals, families and society. In particular, for people at high risk of hearing loss, such as the elderly and people who have been exposed to a noisy environment for a long time, it is particularly important to detect hearing problems and intervene in a timely manner.

[0003] Currently, hearing tests are mainly divided into objective audiometry and subjective audiometry. Objective audiometry, although more accurate, requires the use of expensive equipment in professional hospitals, and the results are usually only used as an auxiliary judgment. Subjective audiometry, such as pure tone audiometry, relies on user cooperation, and for people at high risk of hearing loss, such as the elderly, on the one hand, they pay less attention to hearing health, and the proportion of active regular detection is low, on the other hand, some elderly people have poor subjective cooperation willingness and have a rejection psychology towards active hearing detection, which leads them to fail to detect hearing loss problems in a timely manner and miss the best treatment opportunity.

[0004] Therefore, how to improve the convenience, accuracy and popularity of hearing loss risk assessment is a problem that needs to be considered. SUMMARY

[0005] The embodiments of the present application provide a hearing loss risk assessment method, system, intelligent device and storage medium, which can effectively improve the convenience, accuracy and popularity of hearing loss risk assessment.

[0006] In a first aspect, the embodiments of the present application provide a hearing loss risk assessment method, which is applied to a near-ear wearable device, and the method comprises:

[0007] Acquiring an audio signal through an acoustic sensor configured on the near-ear wearable device;

[0008] Based on the current working mode of the near-ear wearable device, classifying and marking the audio signal according to the sound source;

[0009] According to a set time interval or trigger condition, the signal strength average of each sound source type audio signal is counted;

[0010] transmit the sound source classification label and the statistical signal intensity average to a cloud server corresponding to the user, and instruct the cloud server to perform hearing loss risk assessment on the user according to the sound source classification label and the statistical signal intensity average.

[0011] In a possible implementation of the first aspect, the sound source classification label is based on the current working mode of the near-ear wearable device, and the sound source classification label includes:

[0012] inputting the collected audio signal into a sound source type recognition model to obtain a sound source type of the audio signal, the sound source type recognition model being a machine learning model that has been trained and used to identify a sound source type;

[0013] based on the current working mode of the near-ear wearable device and the sound source type, performing sound source classification labeling on the audio signal.

[0014] In a possible implementation of the first aspect, the acoustic sensor includes a microphone and a human body vibration sensor, the microphone being configured to collect an audio signal, and the human body vibration sensor being configured to detect a bone conduction vibration signal; and the sound source classification label is based on the current working mode of the near-ear wearable device and the sound source type, and includes:

[0015] correcting the sound source classification label according to the bone conduction vibration signal.

[0016] In a possible implementation of the first aspect, the method further includes:

[0017] performing bone voiceprint analysis according to the bone conduction vibration signal to determine whether a current user is a registered user of the near-ear wearable device;

[0018] if the current user is not the registered user of the near-ear wearable device, prompting the current user to create a new user account, or directly triggering an associated application to create a new user account and apply for independent cloud server resources.

[0019] In a second aspect, the embodiments of the present application provide a hearing loss risk assessment system, including a near-ear wearable device configured with an acoustic sensor and a cloud server, where:

[0020] the near-ear wearable device is configured to collect an audio signal through the acoustic sensor, perform sound source classification labeling on the audio signal based on a current working mode, and upload the sound source classification label and a statistical signal intensity average to a cloud server corresponding to the user according to a set time interval or a triggering condition:

[0021] The cloud server is configured to perform hearing loss risk assessment on the user according to the sound source classification labels and the statistical signal intensity mean.

[0022] In a possible implementation of the first aspect, the cloud server is further configured to receive and integrate sound source classification labels and signal intensity mean uploaded by a plurality of near-ear wearable devices worn by the user, and generate a comprehensive hearing loss risk assessment result of the user based on the sound source classification labels and the signal intensity mean uploaded by the plurality of near-ear wearable devices.

[0023] In a third aspect, an embodiment of the present application provides a hearing loss risk assessment device applied to a near-ear wearable device, and the device comprises:

[0024] an audio acquisition unit configured to acquire audio signals through an acoustic sensor arranged on the near-ear wearable device;

[0025] a classification labeling unit configured to perform sound source classification labeling on the audio signals based on a current working mode of the near-ear wearable device;

[0026] a signal statistics unit configured to statistically calculate signal intensity mean of audio signals of each sound source type according to a set time interval or a trigger condition;

[0027] a hearing loss assessment unit configured to send the sound source classification labels and the statistical signal intensity mean to a cloud server corresponding to the user, and instruct the cloud server to perform hearing loss risk assessment on the user according to the sound source classification labels and the statistical signal intensity mean.

[0028] In a fourth aspect, an embodiment of the present application provides an intelligent device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the hearing loss risk assessment method of the first aspect when executing the computer program.

[0029] In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the hearing loss risk assessment method of the first aspect.

[0030] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when running on an intelligent device, enables the intelligent device to perform the hearing loss risk assessment method of the first aspect.

[0031] In the embodiments of the present application, the audio signal is automatically collected through the acoustic sensor on the near-ear wearable device, which can realize unobtrusive data collection in the process of the user using the near-ear wearable device in daily life, without the need for the user to perform additional complex operations or go to a professional institution for a special hearing test, thereby improving the convenience of hearing loss risk assessment. The near-ear wearable device intelligently classifies and marks the sound sources based on the real-time working mode of the device, which helps to improve the accuracy of the assessment. At the same time, by periodically counting the mean values of the signal strengths of each sound source type, the hearing conditions of the user in different time and different sound source environments can be dynamically reflected, making the collected data more representative and timely, and further enhancing the reliability and accuracy of the assessment results. The sound source classification mark and the signal strength mean value are automatically uploaded to the cloud server for professional analysis, and the cloud computing power is used to realize dynamic trend assessment, which significantly improves the accuracy of hearing loss risk assessment. The whole process does not need the user to actively intervene, and through the collaborative processing of the device end and the cloud end, the hearing monitoring capability is expanded to a wider user group, effectively improving the popularity. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0033] Figure 1 is a system architecture diagram of a hearing loss risk assessment system provided by the embodiments of the present application;

[0034] Figure 2 is a schematic diagram of an earphone structure provided by the embodiments of the present application;

[0035] Figure 3.1 is a schematic diagram of an application scenario of a hearing loss risk assessment system provided by the embodiments of the present application;

[0036] Figure 3.2 is another schematic diagram of an application scenario of a hearing loss risk assessment system provided by the embodiments of the present application;

[0037] Figure 4 is an implementation flowchart of a hearing loss risk assessment method provided by the embodiments of the present application;

[0038] Figure 5 is a specific implementation flowchart of step S402 in the hearing loss risk assessment method provided by the embodiments of the present application;

[0039] Figure 6is a specific implementation flowchart of verifying a user identity in a hearing loss risk assessment method provided by an embodiment of the present application;

[0040] Figure 7 is a structural block diagram of a hearing loss risk assessment device provided by an embodiment of the present application;

[0041] Figure 8 is a schematic diagram of a smart device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0042] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0043] It should be understood that the term "comprises" as used in the specification and the appended claims indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0044] It should also be understood that the term "and / or" as used herein refers to any one of the associated listed items, combinations of one or more of the associated listed items, and all possible combinations thereof.

[0045] As used in the description of the application and the appended claims, the term "if" can be interpreted as meaning "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted as meaning "upon determining" or "in response to determining" or "upon detecting [a described condition or event]" or "in response to detecting [a described condition or event]", depending on the context.

[0046] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third", etc. are only used for differentiation in description, and cannot be understood as indicating or implying relative importance.

[0047] Reference within the specification of this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in additional embodiments," and so on, in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise specifically specified

[0048] The hearing loss risk assessment method provided by the embodiments of the application is suitable for various types of intelligent devices that need to perform hearing loss risk assessment, and the embodiments of the application do not make any limitation on the specific type of intelligent device.

[0049] Figure 1 A system architecture diagram of a hearing loss risk assessment system provided by the embodiments of the application is shown, and the details are as follows: For the convenience of description, only the part related to the embodiments of the application is shown.

[0050] With reference to Figure 1 , the hearing loss risk assessment system comprises a near-ear wearable device 1 and a cloud server 2, and the wearable device 1 is configured with an acoustic sensor 1-1, wherein:

[0051] The near-ear wearable device 1 is used to collect audio signals through the acoustic sensor 1-1, and based on the current working mode, the audio signals are classified and marked by sound sources; according to the set time interval or trigger condition, the signal strength average of each sound source type audio signal is counted, and the sound source classification mark and the counted signal strength average are uploaded to the cloud server 2 corresponding to the user:

[0052] The cloud server 2 is used to assess the hearing loss risk of the user according to the sound source classification mark and the counted signal strength average.

[0053] The near-ear wearable device 1 refers to a wearable product worn close to or in contact with the ear, including but not limited to open earphones, in-ear earphones and bone conduction earphones, smart earmuffs, smart hearing aids, AR / VR headsets and other ear-hung electronic devices with near-ear acoustic signal acquisition capability. The acoustic sensor 1-1 is used to collect audio signals in the user's surrounding environment, including ambient environmental noise, music played by earphones, dialogue voice and other audio signals.

[0054] As a possible implementation of the present application, the near-ear wearable device 1 further comprises a computing unit configured to input the collected audio signal into a sound source type identification model to obtain the sound source type of the audio signal, the sound source type identification model being a machine learning model trained to identify the sound source type; and perform sound source classification labeling on the audio signal based on the current working mode of the near-ear wearable device 1 and the sound source type. The computing unit can be a SOC chip or an NPU.

[0055] As a possible implementation of the present application, the acoustic sensor 1-1 comprises a microphone 1-11 and a human body vibration sensor 1-12. The microphone 1-11 is configured to collect an audio signal; and the human body vibration sensor 1-12 is configured to detect a bone conduction vibration signal.

[0056] In a possible implementation, the microphone 1-11 is arranged at a position close to the ear canal of the device.

[0057] In a possible implementation, the human body vibration sensor 1-12 is arranged at a position in the concha cavity, at the top of the ear hook, or at a position clamped by the pinna.

[0058] In a possible implementation, the near-ear wearable device 1 is further configured to correct the sound source classification labeling based on the bone conduction vibration signal detected by the human body vibration sensor 1-12.

[0059] The human body vibration sensor can be placed at different positions according to different earphone shapes, which are in contact with the human body and are easy to receive the human body vibration caused by speaking. If the earphone is in an ear hook shape and the sound emitting front shell of the earphone is far away from the ear canal, the human body vibration sensor can be placed at the top of the ear hook close to the temple. If the sound emitting front shell of the earphone is close to the ear canal and the front shell is directly in contact with the concha cavity, the human body vibration sensor can be directly placed at the contact position of the concha cavity. If the earphone is in a clamping ear shape, the human body vibration sensor can be directly placed at the position in close contact with the pinna.

[0060] For example, as shown in Figure 2 , the near-ear wearable device 1 is an open earphone, which comprises a human body vibration sensor arranged at the top of the ear hook in an ear hook shape, a front cavity microphone, and an earphone speaker.

[0061] As a possible implementation of the present application, the near-ear wearable device 1 is further configured to perform bone voiceprint analysis based on the bone conduction vibration signal to determine whether the current user is a registered user of the near-ear wearable device; and if the current user is not a registered user of the near-ear wearable device, prompting the current user to create a new user account, or directly triggering an associated application to create a new user account and apply for independent cloud server resources 2.

[0062] The hearing loss risk assessment system can establish a wireless connection between the cloud server 2 and one or more near-ear wearable devices 1 by using Bluetooth technology, WIFI technology, or 3G / 4G / 5G technology, etc.

[0063] As a possible implementation of the present application, the near-ear wearable device 1 is connected to a smart device in a wired or wireless manner, and the smart device is connected to the cloud server 2.

[0064] In a possible implementation, the near-ear wearable device 1 can also be connected to a computer through a USB interface, and the cloud server is connected through a computer client program. The communication connection mode of the near-ear wearable device 1 and the cloud server in the embodiments of the present application is not limited.

[0065] In a possible implementation, when the smart device (such as a smart television, a computer, etc.) connected to the near-ear wearable device 1 has a matching software, the collected hearing data is directly uploaded to the corresponding cloud server of the user through the smart device. If there is no matching software on the smart device, the collected hearing data is temporarily stored in the local of the near-ear wearable device 1, and when the near-ear wearable device 1 is connected to the smart device with the matching software, the hearing data is uploaded to the corresponding cloud server.

[0066] As a possible implementation of the present application, the cloud server 2 is also used for receiving and integrating the hearing data such as the sound source classification label and the signal strength average uploaded by the multiple near-ear wearable devices worn by the user; based on the sound source classification label and the signal strength average uploaded by the multiple near-ear wearable devices, a comprehensive hearing loss risk assessment result of the user is generated. The cloud server 2 receives the sound source classification label and the statistical signal strength average uploaded by the different near-ear wearable devices worn by the user. The cloud server 2 is bound to the personal account of the user, and provides cloud resources exclusively for the personal account of the user, and is independent of the near-ear wearable device 1. That is, the cloud server 2 can receive the hearing data uploaded by the different near-ear wearable devices 1 bound to the same personal account of the user.

[0067] Exemplarily, with reference to Figure 3.1, user 1 owns earphone 1, earphone 2 and earphone 3 which are bound to the same personal account, and the multiple earphones of user 1 can communicate with the cloud resource corresponding to user 1 through multiple smart devices (TV, mobile phone 1 and mobile phone 2) of different types. Specifically, earphone 1 communicates with the cloud resource exclusive to user 1 through TV 1 of user 1, earphone 2 communicates with the cloud resource exclusive to user 1 through mobile phone 1 of user 1, and earphone 3 communicates with the cloud resource exclusive to user 1 through mobile phone 2 of user 1, and TV 1, mobile phone 1 and mobile phone 2 are provided with matching software. User 2 owns mobile phone 1 which is bound to a personal account, and earphone 1 of user 2 communicates with the cloud resource exclusive to user 2 through mobile phone 1 of user 2. The hearing data collected by the multiple earphones of user 1 can be uploaded to the same cloud resource corresponding to the personal account of user 1, so as to ensure the accuracy and integrity of the hearing data used for hearing loss risk assessment, and the hearing data collected by the earphones of user 1 and user 2 are uploaded to different cloud resources, so as to ensure the safety and privacy of the data.

[0068] In a possible implementation, the same near-ear wearable device 1 can be shared by different users, and the cloud server 2 to which the near-ear wearable device 1 is connected for uploading hearing data is also different during use by different users.

[0069] Exemplarily, with reference to Figure 3.2 , earphone 1 of user 1 can be shared by user 1, user 2 and user 3, earphone 1 of user 1 uploads the hearing data of user 1 to the cloud resource exclusive to user 1 through mobile phone 1 of user 1, and it is confirmed through bone voiceprint analysis that the current wearer of earphone 1 of user 1 is new user 2 (not user 1), then a cloud resource exclusive to new user 2 is created, and the hearing data collected by user 1 using earphone 1 is uploaded to the cloud resource of user 2, earphone 1 of user 1 can also be lent to user 3 to wear, and if it is confirmed through bone voiceprint analysis that user 3 is a new user, a cloud resource exclusive to new user 3 is created, and the hearing data is uploaded to the cloud resource exclusive to user 3 through mobile phone 1 of user 3.

[0070] In a possible implementation, the cloud server 2 is further configured to send a hearing loss risk assessment result to the near-ear wearable device 1, and the near-ear wearable device 1 further receives the hearing loss risk assessment result returned by the cloud server 2 and prompts the user based on the hearing loss risk assessment result.

[0071] In the embodiments of the present application, through the cooperative architecture of the near-ear wearable device and the cloud server, the device end completes the audio signal collection, the intelligent marking of the sound source based on the working mode, and the periodic signal mean statistics, realizes the lightweight processing of the front-end data, reduces the device power consumption, the cloud server executes the professional risk assessment algorithm based on the data uploaded by the near-ear wearable device, analyzes the long-term dynamic changes by using the high-performance computing resources, and ensures the accuracy and reliability of the evaluation results; the division and cooperation of the device and the cloud not only guarantee the real-time performance of data collection, but also support multi-user expansion through centralized processing of the cloud, significantly improve the popularity of the system, and automatically run the whole process, so that the user does not need to intervene, and the convenience, accuracy and popularity of the hearing loss risk assessment can be effectively improved.

[0072] Figure 4 The implementation process of the hearing loss risk assessment method provided by the embodiments of the present application is shown, and the method process includes steps S401 to S404. The execution end of the method process is the near-ear wearable device 1 shown in the figure, and the specific implementation principle of each step is as follows: Figure 1

[0073] Step S401: Collecting an audio signal through an acoustic sensor configured on the near-ear wearable device.

[0074] The near-ear wearable device such as an open earphone, a bone conduction earphone, etc. is configured with an acoustic sensor, which can be attached to the ear to collect audio signals in the user's surrounding environment in real time through the acoustic sensor. The audio signal includes ambient noise, music played by the earphone, conversation voice, etc. The user can automatically collect data when wearing the near-ear wearable device in daily life, without the need for additional complex operations or going to a professional institution for special hearing tests, thereby effectively reducing the use threshold.

[0075] In a possible implementation, the acoustic sensor includes a microphone and a human body vibration sensor, and the audio signal is collected through the microphone. The acoustic sensor also includes a human body vibration sensor for detecting bone conduction vibration signals by attaching to the human body bone (such as the temple, the concha). For example, when the user speaks, the vocal cords vibrate and conduct through the skull to the sensor, generating a characteristic signal that is different from the environmental sound. The dual-sensor design of the microphone plus the human body vibration sensor can not only collect regular audio, but also accurately distinguish the user's own voice from external sounds by using bone conduction signals, thereby improving the accuracy of data collection from the source.

[0076] Step S402: Classifying and marking the sound source based on the current working mode of the near-ear wearable device.

[0077] The sound source classification marking at least includes information of the device working mode and the sound source type. In a possible implementation, the near-ear wearable device obtains the current working mode through a built-in system state monitoring module. ​

[0078] The device working modes include, but are not limited to, a music playing mode, a telephone call mode, a standby mode, and a voice assistant mode. For example, in the music playing mode, the audio output channel of the device is in the working mode; in the telephone call mode, the device interacts with the calling party according to a specific communication protocol; in the standby mode, the device is in a low-power consumption state without audio output; and in the voice assistant mode, the audio output channel of the device is in the working mode, and the device interacts with the cloud voice assistant server. The sound source types include, but are not limited to, environmental noise, earphone playing audio, user voice in a telephone call, opposite party voice in a telephone call, user voice in a face-to-face conversation, and other person voice in a face-to-face conversation.

[0079] In different working modes, the sound environment of the user and the source of the audio signal are quite different. The sound source classification marking method combining the device working mode and the sound source type can more comprehensively and specifically reflect the source and nature of the audio signal.

[0080] For example, in the telephone call mode, the sound source classification marking can be specifically "user voice in a telephone call" or "opposite party voice in a telephone call"; and in the music playing mode, the sound source classification marking is "earphone playing audio".

[0081] As a possible implementation manner of the present application, Figure 5 An implementation process of step S402 in the hearing loss risk assessment method provided by the embodiment of the present application is shown, and is described in detail as follows:

[0082] A1: input the collected audio signal into a sound source type recognition model to obtain the sound source type of the audio signal.

[0083] In a possible implementation manner, the collected audio signal is preprocessed and then input into the sound source type recognition model. The preprocessing operation includes signal noise reduction processing, signal standardization and normalization, feature enhancement, etc. By preprocessing the audio signal collected by the acoustic sensor, the interference is filtered out, the signal quality is optimized, and unnecessary information is removed.

[0084] The sound source type recognition model is a machine learning model that has been trained and used to recognize the sound source type. The audio signal is converted into a feature vector form that can be processed by the model, and the sound source type recognition model outputs the corresponding sound source type by analyzing the pattern and rule of the feature vector. The machine learning model is trained by a large amount of labeled training data to obtain a sound source type recognition model that can recognize the sound source type of the audio signal, wherein the training data covers a plurality of common sound source types, such as environmental noise (street noise, indoor background noise, etc.), earphone playing audio (music, audio book, etc.), user voice and opposite party voice in a telephone call, user voice and other person voice in a face-to-face conversation, etc.

[0085] In a possible implementation, the sound source type recognition model includes a spectral feature extraction model, a temporal feature extraction model, and a pre-trained classification model. The spectral feature extraction model is a machine learning model for analyzing frequency domain characteristics of an audio signal; the temporal feature extraction model is a machine learning model for analyzing change characteristics of an audio signal in the time domain. The pre-trained classification model is a machine learning model trained based on a large amount of audio data with labeled categories.

[0086] In the model training phase, the spectral feature extraction model is used to extract frequency domain feature information, and the temporal feature extraction model is used to extract time domain feature information; the extracted frequency domain feature information and time domain feature information are respectively given corresponding weights and then weighted and fused to generate a feature distribution histogram; the weighted fusion can comprehensively consider the frequency domain and time domain features, fully play the advantages of the two kinds of features, make up for the shortcomings of a single feature, generate a more representative and recognizable feature description, and visually display the fused feature information in the form of a histogram through the feature distribution histogram. Then, the pre-trained classification model is used to classify the input feature distribution histogram to determine which sound source type it belongs to. The model trained by a large amount of data has good recognition effect on signals of different scenes, different accents, and different audio qualities, can adapt to complex and changeable actual use environment, and ensures reliable sound source type judgment in various situations.

[0087] A2: Based on the current working mode of the near-ear wearable device and the sound source type, performing sound source classification marking on the audio signal.

[0088] In the embodiments of the present application, the powerful pattern recognition capability of the machine learning model can accurately extract subtle feature differences in the audio signal, and can more accurately distinguish similar sound source types compared with traditional simple rule judgment (such as distinguishing based on frequency range). For example, effectively distinguishing human voice noise in the environment from voice content played by earphones, and reducing the misjudgment rate of sound source classification. By comprehensively considering the working mode and the sound source type to perform sound source classification marking on the audio signal, similar sound source types can be more accurately distinguished, and misclassification can be reduced. For example, in a face-to-face conversation scene, the working mode can provide context information of the conversation, such as the conversation in the voice assistant mode and the conversation in the ordinary conversation mode can have different characteristics, and the working mode can be combined to better classify and mark user voice and other people's voice. This provides a more accurate data basis for subsequent hearing loss risk assessment for different sound source types, and helps to improve the performance and reliability of the entire evaluation system.

[0089] In a possible implementation, the sound source classification label further comprises a scene, and a classification label strategy is dynamically adjusted for different use scenarios (such as commuting, office work, and exercise) of the near-ear wearable device, which can improve the accuracy of sound source identification in a complex environment.

[0090] As a possible implementation of the present application, the sound source type label is corrected according to the bone conduction vibration signal detected by the human body vibration sensor.

[0091] When the bone conduction vibration signal is detected by the human body vibration sensor, it indicates that the user is speaking, and the corresponding audio signal can be labeled as a sound source type related to the user's voice, such as "user's voice in a telephone call" or "user's voice in a face-to-face conversation". If no bone conduction vibration signal is detected, it indicates that the current audio signal may come from other sound sources, such as environmental noise, earphone playback audio, the other party's voice in a telephone call, or other people's voice in a face-to-face conversation, and the preliminary sound source classification label is adjusted and corrected accordingly.

[0092] In actual scenarios, there may be some cases that are difficult to accurately distinguish by relying only on audio signals and working states, for example, in a noisy environment, the voice signal collected by the microphone may contain both the user's own voice and the voices of surrounding people. At this time, the bone conduction vibration signal can accurately determine whether the user is speaking, and then accurately distinguish the user's own voice from other voice signals. Using the bone conduction vibration signal to correct the sound source classification label can effectively improve the accuracy of the sound source classification label.

[0093] Step S403: According to the set time interval or trigger condition, the signal intensity average of each sound source type audio signal is counted.

[0094] The set time interval can be flexibly configured according to actual needs, such as every 30 minutes, 1 hour, 12 hours, or 24 hours, for periodic statistics of audio signal intensity. The trigger condition can be that the user enters a specific scene (such as a noisy environment, a concert, etc.), the audio signal intensity exceeds a certain threshold, the device detects a specific event (such as the user starting or ending a call, starting or closing music playback, etc.), and the like. The signal intensity average refers to the average value of the audio signal intensity of each sound source type within the statistical period, which is used to reflect the overall sound intensity level of the sound source type in that time period.

[0095] By counting according to the set time interval or trigger condition, the hearing condition of the user in different time periods and different sound source environments can be dynamically tracked, so that the collected data is more timely and representative. The periodic or condition-triggered statistical method can more comprehensively reflect the change trend of the hearing condition of the user in daily life, avoiding evaluation deviation caused by accidental factors. At the same time, the average signal strength of each sound source type audio signal can simplify the complex sound environment into representative numerical values, facilitating subsequent analysis and evaluation, and providing more reliable data basis for accurate evaluation of hearing loss risk.

[0096] Step S404: sending the sound source classification label and the statistical average signal strength to the cloud server corresponding to the user, and instructing the cloud server to evaluate the hearing loss risk of the user according to the sound source classification label and the statistical average signal strength.

[0097] The cloud server has strong computing and storage capabilities, and can receive and process a large amount of data uploaded by devices. It combines hearing data such as sound source classification labels and average signal strengths to comprehensively analyze and evaluate the hearing loss risk of the user according to a pre-set hearing loss risk evaluation model or algorithm. The hearing loss risk evaluation includes the evaluation level of the production hearing loss risk (such as low, medium, and high risk).

[0098] In the embodiments of the present application, the near-ear wearable device uploads the sound source classification label and the statistical average signal strength to the cloud server corresponding to the user. The cloud server corresponding to the user is a cloud computing and storage resource associated with the user, which is bound to the user's personal account and used to store, process and manage the hearing loss risk evaluation data of the user. When the user uses the near-ear wearable device for the first time and registers a personal account, the system will automatically create a dedicated server resource for the user in the cloud, and bind the user's personal account to the dedicated server resource. The audio signals, sound source classification labels, average signal strengths and other data collected by the near-ear wearable device are marked with the identifier of the user's personal account, and then sent to the cloud server corresponding to the personal account identifier for storage and processing. The cloud server will establish a separate data storage area for each user to ensure that the data of the users are independent and secure. For example, all data of user A are stored in the personal data storage space allocated by the cloud server for user A, and are strictly isolated from the data of user B.

[0099] The cloud server provides more personalized and accurate services for the user according to the bound user personal account. In a possible implementation, in the hearing loss risk assessment, the cloud server will comprehensively analyze the preset evaluation model according to the user's historical hearing data, device usage, and the current uploaded data of the near-ear wearable device, generate a hearing loss risk assessment result that is exclusive to the user, and the assessment result will be bound to the user account, so that the user can check and track his / her hearing health status at any time. The user can access the personal data on the cloud server through the associated mobile application or other terminal devices, and the binding of the user personal account also ensures the security and privacy of the data, so that only the devices and operations authorized by the user can access and manage the data.

[0100] In a possible implementation, the cloud server resource dedicated to the user personal account automatically analyzes the sound size and change of the user in different environmental noises and different device working modes by using the current uploaded and historical uploaded data of the near-ear wearable device of the user, and determines the personal hearing loss risk assessment level of the user by referring to a preset level judgment table. The level judgment table can be established based on a large amount of clinical data, audiology research results, and historical data of the user, and can comprehensively consider the potential influence of various factors such as different sound source types, signal intensity, and change trend on hearing. For example, the sound size and change of the near-ear wearable device playing sound and the sound size and change of the user speaking in different noise environments are automatically analyzed.

[0101] Exemplarily, the level judgment tables of some cases are shown in Tables 1 to 4. The judgment standard tables of the remaining cases can also be designed accordingly.

[0102] Table 1, hearing loss risk assessment table of playing sound (music playing mode)

[0103]

[0104] * The level of the remaining cases is low

[0105] Table 2, hearing loss risk assessment table of playing sound (opponent voice in telephone call mode)

[0106]

[0107]

[0108] * The level of the remaining cases is low

[0109] Table 3, hearing loss risk assessment table of speaking sound (user voice in telephone call mode)

[0110]

[0111] * The rest of the cases are low

[0112]

[0113]

[0114] * The rest of the cases are low

[0115] In one possible implementation, the near-ear wearable device receives the hearing loss risk assessment result returned by the cloud server, and prompts the user based on the hearing loss risk assessment result.

[0116] As one possible implementation of the present application, the cloud server receives and integrates the sound source classification labels and signal intensity averages uploaded by multiple near-ear wearable devices worn by the same user; based on the sound source classification labels and signal intensity averages uploaded by the multiple near-ear wearable devices, the cloud server generates a comprehensive hearing loss risk assessment result for the user.

[0117] In one possible implementation, the cloud server assigns weights to the hearing data uploaded by the user through different types of devices according to the device usage time, device type, etc., and combines the device weights and the hearing data uploaded by the devices with historical hearing data to assess the hearing loss.

[0118] The embodiments of the present application can fully utilize the computing power and storage capacity of the cloud to comprehensively and deeply analyze the hearing loss risk of the user, improve the accuracy and reliability of the assessment, and better meet the needs of the user by sending the sound source classification labels and signal intensity averages to the cloud server for centralized assessment. At the same time, the cloud server can provide the user with device personalized hearing loss risk assessment results and suggestions according to the individual differences (such as age, gender, health status, etc.) and historical data of different users, better meeting the needs of the user. In addition, with the popularity and scalability of the cloud server, the convenience and popularity of the hearing loss risk assessment can be greatly improved, providing timely and effective early warning and diagnosis basis for people with high hearing loss risk.

[0119] As one possible implementation of the present application, Figure 6 A specific implementation process of verifying the user's identity in the hearing loss risk assessment method provided by the embodiments of the present application is shown, and is described in detail as follows:

[0120] B1: Perform bone voiceprint analysis according to the bone conduction vibration signal to determine whether the current user is a registered user of the near-ear wearable device.

[0121] Bone voiceprint analysis is a biometric technology based on the characteristics of human bone conduction vibration. The bone structure and vocal cord vibration characteristics of each person are unique and difficult to fake, so the bone voiceprint can be used as a basis for identity recognition. When a user wears a near-ear wearable device and makes a sound, the human vibration sensor detects the corresponding bone conduction vibration signal.

[0122] In a possible implementation, bone voiceprint feature extraction is performed according to the detected bone conduction vibration signal, and comparison is made with a pre-stored bone voiceprint template to determine whether the user currently wearing the device is a registered user of the near-ear wearable device. By feature extraction and analysis, and comparison with the bone voiceprint template of the registered user pre-stored in the device, it is determined whether the current user is a registered user of the device.

[0123] B2: If the current user is not a registered user of the near-ear wearable device, the current user is prompted to create a new user account, or a new user account is directly triggered to be created by an associated application and independent cloud server resources are applied for.

[0124] When the bone voiceprint analysis determines that the current user is not a registered user of the device, the system triggers a corresponding user account creation and cloud resource application process. In response to the operation of the current user creating a new user account, the cloud server allocates independent cloud storage space and evaluation model instances for the new user account.

[0125] In a possible implementation, prompting the current user to create a new account can be achieved by displaying prompt information in the application program (such as a mobile phone APP) matched with the wearable device, to guide the user to complete the registration process.

[0126] In a possible implementation, directly triggering the creation of a new user account and the application of independent cloud server resources is achieved by a background automatic program, which automatically generates a new user account according to pre-set rules and logic, and allocates independent cloud storage resources for the user account to store and process hearing data of the user.

[0127] The embodiments of the present application allow a near-ear wearable device to be used by multiple users, and the user identity is verified by bone voiceprint analysis, without the need for the user to manually switch accounts or input passwords. The device automatically identifies the user identity and completes data isolation without sensing, thereby avoiding evaluation deviation caused by data confusion.

[0128] It can be seen from the above that, in the embodiment of the present application, the audio signal is automatically collected by the acoustic sensor on the near-ear wearable device, which can realize unobtrusive data collection in the process of the user using the near-ear wearable device in daily life, without the need for the user to perform additional complex operations or go to a professional institution for a special hearing test, thereby improving the convenience of hearing loss risk assessment. The near-ear wearable device intelligently classifies and marks the sound sources based on the real-time working mode of the device, which helps to improve the accuracy of the assessment. At the same time, by periodically counting the mean values of the signal strengths of each sound source type, the hearing conditions of the user in different time and different sound source environments can be dynamically reflected, making the collected data more representative and timely, and further enhancing the reliability and accuracy of the assessment results. The sound source classification mark and the signal strength mean value are automatically uploaded to the cloud server for professional analysis, and the cloud computing power is used to realize dynamic trend assessment, which significantly improves the accuracy of hearing loss risk assessment. The whole process does not need the user to actively intervene, and through the collaborative processing of the device end and the cloud end, the hearing monitoring capability is expanded to a wider user group, effectively improving the popularity.

[0129] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0130] According to the hearing loss risk assessment method described in the above embodiment, Figure 7 The structure block diagram of the hearing loss risk assessment device provided by the embodiment of the present application is shown, and only the part related to the embodiment of the present application is shown for ease of description.

[0131] Referring to Figure 7 The hearing loss risk assessment device is applied to a near-ear wearable device and includes an audio collection unit 71, a classification marking unit 72, a signal statistics unit 73, and a hearing loss assessment unit 74, wherein:

[0132] The audio collection unit 71 is configured to collect an audio signal through an acoustic sensor arranged on the near-ear wearable device;

[0133] The classification marking unit 72 is configured to classify and mark the sound sources of the audio signal based on the current working mode of the near-ear wearable device;

[0134] The signal statistics unit 73 is configured to count the mean values of the signal strengths of the audio signals of each sound source type according to a set time interval or a trigger condition;

[0135] The hearing loss evaluation unit 74 is configured to send the sound source classification label and the statistical mean of the signal strength to a cloud server corresponding to the user, and instruct the cloud server to evaluate the hearing loss risk of the user according to the sound source classification label and the statistical mean of the signal strength.

[0136] As a possible implementation of the present application, the classification labeling unit 72 comprises:

[0137] A type identification module is configured to input the collected audio signal into a sound source type identification model to obtain the sound source type of the audio signal, wherein the sound source type identification model is a machine learning model that has been trained and is used to identify the sound source type.

[0138] A classification labeling module is configured to perform sound source classification labeling on the audio signal based on the current working mode of the near-ear wearable device and the sound source type.

[0139] As a possible implementation of the present application, the acoustic sensor comprises a microphone and a human body vibration sensor, wherein the microphone is configured to collect an audio signal, and the human body vibration sensor is configured to detect a bone conduction vibration signal; the classification labeling module is further configured to correct the sound source classification label according to the bone conduction vibration signal; and the sound source classification label comprises the working mode of the device and the sound source type.

[0140] As a possible implementation of the present application, the hearing loss risk evaluation device further comprises:

[0141] An identity verification processing unit is configured to perform bone voiceprint analysis according to the bone conduction vibration signal to determine whether the current user is a registered user of the near-ear wearable device; if the current user is not a registered user of the near-ear wearable device, the current user is prompted to create a new user account, or a related application program is directly triggered to create a new user account and apply for independent cloud server resources.

[0142] It can be seen from the above that, in the embodiment of the application, the audio signal is automatically collected by the acoustic sensor on the near-ear wearable device, and the data collection can be realized without the user's awareness in the process of the user using the near-ear wearable device in daily life, without the user's additional complex operation or going to a professional institution for a special hearing test, thereby improving the convenience of the hearing loss risk assessment. The near-ear wearable device intelligently classifies and marks the sound sources based on the real-time working mode of the device, which helps to improve the accuracy of the assessment. At the same time, by periodically counting the mean values of the signal strengths of each sound source type, the hearing conditions of the user in different time and different sound source environments can be dynamically reflected, so that the collected data is more representative and timely, further enhancing the reliability and accuracy of the assessment results. The sound source classification mark and the signal strength mean value are automatically uploaded to the cloud server for professional analysis, and the cloud computing power is used to realize dynamic trend assessment, which significantly improves the accuracy of the hearing loss risk assessment. The whole process does not need the user's active intervention, and through the collaborative processing of the device and the cloud, the hearing monitoring capability is expanded to a wider user group, effectively improving the popularity.

[0143] It should be noted that the information interaction, execution process and the like between the above devices / units are based on the same concept as the method embodiments of the application, and the specific functions and the technical effects brought by the same can be referred to the method embodiments part, which will not be described here.

[0144] The embodiment of the application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by a processor to realize the steps of any one of the hearing loss risk assessment methods as Figures 4 to 6 indicated.

[0145] The embodiment of the application further provides an intelligent device, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor realizes the steps of any one of the hearing loss risk assessment methods as Figures 4 to 6 indicated when executing the computer program.

[0146] The embodiment of the application further provides a computer program product, when the computer program product runs on an intelligent device, so that the intelligent device executes the steps of any one of the hearing loss risk assessment methods as Figures 4 to 6 indicated.

[0147] Figure 8 is a schematic diagram of an intelligent device provided by an embodiment of the application. As Figure 8As shown, the smart device 8 of this embodiment includes a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80. The processor 80 implements the steps in each of the above hearing loss risk assessment method embodiments when executing the computer program 82, for example Figure 1 As shown, the processor 80 implements the functions of each of the modules / units in each of the above device embodiments when executing the computer program 82, for example Figure 7 As shown, the processor 80 implements the functions of each of the modules / units in each of the above device embodiments when executing the computer program 82, for example

[0148] For example, the computer program 82 can be segmented into one or more modules / units stored in the memory 81 and executed by the processor 80 to complete the present application. The one or more modules / units can be a series of computer readable instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 82 in the smart device 8.

[0149] The smart device 8 can include, but is not limited to, the processor 80 and the memory 81. Those skilled in the art can understand that Figure 8 The smart device 8 is only an example and does not constitute a limitation on the smart device 8, and can include more or fewer components than shown, or combine certain components, or different components, for example, the smart device 8 can also include an input / output device, a network access device, a bus, etc.

[0150] The processor 80 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0151] The memory 81 can be an internal storage unit of the smart device 8, for example, a hard disk or a memory of the smart device 8. The memory 81 can also be an external storage device of the smart device 8, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the smart device 8. Further, the memory 81 can also include both the internal storage unit and the external storage device of the smart device 8. The memory 81 is used to store the computer program and other programs and data required by the smart device. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0152] It should be noted that the information interaction, execution process, etc. between the above devices / units, since based on the same concept as the method embodiments of the present application, the specific functions and the technical effects brought by them can be referred to the method embodiments part, and will not be repeated here.

[0153] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit and module is only for convenient distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the method embodiments, and will not be repeated here.

[0154] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the device / smart device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0155] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0156] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A hearing loss risk assessment method, characterized in that: Applied to a near-ear wearable device, the method includes: collecting audio signals through an acoustic sensor configured on the near-ear wearable device; Based on the current operating mode of the near-ear wearable device, the audio signal is classified and labeled as a sound source; According to the set time interval or trigger condition, the signal strength average of the audio signal of each sound source type is counted; The sound source classification label and the statistical signal strength mean are sent to the cloud server corresponding to the user, and the cloud server is instructed to perform a hearing loss risk assessment on the user based on the sound source classification label and the statistical signal strength mean.

2. The method according to claim 1, characterized in that The performing sound source classification and labeling on the audio signal based on the current operating mode of the near-ear wearable device includes: Inputting the collected audio signal into a sound source type recognition model to obtain the sound source type of the audio signal, wherein the sound source type recognition model is a machine learning model that has been trained to identify the sound source type; Based on the current operating mode of the near-ear wearable device and the sound source type, the audio signal is subjected to sound source classification and labeling.

3. The method according to claim 1, characterized in that The acoustic sensor includes a microphone and a human body vibration sensor, the microphone is used to collect audio signals, and the human body vibration sensor is used to detect bone conduction vibration signals; The performing sound source classification and labeling on the audio signal based on the current operating mode of the near-ear wearable device and the sound source type includes: The sound source classification label is modified according to the bone conduction vibration signal.

4. The method according to claim 3, characterized in that The method further comprises: Performing bone voiceprint analysis based on the bone conduction vibration signal to determine whether the current user is a registered user of the near-ear wearable device; If the current user is not a registered user of the near-ear wearable device, the current user is prompted to create a new user account, or the associated application is directly triggered to create a new user account and apply for independent cloud server resources.

5. A hearing loss risk assessment system, characterized in that: The invention comprises a near-ear wearable device equipped with an acoustic sensor and a cloud server, wherein: The near-ear wearable device is configured to collect audio signals through the acoustic sensor, classify and label the audio signals based on the current working mode, calculate the average signal strength of the audio signals of each sound source type according to a set time interval or trigger condition, and upload the sound source classification label and the calculated average signal strength to the cloud server corresponding to the user: The cloud server is used to perform a hearing loss risk assessment on the user based on the sound source classification labels and the statistical signal strength mean.

6. The system according to claim 5, characterized in that The acoustic sensor includes a microphone and a human body vibration sensor; The microphone is used to collect audio signals; The human body vibration sensor is used to detect bone conduction vibration signals; The near-ear wearable device is further used to correct the sound source classification mark according to the bone conduction vibration signal; the sound source classification mark includes the device operating mode and the sound source type.

7. The system according to claim 6, characterized in that The near-ear wearable device is further used to perform bone voiceprint analysis based on the bone conduction vibration signal to determine whether the current user is a registered user of the near-ear wearable device; if the current user is not a registered user of the near-ear wearable device, the current user is prompted to create a new user account, or an associated application is directly triggered to create a new user account and apply for independent cloud server resources.

8. The system according to claim 5, wherein: The cloud server is also used to receive and integrate the sound source classification labels and signal strength averages uploaded by multiple near-ear wearable devices worn by the user; based on the sound source classification labels and signal strength averages uploaded by the multiple near-ear wearable devices, generate a comprehensive hearing loss risk assessment result for the user; and receive the sound source classification labels and statistical signal strength averages uploaded by the user wearing different near-ear wearable devices.

9. An intelligent device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the hearing loss risk assessment method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the hearing loss risk assessment method according to any one of claims 1 to 4 is implemented.

Images