Hearing test processing method, apparatus, device, and computer-readable storage medium

Abstract

The application provides a hearing test processing method, device and equipment and a computer readable storage medium; the method comprises the following steps: presenting a hearing test interface, the hearing test interface comprising a perceivable control and an imperceivable control; when the environment of the hearing test meets the environmental test requirements, outputting a test sound corresponding to an initial sound pressure; in response to a triggering operation for the perceivable control, reducing the initial sound pressure; in response to no triggering operation for the perceivable control being received or a triggering operation for the imperceivable control being received, increasing the initial sound pressure; when a triggering operation for the perceivable control is received for the second time at any sound pressure level, taking the current sound pressure level as the hearing value of the current test ear at the current frequency band, and performing audiometry on the next frequency band; after the audiometry is completed, a hearing graph of the hearing test is displayed, and the hearing graph comprises a hearing curve in the full frequency band. Through the application, remote communication with a doctor can be realized, and hearing aid efficiency can be improved.

Description

[0001] This application is a divisional application of the Chinese patent application filed on September 16, 2021, with application number 202111085866.3 and title "Hearing Assistive Method, Apparatus, Device and Computer-Readable Storage Medium". Technical Field

[0002] This application relates to Internet technology, and more particularly to a hearing test processing method, apparatus, device, and computer-readable storage medium. Background Technology

[0003] Currently, about one-fifth of the world's population has hearing loss. Of this population, only a small percentage will be fitted with specialized devices such as hearing aids or cochlear implants.

[0004] Hearing-impaired users wear hearing aids or cochlear implants, and the internal parameters are personalized. This means that hearing aid strategies need to be set according to the actual hearing loss in both ears to achieve the ideal auditory experience. Therefore, the devices need to be adjusted periodically to ensure good quality. On the one hand, face-to-face communication between doctors and users is not always feasible in some situations; on the other hand, the travel costs for users traveling from different locations cannot be ignored. Currently, dedicated hearing aid apps do not meet the needs of remote rehabilitation services in terms of device adjustment and communication with doctors. Summary of the Invention

[0005] This application provides a hearing test processing method, device, and computer-readable storage medium, which enables remote communication with doctors and improves the efficiency of hearing assistance.

[0006] The technical solution of this application embodiment is implemented as follows: This application provides a hearing test processing method, including: A hearing test interface is presented, which includes perceptible controls and non-perceptible controls; When the environment for the hearing test meets the environmental test requirements, the test tone corresponding to the initial sound pressure is output. In response to a triggering operation on the perceived control, the initial sound pressure level is reduced; In response to not receiving a trigger operation for the perceived control or receiving a trigger operation for the imperceptible control, the initial sound pressure level is increased; When a trigger operation for the perceptible control is received for the second time at any sound pressure level, the current sound pressure level is used as the hearing value of the current test ear in the current frequency band, and the hearing test is performed in the next frequency band. After the hearing test is completed, an audiogram of the hearing test is displayed, which includes hearing curves across the entire frequency band.

[0007] This application provides a hearing test processing device, which includes: The first presentation module is used to present a hearing test interface, which includes perceptible and non-perceptible controls. When the hearing test environment meets the environmental test requirements, it outputs a test tone corresponding to the initial sound pressure level. In response to a trigger operation on the perceptible control, it lowers the initial sound pressure level. In response to no trigger operation on the perceptible control or a trigger operation on the non-perceptible control, it raises the initial sound pressure level. When a trigger operation on the perceptible control is received for the second time at any sound pressure level, the current sound pressure level is used as the hearing value of the current test ear in the current frequency band, and the hearing test is performed in the next frequency band. After the hearing test is completed, the audiogram of the hearing test is displayed, which includes a hearing curve across the entire frequency band.

[0008] In some embodiments, the device further includes: The third presentation module is used to respond to the received appointment operation instruction and present the payment page, which displays the consultation amount and payment entry corresponding to the target doctor information; The fourth presentation module is used to present a payment method selection interface in response to a trigger operation on the payment portal; The second determining module is used to determine the target payment method in response to the selection operation triggered by the payment method selection interface, and to make payment based on the target payment method.

[0009] In some embodiments, the device further includes: The output module is used to output a prompt message when the time interval between the current time and the first appointment time is less than the interval threshold. The prompt message is used to indicate that the first appointment time for remote consultation is about to arrive. The fifth presentation module is used to present an audio / video call interface in response to a trigger operation on the entry link when the first appointment time is reached. The first acquisition module is used to activate the image acquisition device and the voice acquisition device to acquire the first image data acquired by the image acquisition device and the first voice data acquired by the voice acquisition device. The sixth presentation module is used to present the first image data and the second image data obtained from the doctor's client in the audio and video call interface; The second sending module is used to send the first voice data to the doctor's client and play the second voice data sent by the doctor's client.

[0010] In some embodiments, the device further includes: The seventh presentation module is used to establish a communication connection between the user terminal and the hearing aid device in response to the device connection operation when it is necessary to adjust the parameters of the hearing aid device, and to present the settings interface of the hearing aid device. The third determining module is used to determine the adjustment parameters and the target values ​​of the adjustment parameters based on the adjustment information obtained from the doctor's client. The first receiving module is used to receive adjustment operations on the adjustment parameters through the setting interface, so as to adjust the adjustment parameters to the target value.

[0011] In some embodiments, the device further includes: The second receiving module is used to receive remote control requests sent by the doctor's client; The eighth presentation module is used to present remote control prompt information in response to the remote control request; The third sending module is used to send a response message allowing remote control to the doctor's client when a positive feedback instruction for the remote control prompt is received, so that the doctor's terminal can adjust the parameters of the hearing aid device through remote control.

[0012] In some embodiments, the device further includes: The third receiving module is used to receive a first remote consultation response sent by the doctor's client. The first remote consultation response indicates that the doctor's client has not accepted the remote consultation request. The first remote consultation response carries a second appointment time. The ninth presentation module is used to respond to the viewing operation of the first remote consultation response, present the remote consultation response interface, and present the second appointment time through the remote consultation response interface; The fourth sending module is used to send a second remote consultation response to the doctor's client when a trigger operation to accept the second preset time is received through the remote consultation response interface.

[0013] In some embodiments, the device further includes: The fourth determination module is used to determine the selected target debugging personnel in response to the selection operation triggered by the doctor information display interface; The fifth sending module is used to send a device adjustment request to the device adjustment client in response to the received reservation operation instruction. The device adjustment request carries the first reservation time and the entry link for conducting real-time audio and video calls. The fourth receiving module is used to receive a second remote control request sent by the tuning client after the real-time audio and video call starts at the first scheduled time and the user terminal establishes a communication connection with the hearing aid device. The tenth presentation module is used to present a second remote control prompt message in response to the second remote control request; The sixth sending module is used to send a second response message allowing remote control to the tuning client when a positive feedback instruction for the second remote control prompt information is received, so that the tuning terminal can adjust the parameters of the hearing aid device remotely based on the adjustment instruction of the doctor terminal.

[0014] In some embodiments, the device further includes: The eleventh presentation module is used to present a near hearing aid interface in response to a trigger operation on the second function entry. The near hearing aid interface presents a hearing configuration entry, a first hearing aid switch control, and a second hearing aid switch control. The third acquisition module is used to acquire the user's hearing test results in response to a trigger operation on the hearing configuration entry. The noise reduction module is used to perform noise reduction processing on the collected voice data when the first auxiliary hearing function is determined to be turned on based on the trigger operation of the first auxiliary hearing switch, so as to obtain the noise-reduced voice data. An equalization processing module is used to perform equalization processing on the noise-reduced speech data based on the hearing test results when the second auxiliary hearing function is determined to be activated based on the trigger operation of the second auxiliary hearing switch, so as to obtain target speech data. The second output module is used to output the target speech data.

[0015] In some embodiments, the noise reduction module is further configured to: The collected speech data is converted to the frequency domain to obtain frequency domain speech data, and the phase information and amplitude information of the frequency domain speech data are obtained. Obtain the trained speech enhancement model, input the amplitude information into the trained speech enhancement model, and obtain the gain information; Based on the amplitude information and the gain information, determine the amplitude information after noise reduction; Based on the amplitude information and the phase information, the denoised frequency domain speech data is determined; The denoised frequency domain speech data is converted to the time domain to obtain the denoised speech data.

[0016] In some embodiments, the device further includes: The feature extraction module is used to extract the speech features of the collected speech data; The scene recognition module is used to input the speech features into the trained scene recognition model to obtain the recognized scene information; Correspondingly, this noise reduction module is also used for: Based on the scene information, obtain the trained speech enhancement model corresponding to the scene information.

[0017] In some embodiments, the equalization processing module is further configured to: The sound pressure level is determined based on the collected speech data, and the hearing threshold corresponding to each frequency band is determined based on the user's hearing test results. Based on the sound pressure level and the hearing threshold corresponding to each frequency band, the gain value corresponding to each frequency band is determined; The parameters of the equalization filter are determined based on the gain values ​​corresponding to each frequency band. The noise-reduced speech data is filtered using the equalization filter to obtain filtered speech data. The filtered speech data is subjected to dynamic range control to obtain the target language data.

[0018] This application provides a terminal device, including: Memory, used to store executable instructions; The processor, when executing executable instructions stored in the memory, implements the hearing assistance method provided in the embodiments of this application.

[0019] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the hearing assistance method provided in this application.

[0020] This application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the method described in any one of claims 1 to 11.

[0021] The embodiments of this application have the following beneficial effects: In response to a trigger operation on the hearing aid client, the user terminal presents a hearing aid interface. This interface includes at least a first function entry point for remote consultation and a second function entry point for near-field hearing aids. When a user wants to perform near-field hearing aids, they can trigger a click or touch operation on the first function entry point. In response to this click or touch operation, a doctor information display interface is presented. Through this doctor information display interface, the user can select the target doctor's information and the first appointment time. After receiving the appointment operation instruction, the server sends a remote consultation request to the doctor's client. This remote consultation request includes the first appointment time and an entry link for conducting real-time audio and video calls. This allows the doctor's client and the user's hearing aid client to conduct real-time audio and video calls through the entry link when the first appointment time is reached, thereby realizing remote consultation. This not only reduces the time and transportation costs for users when seeking medical advice but also improves consultation efficiency. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the architecture of the hearing assistance system provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application; Figure 3 A schematic diagram illustrating an implementation flow of the hearing assistance method provided in this application embodiment; Figure 4 A schematic diagram illustrating another implementation flow of the hearing assistance method provided in the embodiments of this application; Figure 5 A schematic diagram illustrating another implementation of the hearing assistance method provided in the embodiments of this application; Figure 6A A schematic diagram of the interface of the near-field hearing aid and remote rehabilitation App provided in the embodiments of this application; Figure 6B Another schematic diagram of the interface of the near-field hearing aid and remote rehabilitation App provided in the embodiments of this application; Figure 7A This is a schematic diagram of the near-field hearing aid interface provided in an embodiment of this application; Figure 7B This is a schematic diagram of a remote rehabilitation interface provided in an embodiment of this application; Figure 7C This is a schematic diagram of a hearing test interface provided in an embodiment of this application; Figure 8A This is a schematic diagram illustrating the implementation process of a personalized hearing test provided in an embodiment of this application. Figure 8B This is a schematic diagram of pure-tone hearing test results provided in an embodiment of this application; Figure 9 This is a schematic diagram of the sound scene recognition process provided in an embodiment of this application; Figure 10 A schematic diagram illustrating another implementation process of sound scene recognition provided in this application embodiment; Figure 11 A schematic diagram illustrating the implementation process of the simplified rise method for audiometry provided in this application embodiment; Figure 12 A schematic diagram illustrating the implementation of AI voice enhancement provided in an embodiment of this application; Figure 13 A schematic diagram illustrating another implementation process for AI voice provided in the embodiments of this application; Figure 14 A schematic diagram illustrating the implementation process of personalized load balancing provided in this application embodiment; Figure 15A A schematic diagram of the frequency response curve of the filter bank obtained by directly calculating the filter parameters; Figure 15BA schematic diagram of the frequency response curve of the filter bank obtained by inverse calculation of filter parameters; Figure 16 This is a schematic diagram illustrating the implementation process of remote rehabilitation provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0025] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0027] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0028] 1) Sound Pressure Level (SPL) is a physical quantity used to describe the magnitude of sound pressure. The range of sound pressure levels audible to the human ear is 2 × 10⁻⁶. -5 Pa~20Pa corresponds to a sound pressure level range of 0~120dB. Introducing the concept of sound pressure level makes it easier to describe sound pressure that changes linearly.

[0029] 2) Pure-tone hearing threshold measurement (also known as pure-tone hearing test) is the lightest sound that the tested ear can just hear from pure tones of different frequencies, and it is the standard for judging hearing sensitivity. During the test, the user's left and right ears are tested separately; a pure tone of a specific frequency is selected, and the minimum sound pressure level that the user can perceive is measured to obtain the hearing status of the left and right ears (also referred to as personalized hearing status in this embodiment of the application).

[0030] 3) Speech Enhancement: This technique extracts useful speech signals from noisy backgrounds and suppresses or reduces noise interference when speech signals are interfered with or even submerged by various noises. In other words, it extracts the purest possible original speech from noisy speech.

[0031] To better understand the hearing test processing method provided in the embodiments of this application, the technical needs of hearing-impaired users and the current status and existing defects of related technologies are explained.

[0032] The technical needs of hearing-impaired users can be summarized as follows: First, users expect to conduct self-tests at any time to understand their current hearing status. Furthermore, for ordinary users (e.g., those with healthy hearing or those not yet clinically diagnosed with hearing loss), the ability to check their hearing status anytime using a lightweight app is also beneficial to their hearing health.

[0033] Second, targeted hearing aids will be provided based on the user's individual hearing condition, including: ① Hearing aids targeting the frequencies where hearing loss occurs are more effective.

[0034] ② Hearing-impaired users experience an extreme decrease in speech intelligibility when exposed to noise. Speech enhancement is necessary.

[0035] ③ Provide targeted hearing aids for users in different acoustic scenarios (e.g., the intensity or strategy of voice enhancement).

[0036] Third, when face-to-face communication is inconvenient, users can contact doctors remotely to complete basic remote rehabilitation and device adjustments.

[0037] Currently, pure-tone audiometry-based hearing aids are a typical example in the industry. The implementation first involves performing a pure-tone audiometry test on the user to obtain a personalized hearing status. Then, based on this personalized hearing status, specific frequencies with hearing loss are amplified (in some implementations, this is also called personalized equalization) to improve the user's hearing experience.

[0038] In addition, environmental noise can affect the intelligibility of hearing-impaired users. Therefore, some hearing aids or cochlear implant products integrate noise reduction algorithms; at the same time, some hearing aid software also integrates corresponding noise reduction algorithms.

[0039] In terms of assistive hearing, personalized equalization based on pure-tone audiometry is relatively easy to implement. However, for noise cancellation, current solutions for hearing-impaired users still have the following shortcomings: First, schemes based on pure signal processing are generally ineffective at suppressing transient and other non-stationary noise. Second, some noise reduction algorithms damage the speech structure, making it more difficult for users to perceive; Third, due to limitations in computing power, the solutions integrated into hearing aids or cochlear implants are all lightweight noise reduction algorithms, and their effectiveness is generally limited.

[0040] In terms of remote rehabilitation and device adjustment, the current common solution is to use third-party audio and video calling software to enable real-time communication between doctors and users. These software programs are designed for general scenarios and are not integrated into dedicated hearing aid apps, so some processes do not meet the business needs of remote rehabilitation.

[0041] Therefore, in the hearing test processing method of this application embodiment, integrating near-field hearing aids and remote rehabilitation into one application is beneficial to improving the user's auditory experience.

[0042] The following describes exemplary applications of the terminal devices provided in the embodiments of this application. The terminal devices provided in the embodiments of this application can be implemented as various types of user terminals such as laptops, tablets, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), vehicle terminals, and smart TVs.

[0043] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the hearing assistance system 100 provided in the embodiments of this application, as shown below. Figure 1 As shown, the hearing assistance system 100 includes: a doctor terminal 200, a server 300, and a user terminal 400. The doctor terminal 200 and the user terminal 400 can communicate via a network (…). Figure 1 (Not shown in the image) Establish a communication connection. The network can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0044] The user terminal 400 may have a hearing aid client installed. This hearing aid client may be divided into a user version and a doctor version. The doctor terminal 200 has a hearing aid doctor version client installed (hereinafter referred to as the doctor client).

[0045] The user terminal 400 can be used by users with hearing impairments who wear hearing aids or cochlear implants. To provide users with a better auditory experience, a hearing assistance client can be activated. In response to a trigger operation on the hearing assistance client, the user terminal 400 displays a hearing assistance interface. This interface includes at least a first function entry for remote consultation and a second function entry for near-field hearing assistance. When a user wants to perform near-field hearing assistance, they can trigger a click or touch operation on the first function entry. In response to this click or touch operation, a doctor information display interface is displayed. Through this interface, the user can select a target doctor and a first appointment time. After receiving the appointment operation instruction, a remote consultation request is sent to the doctor's client via the server. Upon receiving the remote consultation request, the server 300 allocates an entry link for real-time audio and video calls. The server 300 then sends the remote consultation request to the doctor's client, and the remote consultation request carries the first appointment time and the entry link for real-time audio and video calls.

[0046] In some embodiments, after receiving a remote consultation request, if the doctor terminal 200 accepts the request, it returns a response message accepting the remote consultation to the user terminal via the server 300. Upon reaching a first preset time, the doctor and user respectively use their hearing aids to click the entry link to access a video call interface, thereby conducting a remote consultation.

[0047] In some embodiments, server 300 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. User terminal 400 and doctor terminal 200 may be smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, smart vehicle terminals, etc., but are not limited to these. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0048] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the user terminal 400 provided in the embodiments of this application. Figure 2The user terminal 400 shown includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the user terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0049] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0050] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0051] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0052] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0053] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0054] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 452 is used to reach other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc. The input processing module 453 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0055] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A hearing aid 454 stored in memory 450 is shown. This can be software in the form of programs and plug-ins, and includes the following software modules: a first presentation module 4541, a second presentation module 4542, a first determination module 4543, and a first transmission module 4544. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0056] The first presentation module 4541, the second presentation module 4542 and other presentation modules are used to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with the user interface 430 (e.g., a display screen, a speaker, etc.).

[0057] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the hearing assistance method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0058] In some embodiments, the terminal or server can implement the permission granting method provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as a hearing aid APP or an instant messaging APP; it can also be a mini-program, that is, a program that only needs to be downloaded into a browser environment to run; or it can be a mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plugin.

[0059] To better understand the methods provided in the embodiments of this application, we will first explain artificial intelligence, its various branches, and the application fields involved in the methods provided in the embodiments of this application.

[0060] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0061] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning. The solutions provided in this application mainly relate to machine learning techniques in artificial intelligence, which will be described below.

[0062] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning.

[0063] Artificial intelligence cloud services, also commonly known as AI as a Service (AIaaS), are a mainstream service model for artificial intelligence platforms. Specifically, AIaaS platforms break down several common AI services and provide them independently or as bundled services in the cloud. This service model is similar to opening an AI-themed marketplace: all developers can access and use one or more AI services provided by the platform through API interfaces. Some experienced developers can also use the AI ​​framework and AI infrastructure provided by the platform to deploy and maintain their own dedicated cloud AI services. In the hearing assistance method provided in this application embodiment, the data scheduling and distribution system can be deployed and maintained through the AI ​​framework and AI infrastructure provided by the artificial intelligence cloud service.

[0064] The hearing assistance method provided in this application will be described in conjunction with exemplary applications and implementations of the terminal provided in the embodiments of this application.

[0065] This application provides a hearing assistance method applied to a user terminal. Figure 3 This is a schematic diagram illustrating an implementation flow of the hearing assistance method provided in this application embodiment, which will be combined with... Figure 3 The steps shown illustrate the hearing assistance method provided in the embodiments of this application.

[0066] Step S101: In response to a trigger operation on the hearing assistance client, present the hearing assistance interface.

[0067] The user terminal may have a hearing assistance client installed. The triggering operation for the hearing assistance client can be a click or touch operation on the hearing assistance client icon, or a voice command to open the hearing assistance client, or a gesture operation to open the hearing assistance client. After receiving the triggering operation, the user terminal displays a hearing assistance interface, which includes at least a first function entry for remote consultation and a second function entry for near-field hearing assistance.

[0068] Step S102: In response to the trigger operation for the first function entry, present the doctor information display interface.

[0069] The trigger operation for the first function entry can be a click or touch operation on the first function entry. In response to this trigger operation, a doctor information display interface is presented. This doctor information display interface can display information such as the doctor's name, the hospital where the doctor works, the doctor's title, and the doctor's areas of expertise. The doctor information display interface may also include a preset time selection entry. After clicking or touching this selection entry, a floating layer will appear, displaying the doctor's available appointment times. The available appointment times can include the appointment date and appointment time.

[0070] Step S103: In response to the selection operation triggered by the doctor information display interface, determine the selected target doctor information and the first appointment time.

[0071] In this embodiment, the doctor information display interface may further include selection controls for each doctor. Clicking or touching a doctor's selection control updates the doctor's selection status. For example, if a doctor's current selection status is unselected, receiving a click or touch operation on that selection control updates the selection status to selected. Conversely, if the doctor's current selection status is selected, receiving a click or touch operation on that selection control updates the selection status to unselected. After the doctor's updated selection status is selected, an appointment time selection interface is automatically triggered. This appointment time selection interface can be loaded as a floating layer on top of the doctor information display interface. Then, the user terminal determines a first appointment time based on the selection operation received through the appointment time selection interface. This first appointment time includes a first appointment date and a first appointment time period.

[0072] Step S104: In response to the received appointment operation instruction, a remote consultation request is sent to the doctor's client.

[0073] In practice, the remote consultation request can be sent from the server to the doctor's client. First, the user terminal sends the remote consultation request to the server. The remote consultation request sent by the user terminal to the server may include doctor information and a first appointment time. After receiving the remote consultation request, if the server determines that the first appointment time corresponding to the doctor information is available, it generates an entry link for real-time audio and video calls. Then, the server sends the remote consultation request to the doctor's client (i.e., the doctor's terminal). The remote consultation request carries the first appointment time and the entry link for real-time audio and video calls.

[0074] In some embodiments, after the first appointment time is reached, the user terminal and the doctor terminal can conduct real-time audio and video calls through the entry link, thereby realizing remote consultation.

[0075] In the hearing assistance method provided in this application embodiment, the user terminal responds to a trigger operation on the hearing assistance client and presents a hearing assistance interface. The hearing assistance interface includes at least a first function entry for remote consultation and a second function entry for near-field hearing assistance. When the user wants to conduct a remote consultation, they can trigger a click or touch operation on the first function entry. In response to the click or touch operation, a doctor information display interface is presented. Through the doctor information display interface, the user can select the target doctor information and the first appointment time. After receiving the appointment operation instruction, the server sends a remote consultation request to the doctor's client. The remote consultation request includes the first appointment time and an entry link for conducting real-time audio and video calls, so that the doctor's client and the user's hearing assistance client can conduct real-time audio and video calls through the entry link when the first appointment time is reached, thereby realizing remote consultation. This not only reduces the time and transportation costs for users when conducting consultations, but also improves consultation efficiency.

[0076] In some embodiments, after receiving an appointment booking instruction, the following steps may also be performed for paid medical consultations: Step S105: In response to the received reservation operation instruction, the payment page is displayed.

[0077] The payment page displays the consultation amount and payment entry point corresponding to the target doctor's information.

[0078] Step S106: In response to the trigger operation for the payment entry, a payment method selection interface is presented.

[0079] The payment method selection interface can display at least one payment method, such as bank card payment or third-party e-commerce payment. Each payment method has a corresponding selection control. By clicking the selection control, you can select or deselect the corresponding payment method.

[0080] Step S107: In response to the selection operation triggered by the payment method selection interface, determine the target payment method and make payment based on the target payment method.

[0081] Since remote consultations differ from in-person medical visits, where registration and payment can be made on-site at the hospital, the online payment for the consultation fee can be completed after the user selects the doctor and appointment time through steps S105 to S107.

[0082] It should be noted that steps S105 to S107 can be performed before or after step S104.

[0083] Based on the foregoing embodiments, this application further provides a hearing assistance method, applied to... Figure 1The hearing aid system shown Figure 4 This is a schematic diagram illustrating another implementation flow of the hearing assistance method provided in the embodiments of this application, as shown below. Figure 4 As shown, the process includes: In step S201, the user terminal responds to the trigger operation for the hearing assistance client and presents the hearing assistance interface.

[0084] The hearing aid interface includes at least a first function entry for remote inquiry and a second function entry for near-field hearing aids. In some embodiments, the hearing aid interface also includes a third function entry for conducting hearing tests.

[0085] In step S202, the user terminal responds to the trigger operation for the first function entry and presents the doctor information display interface.

[0086] In step S203, the user terminal responds to the selection operation triggered by the doctor information display interface and determines the selected target doctor information and the first appointment time.

[0087] In step S204, the user terminal responds to the received appointment operation instruction by sending a remote consultation request to the server.

[0088] The remote consultation request includes the target doctor's information and the first appointment time.

[0089] Step S205: After receiving the remote consultation request, the server obtains the entry link allocated to the user terminal for real-time audio and video calls.

[0090] In implementation, if the server integrates a real-time audio and video call SDK or underlying library, the server itself can allocate an entry link for real-time audio and video calls to the user terminal. If the server does not integrate a real-time audio and video call SDK or underlying library, the server will send a link allocation request to a third-party audio and video call server, which will then allocate a link to the user terminal and send it to the server.

[0091] Step S206: The server sends a remote consultation request to the doctor's client.

[0092] The remote consultation request includes the entry link and the first appointment time.

[0093] In step S207, the server returns the entry link to the user terminal.

[0094] After receiving the entry link, the user terminal can add a consultation record, which may include, but is not limited to, the target doctor's information, the first appointment time, and the entry link.

[0095] In step S208, after receiving a remote consultation request, the doctor's terminal responds to the viewing operation for the remote consultation request by displaying the remote consultation request interface.

[0096] The remote file interface displays at least the first scheduled time, and in some embodiments, it may also display an entry link.

[0097] In step S209, when the doctor's terminal receives the operation of accepting the remote consultation request through the remote consultation request interface, it sends the third remote consultation response to the user terminal via the server.

[0098] The third remote consultation response is used to notify the user terminal that the doctor has accepted the appointment. In some embodiments, the third remote consultation response may simply be sent to the server, which then updates the available appointment time information.

[0099] In step S210, the server updates the doctor's available appointment time information based on the third remote consultation response.

[0100] When implementing step S210, the server may delete the first appointment time information from the appointment time information corresponding to the target doctor information based on the third remote consultation response.

[0101] Step S211: When it is determined that the time interval between the current time and the first appointment time is less than the interval threshold, the user terminal and the doctor terminal output a prompt message.

[0102] This notification message indicates that the first appointment time for a remote consultation is approaching. This time threshold can be preset, such as half an hour, one hour, or 10 minutes.

[0103] Determining whether the time interval between the current time and the first appointment time is less than the interval threshold can be implemented by setting a timer. The timing of this timer is determined based on the first appointment time and the time threshold. For example, if the first appointment time is 17:00 on September 10, 2021, and the time threshold is half an hour, then the timing time is 16:31 on September 10, 2021. When this timing time is reached, the doctor's terminal and the user terminal output a prompt message.

[0104] In step S212, when the first appointment time is reached, the doctor's terminal and the user terminal respond to the trigger operation for the entry link and present an audio and video call interface.

[0105] The trigger action for the entry link can be clicking the entry link. In response to this action, an audio and video call interface will be displayed on both the user terminal and the doctor terminal.

[0106] Step S213: The user terminal starts the image acquisition device and the voice acquisition device to acquire the first image data acquired by the image acquisition device and the first voice data acquired by the voice acquisition device.

[0107] The image acquisition device can be a camera, with the front-facing camera being the default, and the voice acquisition device can be a microphone.

[0108] Step S214: The doctor's terminal activates the image acquisition device and the voice acquisition device to acquire the second image data acquired by the image acquisition device and the second voice data acquired by the voice acquisition device.

[0109] In step S215, the user terminal sends the first image data and the first voice data to the doctor's terminal.

[0110] In implementation, the user terminal first sends the first image data and the first voice data to the server, and then the server sends the first image data and the first voice data to the doctor's terminal.

[0111] In step S216, the doctor's terminal sends the second image data and the second voice data to the user terminal.

[0112] In practice, the doctor's terminal first sends the second image data and the second voice data to the server, and then the server sends the second image data and the second voice data to the user terminal.

[0113] In step S217, the user terminal presents the first image data and the second image data on the audio and video call interface, and plays the second voice data.

[0114] In step S218, the doctor's terminal presents the first image data and the second image data on the audio and video call interface, and plays the first voice data.

[0115] Through steps S213 to S218, remote consultations between doctors and patients can be achieved. For example, doctors can inquire about the patient's condition and discomfort, and can also provide real-time speech rehabilitation. Doctors can read aloud relevant words, such as "clothes," and users can imitate the lip movements and sounds synchronized with the audio-visual presentation on the screen.

[0116] Step S219: When it is necessary to adjust the parameters of the hearing aid device, the user terminal responds to the device connection operation, establishes a communication connection between the user terminal and the hearing aid device, and presents the settings interface of the hearing aid device.

[0117] In practice, the device connection operation can be either inserting the hearing aid into the external device port of the user terminal via a wired connection, or establishing a communication connection between the hearing aid and the user terminal via a wireless connection.

[0118] Once a communication connection is established between the user terminal and the hearing aid, the hearing aid's settings interface will be displayed. This settings interface can show the configurable parameters and their current values.

[0119] In step S220, the user terminal determines the adjustment parameters and the target values ​​of the adjustment parameters based on the adjustment information obtained from the doctor's client.

[0120] Here, the user terminal can obtain adjustment information based on the voice sent by the doctor during a real-time audio and video call with the doctor's terminal, thereby determining the adjustment parameters and the target values ​​of the adjustment parameters.

[0121] In step S221, the user terminal receives an adjustment operation on the adjustment parameter through the settings interface to adjust the adjustment parameter to the target value.

[0122] In practice, the adjustment operation can be triggered by the user on the user terminal. In some embodiments, if the user feels that it is difficult to adjust the device on their own, they can have an audio-visual call with a doctor together with a professional tuner. The tuner can then trigger the adjustment operation based on the obtained adjustment information, thereby adjusting the adjustment parameters to the target value.

[0123] In the hearing assistance method provided in this application embodiment, the user terminal, in response to a trigger operation on the hearing assistance client, presents a hearing assistance interface. This interface includes at least a first function entry point for remote inquiry and a second function entry point for near-field hearing assistance. When the user wants to conduct a remote consultation, they can trigger a click or touch operation on the first function entry point. In response to this click or touch operation, a doctor information display interface is presented. Through this interface, the user can select target doctor information and a first appointment time. After receiving the appointment operation instruction, the server sends a remote consultation request to the doctor's client. This request includes the first appointment time and an entry link for conducting real-time audio and video calls. After receiving the remote consultation request, the doctor's terminal... If the remote consultation request is accepted, a third remote consultation response can be returned to the server, allowing the server to update the doctor's available appointment time information. When the first appointment time is reached, the doctor's client and the user terminal can conduct a real-time audio and video call through the entry link, thereby realizing remote consultation. This not only reduces the time and transportation costs for users when seeking medical advice but also improves consultation efficiency. In addition, if the consultation determines that the user's hearing aid needs to be adjusted, the adjustment information provided by the doctor can be obtained through real-time audio and video calls. This allows the doctor to determine the adjustment parameters and their target values. Based on this adjustment information, the adjustment operation is triggered, ultimately adjusting the parameters to the target values. This improves the fit of the hearing aid to the user and enhances the user's auditory experience.

[0124] In some embodiments, when it is necessary to adjust the parameters of the hearing aid device, in addition to the steps S220 and S221 described above, the following two implementation methods are also possible: First, doctors can adjust the parameters of the hearing aid device remotely via a user terminal.

[0125] Second, users can remotely connect with professional technicians who will make adjustments based on the parameters provided by the doctor.

[0126] The first method can be achieved through the following steps: Step S231: The user terminal receives the remote control request sent by the doctor's client.

[0127] The remote control request is used to request remote control of the user terminal. In this embodiment of the application, in order to ensure the user's information security, the doctor's terminal can only remotely control the user terminal within the user's hearing aid client and cannot perform other operations.

[0128] In step S232, the user terminal responds to the remote control request and presents the first remote control prompt information.

[0129] The first remote control prompt message is used to remind the user doctor that they will be remotely controlling their terminal. When the first remote control prompt message is presented, there may be a "Accept remote control" selection control and a "Deny remote control" selection control.

[0130] Step S233: When the user terminal receives a positive feedback instruction for the first remote control prompt information, it sends a response message allowing remote control to the doctor's client, so that the doctor's terminal can adjust the parameters of the hearing aid device through remote control.

[0131] If the user terminal receives a click or touch operation for "accept remote control", or receives a voice command to accept remote control, it is determined that a positive feedback instruction for the first remote control prompt information has been received, and a response message allowing remote control is sent to the doctor's client.

[0132] In step S234, the doctor's terminal displays the settings interface of the hearing aid and adjusts the parameters of the hearing aid according to the settings operation received through the settings interface.

[0133] After obtaining remote control permissions to the user's terminal, the doctor's terminal will display the hearing aid's settings interface. Through this settings interface, the doctor can adjust the parameters of the hearing aid to bring the required parameters to the target values.

[0134] Through the above steps S231 to S234, when the user's hearing aid needs parameter adjustment or debugging, the doctor's terminal can send a remote control request. After the user's terminal agrees to remote control, the doctor can debug the hearing aid through remote control, thereby ensuring the accuracy of the device debugging and saving the user's time and transportation costs, thus improving the debugging efficiency.

[0135] The second implementation method can be achieved through the following steps: In step S241, the user terminal responds to the selection operation triggered by the doctor information display interface and determines the selected target debugging personnel.

[0136] In this embodiment of the application, when a user requests a remote consultation, if they need to adjust their hearing aid device, they can not only select a doctor through the doctor information display interface, but also select a person to adjust the hearing aid device so that the selected person can adjust the hearing aid device.

[0137] In step S242, the user terminal responds to the received reservation operation instruction by sending a device adjustment request to the device adjustment client.

[0138] The device setup request carries the first scheduled time and an entry link for real-time audio and video calls. In implementation, the setup request is sent from the server to the setup client. In some embodiments, after receiving the setup request, the server obtains the entry link assigned to the user and sends the setup request carrying the first scheduled time and entry link to the setup client, i.e., the setup terminal.

[0139] Step S243: After the real-time audio and video call starts at the first scheduled time and the user terminal establishes a communication connection with the hearing aid, the user terminal receives a second remote control request sent by the tuning client.

[0140] In some embodiments, when a user is having an audio or video call with a doctor or equipment technician and the call progresses to the equipment debugging stage, the equipment debugging client can initiate a second remote control request to the user terminal. In some embodiments, the second remote control request can also be initiated by the user terminal.

[0141] In step S244, the user terminal responds to the second remote control request and presents the second remote control prompt information.

[0142] The second remote control prompt message is used to remind the user that the operator will remotely control their terminal. Similar to the first remote control prompt message, when the second remote control prompt message is presented, there may be a "Accept Remote Control" selection control and a "Reject Remote Control" selection control.

[0143] Step S245: When a positive feedback instruction for the second remote control prompt information is received, a second response message allowing remote control is sent to the tuning client, so that the tuning terminal can adjust the parameters of the hearing aid device remotely based on the adjustment instruction from the doctor's terminal.

[0144] If the user terminal receives a click or touch operation for "accept remote control" or receives a voice command to accept remote control, it is determined that a positive feedback instruction for the second remote control prompt information has been received, and a response message allowing remote control is sent to the client.

[0145] In step S246, the adjustment terminal displays the setting interface of the hearing aid and adjusts the parameters of the hearing aid according to the setting operation received through the setting interface.

[0146] After obtaining remote control permissions for the user's terminal, the adjustment terminal will display the hearing aid's settings interface. The adjustment personnel can use this interface to adjust the parameters of the hearing aid according to the doctor's instructions, thereby adjusting the parameters to the target values.

[0147] In some embodiments, after the doctor's terminal presents the remote consultation request interface in step S208, if the doctor is unable to see the patient at the first appointment time due to personal reasons, the doctor can refuse the remote consultation request and update the appointment time through the following steps. The following describes this scenario in conjunction with each step.

[0148] In step S301, when the doctor's terminal receives the operation of rejecting the remote consultation request, the appointment time modification interface is displayed.

[0149] In this step, if a doctor is unable to provide consultation services to the user requesting remote consultation at the first scheduled time due to their own reasons, they can refuse the remote consultation request. Upon receiving the operation of refusing the remote consultation request, an appointment time modification interface will be displayed. This voice time modification interface can display other available appointment times for the doctor. In some embodiments, doctors can also input available appointment times themselves.

[0150] In step S302, the doctor's terminal determines the second appointment time based on the modification operation received through the appointment time modification interface.

[0151] This modification operation can be a selection of multiple available appointment times, or an input of appointment times.

[0152] In step S303, the doctor's terminal sends the first consultation response to the user terminal through the server.

[0153] The first remote consultation response indicates that the doctor's client has not accepted the remote consultation request, and the first remote consultation response carries a second appointment time.

[0154] In step S304, the user terminal responds to the viewing operation of the first remote consultation response by displaying the remote consultation response interface and presenting the second appointment time through the remote consultation response interface.

[0155] The remote consultation response interface can display a second appointment time, as well as a "Accept Time Modification" confirmation control and a "Do Not Accept Time Modification" cancellation control. When the user confirms that they can have the consultation at the second appointment time, they can click the confirmation control to accept the second appointment time. If the user cannot have the consultation at the second appointment time, they can click the cancellation control to refuse the second appointment time.

[0156] Step S305: When the user terminal receives the trigger operation to accept the second preset time through the remote consultation response interface, the server sends the second remote consultation response to the doctor client.

[0157] The second remote consultation response is used to indicate that the user accepts the second appointment time. After step S305, step S210 and subsequent steps can be executed. In actual implementation, the first appointment time in step S210 and subsequent steps can be modified to the second appointment time.

[0158] Through steps S301 to S305 described above, when a doctor receives a remote consultation request from a user, if the doctor needs to modify the first appointment time selected by the user for their own reasons, they can reselect from other available appointment times. The doctor can also input a suitable time themselves. After the doctor determines the second appointment time, the server sends a first remote consultation response to the user terminal to notify the user that the doctor has modified the appointment time. When the user terminal receives the first remote consultation response and accepts the doctor's modified second appointment time, it can return a second remote consultation response to the doctor's terminal to notify the doctor that the user has accepted the second appointment time. In the subsequent process, the doctor and the user conduct remote consultation and equipment debugging through a real-time audio and video call entry link at the second preset time, thereby improving the flexibility and freedom of remote consultation time.

[0159] Based on the above embodiments, this application further provides a hearing assistance method, applied to a user terminal and a server. Figure 5 This is a schematic diagram illustrating another implementation flow of a hearing assistance method provided in an embodiment of this application, as shown below. Figure 5 As shown, the process includes: In step S401, the user terminal responds to the trigger operation for the hearing assistance client and presents the hearing assistance interface.

[0160] The hearing aid interface includes at least a first function entry for remote consultation and a second function entry for near-field hearing aids. In some embodiments, the hearing aid interface also includes a third function entry for conducting hearing tests.

[0161] In step S402, the user terminal responds to the trigger operation for the second function entry and presents the near-field hearing aid interface.

[0162] The near-field hearing aid interface includes a hearing configuration entry, a first hearing aid switch, and a second hearing aid switch. The hearing configuration entry allows importing hearing test results; the first hearing aid switch can be used to turn the first hearing aid function on or off. The first hearing aid function can be a noise reduction function; the second hearing aid switch can be used to turn the second hearing aid function on or off, and the second hearing aid function can be a personalized assistance function.

[0163] In step S403, the user terminal responds to the trigger operation for the hearing configuration entry and obtains the user's hearing test results.

[0164] The hearing test results can be obtained through this hearing aid client, or they can be obtained from hearing tests conducted by the user at a professional medical institution. These results can include the user's hearing thresholds at multiple preset frequency points.

[0165] In step S404, when the user terminal determines to enable the first auxiliary hearing function based on the trigger operation of the first auxiliary hearing switch, it performs noise reduction processing on the collected voice data to obtain noise-reduced voice data.

[0166] In this embodiment, a trained speech enhancement module can be used to perform noise reduction processing on the collected speech data to obtain noise-reduced speech data, thereby achieving the effect of speech enhancement.

[0167] The trained speech enhancement model can be a general neural network model. In some embodiments, the trained speech enhancement model can also be determined based on the actual scene in which the user is currently located, thereby ensuring the adaptability of the trained speech enhancement model to the user environment and scene, and improving the robustness of speech enhancement.

[0168] Step S405: When it is determined that the second hearing aid function is turned on based on the trigger operation of the second hearing aid switch, the noise-reduced speech data is equalized based on the hearing test results to obtain the target speech data.

[0169] In implementing step S405, the sound pressure level can first be determined based on the collected speech data, and the hearing threshold corresponding to each frequency band can be determined based on the user's hearing test results. Then, the gain of each frequency band can be determined using the sound pressure level and the hearing threshold corresponding to each frequency band, and the parameters of the equalization filter can be determined using the gain of each frequency band. Finally, the equalization filter can be used to perform personalized equalization on the noise-reduced speech data to obtain the target speech data.

[0170] Since the gain information is determined based on the sound pressure level of the currently acquired speech and the user's personalized hearing test results, the equalization filter parameters determined using this gain correspond to the frequency band where the user has hearing loss. This can effectively compensate for the user's hearing loss and ensure the auditory experience of the target language data.

[0171] Step S406: Output the target speech data.

[0172] The user terminal can output the target speech data either through its own speech output device or by sending the target speech data to a hearing aid that has established a communication connection with the user terminal. This hearing aid can be worn on the user's ear.

[0173] In the hearing assistance method provided in this application embodiment, when the user terminal receives a trigger operation for the hearing assistance client, a hearing assistance interface is presented in response to the trigger operation. When a trigger operation for the second function entry is received through the hearing assistance interface, a near-field hearing assistance interface is presented. The user terminal can skip the near-field hearing assistance interface to import hearing test results, turn on the first hearing assistance switch and the second hearing assistance switch, so that the user terminal can perform noise reduction processing and personalized equalization processing on the collected speech data. This can not only effectively improve speech intelligibility, but also effectively suppress environmental noise and improve the auditory experience.

[0174] In some embodiments, the step S404 above, "performing noise reduction on the acquired speech data to obtain noise-reduced speech data", can be accomplished through the following steps S4041 to S404: Step S4041: Perform frequency domain conversion on the collected speech data to obtain frequency domain speech data, and obtain the phase information and amplitude information of the frequency domain speech data.

[0175] In this step, the acquired speech data is first sampled to obtain multiple sampling points, and then framed. For example, sampling can be performed at a sampling frequency of 10000 Hz, and frames can be divided into 20 ms, resulting in 200 sampling points per frame. After frame processing, a time-frequency conversion can be performed using Fourier transform to convert the time-domain speech data to the frequency domain, thus obtaining frequency-domain speech data. For example, a fast Fourier transform or a short-time Fourier transform can be used to perform time-frequency conversion on each time-domain frame to obtain the corresponding frequency-domain speech data for each frame.

[0176] The phase and amplitude information of the frequency domain speech data are obtained. In practice, the phase and amplitude information of each frame are obtained separately.

[0177] Step S4042: Obtain the trained speech enhancement model, input the amplitude information into the trained speech enhancement model, and obtain the gain information.

[0178] The trained speech enhancement model can be a neural network model applicable to various scenarios, or it can be a personalized neural network model matched based on the recognized scenario information.

[0179] In implementation, the amplitude information of each frame can be input into a trained speech enhancement model to obtain the gain information of each frame. In some embodiments, considering the correlation of speech, the amplitude information of historical frames and the amplitude information of the current frame can be concatenated into a long amplitude information to obtain better prediction results. For example, when calculating the gain of the nth frame, the amplitude information of the (n-2)th, (n-1)th, and nth frames can be concatenated, and the concatenated vector can be input into the trained speech enhancement model to obtain the gain of the nth frame.

[0180] Step S4043: Determine the noise-reduced amplitude information based on the amplitude information and the gain information.

[0181] Since both amplitude information and gain information are vectors with the same dimension, this step can be implemented by multiplying the amplitude information and the corresponding gain information of each frame to obtain the amplitude information after noise reduction.

[0182] Step S4044: Based on the amplitude information and phase information after noise reduction, determine the frequency domain speech data after noise reduction.

[0183] Once the amplitude and phase information after noise reduction are determined, the frequency domain speech data after noise reduction can be determined.

[0184] Step S4045: Perform time-domain conversion on the noise-reduced frequency-domain speech data to obtain noise-reduced speech data.

[0185] In this step, the denoised frequency domain speech data is converted using inverse Fourier transform to obtain the denoised speech data.

[0186] In steps S4041 to S4045 above, the trained speech enhancement model is used to perform noise reduction processing on the collected speech data. Compared with traditional noise reduction algorithms, it has better computational efficiency and better noise reduction effect. After AI speech enhancement, environmental noise is filtered out, and the speech signal is heard cleaner. Therefore, users can hear clearer speech in noisy environments.

[0187] In some embodiments, before step S4042, scene recognition can be performed based on the collected voice data through the following steps S501 to S502, which will be described in conjunction with each step.

[0188] Step S501: Extract the speech features of the collected speech data.

[0189] In this step, the acquired speech data can first be sampled and framed to obtain multiple time-domain frame signals. Then, Fourier transform is performed on each time-domain frame signal to obtain frequency domain information. Subsequently, the Mel-frequency cepstral coefficients (MFCC) characteristics of the speech signal can be calculated based on the frequency domain signal S(ω). The MFCC calculation process includes Mel filter bank filtering, power spectrum calculation, and discrete Fourier transform.

[0190] Step S502: Input the speech features into the trained scene recognition model to obtain the recognized scene information.

[0191] The scene recognition model is a neural network model, such as a convolutional neural network model or a recurrent neural network model. This scene recognition model may include convolutional layers, pooling layers, long short-term memory (LSTM) layers, and fully connected layers. In this embodiment, the number of layers and the parameters of each layer in the speech enhancement model are not limited; however, considering computational resources, three convolutional layers, three pooling layers, one LSTM layer, and one fully connected layer are used.

[0192] In this embodiment, the identified scene information can be one of four categories: outdoor environment, office, restaurant, and home environment. Each scene category can correspond to different noise reduction strategies, such as different speech enhancement models. These different speech enhancement models can have the same model structure but different model coefficients. In some embodiments, inputting speech features into a trained scene recognition model can also yield environmental sound pressure results. These results are categorized into relatively quiet and relatively noisy, indicating whether the environment meets the requirements for audiometry.

[0193] Correspondingly, after determining the scene information through steps S501 to S502, the "obtaining the trained speech enhancement model" in step S402 can be implemented as follows: based on the scene information, obtain the trained speech enhancement model corresponding to the scene information. In other words, after recognizing the scene information, a speech enhancement model that better matches the scene can be obtained based on the recognized scene information, thereby improving the noise reduction effect.

[0194] In some embodiments, if the trained speech enhancement model is general-purpose, after recognizing scene information, the gain output by the speech enhancement model can be adjusted based on the recognized scene information. For example, the weight corresponding to the recognized scene information can be determined, and then the gain calculated by the speech enhancement model can be multiplied by this weight to obtain the final gain. If the recognized sound scene is an indoor scene with a relatively high signal-to-noise ratio, the weight corresponding to the scene information can be a number less than 1, such as 0.8. In this case, the gain output by the speech enhancement model can be multiplied by this weight to obtain the final gain, thereby making the speech enhancement processing strategy more moderate at high signal-to-noise ratios and avoiding speech clipping.

[0195] In some embodiments, the step S405 above, "performing equalization processing on the noise-reduced speech data based on the hearing test results to obtain target speech data," can be achieved through the following steps: Step S4051: Determine the sound pressure level based on the collected speech data, and determine the hearing threshold corresponding to each frequency band based on the user's hearing test results.

[0196] In implementing this step, the collected speech data also needs to be sampled and framed to obtain multiple frames of speech signals. Then, for the nth frame of speech signal, the nth frame and the (n-1)th frame can be spliced ​​together, and the spliced ​​speech data is then input into the sound pressure level calculation module to obtain the sound pressure level of the nth frame signal.

[0197] Step S4052: Based on the sound pressure level and the hearing threshold corresponding to each frequency band, determine the gain value corresponding to each frequency band.

[0198] When implementing step S4052, the gain range is first determined based on the sound pressure level. This range may include three gain ranges: the low-intensity sound gain range, the comfort-range sound gain range, and the high-intensity sound gain range. Each gain range corresponds to a sound pressure level range. When step S4051 falls into the sound pressure level range corresponding to a gain range, the gain range is determined.

[0199] In this embodiment, the calculation formula for determining the gain value differs for different gain intervals. The actual calculation method can be found in Table 1 of the following embodiments. After determining the gain interval, the gain value corresponding to each frequency band can be determined using the gain calculation formula corresponding to that gain interval and the hearing threshold corresponding to each frequency band.

[0200] To ensure that the equalized signal strength does not cause further hearing loss to the user, in this embodiment, when the sum of the input sound pressure level and the gain exceeds the pain threshold, the portion of the gain exceeding the pain threshold will be removed. For example, if the sound pressure level is 80dB, the gain is 15dB, and the pain threshold is 90dB, then 80+15>90, therefore, the gain will be reduced to 10dB.

[0201] Step S4053: Determine the parameters of the equalization filter based on the gain values ​​corresponding to each frequency band.

[0202] In this embodiment, the "reverse" filter parameter calculation method is adopted. First, the filter parameters corresponding to the high-frequency sub-band are determined, and then the gain of the low-frequency sub-band is calculated based on the frequency response characteristics after filtering, thus obtaining the filter parameters step by step.

[0203] Step S4054: The noise-reduced speech data is filtered using the equalization filter to obtain filtered speech data.

[0204] In implementation, the speech signal to be processed is passed through multiple filters from low frequency to high frequency in sequence to obtain filtered speech data.

[0205] Step S4055: Perform dynamic range control on the filtered speech data to obtain target language data.

[0206] Dynamic Range Control (DRC) maps the dynamic range of an input audio signal to a specified dynamic range. Typically, the mapped dynamic range is smaller than the original dynamic range, hence the term dynamic range compression. Audio signals can be dynamically range controlled as a whole, or divided into several sub-bands for individual dynamic range control.

[0207] Through steps S4051 to S4055, the gain of each frequency band can be determined by the ambient sound pressure level and the hearing threshold corresponding to each frequency band. Then, the parameters of the equalization filter are determined based on the gain values ​​corresponding to each frequency band. The equalization filter is then used to filter the input speech data. In this embodiment, the parameters of the equalization filter are calculated in reverse order from the high-frequency sub-band to the low-frequency sub-band, which can more closely approximate the desired frequency response curve and improve the accuracy of the filter parameters. In order to prevent the output signal from "clipping" and affecting the listening experience, dynamic range control is performed on the filtered speech data in this step to ensure the integrity of the audio signal.

[0208] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0209] The hearing data processing method provided in this application integrates near-field hearing aids and remote rehabilitation solutions. Remote rehabilitation is defined as enabling real-time two-way communication between doctors and users through a remote audio-visual call. Rehabilitation includes doctors guiding users in adjusting their devices (e.g., adjusting internal parameters based on the user's latest hearing status to improve hearing aid effectiveness), and doctors providing related hearing rehabilitation and speech guidance to users. In this application embodiment, the above or similar functions are collectively referred to as "remote rehabilitation."

[0210] The hearing test processing method provided in this application embodiment can be applied to at least two scenarios: near-field hearing aids and remote rehabilitation. In practical applications, near-field hearing aid scenarios may include: 1. In indoor or outdoor scenarios, users hold their phones and face the speaker opposite them; turning on the near-field hearing function in the app allows for better face-to-face communication.

[0211] Second, users can place their mobile phones near the speakers of a TV or radio, turn on near-field assistance, and connect via Bluetooth or wired connection to better listen to and watch news.

[0212] Application scenarios for remote rehabilitation may include: I. Speech Rehabilitation. Doctors can read aloud relevant words, such as "clothes"; users can then imitate the synchronized lip movements and sounds displayed on the screen. Through a remote rehabilitation platform, an effect similar to face-to-face teaching can be established.

[0213] II. Remote Adjustment. Adjustment involves connecting the hearing aid or cochlear implant to a computer and using specialized software (adjustment software) provided by the manufacturer to adjust the internal parameters of the hearing aid or cochlear implant. Before adjustment, a doctor needs to confirm the user's current hearing status and determine the adjustment strategy. Generally, this highly specialized operation cannot be performed by the user alone; the current solution requires a specialist to be present to assist with the adjustment.

[0214] This application provides three remote device adjustment solutions: Option 1: The technician is near the user. Using the remote audio / video call system provided in this embodiment, the doctor remotely informs the technician of the desired setup configuration. The technician then completes the setup using the aforementioned software.

[0215] Option 2: Both the doctor and the technician are connected to the remote audio and video call system. The doctor informs the technician of the setup configuration through the audio and video call system. The technician accesses the user's computer through the remote desktop control module built into the audio and video call system and directly operates the aforementioned setup software to complete the setup.

[0216] Option 3: No technician assistance required. Using the remote audio / video call system provided in this application embodiment, the doctor accesses the user's computer via the remote desktop control module built into the system. The doctor directly and remotely operates the aforementioned tuning software to complete the tuning process.

[0217] Figure 6A This is a schematic diagram of the interface of the near-field hearing aid and remote rehabilitation App provided in the embodiments of this application, as shown below. Figure 6A As shown, the App homepage contains at least two function buttons: "Near-field hearing aid" (601) and "Remote rehabilitation" (602). Figure 6B This is another schematic diagram of the interface of the near-field hearing aid and remote rehabilitation App provided in the embodiments of this application, such as... Figure 6B As shown, a separate "Personalized Hearing Test" button (603) has been added to facilitate users' frequent use of the hearing test function and to obtain the latest hearing status.

[0218] If the user clicks Figure 6A or Figure 6B After clicking the "Near-field Hearing Aid" 601 function button, the system will automatically load the latest personalized hearing status, or the user can select any personalized hearing status (such as accurate measurement results obtained from a professional medical institution). At this time, it can display as follows: Figure 7A The interface shown. In Figure 7AThe device includes a hearing configuration button 701, a first auxiliary hearing switch button 702, and a second auxiliary hearing switch button 703. The hearing configuration button allows selection of a pure-tone audiogram, which can be obtained through the "Personalized Audiometry" function in the app or through precise measurement results obtained and uploaded by the user from a professional medical institution. The first auxiliary hearing switch corresponds to AI noise reduction on / off; the second auxiliary hearing switch indicates personalized equalization on / off.

[0219] If the user finds the environment too noisy, they can click the function button on the first auxiliary hearing switch 702 to activate the noise reduction function. When the user activates the second auxiliary hearing switch 703, the background system calculates and obtains the current personalized equalization parameters to complete the second auxiliary hearing function.

[0220] After activating the primary hearing aid function, the user can select two or more noise reduction modes. These modes include, but are not limited to, noise reduction strength and whether it's outdoor or indoor noise reduction. Based on the selected noise reduction mode, the background system invokes different noise reduction strategies. In this embodiment, a pre-installed sound scene recognition module can also be used to detect the current environment or ambient noise level; the determination result is then transmitted to the noise reduction module to configure the parameters of the noise reduction algorithm.

[0221] If the user clicks Figure 6A or Figure 6B After clicking the "Remote Rehabilitation" function button (602), you can proceed through five steps: selecting an expert, selecting the expert's available service time, selecting payment, final confirmation, and receiving a receipt. Figure 7B This is a schematic diagram of the interface for selecting a specialist. The interface can display information on multiple doctors, allowing users to select the doctor they wish to consult.

[0222] In some embodiments, after selecting a doctor and completing payment, the user will be assigned a scheduled meeting number through the audio and video conferencing system integrated into the App; at the scheduled time, the user will enter the meeting system and receive remote rehabilitation services.

[0223] If the user clicks Figure 6B After clicking the "Personalized Hearing Test" 603 function button, you can see... Figure 7C The interface shown allows you to determine whether the current ear being tested is the left or right ear via the "current test ear" indicator (721). The display box (722) shows the current frequency, which can be one of 125Hz, 250Hz, 500Hz, 1000Hz, 2000Hz, 4000Hz, or 8000Hz. The test progress can be a scroll bar. The current volume display (723) graphically represents the perceptible and imperceptible results from the previous round of user feedback. [Further details omitted] Figure 7C In the interface shown, when the user can hear the pure tone at the current sound pressure level, they can click or touch the perceptible control 724; when the user cannot hear the pure tone at the current sound pressure level, they can click or touch the insensitive control 725.

[0224] The following describes the technical implementation process of the hearing test processing method provided in the embodiments of this application. The hearing test processing method includes three parts: personalized hearing test, near-field hearing aids, and remote rehabilitation.

[0225] Personalized hearing testing refers to conducting frequency-band hearing tests based on frequently used mobile devices, such as mobile phones and tablets, to obtain a personalized hearing profile for each user. This personalized hearing profile is also known as a personalized hearing status. The purpose of using devices frequently used by the user is to ensure consistency between audiometry and supplemental hearing.

[0226] In this embodiment, the following constants are set: the audio signal sampling rate is 16000Hz; the frame length is 20 milliseconds (ms), that is, the number of samples per frame is 320 points; if overlapping time-frequency transform is used, such as short-term Fourier transform (STFT), the overlap is 50%; therefore, the number of samples (Hop-size) staggered between two adjacent windows is 320 points, and a 640-point discrete Fourier transform is performed.

[0227] Figure 8A This is a schematic diagram of the personalized hearing test implementation process provided in the embodiments of this application, such as... Figure 8A As shown, the process includes: Step S801, Pre-test preparation; Step S802, audiometry; Step S803: Result display and recording.

[0228] like Figure 8A As shown, step S801, when implemented, includes three steps: ambient sound detection, device volume adjustment, and wearing headphones. Pre-test preparation is to ensure the accuracy of the hearing test as much as possible. Professional-grade air-conduction pure-tone audiometry has strict requirements for the testing environment and equipment, but the target scenario of this application embodiment is conventional equipment and everyday environment, so playing through headphones reduces the environmental requirements to some extent.

[0229] Figure 8B This diagram illustrates the results of a pure-tone audiometry test provided in an embodiment of this application. The horizontal axis represents frequency, including six specific frequencies from 250-8000Hz for the pure-tone audiometry test; the vertical axis represents the user's hearing threshold. Figure 8BIt can be seen that the right ear has a relatively high hearing threshold at 4000Hz and 8000Hz in the test results, indicating that there is some hearing loss.

[0230] In this embodiment, when the average sound pressure level of the environment is less than a first sound pressure level threshold within a certain period of time, it is considered to meet the environmental testing requirements. In implementation, the first sound pressure level threshold can be set to 40 dB.

[0231] In this embodiment, ambient sound detection can be achieved through sound scene recognition based on deep learning. Sound scene recognition uses a deep learning network to identify and classify the current acoustic environment (here, the deep learning network is a classifier). For example, it outputs whether the current environment is indoors or outdoors. Therefore, ambient sound detection, in addition to outputting whether the current ambient sound pressure level meets the test requirements, also outputs the scene recognition result.

[0232] Figure 9 This is a schematic diagram of the sound scene recognition process provided in the embodiments of this application, such as... Figure 9 As shown, the process includes: Step S901, Feature extraction.

[0233] Audio signals possess various features, such as spectrum, phase, Linear Predictive Coding (LPC) coefficients, and Mel-frequency cepstral curves (MFCCs). Different features are suitable for different task objectives, and feature extraction is one of the important factors affecting model performance. In the embodiments of this application, one or more of the above features can be extracted and combined.

[0234] Step S902, network prediction.

[0235] In this step, the extracted features are input into the scene recognition module to identify the audio scene. For example... Figure 9 As shown, this scene recognition module includes convolutional layers, pooling layers, LSTM layers, and fully connected layers. The goal of the convolutional layers is to learn the cross-correlation between features and map them to a high-dimensional space. Convolutional layers are generally used in conjunction with pooling layers, which can effectively reduce the size of the parameter matrix, thereby reducing the number of parameters in the final connected layers. Adding pooling layers can speed up computation and prevent overfitting. The purpose of introducing LSTM layers is to learn the inter-frame features of audio signals, which can distinguish the audio structural features corresponding to different audio scenes and improve recognition accuracy. Finally, a fully connected layer is used as the output layer to display the recognition results.

[0236] Step S903, Identify and output.

[0237] This outputs the network's prediction results. These results will be used in the AI ​​noise reduction module to select the mode and determine the degree of noise reduction, providing professional noise reduction capabilities in different sound scenarios.

[0238] Figure 10 This is a schematic diagram illustrating another implementation process of sound scene recognition provided in an embodiment of this application, as shown below. Figure 10 The steps shown are as follows: Step S1001: Convert the time-domain speech to time-frequency.

[0239] Here, s(n) represents the time-domain speech signal of the nth frame. In this embodiment, STFT can be used for video conversion, and the STFT uses a 50% overlap method. s(n) and s(n-1) are concatenated to form a 640-point sequence. After calling the STFT, the frequency domain representation can be obtained. Considering the symmetry in the frequency domain, only the first 321 points need to be taken.

[0240] Step S1002: Determine the MFCC features of the speech signal.

[0241] Based on frequency domain signals The MFCC features of a speech signal can be calculated. The MFCC calculation process includes Mel filter bank filtering, power spectrum calculation, and DCT.

[0242] Step S1003: Input the obtained MFCC features into the N-frame feature buffer.

[0243] N is an integer greater than 1. In implementation, the MFCC coefficients of the current frame are obtained through step S1002. Then, the input is stored in the feature buffer and, together with the MFCC coefficients of the previous (N-1) frames, forms an N-dimensional MFCC feature. It should be noted that this embodiment uses N frames of features as input to utilize the LSTM network to learn the inter-frame characteristics of the signal, thereby improving recognition accuracy. Furthermore, this embodiment does not limit the value of N, but considering computational resources, the value of N can be set to 30.

[0244] Step S1004: Input the N frame features into the scene recognition module.

[0245] The scene recognition module outputs environmental sound pressure results and scene recognition results based on the input N-dimensional features. Considering the application scenario of this embodiment, the environmental sound pressure results are divided into two categories: relatively quiet and relatively noisy, indicating whether the environment meets the requirements of the audiometry test. The scene recognition results can be divided into four categories: outdoor environment, office, restaurant, and home environment, each corresponding to a different noise reduction strategy. Furthermore, the scene recognition module includes convolutional layers, pooling layers, LSTM layers, and fully connected layers. This embodiment does not limit the number of layers or the parameters of each layer in the network model; however, considering computational resources, this embodiment recommends using 3 convolutional layers and 3 pooling layers, one LSTM layer and one fully connected layer; the convolutional layers have 32 channels and the convolutional kernels have 3... 3; The pooling layer uses max pooling, and the filter is 2. 2, step size is 2; number of hidden nodes in LSTM is 256; the fully connected output dimension is 3, the first dimension is the ambient sound pressure level, and the last two dimensions are the scene recognition result.

[0246] The above step S802 "auditory test" is implemented by using a frequency band-based rise method to test hearing in each frequency band. Based on the human ear's response characteristics to different frequencies, the frequency range is divided into 6 sub-bands with center frequencies of 250Hz, 500Hz, 1000Hz, 2000Hz, 4000Hz, and 8000Hz.

[0247] In this embodiment, the audiometry test utilizes a simplified rising method to perform hearing tests on each frequency band of the left and right ears, totaling 12 sets of tests. A complete rising method requires the subject to respond five times at the same sound pressure level, and testing a complete binaural audiogram is time-consuming. Therefore, the hearing test method in this embodiment simplifies the rising method to meet the needs of general users. Figure 11 This is a schematic diagram illustrating the implementation process of a simplified rise method for audiometry provided in an embodiment of this application, as shown below. Figure 11 As shown, the process includes: Step S1101: Determine the preset first sound pressure level as the initial sound pressure.

[0248] Continuing with the example above, the first sound pressure level is 40dB, and the initial sound pressure level is set to 40dB in this step.

[0249] Step S1102: Output the test tone corresponding to the initial sound pressure.

[0250] Step S1103: Determine whether the subject heard the test sound.

[0251] Here, when it is determined that a trigger operation is received for a "perceptible" control on the App interface, it is determined that the subject can hear the test sound, and then proceed to step S1104; if no trigger operation is received for a "perceptible" control on the App interface, or if a trigger operation is received for an "invisible" control, it is determined that the subject does not hear the test sound, and then proceed to step S1105.

[0252] Step S1104: Reduce the initial sound pressure level by 10 dB.

[0253] Step S1105: Increase the initial sound pressure level by 5 dB.

[0254] Step S1106: Determine whether the sound pressure level is the second time it has been heard.

[0255] If the sound pressure level is heard for the second time, proceed to step S1107; if the sound pressure level is not heard for the second time, return to step S1105.

[0256] Step S1107: Record the current frequency band hearing threshold.

[0257] Step S1108: Perform audiometry on the next frequency band.

[0258] In other words, the simplified ascending method involves presenting the test tone to the subject at a preset first sound pressure level in each set of audiometry. If the subject selects "imperceptible", the test sound pressure level is increased by 5 dB; if the user selects "perceptible", the test sound pressure level is decreased by 10 dB. This process is repeated until the user selects "perceptible" for the second time at a certain sound pressure level. The current sound pressure level is then recorded as the hearing value of the subject's ear in the current frequency band, and the test proceeds to the next set of tests until a total of 12 sets of tests are completed for both ears.

[0259] In step S803, "Result Display and Recording," the complete audiogram obtained in step S802 is displayed, and a hearing curve across the entire frequency band is plotted. Based on the current hearing curve, the system determines whether the subject has hearing loss and provides recommendations. Furthermore, the complete results of the test are saved for developing personalized hearing aid plans.

[0260] Personalized hearing aids refer to developing a personalized equalization scheme based on the user's individual hearing loss condition, combined with speech enhancement to provide professional-grade hearing aid effects. In implementation, speech enhancement is performed first to remove noise from the input signal; then, the personalized equalization scheme is applied to further improve the listening experience based on the individual's hearing loss condition.

[0261] In related technologies, signal processing-based speech enhancement methods, such as spectral subtraction, first analyze the signal to estimate the noise energy; then, based on signal additivity, the estimated noise component is removed from the original input to achieve noise reduction. While this approach is simple, it cannot handle complex noise types, such as transient noise from keyboard input or outdoor noise like wind noise. In recent years, with the development of deep learning technology, AI-based speech enhancement has become more popular and effective. Therefore, this application employs an AI-based speech enhancement method. Figure 12 This is a schematic diagram illustrating the implementation of AI voice enhancement provided in an embodiment of this application, such as... Figure 12 As shown, the process includes: Step S1201: Perform time-frequency transformation (e.g., STFT short-time Fourier transform) on the input signal to obtain noisy spectrum information.

[0262] Step S1202: Input the noisy spectrogram into a specific spectral regression network.

[0263] This spectral regression network includes, but is not limited to, multi-layer feedforward networks, multi-layer LSTM, multi-layer CNN, or combinations thereof. The spectral regression network outputs a set of gains G, corresponding to the gain value for each frequency point of the noisy spectrum.

[0264] Step S1203: Apply G to the noisy spectrum to obtain the enhanced spectrogram.

[0265] Step S1204: The enhanced spectrogram is transformed to the time domain through inverse Fourier transform, and the enhanced speech signal is output.

[0266] This application does not limit the implementation of any specific AI voice enhancement scheme. After AI voice enhancement, the voice signal becomes clearer. Therefore, users can hear clearer voices in noisy environments.

[0267] Figure 13 Another implementation flowchart for the AI ​​voice provided in the embodiments of this application is illustrated below. Figure 13 As shown, the process includes: Step S1301: Perform time-domain conversion on the speech signal.

[0268] Where s(n) represents the time-domain signal of the nth frame. Figure 13 The example illustrates time-domain conversion using STFT. Based on the characteristics of STFT, s(n) and s(n-1) are concatenated into a 640-point sequence. After calling STFT, the frequency domain representation S(n) can be obtained. Considering the characteristics of DFT, only the first 321 points are needed.

[0269] Step S1302: Determine the amplitude spectrum |S(n)| based on S(n).

[0270] Step S1303: Determine the phase spectrum P(n) based on S(n).

[0271] Step S1304: Input |S(n)| into the first enhancement model to predict the gain value G(n) for speech enhancement.

[0272] In this embodiment of the application, considering that |S(n)| is 1 The first augmented neural network model, with 321 dimensions, also outputs G(n) as 1. 321 dimensions. The first augmented neural network model can be a convolutional neural network (CNN), LSTM, multilayer perceptron (MLP), etc. This application embodiment suggests using a feedforward network with four hidden layers, each layer having 1024 hidden nodes. However, this application embodiment does not limit the specific network operators or the number of nodes in each layer.

[0273] In this step, considering the correlation of speech, the amplitude spectrum of historical frames, |S_pre(n)|, can be concatenated with |S(n)| to form a longer input, resulting in better prediction performance. In practice, |S_pre(n)| can be the amplitude spectrum of the (n-1)th and (n-2)th frames. Therefore, the input vector dimension of the first augmented neural network model is 1. 963. This embodiment suggests using a feedforward network with four hidden layers, each with 2048 hidden nodes. However, this embodiment does not limit the number of network operators or the number of nodes in each layer.

[0274] Step S1305: Input the gain G(n) into the gain processing module for gain processing.

[0275] In this step, G(n) and |S(n)| are multiplied point by point to obtain the enhanced amplitude spectrum |S_e(n)|.

[0276] Step S1306: Based on the enhanced amplitude spectrum |S_e(n)| and the phase spectrum of the input signal, perform an inverse STFT transform to obtain the enhanced time-domain signal.

[0277] In this step, the enhanced Fast Fourier Transform (FFT) coefficients are first determined based on the phase spectrum of the input signal and the enhanced amplitude spectrum; then, the enhanced time-domain signal is obtained by calling iSTFT.

[0278] In this embodiment, a sound scene recognition scheme can be combined to further improve the effect of AI speech enhancement. In implementation, the AI ​​speech enhancement strategy can be refined based on the sound scene recognition results. For example, if the recognized sound scene has a high signal-to-noise ratio (SNR), the G value can be multiplied by a number less than 1 (e.g., 0.8). This softens the processing strategy at high SNR levels during speech enhancement, avoiding speech clipping. Another approach is to use different models for speech enhancement based on the sound scene recognition results. In this way, the AI ​​speech enhancement algorithm selects a model trained on more suitable data based on the scene, resulting in better performance.

[0279] Personalized equalization solutions refer to calculating the gain of each frequency band based on the user's individual hearing condition. By differentiating the gain of each frequency band, the system compensates for the user's hearing loss across the entire frequency range, improves the user's perception of the hearing-impaired frequency band, and thus enhances the user's speech intelligibility, meeting the needs of daily communication.

[0280] Figure 14 This is a schematic diagram illustrating the implementation process of personalized load balancing provided in an embodiment of this application, such as... Figure 14 As shown, the process includes: Step S1401, sound pressure level calculation.

[0281] In implementation, the time-domain signal s(n) of the nth frame can be concatenated with s(n-1) into a 640-dimensional array for processing. s(n) and s(n-1) are then input into the sound pressure level calculation module, and the sound pressure level of the current frame signal is obtained using formula (1-1): (1-1); Step S1402: Calculate the gain based on the sound pressure level and individual hearing status.

[0282] In this step, the user's pure-tone audiogram can be loaded first. This audiogram can be the result obtained through the personalized hearing test mentioned above, or it can be imported from a third party (such as a precise pure-tone audiogram obtained from a professional institution). Then, the gain value is determined using the gain calculation formula shown in Table 1.

[0283] Table 1. Gain Calculation Formula

[0284] Where TH represents the hearing threshold corresponding to each frequency band, and G is the calculated gain.

[0285] When implementing this step, the sound intensity is confirmed based on the sound pressure level obtained in step S1401, and then the gain range of the formula in Table 1 is determined based on the sound intensity. In this embodiment of the application, a sound pressure level below 40dB SPL is considered a low-intensity sound; 40dB to 65dB SPL is considered a comfort-range sound; and 65dB to 90dB SPL is considered a high-intensity sound.

[0286] For example, the sound pressure level is determined to be 60dB by formula (1-1), and the hearing threshold of the left ear corresponding to the 6 frequency bands is pta_L=[30 35 35 40 45 45]. First, the gain range is determined according to the sound pressure level. The input sound pressure level of 60dB corresponds to the comfort range sound. Then, the gain is calculated according to the hearing threshold of each frequency band and Table 1. Here, we take the first frequency band as an example. The hearing threshold is 30dB. Then, the formula (2) for the gain of the comfort range sound is substituted into the formula to calculate g[1]=0.6×(30-20)=6dB. The gain values ​​of the other 5 sub-bands are calculated in turn to obtain g[2]=9dB, g[3]=9dB, g[4]=12dB, g[5]=15dB, and g

[16] =15dB respectively.

[0287] It should be noted that, in order to ensure that the equalized signal strength will not cause further damage to the user's hearing in this embodiment of the application, when the sum of the input sound pressure and the gain exceeds the pain threshold, the part of the gain that exceeds the pain threshold will be removed.

[0288] Step S1403: Calculate the equalization filter parameters based on the gain.

[0289] In this step, the calculated six subband gain values ​​are substituted into the filter calculation function to obtain the equalization filter bank coefficients. The filter bank is composed of shelving filters and peaking filters. The shelving filters include high shelf filters and low shelf filters. The functions of these three types of filters are as follows: Low shelf filter: The high-frequency section is direct, while the low-frequency section is adjustable. It can be used to adjust the gain of the low-frequency subband.

[0290] High shelf filter: The low-frequency section is direct-through, while the high-frequency section is adjustable. It can be used to adjust the gain of the high-frequency subband.

[0291] Peaking filter: Raises the center frequency response. Located between the low shelf filter and the high shelf filter, it is used to adjust the gain of the intermediate subband.

[0292] In this embodiment, when calculating the parameters of the equalization filter based on the gain, a "reverse" calculation method is used to approximate the desired response curve. That is, firstly, the parameters of the high shelf filter are calculated based on g[6], and then the parameters of the fifth frequency band filter are calculated based on the difference g^'[5] between the desired gain g[5] and the frequency response h_6[5] of the filter at the fifth frequency band; and so on, the parameters of the entire filter bank can be obtained.

[0293] Compared to methods that directly calculate filter parameters individually, the "reverse" calculation method used in this application embodiment can more closely approximate the desired frequency response curve. Figure 15A This is a schematic diagram of the frequency response curve of the filter bank obtained by directly calculating the filter parameters. Figure 15B This is a schematic diagram of the frequency response curve of the filter bank obtained by inverse calculation of the filter parameters. Figure 15A The dashed circles and × signs in the diagram represent the expected gains of the corresponding sub-bands, respectively. Figure 15A and Figure 15B It can be seen that the filter bank parameters determined by the "reverse" calculation method have a gain at the target frequency band that is closer to the expected value.

[0294] Step S1404, equalize the voice signal.

[0295] In this step, the input speech signal is processed through 6 filters from low frequency to high frequency to obtain the personalized equalized speech signal s'(n).

[0296] Step S1405, dynamic range control.

[0297] Finally, to prevent the output signal from being clipped and affecting the listening experience, this embodiment of the application adds a dynamic range control module after equalization to protect the integrity of the audio signal.

[0298] In practice, it is recommended to first perform AI voice enhancement to eliminate environmental noise; then, activate personalized hearing aids to improve the listening experience. Finally, the signal processed by near-field hearing aids is connected to the terminal device via Bluetooth or wired connection. These terminal devices include, but are not limited to, wired headphones, Bluetooth headphones, hearing aids, and cochlear implants. The near-field hearing aid-processed signal is then transmitted to the user's device for a better listening experience.

[0299] In the process of remote rehabilitation, a connection is established between the doctor and the user based on real-time audio and video calls, that is, the remote rehabilitation process is completed through video calls. The establishment of real-time audio and video calls can be achieved by integrating a software development kit (SDK) or underlying library into the app, or by the app redirecting directly to a third-party audio and video calling app. This application embodiment does not limit the above implementations.

[0300] Figure 16 This is a schematic diagram illustrating the implementation process of remote rehabilitation provided in the embodiments of this application, such as... Figure 16 As shown, this implementation process includes a user-side component and a doctor-side component. Generally, remote rehabilitation is initiated by the user. The following section combines... Figure 16 The implementation process on the user side and the doctor side is explained.

[0301] Step S1601: Based on the user's selection operation in the doctor interface of the App, determine the selected doctor.

[0302] Step S1602, leave a message for consultation.

[0303] In implementation, after selecting a doctor, the interface provides an entry control for leaving a consultation message. When the user clicks or touches this entry control, a message interface is displayed. Through this message interface, the user can enter a message and then click or touch the "Message" control to send the message to the corresponding doctor.

[0304] Step S1603: Schedule a meeting.

[0305] Step S1604: Enter the meeting.

[0306] After selecting a meeting, an entry point for choosing a time slot will appear. Upon receiving a touch or click action through this entry point, the doctor's available time slots will be displayed for the user to choose from. After selecting a time slot, the user clicks "confirm," which will send a meeting invitation to the doctor. Furthermore, this embodiment does not limit the user from simultaneously scheduling multiple meetings. For example, a user can simultaneously schedule meetings with an audiologist and an audiologist.

[0307] For paid services, such as remote rehabilitation, users may be redirected to a payment page after scheduling a meeting; they will then complete the payment as prompted.

[0308] After the user completes payment, a new record will be added to the user's booked rehabilitation services page. This new record includes, but is not limited to: the booked service time, doctor's name, meeting access number, and access link.

[0309] The implementation process on the doctor's side includes: Step S1611: If a message notification is received, reply to the message.

[0310] According to the app's prompts, when the doctor clicks the message entry control, a message interface is displayed. The doctor can read the message information and reply to the message through this message interface.

[0311] In step S1612, if a meeting appointment reminder is received, the doctor makes the appointment feedback through the client.

[0312] The appointment confirmation can be either an acceptance or a rejection of the meeting invitation. If the doctor accepts the invitation, the back-end payment system will execute the transfer, and the doctor's account will receive the consultation fee. Furthermore, this embodiment does not limit the following function: if the time is inconvenient, the doctor can propose a new meeting time and send it to the user for secondary confirmation.

[0313] Step S1613: Enter the meeting.

[0314] After a doctor accepts the meeting invitation, a new record will be added to the page for pending rehabilitation services. This new record includes, but is not limited to: the scheduled service time, user name, meeting access number, and access link.

[0315] This application provides a system integrating near-field hearing aids and remote hearing rehabilitation and adjustment. Through an application, users can perform audiometry at any time to obtain their personalized hearing status, which helps protect hearing health. Furthermore, based on this personalized status, near-field hearing aids can be performed, effectively improving speech intelligibility. In addition, near-field hearing aids can effectively suppress environmental noise, enhancing the auditory experience; on the other hand, through the remote rehabilitation platform, users can communicate remotely with doctors, improving consultation efficiency.

[0316] The following description continues to illustrate the exemplary structure of the hearing assistive device 454 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the hearing aid 454 in the memory 440 may include: The first presentation module 4541 is used to present a hearing assistance interface in response to a trigger operation for a hearing assistance client. The hearing assistance interface includes at least a first function entry for remote inquiry and a second function entry for near-field hearing assistance. The second presentation module 4542 is used to present a doctor information display interface in response to a trigger operation on the first function entry point; The first determining module 4543 is used to determine the selected target doctor information and the first appointment time in response to the selection operation triggered by the doctor information display interface; The first sending module 4544 is used to send a remote consultation request to the doctor's client in response to the received appointment operation instruction. The remote consultation request carries the first appointment time and the entry link for conducting real-time audio and video calls.

[0317] In some embodiments, the device further includes: The third presentation module is used to respond to the received appointment operation instruction and present the payment page, which displays the consultation amount and payment entry corresponding to the target doctor information; The fourth presentation module is used to present a payment method selection interface in response to a trigger operation on the payment portal; The second determining module is used to determine the target payment method in response to the selection operation triggered by the payment method selection interface, and to make payment based on the target payment method.

[0318] In some embodiments, the device further includes: The output module is used to output a prompt message when the time interval between the current time and the first appointment time is less than the interval threshold. The prompt message is used to indicate that the first appointment time for remote consultation is about to arrive. The fifth presentation module is used to present an audio / video call interface in response to a trigger operation on the entry link when the first appointment time is reached. The first acquisition module is used to activate the image acquisition device and the voice acquisition device to acquire the first image data acquired by the image acquisition device and the first voice data acquired by the voice acquisition device. The sixth presentation module is used to present the first image data and the second image data obtained from the doctor's client in the audio and video call interface; The second sending module is used to send the first voice data to the doctor's client and play the second voice data sent by the doctor's client.

[0319] In some embodiments, the device further includes: The seventh presentation module is used to establish a communication connection between the user terminal and the hearing aid device in response to the device connection operation when it is necessary to adjust the parameters of the hearing aid device, and to present the settings interface of the hearing aid device. The third determining module is used to determine the adjustment parameters and the target values ​​of the adjustment parameters based on the adjustment information obtained from the doctor's client. The first receiving module is used to receive adjustment operations on the adjustment parameters through the setting interface, so as to adjust the adjustment parameters to the target value.

[0320] In some embodiments, the device further includes: The second receiving module is used to receive remote control requests sent by the doctor's client; The eighth presentation module is used to present remote control prompt information in response to the remote control request; The third sending module is used to send a response message allowing remote control to the doctor's client when a positive feedback instruction for the remote control prompt is received, so that the doctor's terminal can adjust the parameters of the hearing aid device through remote control.

[0321] In some embodiments, the device further includes: The third receiving module is used to receive a first remote consultation response sent by the doctor's client. The first remote consultation response indicates that the doctor's client has not accepted the remote consultation request. The first remote consultation response carries a second appointment time. The ninth presentation module is used to respond to the viewing operation of the first remote consultation response, present the remote consultation response interface, and present the second appointment time through the remote consultation response interface; The fourth sending module is used to send a second remote consultation response to the doctor's client when a trigger operation to accept the second preset time is received through the remote consultation response interface.

[0322] In some embodiments, the device further includes: The fourth determination module is used to determine the selected target debugging personnel in response to the selection operation triggered by the doctor information display interface; The fifth sending module is used to send a device adjustment request to the device adjustment client in response to the received reservation operation instruction. The device adjustment request carries the first reservation time and the entry link for conducting real-time audio and video calls. The fourth receiving module is used to receive a second remote control request sent by the tuning client after the real-time audio and video call starts at the first scheduled time and the user terminal establishes a communication connection with the hearing aid device. The tenth presentation module is used to present a second remote control prompt message in response to the second remote control request; The sixth sending module is used to send a second response message allowing remote control to the tuning client when a positive feedback instruction for the second remote control prompt information is received, so that the tuning terminal can adjust the parameters of the hearing aid device remotely based on the adjustment instruction of the doctor terminal.

[0323] In some embodiments, the device further includes: The eleventh presentation module is used to present a near hearing aid interface in response to a trigger operation on the second function entry. The near hearing aid interface presents a hearing configuration entry, a first hearing aid switch control, and a second hearing aid switch control. The third acquisition module is used to acquire the user's hearing test results in response to a trigger operation on the hearing configuration entry. The noise reduction module is used to perform noise reduction processing on the collected voice data when the first auxiliary hearing function is determined to be turned on based on the trigger operation of the first auxiliary hearing switch, so as to obtain the noise-reduced voice data. An equalization processing module is used to perform equalization processing on the noise-reduced speech data based on the hearing test results when the second auxiliary hearing function is determined to be activated based on the trigger operation of the second auxiliary hearing switch, so as to obtain target speech data. The second output module is used to output the target speech data.

[0324] In some embodiments, the noise reduction module is further configured to: The collected speech data is converted to the frequency domain to obtain frequency domain speech data, and the phase information and amplitude information of the frequency domain speech data are obtained. Obtain the trained speech enhancement model, input the amplitude information into the trained speech enhancement model, and obtain the gain information; Based on the amplitude information and the gain information, determine the amplitude information after noise reduction; Based on the amplitude information and the phase information, the denoised frequency domain speech data is determined; The denoised frequency domain speech data is converted to the time domain to obtain the denoised speech data.

[0325] In some embodiments, the device further includes: The feature extraction module is used to extract the speech features of the collected speech data; The scene recognition module is used to input the speech features into the trained scene recognition model to obtain the recognized scene information; Correspondingly, this noise reduction module is also used for: Based on the scene information, obtain the trained speech enhancement model corresponding to the scene information.

[0326] In some embodiments, the equalization processing module is further configured to: The sound pressure level is determined based on the collected speech data, and the hearing threshold corresponding to each frequency band is determined based on the user's hearing test results. Based on the sound pressure level and the hearing threshold corresponding to each frequency band, the gain value corresponding to each frequency band is determined; The parameters of the equalization filter are determined based on the gain values ​​corresponding to each frequency band. The noise-reduced speech data is filtered using the equalization filter to obtain filtered speech data. The filtered speech data is subjected to dynamic range control to obtain the target language data.

[0327] It should be noted that the descriptions of the above-described hearing aid device embodiments are similar to the descriptions of the methods described above, and have the same beneficial effects as the method embodiments. For technical details not disclosed in the embodiments of the hearing aid device of this application, those skilled in the art should refer to the descriptions of the method embodiments of this application for understanding.

[0328] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the hearing assistance method described above in this application.

[0329] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the hearing assistance method provided in this application, for example... Figure 3 , Figure 4 and Figure 5 The auditory aids shown are illustrated.

[0330] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0331] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0332] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0333] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0334] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for processing hearing test data, characterized in that, The method includes: A hearing test interface is presented, which includes perceptible controls and non-perceptible controls; When the environment for the hearing test meets the environmental test requirements, the test tone corresponding to the initial sound pressure is output. In response to a triggering operation on the perceived control, the initial sound pressure level is reduced; In response to not receiving a trigger operation for the perceived control or receiving a trigger operation for the imperceptible control, the initial sound pressure level is increased; When a trigger operation for the perceptible control is received for the second time at any sound pressure level, the current sound pressure level is used as the hearing value of the current test ear in the current frequency band, and the hearing test is performed in the next frequency band. After the hearing test is completed, an audiogram of the hearing test is displayed, which includes hearing curves across the entire frequency band.

2. The method according to claim 1, characterized in that, The hearing test interface also includes the current test ear, current test frequency, test progress, and current volume; The current test ear is used to determine whether the current hearing test is being conducted on the left or right ear. The current test frequency is used to characterize the frequency of the current hearing test. The test progress is used to characterize the hearing test progress. The current volume is used to graphically represent the perceptible and imperceptible conclusions from the previous round based on user feedback.

3. The method according to claim 1, characterized in that, Before outputting the test tone corresponding to the initial sound pressure when the environment of the hearing test meets the environmental testing requirements, the method further includes: Before the hearing test, pre-test preparation is performed, which includes ambient sound detection, device volume adjustment, and headphone wearing information. This pre-test preparation is used to ensure the accuracy of the hearing test.

4. The method according to claim 1, characterized in that, Before the test tone corresponding to the initial sound pressure level is output, the method further includes: The current acoustic environment is classified by a deep learning network to obtain environmental sound pressure results and scene recognition results. The environmental sound pressure results represent whether the acoustic environment meets the requirements of the audiometry test.

5. The method according to claim 4, characterized in that, The process of classifying the current acoustic environment using a deep learning network to obtain environmental sound pressure results and scene recognition results includes: The collected speech data is converted to the frequency domain to obtain frequency domain speech data; Determine the Mel frequency cepstral coefficient characteristics of the speech data based on the frequency domain speech data; The Mel frequency cepstral coefficient features are combined with the Mel frequency cepstral coefficient features of historical frames to form multi-frame features; Scene recognition is performed on the multi-frame features to obtain environmental sound pressure results and scene recognition results.

6. The method according to claim 1, characterized in that, The method further includes: In response to a trigger operation on a hearing-assisted client, a hearing-assisted interface is presented, the hearing-assisted interface including at least a first function entry for remote inquiry and a second function entry for near-field hearing assistance; In response to a trigger operation targeting the first function entry point, a doctor information display interface is presented; In response to a selection operation triggered through the doctor information display interface, the selected target doctor information and the first appointment time are determined; In response to the received appointment operation instruction, a remote consultation request is sent to the doctor's client. The remote consultation request carries the first appointment time and an entry link for conducting a real-time audio and video call.

7. The method according to claim 6, characterized in that, The method further includes: In response to a trigger operation on the entry control for consultation messages in the doctor information display interface, a message interface is presented, wherein the message interface includes a message entered by the user and a message sending control; In response to a trigger operation on the message sending control, the message is sent to the doctor's client corresponding to the target doctor information.

8. The method according to claim 7, characterized in that, When a doctor replies to a message through the message interface on the client, the reply is received; In response to a view operation on the message reply, the message interface is displayed, and the message reply is shown on the message interface.

9. The method according to claim 6, characterized in that, The method further includes: When the doctor's client accepts the remote consultation request and the first appointment time is reached, the audio and video call interface is presented in response to the trigger operation of the entry link; The system receives second image data and second voice data sent by the doctor's client. The second image data includes a mouth shape image of the doctor reading the relevant words aloud, and the second voice data includes the sound of the doctor reading the relevant words aloud. The lip-sync image is displayed and the sound is played on the audio / video call interface so that the user can imitate the relevant words based on the lip-sync image and the sound.

10. The method according to claim 6, characterized in that, The method further includes: In response to the received appointment operation instruction, a payment page is displayed, which shows the consultation amount and payment entry corresponding to the target doctor information; In response to a triggered operation on the payment entry point, a payment method selection interface is displayed; In response to a selection operation triggered by the payment method selection interface, a target payment method is determined, and payment is made based on the target payment method.

11. The method according to claim 6, characterized in that, The method further includes: When the time interval between the current time and the first appointment time is less than the interval threshold, a prompt message is output. The prompt message is used to indicate that the first appointment time for remote consultation is about to arrive. When the first appointment time is determined to be reached, in response to the trigger operation of the entry link, an audio / video call interface is presented; Start the image acquisition device and the voice acquisition device to acquire the first image data acquired by the image acquisition device and the first voice data acquired by the voice acquisition device; The first image data and the second image data obtained from the doctor's client are presented in the audio and video call interface; The first voice data is sent to the doctor's client, and the second voice data sent by the doctor's client is played.

12. The method according to claim 11, characterized in that, The method further includes: When it is necessary to adjust the parameters of the hearing aid, in response to the device connection operation, a communication connection is established between the user terminal and the hearing aid, and the settings interface of the hearing aid is presented. Based on the adjustment information obtained from the doctor's client, the adjustment parameters and the target values ​​of the adjustment parameters are determined; The settings interface allows for the receiving of adjustment operations on the adjustment parameters, which are then adjusted to the target value.

13. The method according to claim 12, characterized in that, The method further includes: Receive remote control requests sent by the doctor's client; In response to the remote control request, a remote control prompt message is displayed; When a positive feedback instruction is received regarding the remote control prompt, a response message allowing remote control is sent to the doctor's client, enabling the doctor's terminal to adjust the parameters of the hearing aid device remotely.

14. The method according to claim 12, characterized in that, The method further includes: In response to a selection operation triggered by the doctor information display interface, the selected target debugging personnel are determined; In response to the received reservation operation instruction, a device switching request is sent to the device switching client. The device switching request carries the first reservation time and the entry link for conducting real-time audio and video calls. After the real-time audio and video call starts at the first scheduled time and the user terminal establishes a communication connection with the hearing aid device, it receives a second remote control request sent by the tuning client. In response to the second remote control request, a second remote control prompt message is displayed; When a positive feedback instruction for the second remote control prompt is received, a second response message allowing remote control is sent to the tuning client, so that the tuning terminal can adjust the parameters of the hearing aid device remotely based on the adjustment instructions from the doctor's terminal.

15. The method according to claim 6, characterized in that, The method further includes: The system receives a first remote consultation response from the doctor's client, the first remote consultation response indicating that the doctor's client has not accepted the remote consultation request, and the first remote consultation response carries a second appointment time. In response to a viewing operation of the first remote consultation response, a remote consultation response interface is displayed, through which the second appointment time is presented; When a trigger operation to accept the second preset time is received through the remote consultation response interface, a second remote consultation response is sent to the doctor's client.

16. The method according to any one of claims 6 to 15, characterized in that, The method further includes: In response to a trigger operation on the second function entry, a near hearing aid interface is presented, which includes a hearing configuration entry, a first hearing aid switch control, and a second hearing aid switch control. In response to a trigger operation on the hearing configuration entry, the user's hearing test results are obtained; When the first hearing aid function is activated based on the trigger operation of the first hearing aid switch, the collected voice data is denoised to obtain denoised voice data. When the activation of the second hearing aid function is determined based on the trigger operation of the second hearing aid switch, the noise-reduced speech data is subjected to equalization processing based on the hearing test results to obtain the target speech data. Output the target speech data.

17. The method according to claim 16, characterized in that, The noise reduction process performed on the collected speech data to obtain the noise-reduced speech data includes: The collected speech data is converted to the frequency domain to obtain frequency domain speech data, and the phase information and amplitude information of the frequency domain speech data are obtained. Obtain the trained speech enhancement model, input the amplitude information into the trained speech enhancement model, and obtain the gain information; Based on the amplitude information and the gain information, determine the amplitude information after noise reduction; Based on the amplitude information and the phase information, the denoised frequency domain speech data is determined; The denoised frequency domain speech data is converted to the time domain to obtain the denoised speech data.

18. The method according to claim 17, characterized in that, The method further includes: Extract the speech features from the collected speech data; The speech features are input into the trained scene recognition model to obtain the recognized scene information; Correspondingly, obtaining the trained speech enhancement model includes: Based on the scene information, obtain the trained speech enhancement model corresponding to the scene information.

19. The method according to claim 16, characterized in that, The process of equalizing the noise-reduced speech data based on the hearing test results to obtain target speech data includes: The sound pressure level is determined based on the collected speech data, and the hearing threshold corresponding to each frequency band is determined based on the user's hearing test results. Based on the sound pressure level and the hearing threshold corresponding to each frequency band, the gain value corresponding to each frequency band is determined; The parameters of the equalization filter are determined based on the gain values ​​corresponding to each frequency band. The noise-reduced speech data is filtered using the equalization filter to obtain filtered speech data. The filtered speech data is subjected to dynamic range control to obtain the target language data.

20. A hearing test processing device, characterized in that, The device includes: The first presentation module is used to present a hearing test interface, which includes perceptible and non-perceptible controls. When the hearing test environment meets the environmental test requirements, it outputs a test tone corresponding to the initial sound pressure level. In response to a trigger operation on the perceptible control, it lowers the initial sound pressure level. In response to no trigger operation on the perceptible control or a trigger operation on the non-perceptible control, it raises the initial sound pressure level. When a trigger operation on the perceptible control is received for the second time at any sound pressure level, the current sound pressure level is used as the hearing value of the current test ear in the current frequency band, and the hearing test is performed in the next frequency band. After the hearing test is completed, the audiogram of the hearing test is displayed, which includes a hearing curve across the entire frequency band.

21. A terminal device, characterized in that, The terminal device includes: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method according to any one of claims 1 to 19.

22. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, they implement the method according to any one of claims 1 to 19.

23. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the method described in any one of claims 1 to 19.

Images