Terminal and terminal pickup method

Abstract

Provided in the present application are a terminal and a terminal pickup method. The terminal comprises a main body portion, a pickup module, and a control module, wherein the main body portion has a first surface, the first surface is a surface facing a user, and the first surface comprises a display surface; the pickup module is disposed in the main body portion, and the pickup module comprises a radar module; the radar module is located on an inner side of the first surface; the radar module is used for emitting a detection beam towards the first surface, and is further used for receiving and processing an echo that passes through the first surface and enters the main body portion, and outputting a first electrical signal outwards; the control module is in communication connection with the pickup module; and the control module is used for receiving the first electrical signal, detecting whether the first electrical signal includes Doppler frequency shift data, and performing analysis on the basis of the Doppler frequency shift data to obtain an audio signal. By means of the terminal and the terminal pickup method provided in the present application, the design complexity of a pickup structure can be reduced to a certain extent.

Description

Terminal and terminal sound pickup method

[0001] This application claims priority to Chinese patent application filed on March 5, 2024, with application number 202410251577.3 and entitled "Terminal and Terminal Sound Pickup Method", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminals, and more particularly to a terminal and a terminal sound pickup method. Background Technology

[0003] To improve user experience, most terminals include a microphone module. This module can be used not only for general audio capture but also for applications such as voice and video calls. Currently, the microphone module used in terminals is typically a microphone. However, using a microphone requires creating a sound inlet or slit in the terminal's casing, increasing the complexity of the microphone design. Summary of the Invention

[0004] This application provides a terminal and a terminal sound pickup method, which improves the problem that using a microphone to pick up sound in existing terminals increases the complexity of the sound pickup structure design.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, a terminal is provided, including a main body, a microphone module, and a control module. The main body has a first surface facing the user, including a display surface. The microphone module is disposed within the main body and includes a radar module. The radar module is located inside the first surface. The radar module emits a detection beam towards the first surface, receives and processes echoes passing through the first surface and entering the main body, and outputs a first electrical signal. The control module is communicatively connected to the microphone module. The control module receives the first electrical signal, detects whether it contains Doppler frequency shift data, and analyzes the Doppler frequency shift data to derive an audio signal. When a user speaks, the skin around the user's mouth, throat, jaw, etc., vibrates. This vibration is generally very small and causes the echoes to exhibit a Doppler effect, resulting in a Doppler frequency shift. Thus, the echoes carry vibration information related to the user's speech, i.e., Doppler frequency shift data. After the echo is received and processed by the radar module, a first electrical signal is generated and transmitted to the control module. The control module analyzes and processes the first electrical signal to extract audio-related features, namely the aforementioned Doppler frequency shift data. Then, a Mel-spectrogram (mel spectrum) is generated based on the Doppler frequency shift data. Speech reconstruction is then performed using the Mel-spectrogram to generate the corresponding audio signal. Finally, the audio signal can be processed with noise reduction and other methods before being output or stored. Therefore, the terminal provided in this embodiment can use a radar module to replace the microphone to achieve sound pickup. The radar module can be installed inside the terminal, and the surface of the terminal does not need to have sound inlets or gaps, or has fewer sound inlets or gaps, making the terminal aesthetically pleasing and complete, and reducing the complexity of the sound pickup structure design to a certain extent. Furthermore, since the radar module is located inside the terminal, it does not need to be placed on the side or bottom of the terminal for aesthetic reasons; it can be positioned directly facing the mouth, meaning the radar module can be placed in an optimal sound pickup position to better pick up the user's voice. Therefore, the terminal provided in this embodiment can improve sound pickup quality to a certain extent.

[0007] In one possible implementation of the first aspect, the main body includes a support device and a screen assembly disposed on the support device, with the display surface being the side of the screen assembly facing away from the support device, and at least a portion of the radar module disposed on the screen assembly. Since the user's face is generally facing the screen assembly during use, the solution provided in this embodiment allows the detection beam emitted by the radar module to reach the parts of the body that vibrate when the user speaks, such as the mouth, throat, and jaw, thereby better receiving the vibration information related to the user's speech, resulting in better audio quality ultimately picked up by the terminal.

[0008] In one possible implementation of the first aspect, the screen assembly includes a screen body and a metal layer, with the metal layer disposed on the side of the screen body facing the support device. The antenna of the radar module is disposed on the screen body or the metal layer. This allows the detection beam generated by the radar module to pass through the corresponding part of the screen assembly more easily without being blocked by the metal layer in the screen assembly, while also making the terminal structure compact and facilitating its miniaturization design.

[0009] In one possible implementation of the first aspect, a through-structure extending through the metal layer along its thickness direction is provided. At least one through-structure is provided. All through-structures form a surrounding structure with at least one opening. The surrounding structure divides the metal layer into a first part and a second part interconnected by a connecting structure. The connecting structure is a structure located at the opening in the metal layer. The first part is the portion of the metal layer surrounded by the through-structure. The first part forms an antenna, and the second part is the portion of the metal layer excluding the first part and the connecting structure. The antenna, using the solution provided in this embodiment, can be manufactured using a portion of the screen assembly, which can reduce the number of components in the terminal to a certain extent, without increasing the thickness of the screen assembly or affecting the display effect, facilitating miniaturization of the terminal design, and reducing the terminal's production cost to a certain extent. Simultaneously, it achieves the purpose of directing the detection beam emitted by the radar module towards the user, achieving multiple benefits.

[0010] In one possible implementation of the first aspect, the through-structure has one element, and it is a strip structure. This structure simplifies the antenna fabrication process and facilitates fabrication.

[0011] In one possible implementation of the first aspect, the screen assembly includes an organic light-emitting diode (OLED) display module, and the metal layer is the cathode layer of the OLED display module. In this embodiment, the antenna can be fabricated using the aforementioned through-type structure. Since the width of the through-type structure is very small, its placement will not affect the normal display of the screen assembly. This allows a terminal with an OLED screen to be integrated with a radar module, thereby enabling the terminal to achieve sound pickup through the radar module.

[0012] In one possible implementation of the first aspect, the screen assembly includes a liquid crystal display (LCD), and the metal layer serves as a reflective layer for the LCD. In this embodiment, the antenna can be fabricated using the aforementioned through-structure method. Since the width of the through-structure is very small, it will not affect the reflective effect of the backlight reflective layer. This allows a terminal with an LCD screen to be integrated with a radar module, thereby enabling the corresponding terminal to achieve sound pickup through the radar module.

[0013] In one possible implementation of the first aspect, the radar module includes a transmitting module and a receiving module. The transmitting module includes a transmitter, a first capacitor, and a transmitting antenna connected in sequence. The receiving module includes a receiving antenna, a second capacitor, and a receiver connected in sequence. Both the first and second capacitors are isolation capacitors with generally very small capacitance values, typically on the order of pF. When the radar module is a millimeter-wave radar, the impedance of the first and second capacitors is very low in the radar module's operating frequency band, approximating a short circuit, while their impedance is very high in the display drive signal frequency band, equivalent to an open circuit. Thus, the first and second capacitors can, to some extent, prevent the display drive signal from the screen assembly from flowing into the transmitter or receiver of the radar module. This can improve the transmission efficiency of the transmitting antenna and the receiving sensitivity of the receiving antenna to some extent.

[0014] In one possible implementation of the first aspect, the main body includes a support device, a screen assembly mounted on the support device, and a bezel surrounding the screen assembly. The display surface is the side of the screen assembly facing away from the support device, and at least a portion of the radar module is disposed on the bezel. This ensures that the placement of the radar module does not damage the internal structure of the screen assembly or affect the normal display of the screen assembly, facilitating the installation of the radar module.

[0015] In one possible implementation of the first aspect, the sound pickup module further includes a microphone, which is disposed within the main body. The main body has a sound-entry structure. The sound-entry structure connects the external space of the main body with the internal space of the main body. The sound-entry structure allows sound waves to pass through for pickup by the microphone. This reduces the number of microphones in the terminal, thereby reducing the number of sound-entry holes or gaps in the terminal, resulting in a more aesthetically pleasing and complete appearance. Furthermore, since the microphone converts all sound signals at its location into electrical signals, it cannot determine which signal is the primary sound source. Using the solution provided in this embodiment, the microphone and radar module can work together. The radar module identifies the primary sound source, thereby filtering out audio signals from non-primary sound sources acquired by the microphone. Simultaneously, the audio signal from the primary sound source acquired by the microphone can be combined with the audio signal acquired by the radar module through AI fusion, noise reduction, and other technologies to achieve joint noise reduction, resulting in a lower noise level in the final audio signal obtained by the terminal.

[0016] In one possible implementation of the first aspect, the control module includes a fusion module, which is communicatively connected to both the microphone and radar modules. The fusion module receives and processes the second electrical signal and the first electrical signal output by the microphone to generate an audio signal. The fusion module can be an AI noise reduction network module, which can be based on computational auditory scene analysis theory, applying deep learning technology, and using a deep neural network to construct a noise reduction model. After training with massive amounts of speech data, it can separate human voice from noise, effectively suppressing various noises in the environment. It can effectively cope with sudden non-stationary noise, and the speech distortion is relatively much smaller, greatly ensuring the sound reproduction accuracy and improving the call experience. Using the solution provided in this embodiment, the data acquired by the radar module and the data acquired by the microphone can be well fused, thereby generating a low-noise audio signal. Compared to a pickup module that only includes a microphone, the solution provided in this embodiment adds an active pickup mode, that is, it can collect vibration signals generated by the sound source through the radar module and perform joint noise reduction with the sound signals collected by the microphone, thereby significantly improving the noise reduction effect.

[0017] In one possible implementation of the first aspect, the control module further includes a voice activity detection module. The voice activity detection module is communicatively connected to the radar module and the fusion module, respectively. The voice activity detection module receives a first electrical signal and analyzes whether Doppler frequency shift data exists in the first electrical signal. The fusion module receives the analysis signal output by the voice activity detection module and processes the first and second electrical signals based on the analysis signal. The voice activity detection module is used to confirm whether the user is speaking. Since vibration only occurs when the user speaks, confirming whether the user is speaking by detecting vibration can be achieved by analyzing whether Doppler frequency shift data exists in the first electrical signal. The specific analysis method is not limited in this application. For example, it can be designed such that when the user speaks, the voice activity detection module outputs a specific signal (e.g., 1); when the user is not speaking, the voice activity detection module outputs a specific signal (e.g., 0); when the fusion module receives the specific signal corresponding to the user speaking, it analyzes and processes the first and second electrical signals; when the fusion module receives the specific signal corresponding to the user not speaking, it deletes the first and second electrical signals. This allows the fusion module to filter out some sound signals that are not emitted by the user, thus achieving noise reduction. At the same time, it can reduce the number of calculations required by the fusion module to a certain extent, thereby improving the working efficiency of the control module.

[0018] In one possible implementation of the first aspect, the control module further includes a noise reduction module. The noise reduction module is communicatively connected to both the microphone and the fusion module. The noise reduction module receives the second electrical signal, performs noise reduction processing on the second electrical signal, and transmits the noise-reduced second electrical signal to the fusion module. The noise reduction module can be an AI noise reduction module or other noise reduction modules, such as a multi-microphone noise reduction module, depending on the specific application requirements. Using the solution provided in this embodiment, the second electrical signal can be denoised once before being transmitted to the fusion module, thereby reducing the computational steps of the fusion module and improving the quality of the audio signal output by the fusion module.

[0019] In one possible implementation of the first aspect, multiple microphones are provided, and a noise reduction module is communicatively connected to each of the multiple microphones. The noise reduction module receives second electrical signals output from the multiple microphones and performs noise reduction based on these signals. The noise reduction module can select a corresponding microphone array noise reduction module based on the number of microphones. For example, the noise reduction module can obtain the pickup range of different microphones, determine the overlapping area of ​​the pickup ranges of different microphones, output the audio signal within the overlapping area as the final audio signal, and remove the audio signal outside the overlapping area, thereby achieving noise reduction. The noise reduction technology provided in this embodiment is mature and easy to design.

[0020] In one possible implementation of the first aspect, the control module further includes a wind noise detection module. This module is communicatively connected to the microphone, noise reduction module, and fusion module. The wind noise detection module receives the second electrical signal output from the microphone, processes the second electrical signal to obtain wind noise data related to the wind sound, and transmits the wind noise data to the noise reduction module and the fusion module. The noise reduction module also receives the wind noise data and performs noise reduction processing on the second electrical signal based on the wind noise data. The fusion module further performs noise reduction processing on the first and second electrical signals based on the wind noise data. The wind noise detection module detects whether wind-related data exists in the data acquired by the microphone and also determines the frequency range and intensity of the wind sound. The wind noise detection module can achieve the above functions in various ways, such as receiving low-frequency signals picked up by the microphone and generating a wearing result signal from the low-frequency signals. The fusion module is an AI noise reduction network that receives the outputs of the wind noise detection module and the voice activity detection module, and further removes noise from the data based on the noise reduction by the noise reduction module. Using the solution provided in this embodiment, the wind noise detection module can detect whether there is wind noise in the audio signal acquired by the microphone, so that the noise reduction module can perform noise reduction processing on the second electrical signal transmitted by the microphone according to the corresponding detection results. At the same time, the fusion module can perform noise reduction processing on the electrical signal transmitted by the noise reduction module and the first electrical signal according to the corresponding detection results, so that the noise reduction effect of the audio signal output by the fusion module is better.

[0021] In one possible implementation of the first aspect, the radar module is a millimeter-wave radar. Millimeter-wave radar features high resolution, strong penetration, ease of integration, and high security. Using millimeter-wave radar in the radar module makes it suitable for various application scenarios.

[0022] Secondly, a terminal sound pickup method is provided, based on a terminal provided by any of the above schemes, including at least the following steps: acquiring Doppler frequency shift data related to the user's speech through a radar module; and analyzing the Doppler frequency shift data to obtain an audio signal.

[0023] The terminal sound pickup method provided in this embodiment can obtain the user's voice by acquiring the vibration of the user's skin (such as the mouth, throat, jaw, etc.) when the user speaks. Compared with acquiring it through a microphone, it can reduce the number of sound holes or gaps to be opened on the terminal surface, making the terminal surface complete and beautiful. It can also reduce the complexity of the sound pickup structure design to a certain extent, and at the same time, it can reduce the noise in the audio signal to a certain extent, such as the wind noise in the environment and the voices of people next to the user.

[0024] In one possible implementation of the second aspect, deriving an audio signal through Doppler frequency shift data analysis includes at least the following steps: receiving a second electrical signal output from a microphone; and obtaining an audio signal based on the second electrical signal and Doppler frequency shift data analysis.

[0025] Since the microphone converts all sound signals at its location into electrical signals (i.e., the second electrical signal), it is impossible to determine which signal is the main sound source. By using the terminal sound pickup method provided in this embodiment, the vibration data related to sound acquired by the radar module is fused with the sound data acquired by the microphone. This can filter out sounds not emitted by the user, resulting in less noise in the acquired audio signal and helping to improve the quality of the audio signal.

[0026] In one possible implementation of the second aspect, before analyzing the audio signal based on the second electrical signal and Doppler frequency shift data, at least the following steps are included: determining whether the distance between the radar module and the user is within a preset distance; if the distance between the radar module and the user is not within the preset distance, then the first electrical signal is eliminated.

[0027] By using the solution provided in this embodiment, a preset distance can be defined based on the general distance between the user and the terminal during use. This ensures that Doppler frequency shift data caused by objects located outside the preset distance will not be mistaken for user voice-related data, thereby achieving the removal of some noise and making the acquired audio information more accurate. Attached Figure Description

[0028] Figure 1 is a schematic diagram of the structure of a condenser microphone in related technologies;

[0029] Figure 2 is a schematic diagram of the structure of a dynamic microphone in related technologies;

[0030] Figure 3 is a circuit diagram of a dynamic microphone in related technologies;

[0031] Figure 4 is a schematic diagram of the structure of a terminal provided in an embodiment of this application;

[0032] Figure 5 is a partial structural diagram of the terminal shown in Figure 4 after the screen component has been removed.

[0033] Figure 6 is a schematic diagram of the structure of a terminal provided in another embodiment of this application;

[0034] Figure 7 is a perspective structural diagram of a terminal provided in an embodiment of this application;

[0035] Figure 8 is a schematic diagram of the usage state structure of a terminal provided in another embodiment of this application;

[0036] Figure 9 is a schematic diagram of the usage state structure of a terminal provided in another embodiment of this application;

[0037] Figure 10 is a schematic diagram of the control principle of a terminal provided in an embodiment of this application;

[0038] Figure 11 is a flowchart illustrating the control module in the terminal provided in an embodiment of this application;

[0039] Figure 12 is a schematic diagram of the circuit principle of the radar module in a terminal provided in an embodiment of this application;

[0040] Figure 13 is a side view of the screen assembly in a terminal provided in an embodiment of this application;

[0041] Figure 14 is a schematic diagram showing the connection relationship between the screen component and the radar module in the terminal provided in the embodiment of this application;

[0042] Figure 15 is a cross-sectional view of a screen assembly in a terminal provided in an embodiment of this application;

[0043] Figure 16 is a cross-sectional view of a screen assembly in a terminal provided in another embodiment of this application;

[0044] Figure 17 is a schematic diagram of the circuit principle of the radar module in a terminal provided in an embodiment of this application;

[0045] Figure 18 is a schematic diagram of the usage state structure of a terminal provided in another embodiment of this application;

[0046] Figure 19 is a schematic diagram of the usage state structure of a terminal provided in another embodiment of this application;

[0047] Figure 20 is a schematic diagram of the control principle of a terminal provided in another embodiment of this application;

[0048] Figure 21 is a schematic flowchart of a terminal sound pickup method provided in an embodiment of this application;

[0049] Figure 22 is a schematic flowchart of a terminal sound pickup method provided in another embodiment of this application;

[0050] Figure 23 is a flowchart illustrating a terminal sound pickup method provided in another embodiment of this application.

[0051] Explanation of reference numerals in the attached figures: 10', condenser microphone; 11', diaphragm; 12', electrode plate; 20', dynamic microphone; 21', diaphragm; 22', coil; 23', magnet; 10, support device; 20, battery; 30, circuit board assembly; 40, screen assembly; 40a, screen body; 40b, metal layer; 41, cathode layer; 411, through-structure; 42, electron injection layer; 43, electron transport layer; 44, light-emitting layer; 45, hole transport layer; 46, hole injection layer; 47, anode layer; 48, panel glass; 41', backlight reflective layer; 42', backlight module; 43', lower polarizer; 44', lower glass substrate; 45', electrode and thin-film transistor; 46', liquid crystal layer; 47', electrode layer; 48', color filter; 49', upper glass substrate; 410', polarizer; 50. Sound pickup module; 51. Radar module; 51a. Radar body; 51b. Antenna; 52. Microphone; 60. Frame; 70. Keyboard; 80. Rear camera; 90. Control module; 91. Fusion module; 92. Voice activity detection module; 93. Noise reduction module; 94. Wind noise detection module; 511. Transmitter; 512. First capacitor; 513. Transmitting antenna; 514. Receiving antenna; 515. Second capacitor; 516. Receiver; 517. Connection structure; 518. Conductive component; m. First surface. Detailed Implementation

[0052] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0053] In the description of this application, it should be understood that the terms "inner" and "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0054] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. For example, the first limiting part and the second limiting part are only used to distinguish different limiting parts and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0055] It should be noted that in this application, the terms "in one embodiment" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "in one embodiment" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "in one embodiment" or "for example" is intended to present the relevant concepts in a specific manner.

[0056] In this application, unless otherwise expressly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments.

[0058] With societal development, smartwatches, mobile phones, laptops, personal computers (PCs), smart cars, wearable devices (smart glasses, smart helmets, etc.), televisions, smart speakers, and car cockpits are increasingly used in various aspects of modern life, work, and communication. To improve user experience, these terminals commonly incorporate microphone modules. These modules can be used not only for general audio capture but also for voice and video calls. Currently, the microphone modules used in terminals are generally microphones, which can be broadly categorized into two types: condenser microphones and dynamic microphones. As shown in Figure 1, the condenser microphone 10' primarily consists of a diaphragm 21' and an electrode plate 12'. The electrode plate 12' is fixedly positioned, and the diaphragm 21' serves as the other electrode plate 12' positioned opposite it. The diaphragm 21', the electrode plate 12', and the air gap between them together form a parallel-plate capacitor, which follows the following formula:

[0059] In the formula, C is the capacitance, S is the area of ​​the diaphragm 21' and the electrode 12' facing each other, Q is the amount of charge stored when the voltage between the diaphragm 21' and the electrode 12' is VC, εr is the dielectric constant of the medium (air) between the diaphragm 21' and the electrode 12', d is the distance between the diaphragm 21' and the electrode 12', and k is an electrostatic constant with a value of 9.0 × 1000000000 N·m. 2 / C 2 The working principle of the condenser microphone 10' is as follows: During use, a stable bias voltage is applied to the parallel plate capacitor. This bias voltage is a direct current (DC) voltage, which keeps the capacitor in a constant charging state, thus ensuring that the amount of charge Q stored between the diaphragm 21' and the electrode 12' remains constant. This function can be achieved by a charge pump. When picking up audio, sound waves enter the terminal through the sound inlet or gap, applying sound pressure F to the diaphragm 21', causing the diaphragm 21' to vibrate. The displacement of the diaphragm 21' causes a change in the distance d between the diaphragm 21' and the electrode 12', thereby causing a change in the capacitance C between them. The change in capacitance C causes a change in the voltage or generates a current between the diaphragm 21' and the electrode 12'. After amplifying the aforementioned voltage or current changes, an electrical signal corresponding to the sound signal can be formed, thus completing the conversion from sound signal to electrical signal.

[0060] The structure of a dynamic microphone 20' is shown in Figure 2. It generally includes a diaphragm 21', a magnet 23' located inside the diaphragm 21', and a coil 22' surrounding the magnet 23'. The coil 22' is electrically connected to an external power supply system and is also connected to the diaphragm 21', moving synchronously with the vibration of the diaphragm 21'. The magnet 23' can be a permanent magnet. The working principle of the dynamic microphone 20' is as follows: When picking up audio, sound waves enter the terminal through the sound inlet or gap, applying a sound pressure F to the diaphragm 21', causing it to vibrate. During this period, the vibration of the diaphragm 21' drives the coil 22' to move relative to the magnet 23'. The movement of the coil 22' cuts the magnetic lines of force generated by the magnet 23', producing an induced electromotive force and induced current related to the sound wave vibration. Amplifying these induced electromotive force and / or induced current generates an electrical signal corresponding to the sound. The circuit diagram corresponding to the above principle is shown in Figure 3.

[0061] Regardless of the type of microphone mentioned above, they are all passive devices. To collect sound signals at the receiving end, air vibrations (i.e., sound waves) are required to conduct to the diaphragm. This necessitates the design of sound inlets or slits on the terminal to allow sound waves to enter. Furthermore, to ensure optimal microphone pickup, multiple microphones and multiple sound inlets or slits are often required in a single terminal. This presents several challenges: first, it compromises the terminal's aesthetic appearance; second, the sound inlets or slits can become clogged by fingers, water, dust, or other impurities, potentially causing the microphone to malfunction; and third, to prevent dust or liquid from entering the terminal through the sound inlets or slits and damaging the microphones, dustproof mesh or membranes are needed within these openings or slits, further increasing the complexity of the pickup structure design.

[0062] Furthermore, to ensure the integrity of the terminal's appearance, the aforementioned sound holes or gaps are generally located at the top, bottom, sides, or hidden in other external structures. For example, in laptops, the sound holes or gaps are typically located on the keyboard surface (C-side) or the front part of the D-side. The D-side refers to the bottom cover of the laptop, the side seen when the laptop is turned upside down. The "front part" refers to the position on the C-side or D-side closest to the user, away from the screen. However, regardless of the design, the microphone cannot be directly pointed at the user's mouth during operation, meaning it cannot be placed in the optimal sound pickup position. This leads to a decrease in microphone pickup performance, especially noticeable in noisy environments. To improve pickup performance, related technologies generally involve placing multiple microphones in the terminal and combining them with sound enhancement algorithms to improve voice quality. This increases the design complexity and cost of the terminal. As people's demands for voice quality continue to increase, sound enhancement algorithms become increasingly complex and computationally intensive. However, in environments with strong noise, strong reverberation, and / or wind, the improvement in voice quality remains limited, and the degradation is significant compared to normal environments.

[0063] To improve, or at least partially improve, the present application provides a terminal. The terminal includes a main body, a microphone module, and a control module. The main body has a first surface facing the user and includes a display surface. The microphone module is disposed within the main body and includes a radar module. The radar module is located inside the first surface. The radar module emits a detection beam toward the first surface, receives and processes echoes passing through the first surface and entering the main body, and outputs a first electrical signal. The control module is communicatively connected to the microphone module. The control module receives the first electrical signal, detects whether it contains Doppler frequency shift data, and analyzes the Doppler frequency shift data to derive an audio signal.

[0064] When a user speaks, the skin around the user's mouth, throat, and jaw vibrates. This vibration, usually very small, causes the Doppler effect in the echoes, resulting in a Doppler frequency shift. The echoes then carry vibration information related to the user's speech, i.e., Doppler frequency shift data. This echo is received and processed by the radar module to generate a first electrical signal, which is transmitted to the control module. The control module analyzes and processes this first electrical signal to extract audio-related features, i.e., the aforementioned Doppler frequency shift data. Then, a Mel-spectrogram (MEL spectrum) can be generated based on the Doppler frequency shift data. Speech reconstruction is then performed using the Mel-spectrogram to generate the corresponding audio signal. Finally, the audio signal can be processed with noise reduction before being output or stored. Therefore, the terminal provided in this application embodiment can use a radar module to replace a microphone to achieve sound pickup. The radar module can be installed inside the terminal, and the terminal surface does not need to have sound inlets or gaps, or has very few sound inlets or gaps, making the terminal aesthetically pleasing and complete, and reducing the complexity of the sound pickup structure design to a certain extent.

[0065] The terminal provided in this application embodiment has a sound pickup function. For ease of explanation, the following embodiments use a mobile phone as an example.

[0066] As shown in Figures 4 and 5, a mobile phone generally includes a support device 10, a battery 20, a circuit board assembly 30, a screen assembly 40, and a microphone module 50. The support device 10, the battery 20, at least a portion of the circuit board assembly 30, and the screen assembly 40 form the main body of the mobile phone. This main body has a first surface, which faces the user and includes a display surface.

[0067] The support device 10 may include at least one frame. When the support device 10 includes multiple frames, adjacent frames can be connected by a hinge structure. The screen assembly 40 generally covers one side of the support device 10, but it can also be provided on both sides of the support device 10, depending on the application requirements. When the screen assembly 40 covers one side of the support device 10, a housing is generally installed on the other side of the support device 10. The housing and the screen assembly 40 can form a closed space, within which the battery 20, circuit board assembly 30, and microphone module 50 are installed. In this case, the side of the screen assembly 40 facing away from the support device 10 is also the display side. If the screen assembly 40 can completely cover an entire side of the support device 10, then the display side is the first side. If the screen assembly 40 cannot completely cover the entire side of the support device 10, a frame 60 surrounding the screen assembly 40 can be provided on the support device 10, as shown in Figure 6. At this time, the front of the bezel 60 and the display surface together form the front of the phone. The front of the phone is the first surface mentioned above. That is, the first surface includes not only the display surface mentioned above, but also the front of the bezel 60.

[0068] Battery 20 provides power to various electronic components and power modules in the mobile phone (such as circuit board assembly 30, screen assembly 40, and microphone module 50). There may be one or more batteries, depending on the usage requirements. Circuit board assembly 30 may include a motherboard assembly and a sub-board assembly. Both the motherboard assembly and the sub-board assembly generally include circuit boards and electronic components mounted on the circuit boards. Electronic components may include, but are not limited to, System on Chip (SoC), UFS (Uniform File System), Computer Aided Manufacturing (CAM), Universal Serial Bus (USB) interface, Subscriber Identity Module (SIM) card, antenna module, Bluetooth module, Wi-Fi module, Global Positioning System (GPS) module, power supply, charging module, screen display and operation module, etc. Screen assembly 40 is generally electrically connected to the aforementioned screen display and operation module to enable screen assembly 40 to perform display or operation functions.

[0069] The microphone module 50 can communicate with at least one chip in the circuit board assembly 30 to cooperate with the circuit board assembly 30 in picking up external audio signals. The number of microphone modules 50 can be one or more, depending on the application requirements. The communication connection can be a wired connection via a wire or a wireless connection via a wireless communication module.

[0070] As shown in Figure 7, the microphone module 50 includes a radar module 51. The radar module 51 is generally located inside the first surface m. It can emit a detection beam towards the first surface m, receive echoes passing through the first surface m and entering the main body, process the echoes to generate a first electrical signal, and output the first electrical signal. The radar module 51 can be located on the back of the screen assembly 40, inside the screen assembly 40, on the support device 10, on the frame 60, or on other components of the terminal. The specific location of the radar module 51 can be flexibly adjusted according to the size of the terminal, as long as the radar module 51 can detect vibrations at the user's mouth, throat, or other positions facing the screen assembly 40. For example, in a mobile phone, the radar module 51 can be located at the bottom or middle of the screen assembly 40, so that when the user holds the phone to make a call, the detection beam emitted by the radar module 51 can be directed towards the user's throat and mouth, as shown in Figure 8. In a laptop computer, the radar module 51 can be designed on the back of the screen assembly 40, so that when the user opens the laptop, the detection beam emitted by the radar module 51 is directed towards the user, as shown in Figure 9.

[0071] Understandably, in order for the detection beam emitted by the radar module 51 to penetrate the first surface m, the material in the mobile phone located between the aforementioned detection beams is generally made of a material that allows the detection beam to pass through, such as glass, plastic, composite materials, etc. Metal materials are generally not used, and the specific choice can be determined according to the penetration performance of the detection beam.

[0072] One or more radar modules 51 can be installed in the same terminal. The radar module 51 is communicatively connected to the circuit board assembly 30, specifically to the system chip and / or other chips in the circuit board assembly 30. The system chip and / or other chips can form a control module, or a module from any of the aforementioned chips can form a control module. This control module can receive the first electrical signal output by the radar module 51, detect whether it contains Doppler frequency shift data, and analyze the Doppler frequency shift data to derive the audio signal.

[0073] As shown in Figure 10, a radar module 51 generally includes a radar body 51a and an antenna 51b connected to the radar body 51a. The antenna 51b can be a transmitting antenna with transmitting function, a receiving antenna with receiving function, or a transceiver antenna with both transmitting and receiving functions, depending on the function and type of the radar module 51. It is understood that when the antenna 51b is a transmitting antenna or a receiving antenna, the same radar module 51 has at least one transmitting antenna and at least one receiving antenna; when the antenna 51b is a transceiver antenna, the same radar module 51 can have one or more transceiver antennas.

[0074] For ease of understanding, the sound pickup principle of the terminal provided in this application embodiment will be explained using radar module 51, which includes a transmitting antenna and a receiving antenna, as an example:

[0075] When a user uses the terminal, they typically face the first surface m. The radar module 51 emits a detection beam towards the first surface m via its transmitting antenna. This beam passes through the first surface m and exits the terminal. Upon contact with the user's skin, the beam creates an echo. This echo passes through the first surface m and returns to the main body of the terminal, where it is received by the receiving antenna of the radar module 51 and transmitted to the radar body 51a. The radar body 51a then processes the received echo to generate a first electrical signal. When the user speaks, the skin around the mouth, throat, and jaw vibrates. This vibration is generally very small and causes the echo to experience a Doppler effect, resulting in a Doppler frequency shift. Thus, the echo carries vibration information related to the user's speech, i.e., Doppler frequency shift data. After the echo is received and processed by the radar module 51, a first electrical signal is generated and transmitted to the control module. The control module analyzes and processes the first electrical signal to extract audio-related features, namely the aforementioned Doppler frequency shift data. Then, a Mel-spectrogram (mel spectrum) can be generated based on the Doppler frequency shift data. Speech recovery is then performed using the Mel-spectrogram to generate the corresponding audio signal. Finally, the audio signal can be processed such as noise reduction before being output or stored, as shown in Figure 11.

[0076] The functions of the aforementioned control module 90 can be implemented based on deep neural network (DNN) networks (such as generative adversarial networks (GANs), which require pre-training), large model techniques, and signal processing techniques.

[0077] Understandably, when multiple radar modules 51 are provided, different radar modules 51 can be positioned at different locations on the main body of the terminal. For example, if two radar modules 51 are provided, one radar module 51 can be positioned at the bottom of the main body to detect vibration information caused by the skin vibration of the throat when the user speaks, and the other radar module 51 can be positioned in the middle of the main body to detect vibration information caused by the skin vibration of the mouth or jaw when the user speaks. Similarly, when the radar module 51 is provided with multiple transmitting antennas and multiple receiving antennas, one transmitting antenna and one receiving antenna can form a transceiver antenna group. In this way, the radar module 51 has multiple transceiver antenna groups, and different transceiver antenna groups can be positioned at different locations on the main body of the terminal. For example, if two transceiver antenna groups are provided, one transceiver antenna group can be positioned at the bottom of the main body to detect vibration information caused by the skin vibration of the throat when the user speaks, and the other transceiver antenna group can be positioned in the middle of the main body to detect vibration information caused by the skin vibration of the mouth or jaw when the user speaks.

[0078] The terminal provided in this application embodiment can replace the microphone 52 with a radar module 51 to achieve sound pickup. The radar module 51 can be installed inside the terminal, and the surface of the terminal does not need to have sound inlets or gaps, or has fewer sound inlets or gaps, making the terminal look aesthetically pleasing and complete, and reducing the complexity of the sound pickup structure design to a certain extent. In addition, since the radar module 51 is located inside the terminal, it does not need to be placed on the side or bottom of the terminal for aesthetic reasons, and can be positioned directly facing the mouth. That is, the radar module 51 can be placed in an optimal sound pickup position to better pick up the user's voice. Therefore, the terminal provided in this embodiment can improve the sound pickup quality to a certain extent.

[0079] As shown in Figure 12, radar module 51 generally includes a transmitting module and a receiving module. The transmitting module includes at least a transmitter 511 and a transmitting antenna 513. The transmitter 511 is used to provide optical signals, and the transmitting antenna 513 is used to receive optical signals and convert them into a detection beam. The receiving module includes at least a receiving antenna 514 and a receiver 516. The receiving antenna 514 is used to receive echoes and conduct them to the receiver 516. The receiver 516 is used to receive and process the echoes conducted by the receiving antenna 514 and generate the aforementioned first electrical signal. The transmitting antenna 513 and the receiving antenna 514 can be a single antenna or can be set separately, depending on the application requirements. When radar module 51 is a coaxial radar module 51, the transmitting antenna 513 and the receiving antenna 514 are a single antenna, which can be called a transceiver antenna; when radar module 51 is a non-coaxial radar module 51, such as a parallel-axis radar module 51, the transmitting antenna 513 and the receiving antenna 514 are set separately, forming two antennas.

[0080] Since the user's face is generally facing the screen assembly 40 during use, in order for the detection beam emitted by the radar module 51 to reach the parts of the body that vibrate when the user speaks, such as the mouth, throat, and jaw, the entire structure of the radar module 51 can be mounted on the screen assembly 40. Alternatively, only the transmitting antenna 513 of the radar module 51 can be mounted on the screen assembly 40, or both the transmitting antenna 513 and the receiving antenna 514 of the radar module 51 can be mounted on the screen assembly 40, depending on the usage requirements. Using the solution provided in this embodiment, the detection beam emitted by the radar module 51 can reach the parts of the body that vibrate when the user speaks, such as the mouth, throat, and jaw, thereby better receiving the vibration information related to the user's speech, resulting in better audio quality ultimately picked up by the terminal.

[0081] As shown in Figure 13, many screen components 40 include a screen body 40a and a metal layer 40b, with the metal layer 40b disposed on the side of the screen body 40a facing the support device 10. Since the detection beam emitted by the radar module 51 generally has difficulty penetrating the metal layer 40b, the antenna in the radar module 51 (which can be a transceiver antenna, a transmitting antenna 513, or a receiving antenna 514) can be disposed on the screen body 40a or the metal layer 40b. It is understood that when the antenna in the radar module 51 is disposed on the metal layer 40b, it can be disposed on the side of the metal layer 40b closest to the screen body 40a, or it can be embedded in the metal layer 40b, as long as the signal emitted by the antenna and the signal to be received are not blocked by the metal layer 40b. This allows the detection beam emitted by the radar module 51 to pass through the corresponding part of the screen component 40 more easily without being blocked by the metal layer 40b in the screen component 40, while also making the terminal structure compact and facilitating its miniaturization design.

[0082] As shown in Figure 14, in some embodiments, the metal layer 40b is provided with a through structure 411 extending through itself along the thickness direction. At least one through structure 411 is provided. All through structures 411 form a surrounding structure with at least one opening. If there is only one through structure 411, it can be a long strip structure, bent or folded, forming a rectangular structure, annular structure, etc., with one opening. When there are multiple through structures 411, each through structure 411 can be a strip structure, which can be a straight structure or a bent structure. An opening can be formed between adjacent through structures 411, thus multiple through structures 411 can form a surrounding structure with multiple openings. The surrounding structure can be rectangular, annular, etc.

[0083] The surrounding structure divides the metal layer 40b into a first part and a second part interconnected by a connecting structure 517. The connecting structure 517 is the structure located at the opening in the metal layer 40b. The first part is the portion of the metal layer 40b surrounded by the through structure 411. This first part forms an antenna. This antenna can be a transmitting antenna 513, a receiving antenna 514, or a transceiver antenna, depending on the type and structure of the radar module 51. The second part is the portion of the metal layer 40b excluding the first part and the connecting structure 517.

[0084] It is understandable that although the first part can be used as an antenna for the radar module 51, its function in the screen assembly 40 is the same as that of the second part, that is, it can cooperate with the structure of the corresponding area in the main screen 40a for display. The through structure 411 can be a regular shape or an irregular shape, depending on the type, shape, structure of the antenna and the display needs of the screen assembly 40. For example, if the outline of the inner side of the through structure 411 forms the outer outline of the antenna, the outline of the inner side of the through structure 411 should be processed according to the shape of the antenna, and its size can be determined according to the operating frequency of the radar module 51 (usually above 20GHz), such as by calculating the wavelength of the detection beam emitted by the radar module 51, for example, by calculating according to the half-wavelength principle or other antenna design theories. As shown in Figure 14, if the first part enclosed by the metal layer 40b is designed as a rectangle with an opening, the diagonal length of the rectangle can be set as half a wavelength. Of course, the first part enclosed by the metal layer 40b can be designed in other shapes (such as circles, irregular shapes, etc.), and its size can be determined according to antenna design theory. The width of the through structure 411 can be determined according to the display requirements of the screen assembly 40 and the circuit design requirements between the antenna and the second part.

[0085] The number, width, and length of the connection structure 517 determine its inductance, which can be calculated using a formula. This formula refers to the formula for calculating the parasitic inductance of a PCB (Printed Circuit Board). During design, the inductance of the connection structure 517 can be made to reach a relatively large value within the operating frequency band of the radar module 51. This allows the first part of the metal layer 40b, which serves as the antenna, to be approximately disconnected from other parts (i.e., the second part) within the operating frequency band of the radar module 51. This allows the detection beam emitted by the radar module 51 to be directly emitted through the transmitting antenna 513, minimizing its reach to the second part of the metal layer 40b, thus improving the transmission efficiency of the radar module 51 to some extent. It also minimizes the conduction of the echo received by the receiving antenna 514 to the second part of the metal layer 40b, thereby improving the receiving sensitivity of the radar module 51 to some extent.

[0086] The antenna adopts the solution provided in this embodiment, which can be made using a part of the screen assembly 40. This can reduce the number of components in the terminal to a certain extent, without increasing the thickness of the screen assembly 40 or affecting the display effect of the screen assembly 40. This facilitates the miniaturization design of the terminal and can also reduce the production cost of the terminal to a certain extent. At the same time, it achieves the purpose of directing the detection beam emitted by the radar module 51 to the user, achieving multiple benefits in one fell swoop.

[0087] In some embodiments, the through structure 411 is a strip structure and is provided. This structure simplifies the antenna fabrication process and facilitates manufacturing.

[0088] Understandably, in order to minimize the impact of the through structure 411 on the display function of the screen component 40 and ensure that the screen component 40 can display normally, the width a of the through structure 411 is generally set to be relatively small. For example, the width a of the through structure 411 can be less than or equal to 10 micrometers or less than or equal to 20 micrometers. The specific width a can be determined according to the shape of the through structure 411, circuit design requirements, etc., and no single limitation is made here.

[0089] Since screen components 40 come in various types, the aforementioned screen body 40a and metal layer 40b can also be configured in various ways. For example, in some embodiments, screen component 40 is an OLED (Organic Light-Emitting Diode, also known as organic electro-laser display or organic light-emitting semiconductor) display module. As shown in Figure 15, the OLED display module includes a cathode layer 41, an electron injection layer 42, an electron transport layer 43, a light-emitting layer 44, a hole transport layer 45, a hole injection layer 46, an anode layer 47, and a panel glass 48, which are stacked sequentially from the inside out. The cathode layer 41 is the aforementioned metal layer 40b, and the other parts of the OLED screen component 40, except for the cathode layer 41, constitute the aforementioned screen body 40a. In this embodiment, the antenna can be fabricated using the aforementioned through-structure 411. Since the width of the through-structure 411 is very small, its configuration will not affect the normal display of the screen component 40. This allows a terminal with an OLED screen to be integrated with the radar module 51, thereby enabling the corresponding terminal to achieve sound pickup through the radar module 51.

[0090] As shown in Figure 16, in some embodiments, the screen assembly 40 includes an LCD (Liquid Crystal Display), comprising, from the inside out, a backlight reflective layer 41', a backlight module 42', a lower polarizer 43', a lower glass substrate 44', electrodes and thin-film transistors 45', a liquid crystal layer 46', an electrode layer 47', a color filter 48', an upper glass substrate 49', and a polarizer 410'. The backlight reflective layer 41' is the aforementioned metal layer 40b. The other parts of the screen assembly 40, except for the backlight reflective layer 41', constitute the aforementioned screen body 40a. In this embodiment, the antenna can be fabricated using the aforementioned through-structure 411. Since the width of the through-structure 411 is very small, it will not affect the reflection effect of the backlight reflective layer 41'. This allows a terminal with an LCD screen to be integrated with a radar module 51, thereby enabling the corresponding terminal to achieve sound pickup through the radar module 51.

[0091] In other embodiments, the screen component 40 may also be configured in other ways, which are not limited here.

[0092] Regardless of the configuration of the screen assembly 40, when the antenna is configured using the through-structure 411 described above, since the antenna is part of the metal layer 40b, in order to reduce the risk of the detection beam being conducted to the second part of the metal layer 40b, in some embodiments, as shown in FIG17, the radar module 51 includes a transmitting module and a receiving module. The transmitting module includes a transmitter 511, a first capacitor 512, and a transmitting antenna 513 connected in sequence. The receiving module includes a receiving antenna 514, a second capacitor 515, and a receiver 516 connected in sequence.

[0093] In this configuration, both the first capacitor 512 and the second capacitor 515 are isolation capacitors with typically very small capacitance values, often reaching the pF level. When the radar module 51 is a millimeter-wave radar, the impedance of the first capacitor 512 and the second capacitor 515 is very low in the operating frequency band of the radar module 51, approximating a short circuit, while the impedance is very high in the display drive signal frequency band, equivalent to an open circuit. Thus, the first capacitor 512 and the second capacitor 515 can, to a certain extent, prevent the display drive signal from the screen assembly 40 from flowing into the transmitter 511 or receiver 516 of the radar module 51. This can, to a certain extent, improve the transmission efficiency of the transmitting antenna 513 and also, to a certain extent, improve the receiving sensitivity of the receiving antenna 514.

[0094] The aforementioned radar module 51 can be a millimeter-wave radar, the aforementioned transmitter 511 can be a millimeter-wave radar transmitting power amplifier, and the aforementioned receiver 516 can be a millimeter-wave radar receiving amplifier. In use, the signal output from the transmitting power amplifier is connected to the transmitting antenna 513 via the first capacitor 512, and the receiving antenna 514 is connected to the millimeter-wave radar receiving amplifier via the second capacitor 515. Connection methods include welding, bonding, and spring-loaded connection.

[0095] The first capacitor 512 and the second capacitor 515 are respectively soldered to the corresponding antennas via conductive components 518. The conductive components 518 can be wires, metal sheets, or other components capable of conducting electricity.

[0096] In addition to being mounted on the screen assembly 40, in some embodiments, the radar module 51 can also be mounted on the bezel 60 surrounding the screen assembly 40. This ensures that the mounting of the radar module 51 does not damage the internal structure of the screen assembly 40 or affect the normal display of the screen assembly 40, and facilitates the installation of the radar module 51.

[0097] The microphone module in the terminal provided in this application embodiment may include only one or more radar modules 51, or it may include both one or more radar modules 51 and one or more microphones 52, as shown in Figure 18. The microphones 52 are generally located inside the main body of the terminal. To facilitate microphone pickup, a sound-entry structure is typically provided near the microphones 52. This sound-entry structure connects the external space and the internal space of the main body, allowing sound waves outside the terminal to pass through and propagate to the location of the microphones 52, where they are picked up. The aforementioned sound-entry structure includes a sound-entry hole and / or gap penetrating the phone casing, frame 60, and / or screen assembly 40 along the thickness direction. As shown in Figure 18, at least one microphone 52 and a corresponding sound-entry hole can be provided at the top and bottom of the phone, respectively. A radar module 51 is provided near the bottom of the screen assembly 40. In use, the user's voice can be picked up through multiple microphones 52 and at least one radar module 51. Other configuration methods can also be used. As shown in Figure 19, a radar module 51 can be installed on the back of the laptop screen assembly 40, and a microphone 52 and a corresponding sound inlet can be installed on the front of the laptop keyboard 70. When the user opens the laptop, the detection beam emitted by the radar module 51 is directed towards the user, and the microphone 52 is positioned close to the user. The front of the keyboard 70 is the end of the keyboard 70 that is furthest from the screen assembly 40 and closest to the user when the laptop is open.

[0098] This reduces the number of microphones 52 in the terminal, thereby reducing the number of sound-entry structures such as sound holes or gaps, resulting in a more aesthetically pleasing and complete appearance. Furthermore, since microphones convert all sound signals at their location into electrical signals, it's impossible to determine which signal is the primary sound source. Using the solution provided in this embodiment, the microphone 52 and radar module 51 can work together. The radar module 51 identifies the primary sound source, allowing the filtering out of audio signals from non-primary sound sources acquired by the microphone 52. Simultaneously, the audio signal from the primary sound source acquired by the microphone 52 can be combined with the audio signal acquired by the radar module 51 using AI fusion and noise reduction technologies to achieve joint noise reduction, resulting in a lower noise level in the final audio signal obtained by the terminal.

[0099] In some embodiments, the sound pickup module includes both a radar module 51 and a microphone 52, and both the radar module 51 and the microphone 52 can be used to pick up audio signals. To better fuse the audio signals picked up by the two, in some embodiments, as shown in FIG20, the control module 90 includes a fusion module 91. The fusion module 91 is communicatively connected to the microphone 52 and the radar module 51 respectively. The fusion module 91 is used to receive and process the first electrical signal and the second electrical signal output by the microphone 52 to generate an audio signal.

[0100] The fusion module 91 can be a functional module in some control chips in the terminal (such as central processing unit (CPU), digital signal processing chip (DSP), network processing unit (NPU) etc.). For example, it can be an AI noise reduction network module. Based on the theory of computational auditory scene analysis, it can apply deep learning technology and use deep neural networks to build a noise reduction model. After training with massive corpora, it can separate human voice and noise, effectively suppress various noises in the environment, effectively deal with sudden non-stationary noise, and the voice distortion is relatively much smaller, which can greatly ensure the sound reproduction and improve the call experience.

[0101] Taking a mobile phone as an example, when a user speaks while holding the phone, the microphone 52 and the radar module 51 operate independently. Since the detection beam emitted by the radar module 51 is directed towards the throat, mouth, and jaw, the vibrations of the skin surface, vocal cords, and other tissues during the user's speech cause a Doppler frequency shift in the echo signal. Therefore, the echo signal carries information related to the user's speech, i.e., Doppler frequency shift data. The microphone 52 can convert the vibration of the diaphragm 21' caused by the sound wave into an electrical signal, i.e., a second electrical signal, which is then transmitted to the fusion module 91. The fusion module 91 can perform noise reduction and fusion processing on the Doppler frequency shift data and the second electrical signal to generate a lower-noise audio signal.

[0102] The solution provided in this embodiment allows for effective fusion of data acquired by the radar module 51 and the microphone 52, resulting in a low-noise audio signal. Compared to the microphone-only pickup module 50, this embodiment adds an active pickup mode, enabling the radar module 51 to collect vibration signals from the sound source and combine them with the sound signals collected by the microphone for noise reduction, thereby significantly improving the noise reduction effect.

[0103] To ensure accurate operation of the control module 90, in some embodiments, the control module 90 further includes a voice activity detection module 92, which is communicatively connected to the radar module 51 and the fusion module 91, respectively. The voice activity detection module 92 receives a first electrical signal and analyzes whether Doppler frequency shift data exists in the first electrical signal. The fusion module 91 receives the analysis signal output by the voice activity detection module 92 and processes the first and second electrical signals based on the analysis signal.

[0104] The voice activity detection module 92 can be a functional module of a chip in the terminal (such as a central processing unit, digital signal processing chip, network processor, etc.), such as a program. The voice activity detection module 92 is used to confirm whether the user is speaking. Since vibration only occurs when the user speaks, confirming whether the user is speaking by detecting vibration can be achieved by analyzing whether Doppler frequency shift data exists in the first electrical signal. The specific analysis method is not limited in this application. For example, it can be designed such that when the user speaks, the voice activity detection module 92 outputs a specific signal (e.g., 1), and when the user is not speaking, the voice activity detection module 92 outputs a specific signal (e.g., 0). When the fusion module 91 receives the specific signal 1 corresponding to the user speaking, it analyzes and processes the first and second electrical signals. When the fusion module 91 receives the specific signal 0 corresponding to the user not speaking, it deletes the first and second electrical signals. This allows the fusion module 91 to filter out some sound signals that are not emitted by the user, thus achieving noise reduction. Compared with sound pickup through a microphone, it can filter out ambient sounds other than the user's (such as people next to the user, ambient noise, etc.), making the audio signal acquired by the terminal more accurate. At the same time, it can reduce the number of calculations by the fusion module 91 to a certain extent and improve the working efficiency of the control module 90.

[0105] In some embodiments, the control module 90 further includes a noise reduction module 93, which is communicatively connected to the microphone and the fusion module 91 respectively. The noise reduction module 93 is used to receive the second electrical signal, perform noise reduction processing on the second electrical signal, and transmit the noise-reduced second electrical signal to the fusion module 91.

[0106] The noise reduction module 93 can be a functional module of a chip in the terminal (such as a central processing unit, digital signal processing chip, network processor, etc.). For example, it can be an AI noise reduction module or other noise reduction modules, such as a multi-microphone noise reduction module or a microphone array module. The specific module can be determined according to the usage requirements.

[0107] By adopting the solution provided in this embodiment, the second electrical signal can be denoised once before being transmitted to the fusion module 91, thereby reducing the calculation steps of the fusion module 91 and improving the quality of the audio signal output by the fusion module 91.

[0108] In some embodiments, multiple microphones 52 are provided. The noise reduction module 93 is communicatively connected to each of the multiple microphones 52. The noise reduction module 93 is used to receive second electrical signals output by the multiple microphones and to perform noise reduction based on the multiple second electrical signals.

[0109] The noise reduction module 93 is generally a functional module of a chip in the terminal (such as a central processing unit, digital signal processing chip, network processor, etc.). The corresponding microphone array noise reduction module can be selected according to the number of microphones. For example, the noise reduction module 93 can obtain the pickup range of different microphones, determine the overlapping area of ​​the pickup range of different microphones, output the audio signal in the overlapping area as the final audio signal, and remove the audio signal outside the overlapping area to achieve noise reduction.

[0110] The noise reduction technology provided in this embodiment is mature and easy to design.

[0111] In some embodiments, the control module 90 further includes a wind noise detection module 94. The wind noise detection module 94 is communicatively connected to the microphone 52, the noise reduction module 93, and the fusion module 91, respectively. The wind noise detection module 94 receives the second electrical signal output by the microphone 52, processes the second electrical signal to obtain wind noise data related to wind sound, and transmits the wind noise data to the noise reduction module 93 and the fusion module 91. The noise reduction module 93 is also used to receive the wind noise data and perform noise reduction processing on the second electrical signal based on the wind noise data. The fusion module 91 is also used to perform noise reduction processing on the first electrical signal and the second electrical signal based on the wind noise data.

[0112] The wind noise detection module 94 is generally a functional module of a chip in the terminal (such as a central processing unit, digital signal processing chip, network processor, etc.). It is used to detect whether there is wind-related data in the data acquired by the microphone 52, and also to determine the frequency range and intensity of the wind sound. The wind noise detection module 94 can achieve the above functions in various ways, such as receiving low-frequency signals picked up by the microphone 52 and generating a wearing result signal from the low-frequency signals.

[0113] The fusion module 91 is an AI noise reduction network that receives the outputs of the wind noise detection module 94 and the voice activity detection module 92, and further removes noise from the above data based on the noise reduction of the noise reduction module.

[0114] Using the solution provided in this embodiment, the wind noise detection module 94 can detect whether there is wind noise in the audio signal acquired by the microphone, so that the noise reduction module 93 can perform noise reduction processing on the second electrical signal transmitted by the microphone according to the corresponding detection results. At the same time, the fusion module 91 can perform noise reduction processing on the electrical signal and the first electrical signal transmitted by the noise reduction module 93 according to the corresponding detection results, so that the noise reduction effect of the audio signal output by the fusion module 91 is better.

[0115] In some embodiments, radar module 51 is a millimeter-wave radar.

[0116] Millimeter-wave radar is a type of radar that operates in the millimeter-wave band. Millimeter waves typically refer to the 30–300 GHz frequency range (wavelength 1–10 mm). Since the wavelength of millimeter waves falls between microwaves and centimeter waves, millimeter-wave radar combines some advantages of both microwave and electro-optical radars, such as high resolution, strong penetration, ease of integration, and high security. Radar module 51, employing millimeter-wave radar, is suitable for various application scenarios.

[0117] Another embodiment of this application provides a terminal sound pickup method. As shown in FIG21, this terminal sound pickup method, based on the terminal provided in any of the above embodiments, includes at least the following steps:

[0118] S1. Acquire Doppler frequency shift data related to the user's speech through the radar module.

[0119] As mentioned above, the Doppler frequency shift data in this step can be carried in the echo received by the radar module, and the control module can obtain it by analyzing and processing the first electrical signal output by the radar module.

[0120] S2. Audio signals are obtained through Doppler frequency shift data analysis.

[0121] The Mel spectrum can be obtained by processing Doppler shift data, and then the audio data can be obtained from the Mel spectrum.

[0122] The terminal sound pickup method provided in this embodiment can obtain the user's voice by acquiring the vibration of the user's skin (such as the mouth, throat, jaw, etc.) when the user speaks. Compared with acquiring it through a microphone, it can reduce the number of sound holes or gaps to be opened on the terminal surface, making the terminal surface complete and beautiful. It can also reduce the complexity of the sound pickup structure design to a certain extent, and at the same time, it can reduce the noise in the audio signal to a certain extent, such as the wind noise in the environment and the voices of people next to the user.

[0123] As shown in Figure 22, step S2 above includes at least the following steps:

[0124] S21, Receive the second electrical signal output from the microphone.

[0125] This step can be achieved by connecting the microphone to the control module via a wired or wireless connection.

[0126] S22. The audio signal is obtained by analyzing the second electrical signal and Doppler frequency shift data.

[0127] As mentioned above, a fusion module can be set in the control module to reduce noise and fuse the second electrical signal and Doppler frequency shift data through AI noise reduction in order to obtain the final audio signal.

[0128] Since the microphone converts all sound signals at its location into electrical signals (i.e., the second electrical signal), it is impossible to determine which signal is the main sound source. By using the terminal sound pickup method provided in this embodiment, the vibration data related to sound acquired by the radar module is fused with the sound data acquired by the microphone. This can filter out sounds not emitted by the user, resulting in less noise in the acquired audio signal and helping to improve the quality of the audio signal.

[0129] As shown in Figure 23, at least the following steps are included before step S22:

[0130] S3. Determine whether the distance between the radar module and the user is within the preset distance.

[0131] This step can be achieved using the ranging function of the radar module. The preset distance can be determined according to usage needs. For example, when the terminal is a mobile phone, the distance between the user and the phone is generally within 20cm, so the preset distance can be 20cm; when the terminal is a laptop, the distance between the user and the laptop screen is generally within 50cm, so the preset distance can be 50cm.

[0132] S4. If the distance between the radar module and the user is not within the preset distance, then eliminate the Doppler shift data.

[0133] Steps S3 and S4 can be set between steps S1 and S2, or between steps S21 and S22, depending on the specific needs.

[0134] By using the solution provided in this embodiment, a preset distance can be defined based on the general distance between the user and the terminal during use. This ensures that Doppler frequency shift data caused by objects located outside the preset distance will not be mistaken for user voice-related data, thereby achieving the removal of some noise and making the acquired audio information more accurate.

[0135] Finally, it should be noted that the above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A terminal, characterized in that, include: The main body has a first surface, which faces the user, and the first surface includes a display surface. A sound pickup module is disposed inside the main body. The sound pickup module includes a radar module located inside the first surface. The radar module is used to emit a detection beam toward the first surface, and is also used to receive and process the echo that passes through the first surface and enters the main body, and output a first electrical signal to the outside. as well as The control module is communicatively connected to the pickup module. The control module is used to receive the first electrical signal, detect whether it contains Doppler frequency shift data, and analyze the audio signal based on the Doppler frequency shift data.

2. The terminal according to claim 1, characterized in that, The main body includes a support device and a screen assembly disposed on the support device, the display surface being the side of the screen assembly facing away from the support device, and at least a portion of the radar module being disposed on the screen assembly.

3. The terminal according to claim 2, characterized in that, The screen assembly includes a screen body and a metal layer. The metal layer is disposed on the side of the screen body facing the support device, and the antenna of the radar module is disposed on the screen body or the metal layer.

4. The terminal according to claim 3, characterized in that, The metal layer has a through structure extending through itself along the thickness direction. There is at least one through structure. All the through structures form a surrounding structure with at least one opening. The surrounding structure divides the metal layer into a first part and a second part that are interconnected by a connecting structure. The connecting structure is the structure in the metal layer located at the opening. The first part is the part of the metal layer surrounded by the through structure and forms the antenna. The second part is the part of the metal layer excluding the first part and the connecting structure.

5. The terminal according to claim 4, characterized in that, The through structure is a single, strip-shaped structure.

6. The terminal according to any one of claims 3-5, characterized in that, The screen assembly includes an organic light-emitting diode (OLED) display module, and the metal layer is the cathode layer of the OLED display module.

7. The terminal according to any one of claims 3-5, characterized in that, The screen assembly includes a liquid crystal display, and the metal layer is the reflective layer of the liquid crystal display.

8. The terminal according to any one of claims 3-7, characterized in that, The radar module includes a transmitting module and a receiving module. The transmitting module includes a transmitter, a first capacitor, and a transmitting antenna connected in sequence. The receiving module includes a receiving antenna, a second capacitor, and a receiver connected in sequence.

9. The terminal according to claim 1, characterized in that, The main body includes a support device, a screen assembly disposed on the support device, and a frame surrounding the screen assembly. The display surface is the side of the screen assembly facing away from the support device, and at least a portion of the radar module is disposed on the frame.

10. The terminal according to any one of claims 1-9, characterized in that, The sound pickup module also includes a microphone, which is disposed inside the main body. The main body has a sound intake structure that connects the external space of the main body with the internal space of the main body. The sound intake structure is used to allow sound waves to pass through so that they can be picked up by the microphone.

11. The terminal according to claim 10, characterized in that, The control module includes a fusion module, which is communicatively connected to the microphone and the radar module respectively. The fusion module is used to receive and process the second electrical signal output by the microphone and the first electrical signal to generate an audio signal.

12. The terminal according to claim 11, characterized in that, The control module further includes a voice activity detection module, which is communicatively connected to the radar module and the fusion module respectively. The voice activity detection module is used to receive the first electrical signal and analyze whether there is Doppler frequency shift data in the first electrical signal. The fusion module is used to receive the analysis signal output by the voice activity detection module and process the first electrical signal and the second electrical signal according to the analysis signal.

13. The terminal according to claim 11 or 12, characterized in that, The control module further includes a noise reduction module, which is communicatively connected to the microphone and the fusion module respectively. The noise reduction module is used to receive the second electrical signal, perform noise reduction processing on the second electrical signal, and transmit the noise-reduced second electrical signal to the fusion module.

14. The terminal according to claim 13, characterized in that, The microphone is provided in multiple ways, and the noise reduction module is communicatively connected to each of the microphones. The noise reduction module is used to receive the second electrical signals output by the multiple microphones and to perform noise reduction based on the multiple second electrical signals.

15. The terminal according to claim 13 or 14, characterized in that, The control module further includes a wind noise detection module, which is communicatively connected to the microphone, the noise reduction module, and the fusion module. The wind noise detection module receives a second electrical signal output by the microphone, processes the second electrical signal to obtain wind noise data related to wind sound, and transmits the wind noise data to the noise reduction module and the fusion module. The noise reduction module also receives the wind noise data and performs noise reduction processing on the second electrical signal based on the wind noise data. The fusion module also performs noise reduction processing on the first electrical signal and the second electrical signal based on the wind noise data.

16. The terminal according to any one of claims 1-15, characterized in that, The radar module is a millimeter-wave radar.

17. A terminal sound pickup method, based on the terminal according to any one of claims 1-16, characterized in that, include: The radar module acquires Doppler frequency shift data related to the user's speech. The audio signal is obtained by analyzing the Doppler frequency shift data.

18. The terminal sound pickup method according to claim 17, characterized in that, The audio signal obtained through the analysis of the Doppler frequency shift data includes: Receive the second electrical signal output from the microphone; The audio signal is obtained by analyzing the second electrical signal and the Doppler frequency shift data.

19. The terminal sound pickup method according to claim 18, characterized in that, Before obtaining the audio signal based on the second electrical signal and the Doppler frequency shift data, the method further includes: Determine whether the distance between the radar module and the user is within a preset distance; If the distance between the radar module and the user is not within the preset distance, the first electrical signal is eliminated.

Images