Terminal and terminal pickup method

Abstract

The invention provides a terminal and a terminal pickup method. The terminal comprises a main body part, a pickup module and a control module. Wherein the main body part is provided with a first surface, the first surface is a surface facing a user, and the first surface comprises a display surface. The pickup module is arranged in the main body part, and the pickup module comprises a radar module. The radar module is located inside the first surface. The radar module is used for emitting a detection wave beam towards the first surface, and is also used for receiving and processing echoes entering the main body part through the first surface, and outputting a first electric signal to the outside. The control module is in communication connection with the pickup module. The control module is used for receiving the first electric signal, detecting whether the first electric signal contains Doppler frequency shift data or not, and obtaining an audio signal through analysis according to the Doppler frequency shift data. According to the terminal and the terminal pickup method provided by the invention, the design complexity of a pickup structure can be reduced to a certain extent.

Description

Terminal and terminal sound pickup method

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on March 5, 2024, with application number 202410251577.3 and application name “Terminal and Terminal Sound Pickup Method”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminals, and in particular to a terminal and a terminal sound pickup method. Background Art

[0003] To enhance the user experience, terminals are often equipped with sound pickup modules. These modules can be used not only for general audio capture but also for voice and video calls. Currently, these modules are typically microphones. However, using a microphone requires openings or slots in the terminal housing, which complicates the design of the sound pickup mechanism. Summary of the Invention

[0004] The present application provides a terminal and a terminal sound pickup method, which improves the problem that the use of a microphone for sound pickup in an existing terminal increases the complexity of the sound pickup structure design.

[0005] To achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, a terminal is provided, comprising a main body, a sound pickup module, and a control module. The main body has a first surface, which faces the user and includes a display surface. The sound pickup module is disposed within the main body and includes a radar module. The radar module is located on the inner side of the first surface. The radar module is configured to emit a detection beam toward the first surface, receive and process echoes that pass through the first surface and enter the main body, and output a first electrical signal. The control module is communicatively connected to the sound pickup module. The control module is configured to receive the first electrical signal, detect whether it contains Doppler shift data, and analyze the Doppler shift data to derive an audio signal. When a user speaks, the skin in areas such as the mouth, throat, and jaw vibrates. This vibration is generally minimal, causing the echo to undergo a Doppler effect, resulting in a Doppler shift. This echo then carries vibration information related to the user's speech, namely, the Doppler shift data. After the echo is received and processed by the radar module, a first electrical signal can be generated and transmitted to the control module. The control module analyzes and processes the first electrical signal to extract audio-related features, namely the above-mentioned Doppler shift data. Then, a Mel-spectrogram (Mel spectrum for short) can be generated based on the Doppler shift data. Then, the speech is restored by the Mel-spectrogram to generate a corresponding audio signal. Finally, the above-mentioned audio signal can be processed by noise reduction and other processing before output or storage. Therefore, using the terminal provided by the embodiment of the present application, the radar module can replace the microphone to achieve the sound pickup function. The above-mentioned radar module can be installed inside the terminal, and the surface of the terminal can be provided with no sound inlet holes or gaps, or with fewer sound inlet holes or gaps, so that the appearance of the terminal is beautiful and complete, and the complexity of the sound pickup structure design can be reduced to a certain extent. In addition, since the radar module is located inside the terminal, it does not need to be set on the side or bottom of the terminal for aesthetic reasons. It can be set directly opposite the human mouth, that is, the radar module can be set at a better sound pickup position to better pick up the sound emitted by the user. Therefore, using the terminal provided by this embodiment, the sound pickup quality can be improved to a certain extent.

[0007] In one possible implementation of the first aspect, the main body includes a support device and a screen assembly mounted on the support device. The display surface is the side of the screen assembly facing away from the support device, and at least a portion of the radar module is mounted on the screen assembly. Since the user's face is generally facing the screen assembly during use, the solution provided in this embodiment allows the detection beam emitted by the radar module to reach areas that vibrate when the user speaks, such as the mouth, throat, and jaw. This allows for better reception of vibration information associated with the user's speech, resulting in better audio quality ultimately picked up by the terminal.

[0008] In one possible implementation of the first aspect, the screen assembly includes a screen body and a metal layer, the metal layer being disposed on a side of the screen body facing the support device, and the radar module's antenna being disposed on the screen body or the metal layer. This allows the detection beam generated by the radar module to more easily pass through a corresponding portion of the screen assembly without being blocked by the metal layer in the screen assembly. This also makes the terminal structure more compact, facilitating miniaturization.

[0009] In a possible implementation of the first aspect, a through structure is provided on the metal layer that penetrates the metal layer in the thickness direction. There is at least one through structure. All the through structures form a surrounding structure having at least one opening. The surrounding structure separates the metal layer into a first part and a second part that are interconnected by a connecting structure. The connecting structure is a structure located at the opening in the metal layer. The first part is the part of the metal layer surrounded by the through structure. The first part forms an antenna, and the second part is the part of the metal layer other than the first part and the connecting structure. The antenna adopts the solution provided by this embodiment and can be manufactured using a part of the screen assembly. This can reduce the number of components in the terminal to a certain extent, and can not increase the thickness of the screen assembly, does not affect the display effect of the screen assembly, facilitates the miniaturized design of the terminal, and can also reduce the production cost of the terminal to a certain extent. At the same time, it achieves the purpose of pointing the detection beam emitted by the radar module to the user, achieving multiple goals at one stroke.

[0010] In a possible implementation of the first aspect, there is one through structure, which is a strip-shaped structure. This structure simplifies the antenna preparation and facilitates processing.

[0011] In one possible implementation of the first aspect, the screen assembly includes an organic light-emitting diode (OLED) display module, and the metal layer is the cathode layer of the OLED display module. In this embodiment, the antenna can be fabricated using the aforementioned through-hole structure. Because the through-hole structure is very narrow, its installation does not affect the normal display of the screen assembly. This allows a terminal with an OLED screen to be integrated with a radar module, thereby enabling the terminal to pick up sound through the radar module.

[0012] In one possible implementation of the first aspect, the screen assembly includes a liquid crystal display (LCD), and the metal layer serves as a reflective layer for the LCD. In this embodiment, the antenna can be fabricated using the aforementioned through-hole structure. Because the through-hole structure is very narrow, it does not affect the reflective effect of the backlight reflective layer. This allows a terminal with an LCD screen to be integrated with a radar module, enabling the terminal to pick up sound through the radar module.

[0013] In a possible implementation of the first aspect, the radar module includes a transmitting module and a receiving module, the transmitting module includes a transmitter, a first capacitor, and a transmitting antenna electrically connected in sequence, and the receiving module includes a receiving antenna, a second capacitor, and a receiver electrically connected in sequence. The first capacitor and the second capacitor are both isolation capacitors, and their capacitance is generally very small, such as can reach the pF level. When the radar module is a millimeter wave radar, the impedance of the first capacitor and the second capacitor is very low in the operating frequency band of the radar module, which is approximately a short circuit, and the impedance is very high in the display drive signal band, which is equivalent to being disconnected. In this way, the first capacitor and the second capacitor can prevent the display drive signal of the screen assembly from flowing into the transmitter or receiver of the radar module to a certain extent. This can improve the transmission efficiency of the transmitting antenna to a certain extent, and can also improve the receiving sensitivity of the receiving antenna to a certain extent.

[0014] In one possible implementation of the first aspect, the main body includes a support device, a screen assembly mounted on the support device, and a frame surrounding the screen assembly. The display surface is the side of the screen assembly facing away from the support device, and at least a portion of the radar module is mounted on the frame. This ensures that the radar module can be installed without disrupting the internal structure of the screen assembly or affecting the normal display of the screen assembly, facilitating installation of the radar module.

[0015] In a possible implementation of the first aspect, the sound pickup module also includes a microphone, which is arranged in the main body. A sound intake structure is provided on the main body. The sound intake structure connects the external space of the main body with the internal space of the main body. The sound intake structure is used to allow sound waves to pass through so as to be picked up by the microphone. This can reduce the number of microphones in the terminal, and further reduce the number of sound intake structures such as sound intake holes or gaps in the terminal, so that the appearance of the terminal is more beautiful and complete. In addition, since the microphone will convert all sound signals at its location into electrical signals, it is impossible to determine which signal is the main sound source. By adopting the solution provided in this embodiment, the microphone and the radar module can be used in combination to confirm the main sound source through the radar module, so that the audio signal corresponding to the non-main sound source obtained by the microphone can be filtered out. At the same time, the audio signal corresponding to the main sound source obtained by the microphone can be combined with the audio signal obtained by the radar module through AI fusion, noise reduction and other technologies to achieve joint noise reduction, so that the noise in the audio signal finally obtained by the terminal is smaller.

[0016] In one possible implementation of the first aspect, the control module includes a fusion module, which is communicatively connected to the microphone and the radar module, respectively, and is configured to receive and process the second electrical signal and the first electrical signal output by the microphone to generate an audio signal. The fusion module can be an AI noise reduction network module. Based on computational auditory scene analysis theory, it can apply deep learning technology and utilize a deep neural network to construct a noise reduction model. After training with a large amount of corpus, it can separate human voices from noise, effectively suppress various noises in the environment, and effectively cope with sudden non-stationary noise. The voice distortion is relatively much smaller, which can greatly ensure the sound restoration and enhance the call experience. The solution provided by this embodiment can effectively integrate the data obtained by the radar module with the data obtained by the microphone, thereby generating an audio signal with lower noise. Compared to the sound pickup module that only includes a microphone, the solution provided by this embodiment adds an active sound pickup mode, that is, the vibration signal generated by the sound source can be collected by the radar module and combined with the sound signal collected by the microphone for noise reduction, thereby significantly improving the noise reduction effect.

[0017] In one possible implementation of the first aspect, the control module further includes a voice activity detection module. The voice activity detection module is communicatively connected to the radar module and the fusion module, respectively. The voice activity detection module is configured to receive a first electrical signal and analyze whether Doppler shift data is present in the first electrical signal. The fusion module is configured to receive an analysis signal output by the voice activity detection module and process the first and second electrical signals based on the analysis signal. The voice activity detection module is configured to determine whether the user is speaking. Because vibration occurs only when the user is speaking, determining whether the user is speaking by detecting vibration can be achieved by analyzing whether Doppler shift data is present in the first electrical signal. The specific analysis method is not limited in this application. For example, the voice activity detection module can output a specific signal (e.g., 1) when the user is speaking and a specific signal (e.g., 0) when the user is not speaking. When the fusion module receives the specific signal corresponding to the user speaking, it analyzes and processes the first and second electrical signals. When the fusion module receives the specific signal corresponding to the user not speaking, it deletes the first and second electrical signals. This allows the fusion module to filter out some sound signals that are not emitted by the user, thereby achieving noise reduction. At the same time, it can reduce the number of calculations of the fusion module to a certain extent and improve the working efficiency of the control module.

[0018] In a possible implementation of the first aspect, the control module also includes a noise reduction module. The noise reduction module is communicatively connected to the microphone and the fusion module respectively. The noise reduction module is used to receive the second electrical signal, perform noise reduction processing on the second electrical signal, and transmit the second electrical signal after noise reduction processing to the fusion module. The noise reduction module can adopt an AI noise reduction module, or other noise reduction modules, such as a multi-microphone noise reduction module, etc., which can be determined according to the specific needs of use. By adopting the solution provided in this embodiment, the second electrical signal can be subjected to a noise reduction once before being transmitted to the fusion module, so as to reduce the calculation steps of the fusion module and improve the quality of the audio signal output by the fusion module.

[0019] In a possible implementation of the first aspect, a plurality of microphones are provided, and the noise reduction module is communicatively connected to the plurality of microphones respectively. The noise reduction module is used to receive the second electrical signals output by the plurality of microphones, and perform noise reduction based on the plurality of second electrical signals. The noise reduction module can select a corresponding microphone array noise reduction module according to the number of microphones. For example, the noise reduction module can obtain the pickup ranges of different microphones, determine the overlapping area of ​​the pickup ranges of different microphones, output the audio signal within the overlapping area as the final audio signal, and remove the audio signal outside the overlapping area, thereby achieving noise reduction. By adopting the solution provided in this embodiment, the noise reduction technology is mature and easy to design.

[0020] In a possible implementation of the first aspect, the control module also includes a wind noise detection module, which is communicatively connected to the microphone, the noise reduction module, and the fusion module respectively. The wind noise detection module is used to receive the second electrical signal output by the microphone, process the second electrical signal to obtain wind noise data related to the wind, and transmit the wind noise data to the noise reduction module and the fusion module. The noise reduction module is also used to receive the wind noise data and perform noise reduction on the second electrical signal based on the wind noise data. The fusion module is also used to perform noise reduction on the first electrical signal and the second electrical signal based on the wind noise data. The wind noise detection module is used to detect whether there is wind-related data in the data obtained by the microphone, and is also used to determine the frequency range, intensity, etc. of the wind noise. For example, the wind noise detection module can implement the above functions in a variety of ways, such as receiving a low-frequency signal picked up by the microphone and generating a wearing result signal based on the low-frequency signal. The fusion module is an AI noise reduction network that receives the outputs of the wind noise detection module and the voice activity detection module, and further removes noise from the above data based on the noise reduction of the noise reduction module. By adopting the solution provided in this embodiment, the setting of the wind noise detection module can detect whether there is wind sound in the audio signal obtained by the microphone, so that the noise reduction module can perform noise reduction processing on the second electrical signal transmitted by the microphone according to the corresponding detection results. At the same time, the fusion module can perform noise reduction processing on the electrical signal conducted by the noise reduction module and the first electrical signal according to the corresponding detection results, so that the noise reduction effect of the audio signal output by the fusion module is better.

[0021] In one possible implementation of the first aspect, the radar module is a millimeter-wave radar. Millimeter-wave radars have the characteristics of high resolution, strong penetration, easy integration, and high security. Using millimeter-wave radar in a radar module can be applicable to a variety of application scenarios.

[0022] In a second aspect, a terminal sound pickup method is provided, which is based on a terminal provided by any of the above solutions and includes at least the following steps: obtaining Doppler frequency shift data related to the user's speech through a radar module; and obtaining an audio signal through Doppler frequency shift data analysis.

[0023] By adopting the terminal sound pickup method provided in this embodiment, the vibration of the user's skin (such as the mouth, throat, jaw, etc.) when the user speaks can be obtained to obtain the voice uttered by the user. Compared with obtaining through a microphone, the number of sound inlet holes or gaps opened on the terminal surface can be reduced, making the terminal surface complete and beautiful, and the complexity of the sound pickup structure design can be reduced to a certain extent. At the same time, the noise in the audio signal can be reduced to a certain extent, such as the wind sound in the environment, the voice of people next to the user, etc.

[0024] In a possible implementation of the second aspect, obtaining an audio signal through Doppler frequency shift data analysis includes at least the following steps: receiving a second electrical signal output by a microphone; and obtaining an audio signal based on the second electrical signal and Doppler frequency shift data analysis.

[0025] Since the microphone will convert all sound signals at its location into electrical signals (i.e., the second electrical signal), it is impossible to determine which signal is the main sound source. By using the terminal sound pickup method provided in this embodiment, the sound-related vibration data obtained by the radar module is integrated with the sound data obtained by the microphone. This can filter out sounds not emitted by the user, making the noise of the obtained audio signal smaller, which helps to improve the quality of the audio signal.

[0026] In a possible implementation of the second aspect, before obtaining the audio signal based on the second electrical signal and Doppler frequency shift data analysis, at least the following steps are also included: determining whether the distance between the radar module and the user is within a preset distance; if the distance between the radar module and the user is not within the preset distance, eliminating the first electrical signal.

[0027] By adopting the solution provided in this embodiment, a preset distance can be defined according to the general distance between the user and the terminal during use. In this way, the Doppler frequency shift data caused by objects outside the preset distance will not be mistaken for user voice-related data, thereby achieving the removal of some noise and making the acquired audio information more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] FIG1 is a schematic structural diagram of a condenser microphone in the related art;

[0029] FIG2 is a schematic structural diagram of a dynamic microphone in the related art;

[0030] FIG3 is a circuit diagram of a dynamic microphone in the related art;

[0031] FIG4 is a schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0032] FIG5 is a schematic diagram of a partial structure of the terminal shown in FIG4 with the screen assembly removed;

[0033] FIG6 is a schematic structural diagram of a terminal provided in another embodiment of the present application;

[0034] FIG7 is a perspective structural diagram of a terminal provided in one embodiment of the present application;

[0035] FIG8 is a schematic diagram of a terminal in use state according to another embodiment of the present application;

[0036] FIG9 is a schematic diagram of a terminal in use state according to another embodiment of the present application;

[0037] FIG10 is a schematic diagram of the control principle of a terminal provided in an embodiment of the present application;

[0038] FIG11 is a schematic diagram of a flow chart of a control module in a terminal provided in an embodiment of the present application;

[0039] FIG12 is a schematic diagram of the circuit principle of a radar module in a terminal provided in an embodiment of the present application;

[0040] FIG13 is a side view schematic diagram of the structure of a screen assembly in a terminal provided in an embodiment of the present application;

[0041] FIG14 is a schematic diagram showing the connection relationship between the screen assembly and the radar module in the terminal provided in an embodiment of the present application;

[0042] FIG15 is a schematic cross-sectional view of a screen assembly in a terminal according to an embodiment of the present application;

[0043] FIG16 is a schematic cross-sectional view of a screen assembly in a terminal according to another embodiment of the present application;

[0044] FIG17 is a schematic diagram of the circuit principle of a radar module in a terminal provided in an embodiment of the present application;

[0045] FIG18 is a schematic diagram of a terminal in use state according to another embodiment of the present application;

[0046] FIG19 is a schematic diagram of a terminal in use state according to another embodiment of the present application;

[0047] FIG20 is a schematic diagram of the control principle of a terminal provided by another embodiment of the present application;

[0048] FIG21 is a schematic diagram of a flow chart of a terminal sound pickup method provided in one embodiment of the present application;

[0049] FIG22 is a schematic diagram of a flow chart of a terminal sound pickup method provided by another embodiment of the present application;

[0050] FIG23 is a flow chart of a terminal sound pickup method provided in another embodiment of the present application.

[0051] Explanation of reference numerals: 10', condenser microphone; 11', diaphragm; 12', plate; 20', dynamic microphone; 21', diaphragm; 22', coil; 23', magnet; 10, supporting device; 20, battery; 30, circuit board assembly; 40, screen assembly; 40a, screen body; 40b, metal layer; 41, cathode layer; 411, through-hole structure; 42, electron injection layer; 43, electron transport layer; 44, light-emitting layer; 45, hole transport layer; 46, hole injection layer; 47, anode layer; 48, panel glass; 41', backlight reflection layer; 42', backlight module; 43', lower polarizer; 44', lower glass substrate; 45', electrode and thin film transistor; 46', liquid crystal layer; 47', electrode layer; 48', color filter; 49', upper glass substrate; 410', polarizer; 50. Sound pickup module; 51. Radar module; 51a. Radar body; 51b. Antenna; 52. Microphone; 60. Frame; 70. Keyboard; 80. Rear camera; 90. Control module; 91. Fusion module; 92. Voice activity detection module; 93. Noise reduction module; 94. Wind noise detection module; 511. Transmitter; 512. First capacitor; 513. Transmitting antenna; 514. Receiving antenna; 515. Second capacitor; 516. Receiver; 517. Connection structure; 518. Conductive member; m. First surface. DETAILED DESCRIPTION

[0052] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0053] In the description of this application, it should be understood that the terms "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on this application.

[0054] To facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the first limiting portion and the second limiting portion are merely used to distinguish between different limiting portions and do not limit their order. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or execution order, and the words "first" and "second" do not necessarily mean that they are different.

[0055] It should be noted that, in this application, words such as "in one embodiment" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described in this application as "in one embodiment" or "for example" should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "in one embodiment" or "for example" is intended to present the relevant concepts in a concrete manner.

[0056] In this application, unless otherwise specified or limited, the terms "connected" and "connection" should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, indirect connection through an intermediate medium, internal communication between two components, or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on specific circumstances.

[0057] In order to make the purpose, technical solutions and advantages of this application more clear, this application is further described in detail below with reference to the accompanying drawings and embodiments.

[0058] With the development of society, terminals such as smart watches, mobile phones, laptops, personal computers (PCs), smart cars, wearable devices (smart glasses, smart helmets, etc.), televisions, smart speakers, and car cockpits are increasingly used in various aspects of modern life, work communication, etc. In order to improve the user experience, a sound pickup module is generally provided in the terminal. The sound pickup module can not only be used for ordinary audio collection, but also can be used in application environments such as voice calls and video calls. The sound pickup module currently used by the terminal is generally a microphone, and microphones can be mainly divided into two categories: condenser microphones and dynamic microphones. As shown in Figure 1, the main structure of the condenser microphone 10' includes a diaphragm 21' and a plate 12'. Among them, the plate 12' is fixedly arranged, and the diaphragm 21' is another plate 12' arranged opposite to the plate 12'. The diaphragm 21', the plate 12' and the air gap between the two together constitute a parallel plate capacitor, which follows the following formula:

[0059] Where C is the capacitance, S is the area of ​​the diaphragm 21' and the plate 12' facing each other, Q is the amount of charge stored when the voltage between the diaphragm 21' and the plate 12' is VC, εr is the dielectric constant of the medium (air) between the diaphragm 21' and the plate 12', d is the distance between the diaphragm 21' and the plate 12', and k is an electrostatic force constant with a value of 9.0×10000000000 N·m 2 / C 2 . The working principle of the condenser microphone 10' is as follows: when in use, a stable bias voltage is applied to the above-mentioned parallel plate capacitor, and the bias voltage is a direct current (DC) voltage to keep it in a constant charging state, so that the amount of charge Q stored between the diaphragm 21' and the plate 12' remains unchanged. This function can be achieved by a charge pump. When picking up audio, the sound wave enters the terminal through the sound inlet or gap of the terminal, and applies sound pressure F to the diaphragm 21', causing the diaphragm 21' to vibrate. The displacement of the diaphragm 21' will cause the distance d between the diaphragm 21' and the plate 12' to change, thereby causing the capacitance C between the two to change. The change in capacitance C causes the voltage between the diaphragm 21' and the plate 12' to change or generate current. After amplifying the above-mentioned changed voltage or current, an electrical signal corresponding to the sound signal can be formed, thereby completing the conversion from sound signal to electrical signal.

[0060] The structure of a dynamic microphone 20' is shown in Figure 2. It generally includes a diaphragm 21', a magnet 23' located inside the diaphragm 21', and a coil 22' mounted outside the magnet 23'. The coil 22' is electrically connected to an external power supply system and is also connected to the diaphragm 21' and can move synchronously with the vibration of the diaphragm 21'. The magnet 23' can be a permanent magnet. The operating principle of a dynamic microphone 20' is as follows: when picking up audio, sound waves enter the terminal through the sound inlet or gap of the terminal, applying a sound pressure F to the diaphragm 21', causing the diaphragm 21' to vibrate. During this period, the vibration of the diaphragm 21' drives the coil 22' to move relative to the magnet 23'. The movement of the coil 22' cuts through the magnetic lines of force generated by the magnet 23', generating an induced electromotive force and an induced current related to the sound wave vibration. This induced electromotive force and / or induced current are amplified to form an electrical signal corresponding to the sound. The circuit schematic corresponding to this principle is shown in Figure 3.

[0061] Regardless of the above-mentioned type of microphone, they are all passive devices. To collect sound signals at the receiving end, air vibrations (i.e., sound waves) need to be transmitted to the diaphragm. This requires that the terminal be designed with sound inlet holes or gaps for sound waves to enter the terminal. And in order to ensure the sound pickup effect of the microphone, in many cases, multiple microphones and multiple sound inlet holes or gaps need to be designed in a single terminal. On the one hand, this will destroy the integrity of the terminal's appearance; on the other hand, the sound inlet holes or gaps may be blocked by impurities such as fingers, water, and dust, which may cause the terminal's sound pickup function to fail; on the third hand, in order to prevent dust or liquid from entering the terminal through the sound inlet holes or gaps and causing damage to the microphone, a dustproof net or membrane needs to be designed in the sound inlet holes or gaps, which will increase the complexity of the sound pickup structure design.

[0062] In addition, in order to ensure the integrity of the terminal's appearance, the above-mentioned sound intake holes or gaps are generally set at the top, bottom, side of the terminal or hidden in other appearance structures. For example, the sound intake holes or gaps in laptop computers are generally set in the front of the keyboard surface (i.e., C surface) or D surface. Among them, D surface refers to the bottom cover of the laptop computer, which is also the side you see when you turn the laptop over. The above-mentioned front position refers to the position of C surface or D surface away from the screen and close to the user. However, no matter what design is used, the microphone cannot be directly facing the human mouth when working, that is, it cannot be set in a better sound pickup position, which leads to a decrease in the microphone's sound pickup effect, especially in noisy environments. In order to improve the sound pickup effect, in related technologies, multiple microphones are generally set in the terminal and combined with a sound pickup enhancement algorithm to improve the voice quality. This leads to an increase in the design complexity and cost of the terminal. As people's requirements for voice quality continue to increase, the sound pickup enhancement algorithm becomes more and more complex, and the amount of calculation continues to increase. However, the voice quality in strong noise environments, strong reverberation environments and / or windy environments is still limited and significantly attenuated compared to general environments.

[0063] In order to improve the above-mentioned problems or at least partially improve the above-mentioned problems, an embodiment of the present application provides a terminal. The terminal includes a main body, a sound pickup module and a control module. The main body has a first surface, which is the surface facing the user and includes a display surface. The sound pickup module is arranged in the main body, and the sound pickup module includes a radar module. The radar module is located on the inner side of the first surface. The radar module is used to emit a detection beam toward the first surface, and is also used to receive and process the echo that passes through the first surface and enters the main body, and output a first electrical signal to the outside. The control module is communicatively connected to the sound pickup module. The control module is used to receive the first electrical signal, detect whether it contains Doppler frequency shift data, and derive an audio signal based on the Doppler frequency shift data analysis.

[0064] When a user speaks, the skin of the user's mouth, throat, jaw, etc. will vibrate. This vibration is generally very small, which will cause the above-mentioned echo to undergo a Doppler effect and produce a Doppler frequency shift. In this way, the echo carries vibration information related to the user's speech, namely the Doppler frequency shift data. After the echo is received and processed by the radar module, a first electrical signal can be generated, which is transmitted to the control module. The control module analyzes and processes the first electrical signal to extract audio-related features, namely the above-mentioned Doppler frequency shift data. Then, a Mel-spectrogram (Mel spectrum for short) can be generated based on the Doppler frequency shift data. Then, the Mel-spectrogram is used to perform speech recovery to generate the corresponding audio signal. Finally, the above-mentioned audio signal can be processed by noise reduction and other processes before output or storage. Therefore, using the terminal provided in the embodiment of the present application, the radar module can replace the microphone to achieve the sound pickup function. The above-mentioned radar module can be installed inside the terminal, and the surface of the terminal can be provided with no sound inlet or gap, or with fewer sound inlet or gaps, so that the appearance of the terminal is beautiful and complete, and the complexity of the sound pickup structure design can be reduced to a certain extent.

[0065] The terminal provided in the embodiment of the present application has a sound pickup function. For the convenience of description, the following embodiments are described by taking a mobile phone as an example.

[0066] As shown in Figures 4 and 5, a mobile phone generally includes a support device 10, a battery 20, a circuit board assembly 30, a screen assembly 40, and a pickup module 50. The support device 10, the battery 20, at least a portion of the circuit board assembly 30, and the screen assembly 40 form the main body of the mobile phone. The main body has a first surface, which faces the user and includes a display surface.

[0067] The support device 10 may include at least one frame. When the support device 10 includes multiple frames, two adjacent frames may be connected by a hinge structure. The screen assembly 40 generally covers one side of the support device 10, and the screen assembly 40 may also be provided on both sides of the support device 10, which may be determined according to the specific needs of use. When the screen assembly 40 covers one side of the support device 10, the other side of the support device 10 is generally installed with a shell. The shell and the screen assembly 40 can enclose a closed space, and the battery 20, the circuit board assembly 30 and the pickup module 50 are all installed in the above-mentioned closed space. At this time, the side of the screen assembly 40 facing away from the support device 10 is also the side used for display, which is the above-mentioned display surface. If the screen assembly 40 can completely cover an entire side of the support device 10, the above-mentioned display surface is the first side. If the screen assembly 40 cannot completely cover the entire side of the support device 10, a frame 60 surrounding the screen assembly 40 can also be provided on the support device 10, as shown in Figure 6. At this time, the front of the frame 60 and the display surface constitute the front of the mobile phone. The front of the mobile phone is the above-mentioned first surface, that is, the first surface at this time not only includes the above-mentioned display surface, but also includes the front of the frame 60.

[0068] The battery 20 provides power to the various electronic components and power modules (such as the circuit board assembly 30, the screen assembly 40, and the pickup module 50) in the mobile phone. There can be one or more batteries, which can be determined according to the specific needs of use. The circuit board assembly 30 may include a main board assembly and a sub-board assembly. The main board assembly and the sub-board assembly generally include a circuit board and electronic components arranged on the circuit board. The electronic components may include but are not limited to a system on chip (SoC), a UFS (short for UNIX file system), a computer-aided manufacturing (CAM), a universal serial bus (USB) interface, a subscriber identity module (SIM) card, an antenna module, a Bluetooth module, a wireless connection (WIreless-Fidelity, WiFi) module, a global positioning system (GPS) module, a power supply, a charging module, a screen display and operation module, etc. The screen assembly 40 is generally electrically connected to the above-mentioned screen display and operation module so that the screen assembly 40 can realize the display or operation function.

[0069] The sound pickup module 50 can be communicatively connected to at least one chip in the circuit board assembly 30 to cooperate with the circuit board assembly 30 to pick up external audio signals. The number of sound pickup modules 50 can be one or more, depending on the specific needs. The communication connection can be a wired connection via a wire or a wireless connection via a wireless communication module.

[0070] As shown in FIG7 , the sound pickup module 50 includes a radar module 51. The radar module 51 is generally located on the inner side of the first surface m and is capable of emitting a detection beam toward the first surface m. It is also capable of receiving echoes that pass through the first surface m and enter the main body, processing the echoes to generate a first electrical signal, and outputting the first electrical signal. For example, the radar module 51 can be located on the back of the screen assembly 40, within the screen assembly 40, on the support device 10, on the frame 60, or on other components of the terminal. The specific location of the radar module 51 can be flexibly adjusted according to the size of the terminal, as long as the radar module 51 can detect vibrations in the mouth, throat, and other areas of the user facing the screen assembly 40. For example, in a mobile phone, the radar module 51 can be located at the bottom or middle of the screen assembly 40. When the user holds the mobile phone to make a call, the detection beam emitted by the radar module 51 can be directed toward the user's throat and mouth, as shown in FIG8 . For example, in a laptop computer, the radar module 51 can be designed on the back of the screen assembly 40 so that when the user opens the laptop, the detection beam emitted by the radar module 51 is directed toward the user, as shown in FIG9 .

[0071] It can be understood that in order to allow the detection beam emitted by the radar module 51 to pass through the first surface m, the material located between the above-mentioned detection beams in the mobile phone is generally made of a material that allows the detection beam to pass through, such as glass, plastic, composite materials, etc., and metal materials are generally not used. The specific material can be determined according to the penetration performance of the detection beam.

[0072] One or more radar modules 51 can be provided in the same terminal. These radar modules 51 are communicatively connected to the circuit board assembly 30, specifically to the system chip and / or other chips within the circuit board assembly 30. These system chip and / or other chips can constitute a control module, or a module within any of these chips can form the control module. This control module is capable of receiving the first electrical signal output by the radar module 51, detecting whether it contains Doppler shift data, and analyzing the Doppler shift data to derive an audio signal.

[0073] As shown in Figure 10, radar module 51 generally includes a radar body 51a and an antenna 51b connected to radar body 51a. Antenna 51b can be a transmitting antenna with a transmitting function, a receiving antenna with a receiving function, or a transceiver antenna with both transmitting and receiving functions, depending on the function and type of radar module 51. It should be understood that when antenna 51b is a transmitting antenna or a receiving antenna, the same radar module 51 has at least one transmitting antenna and at least one receiving antenna; when antenna 51b is a transceiver antenna, the same radar module 51 can have one or more transceiver antennas.

[0074] For ease of understanding, the radar module 51 including a transmitting antenna and a receiving antenna is used as an example to illustrate the sound pickup principle of the terminal provided in the embodiment of the present application:

[0075] When a user uses the terminal, they typically face the first surface m of the terminal. The radar module 51 can emit a probe beam toward the first surface m via its transmitting antenna. This probe beam can pass through the first surface m and exit the terminal. Upon contacting the user's skin, this probe beam forms an echo. This echo can pass through the first surface m and return to the terminal's main body, where it is received by the radar module 51's receiving antenna and transmitted to the radar body 51a. The radar body 51a then processes the received echo to generate a first electrical signal. When the user speaks, the skin around the user's mouth, throat, jaw, and other areas vibrates. This vibration is generally very small, causing the echo to undergo a Doppler effect, resulting in a Doppler frequency shift. This echo then carries vibration information related to the user's speech, namely, the Doppler frequency shift data. After the echo is received and processed by the radar module 51, a first electrical signal can be generated and transmitted to the control module. The control module analyzes and processes the first electrical signal to extract audio-related features, namely the above-mentioned Doppler frequency shift data. Then, a Mel-spectrogram (Mel spectrum for short) can be generated based on the Doppler frequency shift data, and the speech is restored through the Mel-spectrogram to generate a corresponding audio signal. Finally, the above-mentioned audio signal can be processed such as noise reduction and then output or stored, as shown in Figure 11.

[0076] The functions of the above-mentioned control module 90 can be implemented based on a deep neural network (DNN) network (such as a generative adversarial network (GAN), which requires pre-training), large model technology, and signal processing technology.

[0077] It is understandable that when there are multiple radar modules 51, different radar modules 51 can be set at different positions of the main body of the terminal. For example, if there are two radar modules 51, one radar module 51 can be set at the bottom of the main body to detect vibration information caused by vibration of the throat skin when the user speaks, and the other radar module 51 can be set in the middle of the main body to detect vibration information caused by vibration of the mouth or jaw skin when the user speaks. Similarly, when the radar module 51 is provided with multiple transmitting antennas and multiple receiving antennas, one transmitting antenna and one receiving antenna can form a group of transmitting and receiving antenna groups. In this way, the radar module 51 has multiple groups of transmitting and receiving antenna groups, and different transmitting and receiving antenna groups can be set at different positions of the main body of the terminal. For example, if there are two transmitting and receiving antenna groups, one transmitting and receiving antenna group can be set at the bottom of the main body to detect vibration information caused by vibration of the throat skin when the user speaks, and the other transmitting and receiving antenna group can be set in the middle of the main body to detect vibration information caused by vibration of the mouth or jaw skin when the user speaks.

[0078] Using the terminal provided in the embodiment of the present application, the radar module 51 can replace the microphone 52 to achieve the sound pickup function. The radar module 51 can be installed inside the terminal, and the surface of the terminal can be provided with no sound inlet holes or gaps, or with fewer sound inlet holes or gaps, making the terminal's appearance beautiful and complete, and can reduce the complexity of the sound pickup structure design to a certain extent. In addition, because the radar module 51 is located inside the terminal, it does not need to be placed on the side or bottom of the terminal for aesthetic reasons. It can be placed directly opposite the human mouth. That is, the radar module 51 can be placed in an optimal sound pickup position to better pick up the sound emitted by the user. Therefore, using the terminal provided in this embodiment can improve the sound pickup quality to a certain extent.

[0079] As shown in Figure 12, the radar module 51 generally includes a transmitting module and a receiving module. The transmitting module includes at least a transmitter 511 and a transmitting antenna 513. Transmitter 511 provides an optical signal, while transmitting antenna 513 receives the optical signal and converts it into a detection beam. The receiving module includes at least a receiving antenna 514 and a receiver 516. Receiving antenna 514 receives the echo and transmits it to receiver 516. Receiver 516 receives and processes the echo transmitted by receiving antenna 514 and generates the aforementioned first electrical signal. The transmitting antenna 513 and receiving antenna 514 can be a single antenna or separate antennas, depending on the specific application. When the radar module 51 is a coaxial radar module, the transmitting antenna 513 and receiving antenna 514 are a single antenna, which can be referred to as a transceiver antenna. When the radar module 51 is a non-coaxial radar module, such as a parallel-axis radar module, the transmitting antenna 513 and receiving antenna 514 are separate antennas.

[0080] Since the user's face is generally facing the screen assembly 40 during use, in order to ensure that the detection beam emitted by the radar module 51 can reach the parts of the user that vibrate when speaking, such as the mouth, throat, and jaw, the entire structure of the radar module 51 can be installed on the screen assembly 40, or only the transmitting antenna 513 in the radar module 51 can be installed on the screen assembly 40, or the transmitting antenna 513 and the receiving antenna 514 in the radar module 51 can be installed on the screen assembly 40, and the specific location can be determined according to the needs of use. Using the solution provided in this embodiment, the detection beam emitted by the radar module 51 can reach the parts of the user that vibrate when speaking, such as the mouth, throat, and jaw, so that the vibration information related to the user's speech can be better received, so that the audio quality ultimately picked up by the terminal is better.

[0081] As shown in FIG13 , many screen assemblies 40 include a screen body 40a and a metal layer 40b, with the metal layer 40b being disposed on the side of the screen body 40a facing the support device 10. Since the detection beam emitted by the radar module 51 is generally difficult to penetrate the metal layer 40b, the antenna in the radar module 51 (which may be a transceiver antenna, a transmitting antenna 513, or a receiving antenna 514) may be disposed on the screen body 40a or the metal layer 40b. It is understood that when the antenna in the radar module 51 is disposed on the metal layer 40b, it may be disposed on the side of the metal layer 40b closest to the screen body 40a, or it may be embedded in the metal layer 40b, as long as the signals emitted by the antenna and the signals to be received are not blocked by the metal layer 40b. This allows the detection beam generated by the radar module 51 to more easily pass through the corresponding portion of the screen assembly 40 without being blocked by the metal layer 40b in the screen assembly 40. This also makes the terminal structure compact, facilitating its miniaturization.

[0082] As shown in FIG14 , in some embodiments, a through structure 411 is provided on the metal layer 40b that penetrates the metal layer 40b in the thickness direction. There is at least one through structure 411. All through structures 411 form a surrounding structure having at least one opening. If there is only one through structure 411, the through structure 411 can be a long strip structure, bent or folded to form a rectangular structure, annular structure, etc. having an opening; when there are multiple through structures 411, each through structure 411 can be a strip structure, and the strip structure can be a straight structure or a curved structure. An opening can be formed between two adjacent through structures 411, so that multiple through structures 411 can form a surrounding structure having multiple openings, and the surrounding structure can be a rectangular structure, annular structure, etc.

[0083] The surrounding structure separates the metal layer 40b into a first portion and a second portion, which are connected to each other via a connecting structure 517. The connecting structure 517 is a structure located at an opening in the metal layer 40b. The first portion is the portion of the metal layer 40b surrounded by the through structure 411. The first portion forms an antenna. The antenna referred to here can be a transmitting antenna 513, a receiving antenna 514, or a transceiver antenna, depending on the type and structure of the radar module 51. The second portion is the portion of the metal layer 40b excluding the first portion and the connecting structure 517.

[0084] It is understood that although the first portion can be used as the antenna of the radar module 51, its function in the screen assembly 40 is still the same as that of the second portion, that is, it can cooperate with the structure of the corresponding area in the screen body 40a to perform display. The through-structure 411 can be a regular pattern or an irregular pattern, which can be determined according to the type, shape, structure of the antenna and the display requirements of the screen assembly 40. For example, the contour of the inner side of the through-structure 411 forms the outer contour of the antenna. Therefore, the contour of the inner side of the through-structure 411 should be processed according to the shape of the antenna, and its size can be determined according to the operating frequency of the radar module 51 (usually above 20GHz), such as by calculating the wavelength of the detection beam emitted by the radar module 51, for example, according to the half-wavelength principle or other antenna design theories. As shown in Figure 14, if the first part enclosed by the metal layer 40b is designed as a rectangle with an opening, the diagonal length of the rectangle can be set to half a wavelength. Of course, the first portion enclosed by the metal layer 40b can be designed into other shapes (e.g., circular, irregular, etc.), and its size can be determined based on antenna design theory. The width of the through structure 411 can be determined based on the display requirements of the screen assembly 40 and the circuit design requirements between the antenna and the second portion.

[0085] The number, width, and length of the connecting structures 517 determine their inductance, which can be calculated using a formula. The formula above refers to the parasitic inductance calculation formula for a PCB (Printed Circuit Board). During design, the inductance of the connecting structures 517 can be optimized to achieve a relatively high value within the operating frequency band of the radar module 51. This allows the first portion of the metal layer 40b, which serves as the antenna, to be approximately disconnected from the other portions (i.e., the second portion) within the operating frequency band of the radar module 51. This allows the detection beam emitted by the radar module 51 to be directly emitted through the transmitting antenna 513, minimizing the amount of radiation reaching the second portion of the metal layer 40b. This can improve the transmission efficiency of the radar module 51 to a certain extent. It can also minimize the amount of radiation transmitted by the receiving antenna 514 to the second portion of the metal layer 40b, thereby improving the receiving sensitivity of the radar module 51 to a certain extent.

[0086] The antenna adopts the solution provided in this embodiment and can be manufactured using a part of the screen assembly 40, which can reduce the number of components in the terminal to a certain extent, and does not increase the thickness of the screen assembly 40, does not affect the display effect of the screen assembly 40, facilitates the miniaturization design of the terminal, and can also reduce the production cost of the terminal to a certain extent. At the same time, it achieves the purpose of pointing the detection beam emitted by the radar module 51 to the user, achieving multiple goals at one stroke.

[0087] In some embodiments, the through structure 411 is a strip-shaped structure and is provided with one. This structure makes the antenna preparation method simple and easy to process.

[0088] It can be understood that in order to minimize the impact of the setting of the through structure 411 on the display function of the screen assembly 40 and enable the screen assembly 40 to display normally, the width a of the through structure 411 is generally set to be relatively small. For example, the width a of the through structure 411 can be less than or equal to 10 microns, or less than or equal to 20 microns. The specific width can be determined according to the shape of the through structure 411, circuit design requirements, etc., and is not limited here.

[0089] Since the screen assembly 40 has various types, the above-mentioned screen body 40a and metal layer 40b can also be arranged in various ways. For example, in some embodiments, the screen assembly 40 is an OLED (Organic Light-Emitting Diode, also known as organic laser display, organic light-emitting semiconductor) display module. As shown in Figure 15, the OLED display module includes a cathode layer 41, an electron injection layer 42, an electron transport layer 43, a light-emitting layer 44, a hole transport layer 45, a hole injection layer 46, an anode layer 47 and a panel glass 48, which are stacked in sequence from the inside to the outside. Among them, the cathode layer 41 is the above-mentioned metal layer 40b, and the other parts of the OLED screen assembly 40 except the cathode layer 41 constitute the above-mentioned screen body 40a. In this embodiment, the antenna can be prepared using the above-mentioned through structure 411. Since the width of the through structure 411 is very small, the setting of the through structure 411 will not affect the normal display of the screen assembly 40. This allows the terminal with an OLED screen to be integrated with the radar module 51, thereby achieving the purpose of the corresponding terminal picking up sound through the radar module 51.

[0090] As shown in FIG16 , in other embodiments, the screen assembly 40 includes an LCD (Liquid Crystal Display), including a backlight reflection layer 41′, a backlight module 42′, a lower polarizer 43′, a lower glass substrate 44′, electrodes and thin-film transistors 45′, a liquid crystal layer 46′, an electrode layer 47′, a color filter 48′, an upper glass substrate 49′, and a polarizer 410′, which are stacked in sequence from the inside to the outside. The backlight reflection layer 41′ is the aforementioned metal layer 40b. The remaining parts of the screen assembly 40, except for the backlight reflection layer 41′, constitute the aforementioned screen body 40a. In this embodiment, the antenna can be prepared using the aforementioned through-structure 411. Since the width of the through-structure 411 is very small, it will not affect the reflective effect of the backlight reflection layer 41′. This allows a terminal with an LCD screen to be integrated with a radar module 51, thereby enabling the corresponding terminal to achieve sound pickup through the radar module 51.

[0091] In other embodiments, the screen assembly 40 may also be configured in other ways, which are not limited here.

[0092] Regardless of the configuration of the screen assembly 40, when the antenna is configured using the through-structure 411 described above, since the antenna is part of the metal layer 40b, in order to reduce the risk of the detection beam being conducted to the second portion of the metal layer 40b, in some embodiments, as shown in FIG17 , the radar module 51 includes a transmitting module and a receiving module. The transmitting module includes a transmitter 511, a first capacitor 512, and a transmitting antenna 513, which are electrically connected in sequence. The receiving module includes a receiving antenna 514, a second capacitor 515, and a receiver 516, which are electrically connected in sequence.

[0093] Among them, the first capacitor 512 and the second capacitor 515 are both isolation capacitors, and their capacitance is generally very small, such as can reach the pF level. When the radar module 51 is a millimeter wave radar, the impedance of the first capacitor 512 and the second capacitor 515 is very low in the operating frequency band of the radar module 51, which is approximately a short circuit, and the impedance is very high in the display drive signal frequency band, which is equivalent to being disconnected. In this way, the first capacitor 512 and the second capacitor 515 can prevent the display drive signal of the screen assembly 40 from flowing into the transmitter 511 or receiver 516 of the radar module 51 to a certain extent. This can improve the transmission efficiency of the transmitting antenna 513 to a certain extent, and can also improve the receiving sensitivity of the receiving antenna 514 to a certain extent.

[0094] The radar module 51 may be a millimeter-wave radar, the transmitter 511 may be a millimeter-wave radar transmit power amplifier, and the receiver 516 may be a millimeter-wave radar receiver amplifier. During operation, the signal output by the transmit power amplifier is connected to the transmit antenna 513 via a first capacitor 512, while the receive antenna 514 is connected to the millimeter-wave radar receiver amplifier via a second capacitor 515. Connection methods include welding, bonding, and spring clip connections.

[0095] The first capacitor 512 and the second capacitor 515 are respectively welded to the corresponding antenna via a conductive member 518. The conductive member 518 can be a wire, a metal sheet, or other components capable of conducting electricity.

[0096] In addition to being disposed on the screen assembly 40, in other embodiments, the radar module 51 can also be disposed on the frame 60 surrounding the screen assembly 40. This allows the radar module 51 to be disposed without damaging the structure within the screen assembly 40 or affecting the normal display of the screen assembly 40, thereby facilitating the installation of the radar module 51.

[0097] The sound pickup module in the terminal provided in the embodiment of the present application can include only one or more radar modules 51, or it can include both one or more radar modules 51 and one or more microphones 52, as shown in Figure 18. The microphone 52 is generally arranged in the main body of the terminal. At the same time, in order to cooperate with the microphone 52 to pick up sound, a sound inlet structure is generally provided near the microphone 52. The sound inlet structure can connect the external space of the main body with the internal space of the main body, so that sound waves outside the terminal can pass through and propagate to the location of the microphone 52, thereby being picked up by the microphone 52. The above-mentioned sound inlet structure includes sound inlet holes and / or gaps that pass through the mobile phone housing, frame 60 and / or screen assembly 40 in the thickness direction. As shown in Figure 18, at least one microphone 52 and a sound inlet hole corresponding to the position of the microphone 52 can be respectively arranged at the top and bottom of the mobile phone, and a radar module 51 can be arranged near the bottom of the screen assembly 40 of the mobile phone. When in use, the multiple microphones 52 and at least one radar module 51 can be used to pick up the sound emitted by the user. Other configurations are also possible. As shown in FIG19 , a radar module 51 can be disposed on the back of a laptop computer's screen assembly 40, and a microphone 52 and a sound inlet corresponding to the position of microphone 52 can be disposed on the front of the laptop computer's keyboard 70. This allows the radar module 51 to direct the detection beam toward the user when the user opens the laptop computer, while the microphone 52 is positioned closer to the user. The front end of the keyboard 70 is the end of the keyboard 70 that is away from the screen assembly 40 and closer to the user when the laptop computer is open.

[0098] This can reduce the number of microphones 52 in the terminal, and further reduce the number of sound intake holes or gaps and other sound intake structures in the terminal, making the terminal more beautiful and complete in appearance. In addition, since the microphone converts all sound signals at its location into electrical signals, it is impossible to determine which signal is the main sound source. By adopting the solution provided in this embodiment, the microphone 52 and the radar module 51 can be used in conjunction, and the main sound source can be confirmed by the radar module 51, so that the audio signal corresponding to the non-main sound source obtained by the microphone 52 can be filtered out. At the same time, the audio signal corresponding to the main sound source obtained by the microphone 52 can be combined with the audio signal obtained by the radar module 51 through AI fusion, noise reduction and other technologies to achieve joint noise reduction, so that the noise in the audio signal finally obtained by the terminal is smaller.

[0099] In some embodiments, the sound pickup module includes both the radar module 51 and the microphone 52, and both the radar module 51 and the microphone 52 can be used to pick up audio signals. In order to better integrate the audio signals picked up by the two, in some embodiments, as shown in FIG20 , the control module 90 includes a fusion module 91. The fusion module 91 is communicatively connected to the microphone 52 and the radar module 51, respectively, and is configured to receive and process the first electrical signal and the second electrical signal output by the microphone 52 to generate an audio signal.

[0100] The fusion module 91 can be a functional module in some control chips in the terminal (such as a central processing unit (CPU, English: Central Processing Unit / Processor), a digital signal processing chip (Digital Signal Process, DSP), a network processor (Neural-network Processing Unit, NPU), etc.), such as an AI noise reduction network module. It can be based on the theory of computational auditory scene analysis, apply deep learning technology, and use deep neural networks to build a noise reduction model. After training with massive corpus, it can separate human voice and noise, effectively suppress various noises in the environment, and effectively deal with sudden non-stationary noise. In addition, the voice distortion is relatively much smaller, which can greatly ensure the sound restoration and improve the call experience.

[0101] Taking a mobile phone as an example, when a user speaks while holding the phone, microphone 52 and radar module 51 operate separately. Because the detection beam emitted by radar module 51 is directed toward the throat, mouth, and jaw, the vibrations of the skin surface, vocal cords, and other tissues when the user speaks cause a Doppler shift in the echo signal. Therefore, the echo signal carries information related to the user's speech, namely, the Doppler shift data. Microphone 52 can convert the vibration of diaphragm 21' caused by the sound waves into an electrical signal, namely, a second electrical signal, and transmit the second electrical signal to fusion module 91. Fusion module 91 can perform noise reduction and fusion processing on the Doppler shift data and the second electrical signal to generate an audio signal with lower noise.

[0102] The solution provided by this embodiment allows for a smooth integration of data acquired by the radar module 51 and the microphone 52, thereby generating a low-noise audio signal. Compared to the sound pickup module 50 comprising only a microphone, the solution provided by this embodiment adds an active sound pickup mode. This means that the vibration signal generated by the sound source can be collected by the radar module 51 and combined with the sound signal collected by the microphone for noise reduction, significantly improving the noise reduction effect.

[0103] To ensure accurate operation of control module 90, in some embodiments, control module 90 further includes a voice activity detection module 92, which is communicatively coupled to radar module 51 and fusion module 91. Voice activity detection module 92 is configured to receive the first electrical signal and analyze it for Doppler shift data. Fusion module 91 is configured to receive the analysis signal output by voice activity detection module 92 and process the first and second electrical signals based on the analysis signal.

[0104] The voice activity detection module 92 can be a functional module of a certain chip in the terminal (such as a central processing unit, a digital signal processing chip, a network processor, etc.), such as a program. The voice activity detection module 92 is used to confirm whether the user is speaking. Because there is vibration only when the user is speaking, confirming whether the user is speaking by detecting vibration can be achieved by analyzing whether there is Doppler frequency shift data in the first electrical signal. The specific analysis method is not limited in this application. For example, it can be designed that when the user is speaking, the voice activity detection module 92 outputs a specific signal (for example, 1); when the user is not speaking, the voice activity detection module 92 outputs a specific signal (for example, 0); when the fusion module 91 receives the specific signal 1 corresponding to the user speaking, the first electrical signal and the second electrical signal are analyzed and processed; when the fusion module 91 receives the specific signal 0 corresponding to the user not speaking, the first electrical signal and the second electrical signal are deleted. This allows the fusion module 91 to filter out some sound signals that are not emitted by the user, thereby achieving a noise reduction function. Compared with picking up sound through a microphone, it can filter out environmental sounds other than the user (such as people next to the user, environmental noise, etc.), making the audio signal obtained by the terminal more accurate. At the same time, it can reduce the number of calculations of the fusion module 91 to a certain extent, thereby improving the working efficiency of the control module 90.

[0105] In some embodiments, the control module 90 also includes a noise reduction module 93, which is communicatively connected to the microphone and the fusion module 91 respectively. The noise reduction module 93 is used to receive the second electrical signal and perform noise reduction processing on the second electrical signal, and is also used to transmit the second electrical signal after noise reduction processing to the fusion module 91.

[0106] The noise reduction module 93 can be a functional module of a chip in the terminal (such as a central processing unit, a digital signal processing chip, a network processor, etc.). For example, an AI noise reduction module or other noise reduction modules such as a multi-microphone noise reduction module, a microphone array module, etc. can be used, which can be determined according to usage needs.

[0107] By adopting the solution provided in this embodiment, the second electrical signal can be subjected to noise reduction before being transmitted to the fusion module 91, thereby reducing the calculation steps of the fusion module 91 and improving the quality of the audio signal output by the fusion module 91.

[0108] In some embodiments, there are multiple microphones 52. The noise reduction module 93 is communicatively connected to each of the multiple microphones 52. The noise reduction module 93 is configured to receive the second electrical signals output by the multiple microphones and perform noise reduction based on the multiple second electrical signals.

[0109] The noise reduction module 93 is generally a functional module of a chip in the terminal (such as a central processing unit, a digital signal processing chip, a network processor, etc.). The corresponding microphone array noise reduction module can be selected according to the number of microphones. For example, the noise reduction module 93 can obtain the pickup range of different microphones, determine the overlapping area of ​​the pickup ranges of different microphones, output the audio signal in the above overlapping area as the final audio signal, and remove the audio signal outside the above overlapping area to achieve noise reduction.

[0110] The solution provided in this embodiment is mature in noise reduction technology and easy to design.

[0111] In some embodiments, the control module 90 further includes a wind noise detection module 94. The wind noise detection module 94 is communicatively connected to the microphone 52, the noise reduction module 93, and the fusion module 91. The wind noise detection module 94 is configured to receive the second electrical signal output by the microphone 52, process the second electrical signal to obtain wind noise data related to the wind, and transmit the wind noise data to the noise reduction module 93 and the fusion module 91. The noise reduction module 93 is further configured to receive the wind noise data and perform noise reduction processing on the second electrical signal based on the wind noise data. The fusion module 91 is further configured to perform noise reduction processing on the first electrical signal and the second electrical signal based on the wind noise data.

[0112] The wind noise detection module 94 is typically a functional module of a chip within the terminal (e.g., a central processing unit, a digital signal processing chip, a network processor, etc.). It is used to detect whether there is wind-related data in the data acquired by the microphone 52 and to determine the frequency range and intensity of the wind sound. For example, the wind noise detection module 94 can implement the above functions in various ways, such as receiving a low-frequency signal picked up by the microphone 52 and generating a wearing result signal based on the low-frequency signal.

[0113] The fusion module 91 is an AI noise reduction network that receives the outputs of the wind noise detection module 94 and the voice activity detection module 92, and further removes noise from the above data based on the noise reduction performed by the noise reduction module.

[0114] By adopting the solution provided in this embodiment, the setting of the wind noise detection module 94 can detect whether there is wind sound in the audio signal obtained by the microphone, so that the noise reduction module 93 can perform noise reduction processing on the second electrical signal transmitted by the microphone according to the corresponding detection results. At the same time, the fusion module 91 can perform noise reduction processing on the electrical signal and the first electrical signal transmitted by the noise reduction module 93 according to the corresponding detection results, so that the noise reduction effect of the audio signal output by the fusion module 91 is better.

[0115] In some embodiments, the radar module 51 is a millimeter wave radar.

[0116] Millimeter-wave radar operates in the millimeter wave band. Millimeter waves typically fall within the 30-300 GHz frequency range (wavelengths of 1-10 mm). Millimeter-wave wavelengths lie between microwaves and centimeter waves, so millimeter-wave radar combines the advantages of both microwave and photoelectric radars, such as high resolution, strong penetration, ease of integration, and high security. Radar module 51 employing millimeter-wave radar can be applied to a variety of scenarios.

[0117] Another embodiment of the present application further provides a terminal sound pickup method. As shown in FIG21 , the terminal sound pickup method is based on the terminal provided by any of the above embodiments and includes at least the following steps:

[0118] S1. Obtain Doppler frequency shift data related to the user's speech through the radar module.

[0119] As mentioned above, the Doppler frequency shift data in this step may be carried in the echo received by the radar module, and the control module may obtain the Doppler frequency shift data by analyzing and processing the first electrical signal output by the radar module.

[0120] S2. derive the audio signal through Doppler frequency shift data analysis.

[0121] The Doppler frequency shift data can be processed to obtain the Mel spectrum, and then the audio data can be obtained from the Mel spectrum.

[0122] By adopting the terminal sound pickup method provided in this embodiment, the vibration of the user's skin (such as the mouth, throat, jaw, etc.) when the user speaks can be obtained to obtain the voice uttered by the user. Compared with obtaining through a microphone, the number of sound inlet holes or gaps opened on the terminal surface can be reduced, making the terminal surface complete and beautiful, and the complexity of the sound pickup structure design can be reduced to a certain extent. At the same time, the noise in the audio signal can be reduced to a certain extent, such as the wind sound in the environment, the voice of people next to the user, etc.

[0123] As shown in FIG22 , the above step S2 includes at least the following steps:

[0124] S21. Receive a second electrical signal output by a microphone.

[0125] This step can be achieved by connecting the microphone to the control module via a wired connection or a wireless connection.

[0126] S22. Analyze the second electrical signal and the Doppler frequency shift data to obtain an audio signal.

[0127] As mentioned above, a fusion module can be set in the control module to reduce noise and fuse the second electrical signal and Doppler frequency shift data through AI noise reduction to obtain the final audio signal.

[0128] Since the microphone will convert all sound signals at its location into electrical signals (i.e., the second electrical signal), it is impossible to determine which signal is the main sound source. By using the terminal sound pickup method provided in this embodiment, the sound-related vibration data obtained by the radar module is integrated with the sound data obtained by the microphone. This can filter out sounds not emitted by the user, making the noise of the obtained audio signal smaller, which helps to improve the quality of the audio signal.

[0129] As shown in FIG23 , before the above step S22, at least the following steps are also included:

[0130] S3. Determine whether the distance between the radar module and the user is within a preset distance.

[0131] This step can be achieved through the ranging function of the radar module. The preset distance can be determined according to the actual use. For example, if the terminal is a mobile phone and the distance between the user and the phone is generally within 20 cm, the preset distance can be 20 cm. If the terminal is a laptop and the distance between the user and the laptop screen is generally within 50 cm, the preset distance can be 50 cm.

[0132] S4. If the distance between the radar module and the user is not within the preset distance, the Doppler shift data is eliminated.

[0133] Step S3 and step S4 may be arranged between step S1 and step S2, or between step S21 and step S22, depending on the specific use requirements.

[0134] By adopting the solution provided in this embodiment, a preset distance can be defined according to the general distance between the user and the terminal during use. In this way, the Doppler frequency shift data caused by objects outside the preset distance will not be mistaken for user voice-related data, thereby achieving the removal of some noise and making the acquired audio information more accurate.

[0135] Finally, it should be noted that the above are only specific embodiments of this application, but the scope of protection of this application is not limited to them. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A terminal, characterized in that: include: The main body has a first surface, the first surface is the surface facing the user, and the first surface includes a display surface; a sound pickup module disposed within the main body, the sound pickup module including a radar module, the radar module being located on the inner side of the first surface, the radar module being configured to emit a detection beam toward the first surface, receive and process echoes that pass through the first surface and enter the main body, and output a first electrical signal; as well as A control module is communicatively connected to the sound pickup module, and is used to receive the first electrical signal, detect whether it contains Doppler frequency shift data, and obtain an audio signal based on the Doppler frequency shift data.

2. The terminal according to claim 1, wherein The main body includes a supporting device and a screen assembly arranged on the supporting device. The display surface is a side of the screen assembly facing away from the supporting device. At least a portion of the radar module is arranged on the screen assembly.

3. The terminal according to claim 2, characterized in that The screen assembly includes a screen body and a metal layer. The metal layer is arranged on a side of the screen body facing the supporting device, and the antenna of the radar module is arranged on the screen body or the metal layer. The terminal according to claim 3, wherein: The metal layer is provided with a through structure that penetrates itself along the thickness direction, and there is at least one through structure. All the through structures form a surrounding structure with at least one opening, and the surrounding structure separates the metal layer into a first part and a second part that are interconnected by a connecting structure. The connecting structure is a structure in the metal layer located at the opening, the first part is the part of the metal layer surrounded by the through structure, the first part forms the antenna, and the second part is the part of the metal layer excluding the first part and the connecting structure. The terminal according to claim 4 , wherein: The through structure is provided with one and is a strip-shaped structure.

6. The terminal according to any one of claims 3 to 5, characterized in that: The screen assembly includes an organic light emitting diode display module, and the metal layer is a cathode layer of the organic light emitting diode display module.

7. The terminal according to any one of claims 3 to 5, characterized in that: The screen assembly includes a liquid crystal display, and the metal layer is a reflective layer of the liquid crystal display.

8. The terminal according to any one of claims 3 to 7, characterized in that: The radar module includes a transmitting module and a receiving module. The transmitting module includes a transmitter, a first capacitor and a transmitting antenna electrically connected in sequence. The receiving module includes a receiving antenna, a second capacitor and a receiver electrically connected in sequence.

9. The terminal according to claim 1, wherein: The main body includes a supporting device, a screen assembly arranged on the supporting device, and a frame surrounding the screen assembly. The display surface is a side of the screen assembly facing away from the supporting device, and at least a portion of the radar module is arranged on the frame.

10. The terminal according to any one of claims 1 to 9, characterized in that: The sound pickup module also includes a microphone, which is arranged in the main body. The main body is provided with a sound intake structure, which connects the external space of the main body with the internal space of the main body. The sound intake structure is used to allow sound waves to pass through so as to be picked up by the microphone. The terminal according to claim 10 , wherein: The control module includes a fusion module, which is communicatively connected to the microphone and the radar module respectively. The fusion module is used to receive and process the second electrical signal output by the microphone and the first electrical signal to generate an audio signal.

12. The terminal according to claim 11, characterized in that The control module also includes a voice activity detection module, which is communicatively connected to the radar module and the fusion module respectively. The voice activity detection module is used to receive the first electrical signal and analyze whether Doppler frequency shift data exists in the first electrical signal. The fusion module is used to receive an analysis signal output by the voice activity detection module and process the first electrical signal and the second electrical signal according to the analysis signal.

13. The terminal according to claim 11 or 12, characterized in that: The control module also includes a noise reduction module, which is communicatively connected to the microphone and the fusion module respectively. The noise reduction module is used to receive the second electrical signal, perform noise reduction processing on the second electrical signal, and transmit the second electrical signal after noise reduction processing to the fusion module. The terminal according to claim 13 , wherein: There are multiple microphones, and the noise reduction module is communicatively connected to the multiple microphones respectively. The noise reduction module is used to receive the second electrical signals output by the multiple microphones and perform noise reduction according to the multiple second electrical signals.

15. The terminal according to claim 13 or 14, characterized in that: The control module also includes a wind noise detection module, which is communicatively connected to the microphone, the noise reduction module and the fusion module respectively. The wind noise detection module is used to receive a second electrical signal output by the microphone, process the second electrical signal to obtain wind noise data related to wind sound, and transmit the wind noise data to the noise reduction module and the fusion module. The noise reduction module is also used to receive the wind noise data and perform noise reduction processing on the second electrical signal according to the wind noise data. The fusion module is also used to perform noise reduction processing on the first electrical signal and the second electrical signal according to the wind noise data.

16. The terminal according to any one of claims 1 to 15, characterized in that: The radar module is a millimeter wave radar.

17. A terminal sound pickup method, based on the terminal according to any one of claims 1 to 16, characterized in that: include: Acquiring Doppler frequency shift data related to the user's speech through the radar module; An audio signal is obtained by analyzing the Doppler frequency shift data.

18. The terminal sound pickup method according to claim 17, characterized in that: The obtaining of an audio signal by analyzing the Doppler frequency shift data includes: receiving a second electrical signal output by the microphone; The audio signal is obtained by analyzing the second electrical signal and the Doppler frequency shift data.

19. The terminal sound pickup method according to claim 18, characterized in that: Before obtaining the audio signal by analyzing the second electrical signal and the Doppler frequency shift data, the method further includes: Determining whether the distance between the radar module and the user is within a preset distance; If the distance between the radar module and the user is not within the preset distance, the first electrical signal is eliminated.

Images