Control method and device and terminal equipment

Abstract

The embodiment of the invention provides a control method and device and terminal equipment. The method can be applied to the field of man-machine interaction. The method comprises the steps that the terminal equipment detects touch operation on one or more microphones in a microphone array, the touch operation indicates the terminal equipment to execute a man-machine interaction instruction, and the microphones in the microphone array correspond to different space areas in the terminal equipment; the terminal equipment is controlled to execute the man-machine interaction instruction according to the touch operation. The embodiment of the invention can be applied to terminal equipment including an intelligent automobile or an electric automobile, a user does not need to carry out voice interaction with the terminal equipment, the accuracy of man-machine interaction in a high-noise or high-interference scene is improved, interference to other surrounding users in the voice interaction process is avoided, and the user experience is improved. And therefore, the human-computer interaction experience of the user can be improved.

Description

Control method, device and terminal equipment Technical Field

[0001] The embodiments of the present application relate to the field of human-computer interaction, and more specifically, to a control method, apparatus, and terminal device. Background Art

[0002] With the development of intelligent terminal devices, voice interaction has gradually become the mainstream human-computer interaction method. However, current voice interaction has certain limitations. For example, there is a certain interaction threshold when users interact with terminal devices through voice, requiring users to speak at an appropriate speed and volume. For example, in scenes with high noise or interference, the accuracy of voice interaction is significantly reduced, which affects the user's human-computer interaction experience.

[0003] Summary of the Invention

[0004] The present application provides a control method, apparatus, and terminal device, which help to enhance the user's human-computer interaction experience.

[0005] In a first aspect, a control method is provided, which includes: obtaining first touch control information corresponding to one or more microphones, the first touch control information indicating a first operation, the one or more microphones belonging to a pre-set microphone array, and the microphones in the microphone array corresponding to different spatial areas in a terminal device; according to the first touch control information, controlling the terminal device to perform the first operation.

[0006] Based on the above technical solution, the terminal device is controlled to perform a first operation by first touch control information corresponding to one or more microphones, without the need for the user to interact with the terminal device through voice. This helps improve the accuracy of human-computer interaction in high-noise or high-interference scenarios, avoids interference with other users during voice interaction, and thus helps improve the user's human-computer interaction experience. At the same time, the microphones in the microphone array correspond to different spatial areas of the terminal device. Through touch control information corresponding to the microphones in the microphone array, control of different spatial areas can be achieved, which helps to improve the flexibility of human-computer interaction.

[0007] For example, the microphone array includes microphone 1, microphone 2, and microphone 3, and the spatial area of ​​the terminal device includes spatial area 1, spatial area 2, and spatial area 3.

[0008] In some possible implementations, microphones in the microphone array correspond to different spatial regions in the terminal device, including: certain microphones in the microphone array correspond to different spatial regions in the terminal device. For example, microphone 1 may correspond to spatial region 1, spatial region 2, and spatial region 3.

[0009] In some possible implementations, the microphones in the microphone array correspond to different spatial regions in the terminal device, including: multiple microphones in the microphone array correspond to different spatial regions in the terminal device. For example, microphone 1 may correspond to spatial region 1, microphone 2 may correspond to spatial region 2, and microphone 3 may correspond to spatial region 3.

[0010] In some possible implementations, when multiple microphones in the microphone array correspond to different spatial areas in the terminal device, the multiple microphones can be located in the corresponding spatial areas respectively, such as microphone 1 can be located in spatial area 1, microphone 2 can be located in spatial area 2, and microphone 3 can be located in spatial area 3; or, the multiple microphones can be located in a certain spatial area of ​​the terminal device, for example, microphone 1, microphone 2 and microphone 3 can all be located in spatial area 1.

[0011] In some possible implementations, the first touch control information includes a touch operation corresponding to the one or more microphones, for example, the touch operation includes one or more of a single click, a double click, a continuous click, a long press, and a continuous touch.

[0012] In some possible implementations, taking the terminal device being a laptop computer as an example, the first operation may be to lower or increase the volume intensity of the speaker in the laptop computer; or, taking the terminal device being a smart desk lamp as an example, the first operation may be to lower or increase the brightness of the smart desk lamp.

[0013] In combination with the first aspect, in certain implementations of the first aspect, the terminal device is a vehicle, and the first operation includes controlling the opening or closing of a vehicle window, adjusting the volume intensity of a sound-emitting device in the vehicle, switching multimedia files, controlling the locking or unlocking of a vehicle door, or controlling a display device in the vehicle to display the desktop.

[0014] Based on the above technical solution, different in-cabin devices can be controlled based on the first touch control information corresponding to one or more microphones, helping to enhance the user's driving experience. Furthermore, there is no need to set virtual or physical buttons for specific in-cabin devices in the cabin, which helps reduce vehicle production costs.

[0015] In some possible implementations, the first operation also includes operations such as opening a trunk, folding rearview mirrors, and controlling devices outside the cabin.

[0016] In combination with the first aspect, in certain implementations of the first aspect, the terminal device is a vehicle, the vehicle includes a cabin, the one or more microphones are located in a first area within the cabin, and controlling the terminal device to perform the first operation includes: controlling the cabin device in the first area to perform the first operation.

[0017] Based on the above technical solution, when first touch control information corresponding to one or more microphones in a first area is obtained, the cabin equipment in the first area can be controlled to perform the first operation. This can avoid interference with users in other areas of the cabin and help improve the user's driving experience.

[0018] In some possible implementations, taking the vehicle as an example, the first area may be the driver's seat area, the passenger seat area, the second row left area, or the second row right area. For example, upon obtaining touch control information corresponding to one or more microphones in the driver's seat area, the air conditioning temperature in the driver's seat area, the volume level of a sound device in the driver's seat area (for example, the volume level of a speaker on the driver's seat headrest), or the brightness of the ambient light in the driver's seat area may be adjusted.

[0019] In some possible implementations, for example, where the vehicle is a vehicle, the first area may be an area outside the cabin. For example, the one or more microphones may be located near the trunk. When touch control information corresponding to the one or more microphones is obtained, the trunk may be controlled to open.

[0020] In combination with the first aspect, in certain implementations of the first aspect, the terminal device is a vehicle, the vehicle includes a cabin, the cabin includes a first area and a second area, the one or more microphones are located in the first area, and controlling the terminal device to perform the first operation includes: controlling the cabin device in the second area to perform the first operation.

[0021] Based on the above technical solution, when first touch control information corresponding to one or more microphones in the first area is obtained, the first operation can be performed on the cockpit equipment in the second area. This eliminates the need for the user to enter the second area to control the cockpit equipment, helping to improve the efficiency of the user's control of cockpit equipment in other areas.

[0022] In some possible implementations, taking the vehicle as an example, the first area may be a main driving area, and the second area may be other areas in the cabin except the main driving area.

[0023] Based on the above technical solution, the user in the main driving area of ​​the cockpit can control other areas in the cockpit, which can ensure that the authority of the user in the main driving area is higher than that of users in other areas. For example, the user in the main driving area can light up the reading lights in the second row left area by touching one or more microphones in the main driving area. However, the user in the second row left area cannot light up the reading lights in the main driving area by touching one or more microphones in the second row left area. For another example, the user in the main driving area can unlock the door of the vehicle by touching one or more microphones in the main driving area; while the user in the passenger area cannot unlock the door of the vehicle by touching one or more microphones in the passenger area, or the user in the rear area (for example, the second row left area or the second row right area) cannot unlock the door of the vehicle by touching one or more microphones in the rear area.

[0024] For example, the first area is the main driving area, and the second area is the left side area of ​​the second row. When touch control information corresponding to one or more microphones in the main driving area is obtained, the window of the left side area of ​​the second row can be controlled to close.

[0025] In some possible implementations, taking the vehicle as an example, the first area may be a front area, and the second area may be a rear area.

[0026] In some possible implementations, the first area may be an area inside the cockpit (for example, the main driving area), and the second area may be an area outside the cockpit. For example, when touch control information corresponding to one or more microphones in the main driving area is obtained, the trunk of the vehicle can be controlled to open. In combination with the first aspect, in some implementations of the first aspect, the terminal device is a vehicle, the cockpit of the vehicle includes multiple areas, the one or more microphones include a first microphone, and the first microphone corresponds to the multiple areas. For example, the multiple areas may include the second row left area and the second row right area, and the first microphone is microphone 1. Microphone 1 can correspond to the second row left area and the second row right area.

[0027] The microphone 1 described above can be used in both the second row left and second row right areas, meaning it can be used to control cabin devices in both the second row left and second row right areas. For example, if the cabin device is a speaker, a single click on the microphone 1 can increase the volume of the second row left speaker; a double click on the microphone 1 can decrease the volume of the second row left speaker; a double click on the microphone 1 can increase the volume of the second row right speaker; and a long press on the microphone 1 can decrease the volume of the second row right speaker.

[0028] In combination with the first aspect, in some implementations of the first aspect, the microphone array is configured as a linear array, a circular array, a rectangular array, or a multi-arm array.

[0029] In combination with the first aspect, in certain implementations of the first aspect, before obtaining the first touch control information corresponding to one or more microphones, the method also includes: obtaining a first touch operation of the user on a mechanical transmission device included in the terminal device; and according to the first touch operation, controlling the mechanical transmission device to touch the one or more microphones.

[0030] Based on this technical solution, a mechanical transmission device can be added between the microphone array and the user. This device transmits the user's touch operation to the microphones in the microphone array, replacing the user's direct contact with the microphones. For some special groups (such as children in car seats or users with hand impairments), the mechanical transmission device can help them touch the microphones, helping to improve the user's human-computer interaction experience.

[0031] In some possible implementations, texture markings are provided around the one or more microphones, so as to facilitate touch operations by users in dark environments, or also enable touch operations by blind people.

[0032] In some possible implementations, indicator lights or motors are provided around the one or more microphones. When touch control information corresponding to the one or more microphones is obtained, interactive feedback can be generated by controlling the indicator lights to light up or the motors to vibrate, so that the user can clearly know whether the touch on the one or more microphones is successful, which helps to enhance the user's human-computer interaction experience.

[0033] In combination with the first aspect, in some implementations of the first aspect, the first touch control information includes a second touch operation corresponding to the one or more microphones; controlling the terminal device to perform the first operation based on the first touch control information includes: controlling the terminal device to perform the first operation based on the second touch operation.

[0034] Based on the above technical solution, when a second touch operation is obtained for the one or more microphones, the terminal device can be controlled to perform the first operation. This eliminates the need for the user to interact with the terminal device through voice, helps improve the accuracy of human-computer interaction in high-noise or high-interference scenarios, avoids interference with other users during voice interaction, and thus helps improve the user's human-computer interaction experience.

[0035] The above second touch operation may be performed by a user or a device.

[0036] For example, upon detecting a first touch operation by a user on the mechanical transmission device, the terminal device may control the mechanical transmission device to touch the one or more microphones. During the process of the mechanical transmission device touching the one or more microphones, a second touch operation by the mechanical transmission device on the one or more microphones may be obtained by analyzing the signals collected by the one or more microphones.

[0037] For another example, when detecting that a user has touched one or more microphones, the terminal device may analyze the signals collected by the one or more microphones to obtain the user's second touch operation on the one or more microphones. Alternatively, taking the terminal device as an example of a vehicle, when detecting that a user has touched one or more microphones, the user's second touch operation on the one or more microphones may be determined through data collected by sensors (e.g., cameras) in the cabin.

[0038] In some possible implementations, the terminal device stores a mapping relationship between the type of touch operation and the operation performed by the terminal device. Controlling the terminal device to perform the first operation based on the second touch operation includes: controlling the terminal device to perform the first operation based on the second touch operation and the mapping relationship.

[0039] In combination with the first aspect, in certain implementations of the first aspect, the first touch control information corresponds to a microphone, and the first touch control information represents the energy difference between signals collected by the microphone at different times (or, different time periods), and / or the signal correlation between signals collected by the microphone at different times (or, different time periods).

[0040] Based on the above technical solution, by analyzing the energy differences and / or signal correlations of signals collected by a microphone at different times (or different time periods), the terminal device can be controlled to perform the first operation. In this way, there is no need for the user to interact with the terminal device through voice, which helps to improve the accuracy of human-computer interaction in high-noise or high-interference scenarios, avoids interference with other users during voice interaction, and thus helps to improve the user's human-computer interaction experience.

[0041] In combination with the first aspect, in some implementations of the first aspect, the first touch control information corresponds to multiple microphones, and the first touch control information represents the energy difference between the signals collected by the multiple microphones, and / or the signal correlation between the signals collected by the multiple microphones.

[0042] Based on the above technical solution, by analyzing the energy differences and / or signal correlations of the signals collected by multiple microphones, the terminal device can be controlled to perform the first operation. This eliminates the need for the user to engage in voice interaction with the terminal device, helping to improve the accuracy of human-computer interaction in high-noise or high-interference scenarios, and avoiding interference with other users during voice interaction, thereby helping to enhance the user's human-computer interaction experience.

[0043] In some possible implementations, analyzing the energy difference between signals collected by multiple microphones includes analyzing the energy difference between signals collected by at least two of the multiple microphones at the same time (or the same time period).

[0044] In some possible implementations, analyzing the energy difference between signals collected by at least two of the multiple microphones at the same time (or, the same time period) includes analyzing the energy difference between signals collected by each of the multiple microphones at the same time (or, the same time period).

[0045] In combination with the first aspect, in certain implementations of the first aspect, the multiple microphones include a second microphone and a third microphone; wherein the first operation is an operation corresponding to single-clicking the second microphone, and the first touch control information represents that within a first time period, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets a first preset condition.

[0046] In combination with the first aspect, in certain implementations of the first aspect, the multiple microphones include a second microphone and a third microphone; the first operation is an operation corresponding to continuously clicking or continuously touching the second microphone, and the first touch control information represents that within multiple time periods, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets a first preset condition.

[0047] Based on the above technical solution, by analyzing the energy differences and / or signal correlations between the signals collected by multiple microphones, the terminal device can be controlled to perform the first operation. This eliminates the need for the user to issue voice commands to control the terminal device to perform the corresponding operation, helping to improve the accuracy of human-computer interaction in high-noise or high-interference scenarios, avoiding interference with other users during voice interaction, and thus helping to enhance the user's human-computer interaction experience.

[0048] In some possible implementations, the time interval between every two adjacent time periods in the multiple time periods is less than or equal to a preset time interval.

[0049] Exemplarily, the preset time interval is 0.4 seconds (s).

[0050] In some possible implementations, the continuous clicking includes double clicking or continuous clicking.

[0051] In some possible implementations, according to the first touch control information, the terminal device is controlled to perform the first operation, including: determining to touch the second microphone based on the energy difference between the signals collected by the second microphone and the third microphone, and / or the signal correlation between the signals collected by the second microphone and the third microphone; and controlling the terminal device to perform the first operation corresponding to touching the second microphone.

[0052] In some possible implementations, determining that the second microphone is touched is based on the energy difference between the signals collected by the second microphone and the third microphone, and / or the signal correlation between the signals collected by the second microphone and the third microphone, including: determining that the second microphone is touched when the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets a first preset condition and the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone.

[0053] Based on the above technical solution, when the difference between the energy collected by the second microphone and the energy collected by the third microphone meets the first preset condition and the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, it can be determined that the second microphone is touched. Therefore, the terminal device can perform the first operation corresponding to touching the second microphone.

[0054] In some possible implementations, the energy difference between the signal collected by the second microphone and the signal collected by the third microphone satisfies the first preset condition, including: the energy difference between the signal collected by the second microphone and the signal collected by the third microphone is greater than or equal to the preset difference, and / or the energy ratio between the signal collected by the second microphone and the signal collected by the third microphone is greater than or equal to the preset ratio.

[0055] In some possible implementations, touching the second microphone includes single-clicking the second microphone, double-clicking the second microphone, continuously tapping the second microphone, or continuously touching the second microphone.

[0056] The above touching of the second microphone may be a second touch operation.

[0057] With reference to the first aspect, in certain implementations of the first aspect, the first touch control information indicates that energy of a signal collected by the second microphone is greater than or equal to a first preset threshold.

[0058] Based on the above technical solution, if the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets a preset condition and the energy of the signal collected by the second microphone is greater than or equal to a first preset threshold, it can be determined that the second microphone has been touched. This helps improve the accuracy of microphone touch detection, thereby helping to improve the efficiency and accuracy of human-computer interaction.

[0059] In combination with the first aspect, in certain implementations of the first aspect, the multiple microphones include a fourth microphone and a fifth microphone; wherein the first operation is an operation corresponding to long pressing the fourth microphone, and the first touch control information represents that within the first frequency band, the energy of the signal collected by the fourth microphone is less than the energy of the signal collected by the fifth microphone, and the energy difference between the signal collected by the fourth microphone and the signal collected by the fifth microphone meets the second preset condition.

[0060] In some possible implementations, the terminal device is controlled to perform the first operation based on the first touch control information, including: determining to long press the fourth microphone based on the energy difference between the signals collected by the fourth microphone and the fifth microphone, and / or the signal correlation between the signals collected by the fourth microphone and the fifth microphone; and controlling the terminal device to perform the first operation corresponding to the long press of the fourth microphone.

[0061] The above long pressing of the fourth microphone may be a second touch operation.

[0062] In some possible implementations, determining whether to long press the fourth microphone is based on the energy difference between the signals collected by the fourth microphone and the fifth microphone, and / or the signal correlation between the signals collected by the fourth microphone and the fifth microphone, includes: within the first frequency band, when the energy difference between the signal collected by the fourth microphone and the signal collected by the fifth microphone meets the second preset condition and the energy of the signal collected by the fourth microphone is less than the energy of the signal collected by the fifth microphone, determining whether to long press the fourth microphone.

[0063] In some possible implementations, the first frequency band is a medium to high frequency band. For example, the first frequency band is (100 Hz, 8000 Hz).

[0064] With reference to the first aspect, in certain implementations of the first aspect, the first touch control information indicates that energy of a signal collected by the fourth microphone is less than or equal to a second preset threshold.

[0065] Based on the above technical solution, if the energy difference between the signal collected by the fourth microphone and the signal collected by the fifth microphone meets a preset condition and the energy of the signal collected by the fourth microphone is less than or equal to a second preset threshold, it can be determined that the fourth microphone is being long-pressed. This helps improve the accuracy of microphone long-press recognition, thereby helping to improve the efficiency and accuracy of human-computer interaction.

[0066] In combination with the first aspect, in some implementations of the first aspect, the energy difference includes at least one of a difference in time domain amplitude, a difference in time domain energy, a difference in frequency domain sub-band amplitude, and a difference in frequency band energy.

[0067] Based on the above technical solution, by combining at least one of the time domain amplitude, time domain energy, frequency domain sub-band amplitude and frequency band energy of the signal collected by the microphone, it helps to improve the accuracy of identifying the type of touch operation through the microphone signal.

[0068] In the second aspect, a control device is provided, which includes: an acquisition unit for acquiring first touch control information corresponding to one or more microphones, the first touch control information indicating a first operation, the one or more microphones belonging to a pre-set microphone array, and the microphones in the microphone array correspond to different spatial areas in the terminal device; a control unit for controlling the terminal device to perform the first operation according to the first touch control information.

[0069] In combination with the second aspect, in certain implementations of the second aspect, the terminal device is a vehicle, the vehicle includes a cabin, the one or more microphones are located in a first area within the cabin, and the control unit is used to: control the cabin device in the first area to perform the first operation.

[0070] In combination with the second aspect, in certain implementations of the second aspect, the terminal device is a vehicle, the vehicle includes a cabin, the cabin includes a first area and a second area, the one or more microphones are located in the first area, and the control unit is used to: control the cabin device in the second area to perform the first operation.

[0071] In combination with the second aspect, in some implementations of the second aspect, the terminal device is a vehicle, the vehicle includes a cabin, the cabin includes multiple areas, and the one or more microphones include a first microphone, and the first microphone corresponds to the multiple areas.

[0072] In combination with the second aspect, in certain implementations of the second aspect, the terminal device is a vehicle, and the first operation includes controlling the opening or closing of a vehicle window, adjusting the volume intensity of a sound-emitting device in the vehicle, switching multimedia files, controlling the locking or unlocking of a vehicle door, or controlling a display device in the vehicle to display the desktop.

[0073] In combination with the second aspect, in some implementations of the second aspect, the microphone array is configured as a linear array, a circular array, a rectangular array, or a multi-arm array.

[0074] In combination with the second aspect, in certain implementations of the second aspect, the terminal device also includes a mechanical transmission device, and the acquisition unit is further used to obtain a first touch operation of the user on the mechanical transmission device before obtaining the first touch control information corresponding to one or more microphones; the control unit is further used to control the mechanical transmission device to touch the one or more microphones according to the first touch operation.

[0075] In combination with the second aspect, in some implementations of the second aspect, the first touch control information includes a second touch operation corresponding to the one or more microphones; the control unit is used to: control the terminal device to perform the first operation according to the second touch operation.

[0076] In combination with the second aspect, in certain implementations of the second aspect, the first touch control information corresponds to a microphone, and the first touch control information represents the energy difference between signals collected by the microphone at different times, and / or the signal correlation between signals collected by the microphone at different times.

[0077] In combination with the second aspect, in some implementations of the second aspect, the first touch control information corresponds to multiple microphones, and the first touch control information represents the energy difference between the signals collected by the multiple microphones, and / or the signal correlation between the signals collected by the multiple microphones.

[0078] In combination with the second aspect, in certain implementations of the second aspect, the multiple microphones include a second microphone and a third microphone; wherein the first operation is an operation corresponding to single-clicking the second microphone, and the first touch control information represents that within a first time period, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets a first preset condition.

[0079] In conjunction with the second aspect, in certain implementations of the second aspect, the multiple microphones include a second microphone and a third microphone; the first operation is an operation corresponding to continuously clicking or continuously touching the second microphone; and the first touch control information indicates that, during multiple time periods, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone satisfies a first preset condition.

[0080] In combination with the second aspect, in some implementations of the second aspect, the first touch control information represents that energy of a signal collected by the second microphone is greater than or equal to a first preset threshold.

[0081] In combination with the second aspect, in certain implementations of the second aspect, the multiple microphones include a fourth microphone and a fifth microphone; wherein the first operation is an operation corresponding to long pressing the fourth microphone, and the first touch control information represents that within the first frequency band, the energy of the signal collected by the fourth microphone is less than the energy of the signal collected by the fifth microphone, and the energy difference between the signal collected by the fourth microphone and the signal collected by the fifth microphone meets the second preset condition.

[0082] In combination with the second aspect, in some implementations of the second aspect, the first touch control information represents that energy of the signal collected by the fourth microphone is less than or equal to a second preset threshold.

[0083] In combination with the second aspect, in some implementations of the second aspect, the energy difference includes at least one of a difference in time domain amplitude, a difference in time domain energy, a difference in frequency domain sub-band amplitude, and a difference in frequency band energy.

[0084] In a third aspect, a control device is provided, which includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the control device to perform any possible method in the first aspect.

[0085] In a fourth aspect, a control system is provided, which includes a microphone array and a computing platform, wherein the computing platform includes any possible control device in the second aspect or the third aspect.

[0086] In some possible implementations, the control system further includes cockpit equipment.

[0087] In some possible implementations, the control system further includes the mechanical transmission device in the first aspect described above.

[0088] In some possible implementations, the microphone array is configured as a linear array, a circular array, a rectangular array, or a multi-arm array.

[0089] In a fifth aspect, a terminal device is provided, which includes any possible control device in the second aspect, or includes the control device described in the third aspect, or includes the control system described in the fourth aspect.

[0090] In some possible implementations, the terminal device is a vehicle (eg, a vehicle).

[0091] In some possible implementations, the terminal device is a smart home.

[0092] In a sixth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute any possible method in the first aspect.

[0093] It should be noted that the above-mentioned computer program code can be stored in whole or in part on the first storage medium, wherein the first storage medium can be packaged together with the processor or separately packaged with the processor, and the embodiments of the present application do not specifically limit this.

[0094] In a seventh aspect, a computer-readable medium is provided, wherein the computer-readable medium stores a program code, and when the computer program code is run on a computer, the computer is caused to execute any possible method in the first aspect.

[0095] In an eighth aspect, an embodiment of the present application provides a chip system, which includes a processor for calling a computer program or computer instructions stored in a memory so that the processor executes any possible method in the above-mentioned first aspect.

[0096] In combination with the eighth aspect, in a possible implementation, the processor is coupled to the memory through an interface.

[0097] In combination with the eighth aspect, in one possible implementation, the chip system also includes a memory, in which a computer program or computer instructions are stored. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] FIG1 is a functional block diagram of a terminal device provided in an embodiment of the present application.

[0099] FIG2 is a schematic diagram of the configuration of a microphone array provided in an embodiment of the present application.

[0100] FIG3 is a schematic diagram of the arrangement of the microphone array in a vehicle according to an embodiment of the present application.

[0101] FIG4 is a set of human-machine interaction interfaces HMI provided in an embodiment of the present application.

[0102] FIG5 is another set of HMIs provided in an embodiment of the present application.

[0103] FIG6 is another HMI provided by an embodiment of the present application.

[0104] FIG7 is another schematic diagram of the arrangement of the microphone array in a vehicle according to an embodiment of the present application.

[0105] FIG8 is another set of HMIs provided in an embodiment of the present application.

[0106] FIG9 is another set of HMIs provided in an embodiment of the present application.

[0107] FIG10 is another HMI provided by an embodiment of the present application.

[0108] FIG11 is another set of HMIs provided in an embodiment of the present application.

[0109] FIG12 is another application scenario provided by an embodiment of the present application.

[0110] FIG13 is a schematic flow chart of a control method provided in an embodiment of the present application.

[0111] FIG14 is a curve showing energy changes of signals picked up by microphones in a microphone array provided by an embodiment of the present application when the microphones are not touched and not long pressed.

[0112] FIG15 is a curve showing energy changes of signals picked up by each microphone in the microphone array when Mic1 is clicked, provided by an embodiment of the present application.

[0113] FIG16 is a curve showing energy changes of signals picked up by each microphone in the microphone array when Mic1 is double-clicked according to an embodiment of the present application.

[0114] FIG17 is a curve showing energy changes of signals picked up by each microphone in the microphone array when Mic1 is continuously stroked according to an embodiment of the present application.

[0115] FIG18 is a curve showing energy changes of signals picked up by each microphone in the microphone array when Mic1 is long pressed, provided by an embodiment of the present application.

[0116] FIG19 is a schematic diagram of determining whether a microphone is touched or long pressed through signal correlation according to an embodiment of the present application.

[0117] Figure 20 is a schematic block diagram of the control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0118] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a way to describe the association relationship of associated objects, indicating that there can be three kinds of relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. "At least one" means one or more. For example, "at least one of A and B" is similar to "A and / or B", describing the association relationship of associated objects, indicating that there can be three kinds of relationships, for example, at least one of A and B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0119] In the embodiments of the present application, prefixes such as "first" and "second" are used only to distinguish different description objects and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present application does not constitute a restriction on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and the use of such prefixes should not constitute an unnecessary restriction. In addition, in the description of this embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

[0120] The method provided in this application can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), vehicles, smart homes, etc. The embodiments of this application do not impose any restrictions on the specific type of terminal device.

[0121] The vehicles in this application may include road vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or recreational equipment. For example, the vehicle may be a vehicle, which is a vehicle in a broad sense, and may be a vehicle (such as a commercial vehicle, a passenger car, a motorcycle, a flying car, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), agricultural equipment (such as a mower, a harvester, etc.), amusement equipment, a toy vehicle, etc. The embodiments of this application do not specifically limit the type of vehicle. For another example, the vehicle may be a vehicle such as an airplane or a ship.

[0122] 1 is a functional block diagram of a terminal device 100 according to an embodiment of the present application. The terminal device 100 may include a microphone array 110 and a computing platform 120, wherein the microphone array 110 may include one or more microphone arrays, or the microphone array 110 may include multiple microphones.

[0123] The computing platform 120 may include one or more processors, such as processors 121 to 12n (n is a positive integer). A processor is a circuit with signal processing capabilities. In one implementation, the processor may be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationship of a hardware circuit. The logical relationship of the hardware circuit is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field programmable gate array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. In addition, the computing platform 120 can also include a memory for storing instructions, and some or all of the processors 121 to 12n can call the instructions in the memory and execute the instructions to implement the corresponding functions. In an embodiment of the present application, the computing platform 120 can obtain touch control information corresponding to one or more microphones in the microphone array 110, and control the terminal device to perform corresponding human-computer interaction operations based on the touch control information.

[0124] The connection lines between the microphone array 110 and the computing platform 120 above indicate that the microphone array 110 and the computing platform 120 may be wirelessly connected or wiredly connected.

[0125] Figure 2 shows a schematic diagram of the configuration of the microphone array provided in an embodiment of the present application. A microphone array can be composed of two or more microphones and have a certain spatial configuration. In terms of spatial dimension, microphone arrays are generally divided into one-dimensional linear arrays, two-dimensional planar arrays and three-dimensional stereo arrays. For example, Figure 2 shows microphone arrays configured as linear arrays, circular arrays, rectangular arrays, matrix arrays and multi-arm arrays from left to right. Taking the above-mentioned terminal device 100 as a vehicle as an example, the microphone array can be a large array distributed over a large area in the cabin, or it can be a local small array. Optionally, the microphone array can be multiplexed with the sound pickup microphone or active noise reduction microphone in the cabin.

[0126] Exemplarily, taking the terminal device 100 as a vehicle as an example, FIG3 shows a schematic diagram of the arrangement of the microphone array in the vehicle according to an embodiment of the present application. As shown in FIG3 , the microphone array in the vehicle includes microphone 1, microphone 2, microphone 3, and microphone 4. Microphone 1, microphone 2, microphone 3, and microphone 4 respectively correspond to different spatial areas in the vehicle. For example, microphone 1 can be located on the ceiling of the main driving area. For another example, microphone 2 can be located on the ceiling of the co-pilot area. For another example, microphone 3 can be located on the ceiling safety armrest in the left area of ​​the second row. For another example, microphone 4 can be located on the ceiling safety armrest in the right area of ​​the second row.

[0127] The above arrangements of microphones 1, 2, 3, and 4 within the cabin are merely illustrative and are not intended to limit the present invention. For example, microphone 1 could be located on the steering wheel in the driver's seat, while microphone 3 could be located on the frame of the entertainment screen in the left second row area.

[0128] Exemplarily, FIG4 shows a set of human machine interfaces (HMIs) provided in an embodiment of the present application.

[0129] As shown in (a) in Figure 4, the vehicle's central control screen currently displays a music playback interface, wherein the music playback interface includes singer information 401 (for example, singer A), song information 402 (for example, song B), lyrics information 403, a playback progress bar 404, a like control 405, a control for switching to the previous song 406, a start / pause control 407, and a control for switching to the next song 408.

[0130] As shown in (b) of FIG4 , when a single-click operation of the microphone 1 corresponding to the main driving area is obtained, the vehicle can control the song to switch from the start playing state to the pause playing state.

[0131] When a single-click operation of microphone 1 corresponding to the main driver's area is obtained, the vehicle can control the song to switch from the on-play state to the paused play state, including: determining the single-click operation corresponding to microphone 1 based on the energy difference between the signals collected by microphone 1, microphone 2, microphone 3 and microphone 4, and / or the signal correlation between the signals collected by microphone 1, microphone 2, microphone 3 and microphone 4; and controlling the song to switch from the on-play state to the paused play state based on the single-click operation on microphone 1.

[0132] For example, within a first time period, when the energy of the signal collected by microphone 1 is greater than the energy of the signals collected by microphone 2, microphone 3, and microphone 4, and the energy difference between the signal collected by microphone 1 and the signal collected by microphone 2, the energy difference between the signal collected by microphone 1 and the signal collected by microphone 3, and the energy difference between the signal collected by microphone 1 and the signal collected by microphone 4 meet preset conditions, a single-click operation corresponding to microphone 1 can be determined.

[0133] Alternatively, the cockpit equipment may be directly controlled to perform corresponding operations based on energy differences between signals collected by multiple microphones and / or signal correlations between signals collected by multiple microphones.

[0134] Alternatively, the cockpit device can be directly controlled to perform corresponding operations based on the energy difference between signals collected by a single microphone at different times (or different time periods), and / or the signal correlation between signals collected by a single microphone at different times (or different time periods). For example, if the energy of the signal collected by microphone 1 at a first moment is less than the energy of the signal collected by microphone 1 at a second moment, and the energy difference between the signal collected by microphone 1 at the first moment and the signal collected by microphone 1 at the second moment meets a preset condition, the song can be controlled to switch from the start playback state to the pause playback state, where the first moment can be before the second moment.

[0135] In the following embodiments, an example is given of controlling a terminal device to execute an operation corresponding to the type of touch operation after obtaining the type of touch operation corresponding to one or more microphones.

[0136] In the embodiment of the present application, a single click operation on the microphone can be used to control the song to switch to a paused playback state. This eliminates the need for the user to control the song playback state through voice commands, helps improve the accuracy of human-computer interaction in high-noise or high-interference scenarios, avoids interference with other users during voice interaction, and thus helps improve the user's human-computer interaction experience.

[0137] For example, FIG5 shows another set of HMIs provided by an embodiment of the present application.

[0138] As shown in (a) in Figure 5, when a continuous touch operation corresponding to the microphone 1 of the main driving area is obtained, the vehicle can control the volume of the speaker in the main driving area to be lowered and display a prompt box 501 through the central control screen, wherein the prompt box 501 includes the prompt information "The volume of the speaker in the main driving area has been lowered for you!".

[0139] As shown in (b) in Figure 5, when a long press operation corresponding to the microphone 1 in the main driver's area is obtained, the vehicle can control the volume of the speaker in the main driver's area to increase and display a prompt box 502 through the central control screen, wherein the prompt box 502 includes a prompt message "The volume of the speaker in the main driver's area has been increased for you!".

[0140] In this embodiment of the present application, by detecting a touch or long press on microphone 1 in the driver's seat, the volume of the speaker in the driver's seat can be controlled and adjusted. This eliminates the need for the user to use voice commands to adjust the volume of the speaker in the driver's seat, helping to improve the accuracy of human-computer interaction in high-noise or high-interference environments, and avoiding interference with other users during voice interaction, thereby enhancing the user's human-computer interaction experience. Furthermore, a touch or long press on microphone 1 only adjusts the volume of the speaker in the driver's seat and does not affect the listening experience of users in other areas of the cabin.

[0141] For example, Figure 6 shows another HMI provided by an embodiment of the present application. Upon receiving a double-click on microphone 1 corresponding to the driver's seat, the vehicle can control the central control screen to switch from displaying the music playback interface to displaying the desktop, where the desktop includes a card 601 corresponding to the settings application, a card 602 corresponding to the music application, a card 603 corresponding to the vehicle's remaining battery life and remaining mileage, and a card 604 corresponding to the navigation application.

[0142] In the embodiment of the present application, by detecting a double-click operation on the microphone, the central control screen can be controlled to switch from displaying the music playback interface to displaying the desktop. This eliminates the need to set a virtual button on the central control screen to switch to displaying the desktop, nor does it require the user to control the central control screen from displaying the music playback interface to displaying the desktop through voice commands. This helps improve the accuracy of human-computer interaction in high-noise or high-interference scenarios, avoids interference with other users during voice interaction, and thus helps improve the user's human-computer interaction experience.

[0143] Figures 4 to 6 above illustrate how single-clicking microphone 1 pauses or starts a song, continuously touching microphone 1 lowers the speaker volume, long-pressing microphone 1 increases the speaker volume, and double-clicking microphone 1 switches to the desktop. In this embodiment of the present application, the correspondence between the type of microphone touch operation and the operation performed by the terminal device is not specifically limited. For example, if a continuous touching operation is detected on microphone 1 corresponding to the main driver's seat, the vehicle can control the lowering of the main driver's seat window; if a long-pressing operation is detected on microphone 1 corresponding to the main driver's seat, the vehicle can control the upper position of the main driver's seat window.

[0144] For example, when the operation of double-clicking microphone 1 is obtained, the vehicle can perform the music collection operation; when the operation of continuously tapping microphone 1 is obtained, the vehicle can control the central control screen to display the desktop; when the operation of continuously touching microphone 1 is obtained, the optimal listening configuration of the main driving area can be achieved.

[0145] FIG3 above is an example of a cabin comprising one microphone array. The following describes a scenario in which a cabin comprises multiple microphone arrays in conjunction with FIG7 to FIG9 .

[0146] Figure 7 shows a schematic diagram of the arrangement of the microphone array in the vehicle according to an embodiment of the present application. As shown in Figure 7, the microphone array in the vehicle includes microphone array 1, microphone array 2, microphone array 3 and microphone array 4, and each microphone array includes 4 microphones. Microphone array 1, microphone array 2, microphone array 3 and microphone array 4 respectively correspond to different spatial areas in the vehicle. For example, microphone array 1 can be located on the ceiling of the main driving area. For another example, microphone array 2 can be located on the ceiling of the co-pilot area. For another example, microphone array 3 can be located on the ceiling safety handrail of the second row left area. For another example, microphone array 4 can be located on the ceiling safety handrail of the second row right area.

[0147] FIG7 above illustrates an example in which the microphone arrays arranged in each area of ​​the vehicle cabin are configured as linear arrays, but the embodiments of the present application are not limited thereto. For example, the microphone array in the driver's seat area may be configured as a linear array, the microphone array in the passenger seat area may be configured as a circular array, and the microphone arrays in the second row left area and the second row right area may be configured as rectangular arrays.

[0148] For example, FIG8 shows another set of HMIs provided by an embodiment of the present application.

[0149] As shown in FIG8( a ), the entertainment screen in the left second row of the vehicle currently displays a music playback interface, which includes singer information 801 (e.g., singer A), song information 802 (e.g., song B), lyrics information 803, a playback progress bar 804, a like button 805, a control for switching to the previous song 806, a start / pause button 807, and a control for switching to the next song 808. The microphone array 3 in the left second row of the vehicle can include multiple microphones, such as microphone 5, microphone 6, microphone 7, and microphone 8.

[0150] As shown in FIG8(b), when a left-to-right swipe operation corresponding to a microphone in microphone array 3 (or a swipe operation from microphone 5 to microphone 8) is detected, the vehicle can control the entertainment screen in the left area of ​​the second row to switch to the next song. For example, the vehicle can control the entertainment screen in the left area of ​​the second row to switch from displaying the playback interface of song B to displaying the playback interface of song D.

[0151] The above-obtained operation corresponding to the microphone in the microphone array 3 sliding from left to right can also be understood as obtaining the operation of the user clicking microphones 5 to 8 in sequence.

[0152] In one embodiment, when an operation corresponding to a microphone in the microphone array 3 sliding from right to left (or a sliding operation from microphone 8 to microphone 5) is obtained, the vehicle can control the entertainment screen in the left area of ​​the second row to switch to the previous song.

[0153] In this embodiment of the present application, by detecting a directional sliding operation on multiple microphones in the microphone array in the second row on the left, the song displayed on the entertainment screen on the second row on the left can be switched. This eliminates the need for users to control song switching through voice commands, helping to improve the accuracy of human-computer interaction in high-noise or high-interference scenarios, and avoiding interference with other users during voice interaction, thereby helping to enhance the user's human-computer interaction experience.

[0154] For example, FIG9 shows another set of HMIs provided by an embodiment of the present application.

[0155] As shown in (a) in Figure 9, when an operation of sliding from left to right after clicking microphone 5 (or a sliding operation from microphone 5 to microphone 8) is obtained, the vehicle can control the volume of the speakers in the left area of ​​the second row to increase and control the entertainment screen on the left side of the second row to display a prompt box 901, wherein the prompt box 901 includes a prompt message "The volume of the left area of ​​the second row has been increased for you."

[0156] As shown in (b) in Figure 9, when an operation of sliding from right to left after clicking microphone 8 (or a sliding operation from microphone 8 to microphone 5) is obtained, the vehicle can control the volume of the speakers in the left area of ​​the second row to increase and control the entertainment screen on the left side of the second row to display a prompt box 902, wherein the prompt box 902 includes a prompt message "The volume of the left area of ​​the second row has been lowered for you."

[0157] In the embodiment of the present application, by detecting a single-click followed by a slide operation on the microphone in the second row's left area, the volume of the speakers in the second row's left area can be adjusted. This eliminates the need for users to click virtual or physical buttons to adjust the speaker volume, nor does it require users to use voice commands to adjust the speaker volume. This helps improve the accuracy of human-computer interaction in high-noise or high-interference scenarios, avoids interference with other users during voice interaction, and thus helps enhance the user's human-computer interaction experience.

[0158] The above is explained by taking the example of a sliding operation from microphone 5 to microphone 8 corresponding to switching to the next song, a sliding operation from microphone 8 to microphone 5 corresponding to switching to the previous song, a single-click on microphone 5 followed by a sliding operation from microphone 5 to microphone 8 corresponding to increasing the volume of the speakers in the second row left area, and a single-click on microphone 8 followed by a sliding operation from microphone 8 to microphone 5 corresponding to decreasing the volume of the speakers in the second row left area. In the embodiments of the present application, the correspondence between the type of touch operation and the operation performed by the terminal device is not specifically limited. For example, when an operation corresponding to a sliding operation from left to right of a microphone in microphone array 3 (or a sliding operation from microphone 5 to microphone 8) is obtained, the vehicle can control the closing of the windows in the second row left area; for another example, when an operation of single-clicking microphone 5 followed by double-clicking microphones 5 and 6 is obtained, the vehicle can execute an operation to start / stop recording.

[0159] Figures 5, 8, and 9 above illustrate how a touch operation on a microphone in a specific area can be used to control cabin devices in that area. However, embodiments of the present application are not limited thereto. For example, upon detecting a touch operation on a microphone in a specific area, the vehicle can also control cabin devices in other areas of the cabin.

[0160] For example, FIG10 shows another HMI provided by an embodiment of the present application.

[0161] When a long press operation corresponding to the rightmost microphone 9 in the microphone array 5 is obtained, the vehicle can open the window in the right area of ​​the second row and display a prompt message 1001 "The window of the right rear door is opening for you" through the central control screen.

[0162] In one embodiment, when a continuous touch operation corresponding to the rightmost microphone 9 in the microphone array 5 is obtained, the vehicle can control the windows in the right area of ​​the second row to close and display a prompt message on the central control screen "The window of the right rear door is closing for you."

[0163] In this embodiment of the present application, when a touch operation is detected on the microphone corresponding to the driver's seat, the cabin devices in other areas of the cabin can be controlled to perform corresponding operations. This eliminates the need for virtual or physical buttons in the driver's seat to control cabin devices in other areas, and eliminates the need for users to enter other areas to control cabin devices, thereby improving the efficiency of user control of cabin devices in other areas.

[0164] The above describes the scenario in which the vehicle performs corresponding operations after a touch operation on one or more microphones in a vehicle in conjunction with Figures 3 to 10. The following describes the scenario in conjunction with Figures 11 and 12 using a laptop computer and a smart desk lamp as examples, respectively.

[0165] FIG11 shows another set of HMIs provided by an embodiment of the present application.

[0166] As shown in (a) of FIG11 , a microphone array is included on the frame of the laptop computer. The microphone array includes a microphone 10 , a microphone 11 and a microphone 12 .

[0167] As shown in (b) of FIG11 , when a sliding operation corresponding to the sliding from the microphone 10 to the microphone 12 is obtained, the laptop computer can control the volume of the speaker to increase and display a prompt box 1101 through the display screen, wherein the volume bar 1102 displayed in the prompt box 1101 moves to the right.

[0168] FIG11 illustrates an example of a microphone array being located on the upper frame of a laptop computer, but the present invention is not limited thereto. For example, the microphone array may also be located on the lower frame of the laptop computer.

[0169] In the embodiment of the present application, the volume of the laptop computer can be adjusted by sliding the corresponding microphone in the microphone array. This eliminates the need for the user to click the volume icon and then drag the volume bar to adjust the volume, thus avoiding the user's tedious volume adjustment operations and helping to improve the user experience.

[0170] FIG12 shows another application scenario provided by an embodiment of the present application.

[0171] As shown in (a) of FIG12 , the smart desk lamp is provided with a microphone array, and the configuration of the microphone array is a circular array.

[0172] As shown in (b) of FIG12 , when it is detected that the user touches each microphone in a clockwise manner, the smart desk lamp can control the brightness to gradually increase.

[0173] In one embodiment, when it is detected that the user touches each microphone in a counterclockwise manner, the smart desk lamp can control the brightness to gradually decrease.

[0174] In the embodiment of the present application, the brightness of the smart desk lamp can be adjusted by sliding the microphones in the microphone array clockwise or counterclockwise. This eliminates the need for physical buttons for adjusting the brightness on the smart desk lamp, helping to reduce the production cost of the smart desk lamp.

[0175] In one embodiment, the correspondence between the above touch operation types and the operations performed by the terminal device may be set when the terminal device leaves the factory, or may be set by the user.

[0176] FIG13 shows a schematic flow chart of a control method 1300 provided in an embodiment of the present application. The method 1300 can be executed by a terminal device (e.g., a vehicle), or the method 1300 can be executed by the above-mentioned computing platform, or the method 1300 can be executed by a system consisting of a computing platform and a microphone array, or the method 1300 can be executed by a system-on-a-chip (SoC) in the above-mentioned computing platform, or the method 1300 can be executed by a processor in the computing platform. The method 1300 includes:

[0177] S1310, obtaining first touch control information corresponding to one or more microphones, where the first touch control information indicates a first operation, and the one or more microphones belong to a pre-set microphone array, where the microphones in the microphone array correspond to different spatial areas in the terminal device.

[0178] In one embodiment, the microphones in the microphone array correspond to different spatial areas in the terminal device, including: the microphones in the microphone array are used to control devices of the terminal device in different spatial areas. Taking the terminal device as a vehicle as an example, the microphones in the microphone array can be used to control the vehicle's cabin equipment in the main driver's area, the co-driver's area, the second row left area, and the second row right area. Taking the terminal device as a smart home as an example, the smart home can include smart devices in multiple rooms (for example, air conditioners, lamps, etc.) and a control device in the living room. The control device includes a microphone array, and the microphones in the microphone array can correspond to multiple rooms. When touch control information corresponding to one or more microphones in the microphone array is obtained, the smart furniture system can control the smart devices in the room.

[0179] In one embodiment, the microphones in the microphone array correspond to different spatial areas in the terminal device, including: multiple microphones in the microphone array are respectively located in different areas of the terminal device, and each microphone corresponds to the area where it is located.

[0180] Exemplarily, the microphone array can be the microphone array in Figure 3, which can include microphone 1, microphone 2, microphone 3 and microphone 4, wherein microphone 1 corresponds to the main driver's area in the vehicle, microphone 2 corresponds to the co-driver's area in the vehicle, microphone 3 corresponds to the second row left area in the vehicle and microphone 4 corresponds to the second row right area in the vehicle.

[0181] Exemplarily, the microphone array can be microphone array 1, microphone array 2, microphone array 3 and microphone array 4 in Figure 7, wherein microphone array 1 corresponds to the main driver's area in the vehicle, microphone array 2 corresponds to the co-driver's area in the vehicle, microphone array 3 corresponds to the second row left area in the vehicle and microphone array 4 corresponds to the second row right area in the vehicle.

[0182] In one embodiment, the microphones in the microphone array correspond to different spatial areas in the terminal device, including: multiple microphones in the microphone array are located in the same spatial area of ​​the terminal device, and the multiple microphones respectively correspond to different areas of the terminal device.

[0183] For example, the microphone array may be microphone array 5 located in the driver's seat area in FIG10 , wherein microphone array 5 includes four microphones, which may correspond to the driver's seat area, the passenger seat area, the second row left area, and the second row right area, respectively. For example, the rightmost microphone 9 may correspond to the second row right area.

[0184] In one embodiment, the microphones in the microphone array correspond to different spatial areas in the terminal device, including: a certain microphone in the microphone array corresponds to a different spatial area of ​​the terminal device.

[0185] Exemplarily, the microphone array can be the microphone array in Figure 3, which can include microphone 1, microphone 2, microphone 3 and microphone 4, wherein microphone 1 can correspond to the main driver area and the co-driver area in the vehicle, and microphone 2 can correspond to the second row left area and the second row right area in the vehicle.

[0186] Optionally, before obtaining the first touch control information corresponding to one or more microphones, the method further includes: obtaining a first touch operation of the user on a mechanical transmission device included in the terminal device; and controlling the mechanical transmission device to touch the one or more microphones according to the first touch operation. The mechanical transmission device added between the microphone and the person can transmit the user's touch operation to the microphone through the mechanical transmission device, instead of the user directly touching the microphone. For example, for young children in safety seats or users with hand impairments who may not be able to directly touch the microphone, the mechanical transmission device can help them achieve touch operation on the microphone, thereby helping them achieve human-computer interaction.

[0187] Optionally, special texture markings can be set around the microphones in the microphone array. The texture markings can be used to indicate the position of each microphone, which can facilitate touch operations by users in dark environments, or also facilitate touch operations by blind people.

[0188] Optionally, interactive feedback can be provided to the user's touch operation through indicator lights, buzzer prompts, voice broadcasts, motor vibrations, etc., so that the user can clearly know whether the touch operation on the microphone is successful, which helps to improve the user's human-computer interaction experience.

[0189] S1320: Control the terminal device to perform the first operation according to the first touch control information.

[0190] Optionally, the first touch control information includes a second touch operation corresponding to the one or more microphones; and controlling the terminal device to perform the first operation according to the first touch control information includes: controlling the terminal device to perform the first operation according to the second touch operation.

[0191] Optionally, obtaining first touch control information corresponding to one or more microphones includes: determining a second touch operation corresponding to the one or more microphones based on data collected by a sensor included in the terminal device.

[0192] For example, the sensor includes one or more of a camera, a lidar, and a millimeter-wave radar.

[0193] Exemplarily, the terminal device may include a code consisting of a microphone and a touch operation. For example, the microphone array may include microphone 1 to microphone M (M is a positive integer greater than 1). The touch operation of the microphone is defined, and common touch operations include single-click, double-click (defined as two single-clicks with a time interval between (t1, t2)), continuous click (defined as three or more single-clicks with a time interval between (t1, t2)), long press (defined as continuously pressing the microphone pickup hole for more than a first preset time length) and continuous touch (defined as continuously touching the microphone pickup hole for more than a second preset time length). Among them, the codes for single-click, double-click, continuous click, long press and continuous touch can be A, B, C, D and E respectively. In this way, the touch operation of any microphone can form a code consisting of letters and numbers as shown in Table 1, for example, the code for double-clicking microphone 3 is B3.

[0194] Table 1

[0195] The simultaneous or time-sharing touch operation of one or more microphones can be represented by a combination of the above codes, and the code sequence is in the form of: [code 1 code 2 ...] - [code 3 code 4 ...] - [code 5 code 6 ...] - .... Among them, "[]" is the code for simultaneous operation, and "-" connects the codes for successive operations. For example, if a touch operation is in the following order: (1) single-touch microphone 1; (2) double-touch microphone 1 and microphone 2 at the same time; (3) click microphone 1, microphone 2 and microphone 3 at the same time, the corresponding code sequence is A1-B1B2-C1C2C3.

[0196] Illustratively, the above (t1, t2) may be (0.1s, 0.5s).

[0197] Exemplarily, the first preset time length may be 2 seconds.

[0198] Exemplarily, the second preset time length may be 2 seconds.

[0199] Exemplarily, as shown in FIG4( b ), the code for the single-click operation on the microphone 1 is A1 .

[0200] For example, as shown in FIG8( b ), the code for the sliding operation from microphone 5 to microphone 8 is A5 - A6 - A7 - A8 .

[0201] For example, as shown in (a) of FIG9 , the operation of first clicking the microphone 5 and then sliding from the microphone 5 to the microphone 8 is coded as A5-A5-A6-A7-A8.

[0202] Optionally, the terminal device includes a correspondence between the type of touch operation on the microphone and the operation performed by the terminal device. For example, the type of touch operation on the microphone can be represented by the code shown in Table 1.

[0203] Optionally, the first touch control information corresponds to a microphone, and the first touch control information represents an energy difference between signals collected by the microphone at different times, and / or a signal correlation between signals collected by the microphone at different times.

[0204] Optionally, according to the first touch control information, controlling the terminal device to perform the first operation includes: controlling the terminal device to perform the first operation based on the energy difference between the signals collected by the microphone at different times, and / or the signal correlation between the signals collected by the microphone at different times.

[0205] Taking the scenario shown in Figure 3 as an example, when the energy of the signal collected by microphone 1 at time T1 is less than the energy of the signal collected by microphone 1 at time T2, and the energy difference between the signal collected by microphone 1 at time T1 and the signal collected by microphone 1 at time T2 is greater than or equal to the first preset threshold, the volume of the speaker in the main driver area can be controlled to increase, where time T2 is after time T1.

[0206] Alternatively, when the energy of the signal collected by microphone 1 at time T1 in the first frequency band is greater than the energy of the signal collected by microphone 1 at time T2 in the first frequency band, and the energy difference between the signal collected by microphone 1 at time T1 and the signal collected by microphone 1 at time T2 in the first frequency band is greater than a second preset threshold, the volume of the speaker in the main driver area can be controlled to be reduced.

[0207] For example, the first frequency band is (100 Hz, 8000 Hz).

[0208] Optionally, the first touch control information corresponds to multiple microphones, and the first touch control information represents energy differences between signals collected by the multiple microphones and / or signal correlations between the signals collected by the multiple microphones.

[0209] Touching multiple microphones in a microphone array will cause the microphone array to collect special signals that are different from regular sound signals. For example, for regular sound signals, the signals picked up by each microphone in the microphone array are basically the same or similar. However, when a microphone is touched (for example, including single clicks, double clicks, continuous taps, or continuous touches) or long pressed, a special signal that is different from regular sound signals will be generated by the microphone array, which is manifested in two aspects:

[0210] (1) A microphone that is touched or pressed for a long time will generate special signals such as strong impact or silence. For example, if the pickup hole of a microphone is touched, the microphone will generate a strong impact signal; if the pickup hole of a microphone is pressed for a long time, the microphone will generate a silence signal with a significant increase in low frequency and a significant attenuation in mid- and high-frequency.

[0211] (2) The energy difference between the signals collected by the touched or long-pressed microphone and the other microphones that are not touched and not long-pressed is large, while the energy difference between the signals collected by the other microphones that are not touched and not long-pressed is small.

[0212] Optionally, the multiple microphones include a second microphone and a third microphone; wherein the first operation is an operation corresponding to clicking the second microphone, and the first touch control information represents that within a first time period, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets a first preset condition.

[0213] Taking the microphone array including Mic1 to Mic3 as an example, FIG14 shows an energy change curve of a signal picked up by a microphone when the microphones in the microphone array provided by an embodiment of the present application are not touched and are not long pressed.

[0214] FIG14( a ) shows a time domain amplitude variation curve of the signals collected from Mic1 to Mic3 within 1 s, wherein the abscissa is time and the ordinate is the time domain amplitude.

[0215] Figure 14(b) shows the spectrum energy amplitude variation curve for the signals collected by Mic1 through Mic3 in the dashed box on the left side of Figure 14(a). The horizontal axis represents frequency, and the vertical axis represents spectrum energy amplitude. It can be seen that during the time period (0.2s, 0.23s), since Mic1 through Mic3 were not touched or long-pressed, the spectrum energy amplitudes of the signals collected by Mic1 through Mic3 were not significantly different.

[0216] Figure 14(c) shows the spectral energy amplitude variation curve for the signals collected by Mic1 through Mic3 in the dashed box on the right side of Figure 14(a). The horizontal axis represents frequency, and the vertical axis represents spectral energy amplitude. It can be seen that during the time period (0.8s, 0.83s), because Mic1 through Mic3 were not touched or long-pressed, the spectral energy amplitudes of the signals collected by Mic1 through Mic3 were similar.

[0217] FIG15 shows an energy variation curve of a signal picked up by each microphone in a microphone array when Mic1 is clicked, provided by an embodiment of the present application.

[0218] FIG15( a ) shows a time domain amplitude variation curve of the signals collected from Mic1 to Mic3 within 1 s, wherein the abscissa is time and the ordinate is the time domain amplitude.

[0219] Figure 15(b) shows the spectrum energy amplitude variation curve for the signals collected by Mic1 through Mic3 in the dashed box on the left of Figure 15(a). The horizontal axis represents frequency, and the vertical axis represents spectrum energy amplitude. It can be seen that during the time period (0.2s, 0.23s), because Mic1 through Mic3 were not touched or long-pressed, the spectrum energy amplitudes of the signals collected by Mic1 through Mic3 were similar.

[0220] Figure 15(c) shows the spectral energy amplitude variation curve for the signals collected by Mic1 through Mic3 in the dashed box on the right side of Figure 15(a), with frequency plotted on the horizontal axis and spectral energy amplitude plotted on the vertical axis. It can be seen that during the time period (0.4s, 0.43s), because Mic1 was single-clicked while Mic2 and Mic3 were untouched and not long-pressed, the spectral energy amplitude of the signal collected by Mic1 was greater than that of the signals collected by Mic2 and Mic3.

[0221] Exemplarily, the first time period may be the (0, 1s) time period shown in FIG. 15 .

[0222] Exemplarily, the Mic1 may be the second microphone, and the Mic2 and Mic3 may be the third microphone. The first preset condition may be that the difference between the spectrum energy amplitude of the signal collected by Mic1 and the spectrum energy amplitude of the signal collected by Mic2 is greater than or equal to a preset energy difference, and the difference between the spectrum energy amplitude of the signal collected by Mic1 and the spectrum energy amplitude of the signal collected by Mic3 is greater than or equal to the preset energy difference.

[0223] Exemplarily, the preset energy difference may be 10 dB.

[0224] The above single-clicking of Mic1 may be one of the above-mentioned second touch operations.

[0225] In the above, within (0.4s, 0.43s), it is obtained that the spectrum energy amplitude of the signal collected by Mic1 is greater than the spectrum energy amplitude of the signals collected by Mic2 and Mic3, and when the difference between the spectrum energy amplitude of the signal collected by Mic1 and the spectrum energy amplitude of the signal collected by Mic2 and the difference between the spectrum energy amplitude of the signal collected by Mic1 and the spectrum energy amplitude of the signal collected by Mic3 meet the first preset condition, the terminal device can be controlled to perform the corresponding operation.

[0226] Alternatively, within (0.4s, 0.43s), if the spectrum energy amplitude of the signal collected by Mic1 is greater than the spectrum energy amplitudes of the signals collected by Mic2 and Mic3, and the difference between the spectrum energy amplitudes of the signal collected by Mic1 and the spectrum energy amplitudes of the signal collected by Mic2, and the difference between the spectrum energy amplitudes of the signal collected by Mic1 and the spectrum energy amplitudes of the signal collected by Mic3 meet the first preset condition, the terminal device can determine that Mic1 is single-clicked. Thus, the terminal device can perform the operation corresponding to Mic1 being single-clicked.

[0227] Optionally, the multiple microphones include a second microphone and a third microphone; the first operation is an operation corresponding to continuously clicking or continuously touching the second microphone, and the first touch control information represents that within multiple time periods, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets the first preset condition.

[0228] The above continuously clicking the second microphone may include double-clicking the second microphone or continuously clicking the second microphone.

[0229] FIG16 shows an energy change curve of a signal picked up by each microphone in a microphone array when Mic1 is double-clicked provided by an embodiment of the present application.

[0230] FIG16( a ) shows a time domain amplitude variation curve of the signals collected from Mic1 to Mic3 within 1 second, wherein the abscissa is time and the ordinate is the time domain amplitude.

[0231] Figure 16(b) shows the spectral energy amplitude variation curve for the signals collected by Mic1 through Mic3 in the dashed box on the left of Figure 16(a), with frequency plotted on the horizontal axis and spectral energy amplitude plotted on the vertical axis. It can be seen that during the time period (0.4s, 0.43s), because Mic1 was single-clicked while Mic2 and Mic3 were untouched and not long-pressed, the spectral energy amplitude of the signal collected by Mic1 was greater than that of the signals collected by Mic2 and Mic3.

[0232] Figure 16(c) shows the spectral energy amplitude variation curve for the signals collected by Mic1 through Mic3 in the dashed box on the right side of Figure 16(a), with frequency plotted on the horizontal axis and spectral energy amplitude plotted on the vertical axis. It can be seen that during the period (0.6s and 0.63s) between two consecutive single clicks of Mic1, the spectral energy amplitudes of the signals collected by Mic1 through Mic3 remained largely unchanged, as Mic1 through Mic3 were not touched or long-pressed.

[0233] During the time period (0.6s, 0.63s), since Mic1 was single-clicked again, Mic2 and Mic3 were not touched or long-pressed, the spectrum energy amplitude of the signal collected by Mic1 was greater than the spectrum energy amplitude of the signals collected by Mic2 and Mic3. Since the time interval between the two times when the spectrum energy amplitude of the signal collected by Mic1 was greater than the spectrum energy amplitude of the signals collected by Mic2 and Mic3 was less than the preset time interval, the terminal device can perform the corresponding operation. Alternatively, the terminal device can perform the corresponding operation after determining that Mic1 was double-clicked.

[0234] Exemplarily, the preset time interval may be 0.4s.

[0235] The above double-clicking of Mic1 may be one of the above-mentioned second touch operations.

[0236] The above describes the recognition process of double-clicking Mic1 through signals collected by Mic1 to Mic3 in conjunction with FIG16 . The recognition process of consecutive clicks can also refer to the recognition process of double-clicks, which will not be repeated here.

[0237] FIG17 shows an energy variation curve of a signal picked up by each microphone in a microphone array when Mic1 is continuously stroked according to an embodiment of the present application.

[0238] FIG17( a ) shows a time domain amplitude variation curve of the signals collected from Mic1 to Mic3 within 1 s, wherein the abscissa is time and the ordinate is the time domain amplitude.

[0239] Figure 17(b) shows the spectral energy amplitude variation curve for the signals collected by Mic1 through Mic3 in the dashed box on the left of Figure 17(a), with frequency plotted on the abscissa and spectral energy amplitude plotted on the ordinate. It can be seen that during the time period (0.2s, 0.23s), because Mic1 was being stroked while Mic2 and Mic3 were untouched and not long-pressed, the spectral energy amplitude of the signal collected by Mic1 was greater than that of the signals collected by Mic2 and Mic3.

[0240] Figure 17(c) shows the spectral energy amplitude variation curve for the signals collected by Mic1 through Mic3 in the dashed box on the right side of Figure 17(a), with frequency plotted on the horizontal axis and spectral energy amplitude plotted on the vertical axis. It can be seen that during the time period (0.7s, 0.73s), because Mic1 was being stroked while Mic2 and Mic3 were not being touched or long-pressed, the spectral energy amplitude of the signal collected by Mic1 was greater than that of the signals collected by Mic2 and Mic3.

[0241] The above continuous touching of Mic1 may be one of the above second touch operations.

[0242] Optionally, the first touch control information indicates that energy of the signal collected by the second microphone is greater than or equal to a first preset threshold.

[0243] Taking single-clicking Mic1 in Figure 15 as an example, it can be determined that Mic1 is single-clicked when it is determined that the difference between the spectral energy amplitude of the signal collected by Mic1 and the spectral energy amplitude of the signal collected by Mic2 is greater than or equal to the preset energy difference and the difference between the spectral energy amplitude of the signal collected by Mic1 and the spectral energy amplitude of the signal collected by Mic3 is greater than or equal to the preset energy difference, and the average value of the spectral energy amplitude of the signal collected by Mic1 within (0.4s, 0.43s) is greater than or equal to 10dB.

[0244] The above description uses the example where the average value of the spectrum energy amplitude of the signal collected by Mic1 is greater than or equal to 10 dB, but the embodiments of the present application are not limited thereto. For example, it is also possible to determine whether the total energy of the frequency band of the signal collected by Mic1 is greater than or equal to the first preset threshold. In this case, the first preset threshold may be related to the sensitivity of the microphone, for example, the first preset threshold may be an equivalent sound pressure level of 80 dB.

[0245] Optionally, the multiple microphones include a fourth microphone and a fifth microphone; wherein the first operation is an operation corresponding to long pressing the fourth microphone, and the first touch control information represents that within the first frequency band, the energy of the signal collected by the fourth microphone is less than the energy of the signal collected by the fifth microphone, and the energy difference between the signal collected by the fourth microphone and the signal collected by the fifth microphone meets the second preset condition.

[0246] Exemplarily, the first frequency band is (100 Hz, 8000 Hz).

[0247] FIG18 shows an energy change curve of a signal picked up by each microphone in a microphone array when Mic1 is long pressed, provided by an embodiment of the present application.

[0248] (a) in FIG18 shows a time-domain amplitude variation curve of the signals collected from Mic1 to Mic3 within 1 second, wherein the abscissa is time and the ordinate is the time-domain amplitude.

[0249] Figure 18(b) shows the spectral energy amplitude variation curve for the signals collected by Mic1 through Mic3 in the left dashed box in Figure 18(a), with frequency plotted on the horizontal axis and spectral energy amplitude plotted on the vertical axis. It can be seen that during the time period (0.2s, 0.23s), because Mic1 was long-pressed while Mic2 and Mic3 were untouched and not long-pressed, the spectral energy amplitude of the signal collected by Mic1 in the frequency band (100 Hz, 8000 Hz) was smaller than the spectral energy amplitude of the signals collected by Mic2 and Mic3.

[0250] Figure 18(c) shows the spectral energy amplitude variation curve for the signals collected by Mic1 through Mic3 in the dashed box on the right side of Figure 18(a), with frequency plotted on the horizontal axis and spectral energy amplitude plotted on the vertical axis. It can be seen that during the time period (0.7s, 0.73s), because Mic1 was long-pressed while Mic2 and Mic3 were untouched and not long-pressed, the spectral energy amplitude of the signal collected by Mic1 in the frequency band (100 Hz, 8000 Hz) was smaller than the spectral energy amplitude of the signals collected by Mic2 and Mic3.

[0251] The above Mic1 can be the fourth microphone, and Mic2 and Mic3 can be the fifth microphone. The second preset condition can be that within (100 Hz, 8000 Hz), the difference between the spectrum energy amplitude of the signal collected by Mic2 and the spectrum energy amplitude of the signal collected by Mic1 is greater than or equal to a preset energy difference, and the difference between the spectrum energy amplitude of the signal collected by Mic3 and the spectrum energy amplitude of the signal collected by Mic1 is greater than or equal to the preset energy difference.

[0252] Optionally, the first touch control information indicates that energy of the signal collected by the fourth microphone is less than or equal to a second preset threshold.

[0253] Taking the long press of Mic1 in Figure 18 as an example, it can be determined that Mic1 is long pressed when it is determined that the difference between the spectral energy amplitude of the signal collected by Mic2 and the spectral energy amplitude of the signal collected by Mic1 is greater than or equal to the preset energy difference and the difference between the spectral energy amplitude of the signal collected by Mic3 and the spectral energy amplitude of the signal collected by Mic1 is greater than or equal to the preset energy difference, and the average value of the spectral energy amplitude of the signal collected by Mic1 within (100hz, 8000hz) is less than or equal to -50dB.

[0254] The above description uses the example of the average value of the spectrum energy amplitude of the signal collected by Mic1 being less than or equal to -50 dB, but the embodiments of the present application are not limited thereto. For example, it is also possible to determine whether the total energy of the frequency band of the signal collected by Mic1 is less than or equal to the second preset threshold. In this case, the second preset threshold may be related to the sensitivity of the microphone, for example, the second preset threshold may be an equivalent sound pressure level of 40 dB.

[0255] The above long pressing of Mic1 may be one of the above second touch operations.

[0256] FIG19 shows a schematic diagram of determining whether a microphone is touched or long pressed through signal correlation according to an embodiment of the present application.

[0257] Figure 19(a) shows the frequency domain correlation test curve of the signals collected by Mic1 and Mic2 within 1 second, where the horizontal axis is time and the vertical axis is frequency. It can be seen that the signal correlation between Mic1 and Mic2 is poor within (0.5s, 0.7s).

[0258] Figure 19(b) shows the frequency domain correlation test curve of the signals collected by Mic1 and Mic3 within 1 second, where the horizontal axis is time and the vertical axis is frequency. It can be seen that the signal correlation between Mic1 and Mic3 is poor within (0.5s, 0.7s).

[0259] Figure 19(a) shows the frequency domain correlation test curve of the signals collected by Mic2 and Mic3 within 1 second, with the horizontal axis representing time and the vertical axis representing frequency. It can be seen that the signals of Mic2 and Mic3 are well correlated within (0.5s and 0.7s).

[0260] In (a), (b) and (c) shown in FIG19 above, the brighter the part, the worse the signal correlation.

[0261] When it is obtained that the signal correlation between Mic1 and Mic2 is poor, the signal correlation between Mic1 and Mic3 is relatively good, and the signal correlation between Mic2 and Mic3 is relatively good, it can be determined that Mic1 is touched or long pressed, thereby controlling the terminal device to perform corresponding operations.

[0262] Alternatively, when it is obtained that the signal correlation between Mic1 and Mic2 is poor, the signal correlation between Mic1 and Mic3 is relatively good, and the signal correlation between Mic2 and Mic3 is relatively good, the terminal device can be directly controlled to perform corresponding operations.

[0263] By picking up the signal collected by the microphone array and detecting the special signal in the signal, the type of touch operation on the microphone can be determined. The special signal generally indicates that the microphone pickup hole is touched or pressed for a long time. For example, the signal of each microphone in the microphone array is x m (t), where m = 1, 2, ..., M, t is the time series. The signal is sampled and framed, and the sampling frequency is f s , the frame length is N, then the signal of each frame is Where n=1,2,…,N,l is the frame sequence. Fast Fourier transform to spectrum signal Where k is a frequency domain subband sequence, k = 1, 2, ..., N. The following dimensions can be used to determine whether the microphone is touched or long pressed.

[0264] (1) Time domain amplitude

[0265] Exemplarily, the time domain amplitude is the absolute value of the time domain signal, that is, Select a representative amplitude, such as the maximum amplitude, then the maximum amplitude of the signal collected by the mth microphone is If the maximum amplitude of the signal collected by the mth microphone is greater than or equal to a certain threshold and is significantly greater than the time domain amplitude of the signals collected by one or more other microphones, it can be determined that the mth microphone is touched; if the maximum amplitude of the signal collected by the mth microphone is less than or equal to a certain threshold and is significantly less than the time domain amplitude of the signals collected by one or more other microphones, it can be determined that the mth microphone is long pressed.

[0266] (2) Total energy in time domain

[0267] For example, the total time domain energy of the signals collected by each microphone is Here, p is generally set to 1 or 2. If the total time-domain energy of the signal collected by the mth microphone is greater than or equal to a certain threshold and significantly greater than the total time-domain energy of the signals collected by one or more other microphones, it can be determined that the mth microphone is touched. If the total time-domain energy of the signal collected by the mth microphone is less than or equal to a certain threshold and significantly less than the total time-domain energy of the signals collected by one or more other microphones, it can be determined that the mth microphone is long-pressed.

[0268] (3) Frequency domain subband amplitude

[0269] For example, the frequency domain sub-band amplitude of the signal collected by each microphone is Select a representative frequency domain sub-band amplitude, such as the maximum sub-band amplitude, and the maximum frequency domain sub-band amplitude of the signal collected by each microphone is If the maximum frequency domain sub-band amplitude of the signal collected by the mth microphone is greater than or equal to a certain threshold and is significantly greater than the maximum frequency domain sub-band amplitude of the signals collected by one or more other microphones, it can be determined that the mth microphone is touched; if the maximum frequency domain sub-band amplitude of the signal collected by the mth microphone is less than or equal to a certain threshold and is significantly less than the maximum frequency domain sub-band amplitude of the signals collected by one or more other microphones, it can be determined that the mth microphone is long pressed.

[0270] (4) Total energy of the frequency band

[0271] For example, the total energy of the frequency band of the signal collected by each microphone is Where n1 and n2 are the starting and ending frequencies of the frequency band, and p is generally 1 or 2. If the total frequency band energy of the signal from the mth microphone is greater than or equal to a certain threshold and significantly greater than the total frequency band energy of the signals collected by one or more other microphones, it can be determined that the mth microphone is touched. If the total frequency band energy of the signal collected by the mth microphone is less than or equal to a certain threshold and significantly less than the total frequency band energy of the signals collected by one or more other microphones, it can be determined that the mth microphone is long-pressed.

[0272] (5) Signal correlation

[0273] For example, taking the frequency domain correlation of the signal as an example, the frequency band correlation coefficient of the mth microphone and the other m′th microphones is defined as

[0274] Where η(n) is the frequency band correlation coefficient between the mth microphone and other m′th microphones, (·) * represents conjugate, E{·} represents expectation, represents the spectrum signal of the m′th microphone. The frequency band average correlation coefficient of the mth microphone and the m′th microphone is If the frequency band average correlation coefficient between the m-th microphone and other microphones is less than or equal to a certain threshold, it can be determined that the m-th microphone is touched or long pressed.

[0275] Through the above steps, after obtaining the energy difference and / or signal correlation between multiple microphones, the terminal device can be directly controlled to perform corresponding operations; or, after obtaining the energy difference and / or signal correlation between multiple microphones, the type of touch operation on certain microphones among the multiple microphones can be determined, thereby controlling the terminal device to perform the operation corresponding to the type of touch operation.

[0276] The above method of determining whether the microphone is touched or long pressed is merely illustrative, and the embodiments of the present application are not limited thereto. For example, the type of touch operation on the microphone can also be determined by the average power of the signal collected by the microphone.

[0277] Optionally, the terminal device is a vehicle including a cabin, the one or more microphones are located in a first area within the cabin, and controlling the terminal device to perform the first operation includes: controlling the cabin device in the first area to perform the first operation.

[0278] For example, as shown in (a) of FIG5 , when a continuous touch operation corresponding to the microphone 1 in the main driving area is obtained, the vehicle may control the volume of the speaker in the main driving area to be lowered.

[0279] For example, as shown in (b) of FIG5 , when a long press operation of the microphone 1 corresponding to the main driving area is obtained, the vehicle can control the volume of the speaker in the main driving area to increase.

[0280] For example, as shown in (b) in Figure 8, when a sliding operation corresponding to from microphone 5 to microphone 8 is obtained, the vehicle can control the display screen on the left side of the second row to switch from displaying the playback interface of song B to displaying the playback interface of song D.

[0281] Optionally, the terminal device is a vehicle including a cabin, the cabin including a first area and a second area, the one or more microphones are located in the first area, and controlling the terminal device to perform the first operation includes: controlling the cabin device in the second area to perform the first operation.

[0282] For example, as shown in FIG10 , when a long press operation corresponding to the rightmost microphone 9 in the microphone array 5 is obtained, the vehicle can open the windows in the right area of ​​the second row and display a prompt message 1001 “The window of the right rear door is opening for you” on the central control screen.

[0283] Optionally, the terminal device is a vehicle including a cabin, the cabin including multiple areas, and the one or more microphones include a first microphone corresponding to the multiple areas.

[0284] The above first microphone corresponding to the multiple areas can be understood as the first microphone being able to control the cabin equipment in the multiple areas.

[0285] For example, as shown in Figure 3, microphone 1 can correspond to the second row left area and the second row right area. For example, if the cabin device is a speaker, a single-click on microphone 1 can increase the volume of the second row left speaker; a double-click on microphone 1 can decrease the volume of the second row left speaker; a double-click on microphone 1 can increase the volume of the second row right speaker; and a long-press on microphone 1 can decrease the volume of the second row right speaker.

[0286] Optionally, the terminal device is a vehicle, and the first operation includes controlling the opening or closing of a vehicle window, adjusting the volume intensity of a sound-emitting device in the vehicle, switching multimedia files, controlling the locking or unlocking of a vehicle door, or controlling a display device in the vehicle to display the desktop.

[0287] Optionally, the terminal device is a vehicle, and the first operation includes launching an application. For example, Table 2 shows a correspondence between a touch operation type and an launched application.

[0288] Table 2

[0289] For example, a user in the driver's seat can launch a navigation app by tapping one or more microphones in the microphone array. This eliminates the need for the user to search for the desired app among multiple apps while driving, helping to improve driving safety.

[0290] The correspondence between the touch operation type and the launched application in Table 2 above is merely illustrative and is not specifically limited in this embodiment of the present application.

[0291] The corresponding relationship in Table 2 above may be configured before the vehicle leaves the factory, or may be manually configured by the user. When the corresponding relationship is manually configured by the user, users in different regions may configure their desired corresponding relationship respectively.

[0292] For example, the user in the main driving area can configure the correspondence between a single-click operation and a setting application. In this way, when the user's single-click operation on microphone 1 is obtained, the vehicle can control the central control screen to display the display interface of the setting application.

[0293] For another example, the user in the passenger seat can configure the correspondence between a single-click operation and a video application. In this way, when a single-click operation of the user on the microphone 2 is obtained, the vehicle can control the entertainment screen in the passenger seat to display the display interface of the video application.

[0294] In an embodiment of the present application, when the vehicle obtains different types of touch operations, it can control the display screen to display different types of applications. This eliminates the need for users to search for the application they want to open among multiple applications on the display screen, helping to improve the efficiency of users in launching applications and thus helping to improve the user's human-computer interaction experience.

[0295] Optionally, the microphone array is configured as a linear array, a circular array, a rectangular array or a multi-arm array.

[0296] Figure 20 shows a schematic block diagram of a control device 2000 provided in an embodiment of the present application. As shown in Figure 20, the device 2000 includes: an acquisition unit 2010, configured to acquire first touch control information corresponding to one or more microphones, the first touch control information indicating a first operation, the one or more microphones belonging to a pre-set microphone array, the microphones in the microphone array corresponding to different spatial regions in a terminal device; and a control unit 2020, configured to control the terminal device to perform the first operation based on the first touch control information.

[0297] Optionally, the terminal device is a vehicle including a cabin, the one or more microphones are located in a first area within the cabin, and the control unit 2020 is used to control the cabin equipment in the first area to perform the first operation.

[0298] Optionally, the terminal device is a vehicle including a cabin, the cabin including a first area and a second area, the one or more microphones are located in the first area, and the control unit 2020 is used to control the cabin device in the second area to perform the first operation.

[0299] Optionally, the terminal device is a vehicle, and the first operation includes controlling the opening or closing of a vehicle window, adjusting the volume intensity of a sound-emitting device in the vehicle, switching multimedia files, controlling the locking or unlocking of a vehicle door, or controlling a display device in the vehicle to display the desktop.

[0300] Optionally, the microphone array is configured as a linear array, a circular array, a rectangular array or a multi-arm array.

[0301] Optionally, the terminal device also includes a mechanical transmission device, and the acquisition unit 2010 is further used to obtain the user's first touch operation on the mechanical transmission device before obtaining the first touch control information corresponding to one or more microphones; the control unit 2020 is further used to control the mechanical transmission device to touch the one or more microphones according to the first touch operation.

[0302] Optionally, the first touch control information includes a second touch operation corresponding to the one or more microphones; the control unit 2020 is used to control the terminal device to perform the first operation according to the second touch operation.

[0303] Optionally, the first touch control information corresponds to a microphone, and the first touch control information represents an energy difference between signals collected by the microphone at different times, and / or a signal correlation between signals collected by the microphone at different times.

[0304] Optionally, the first touch control information corresponds to multiple microphones, and the first touch control information represents energy differences between signals collected by the multiple microphones and / or signal correlations between the signals collected by the multiple microphones.

[0305] Optionally, the multiple microphones include a second microphone and a third microphone; wherein the first operation is an operation corresponding to clicking the second microphone, and the first touch control information represents that within a first time period, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets a first preset condition.

[0306] Optionally, the multiple microphones include a second microphone and a third microphone; the first operation is an operation corresponding to continuously clicking or continuously touching the second microphone, and the first touch control information represents that within multiple time periods, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets the first preset condition.

[0307] Optionally, the first touch control information indicates that energy of the signal collected by the second microphone is greater than or equal to a first preset threshold.

[0308] Optionally, the multiple microphones include a fourth microphone and a fifth microphone; wherein the first operation is an operation corresponding to long pressing the fourth microphone, and the first touch control information represents that within the first frequency band, the energy of the signal collected by the fourth microphone is less than the energy of the signal collected by the fifth microphone, and the energy difference between the signal collected by the fourth microphone and the signal collected by the fifth microphone meets the second preset condition.

[0309] Optionally, the first touch control information indicates that energy of the signal collected by the fourth microphone is less than or equal to a second preset threshold.

[0310] Optionally, the energy difference includes at least one of a difference in time domain amplitude, a difference in time domain energy, a difference in frequency domain sub-band amplitude, and a difference in frequency band energy.

[0311] For example, the acquisition unit 2010 may be the computing platform in FIG1 or a processing circuit, processor, or controller in the computing platform. For example, if the acquisition unit 2010 is the processor 121 in the computing platform, the processor 121 may acquire first touch control information corresponding to one or more microphones. For example, the processor 121 may acquire signals collected by the one or more microphones.

[0312] For another example, the determining unit may be the computing platform in Figure 1 or a processing circuit, processor, or controller in the computing platform. For example, if the determining unit is the processor 122 in the computing platform, the processor 122 may analyze the signals collected by the one or more microphones obtained by the processor 121, such as by analyzing energy differences and / or signal correlations in the signals collected by the microphones, to determine the type of touch operation on the one or more microphones.

[0313] For another example, the control unit 2020 may be the computing platform in FIG1 or a processing circuit, processor, or controller in the computing platform. Taking the control unit 2020 as the processor 123 in the computing platform as an example, the processor 123 may control the terminal device to perform an operation corresponding to the type of touch operation on the one or more microphones determined by the processor 122.

[0314] The functions implemented by the above-mentioned acquisition unit 2010, the functions implemented by the determination unit and the functions implemented by the control unit 2020 can be implemented by different processors, or some functions can be implemented by the same processor, or they can also be implemented by the same processor. The embodiments of the present application do not limit this.

[0315] It should be understood that the division of the various units in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a single physical entity, or they may be physically separated. Furthermore, the units in the device may be implemented in the form of a processor calling software; for example, the device includes a processor connected to a memory storing instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or the functions of the various units in the device, where the processor is, for example, a general-purpose processor such as a CPU or a microprocessor, and the memory is a memory within the device or a memory external to the device. Alternatively, the units in the device may be implemented in the form of hardware circuits, and the functions of some or all of the units may be implemented through the design of the hardware circuits. The hardware circuits may be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units may be implemented through the design of the logical relationships between the components within the circuits. In another implementation, the hardware circuit may be implemented using a PLD, such as an FPGA, which may include a large number of logic gate circuits, and the connections between the logic gate circuits may be configured using a configuration file to implement the functions of some or all of the above units. All units of the above apparatus may be implemented entirely in the form of software called by a processor, or entirely in the form of hardware circuits, or partially in the form of software called by a processor and the rest in the form of hardware circuits.

[0316] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and execute instructions, such as a CPU, a microprocessor, a GPU, or a DSP. In another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit may be fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0317] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0318] In addition, the various units in the above apparatus may be fully or partially integrated together, or may be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the various units of the apparatus. The at least one processor may be of different types, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0319] An embodiment of the present application also provides a control device, which includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the device to execute the control method executed by the above embodiment.

[0320] Optionally, if the apparatus is located in a terminal device, the processing unit may be the processor 121 - 12n shown in FIG. 1 .

[0321] An embodiment of the present application further provides a control system, which may include a computing platform and a microphone array, and the computing platform may include the above-mentioned control device 2000.

[0322] Optionally, the control system may also include cockpit equipment.

[0323] Optionally, the control system may further include a mechanical transmission device.

[0324] An embodiment of the present application further provides a terminal device, which may include the above-mentioned control device 2000 or control system.

[0325] Optionally, the terminal device may be a vehicle (eg, a vehicle).

[0326] An embodiment of the present application further provides a computer program product, which includes: computer program code, which, when executed on a computer, enables the computer to execute the control method in the above embodiment.

[0327] An embodiment of the present application further provides a computer-readable medium, wherein the computer-readable medium stores a program code. When the computer program code is run on a computer, the computer executes the control method in the above embodiment.

[0328] An embodiment of the present application further provides a chip, which includes a circuit, and the circuit is used to execute the control method in the above embodiment.

[0329] During implementation, each step of the above method can be completed by hardware integrated logic circuits in a processor or by software instructions. The methods disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware processor, or by a combination of hardware and software modules in the processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or power-on erasable programmable memory, registers, etc. The storage medium is located in a memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not given here.

[0330] It should be understood that in the embodiment of the present application, the memory may include a read-only memory and a random access memory, and provide instructions and data to the processor.

[0331] It should also be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0332] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0333] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0334] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0335] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0336] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0337] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0338] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be covered and fall within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A control method, characterized in that: include: Acquire first touch control information corresponding to one or more microphones, where the first touch control information indicates a first operation, where the one or more microphones belong to a pre-set microphone array, and where microphones in the microphone array correspond to different spatial regions in the terminal device; According to the first touch control information, the terminal device is controlled to perform the first operation.

2. The method according to claim 1, characterized in that The terminal device is a vehicle, the vehicle includes a cabin, the one or more microphones are located in a first area in the cabin, and controlling the terminal device to perform the first operation includes: Controlling cockpit equipment in the first area to perform the first operation.

3. The method according to claim 1, characterized in that The terminal device is a vehicle, the vehicle includes a cabin, the cabin includes a first area and a second area, the one or more microphones are located in the first area, and controlling the terminal device to perform the first operation includes: Controlling the cockpit equipment in the second area to perform the first operation.

4. The method according to claim 1, characterized in that The terminal device is a vehicle, the vehicle includes a cabin, the cabin includes multiple areas, the one or more microphones include a first microphone, and the first microphone corresponds to the multiple areas.

5. The method according to any one of claims 1 to 4, characterized in that The terminal device is a vehicle, and the first operation includes controlling the opening or closing of a window, adjusting the volume intensity of a sound device in the vehicle, switching multimedia files, controlling the locking or unlocking of a door, or controlling a display device in the vehicle to display a desktop.

6. The method according to any one of claims 1 to 5, characterized in that The microphone array is configured as a linear array, a circular array, a rectangular array or a multi-arm array.

7. The method according to any one of claims 1 to 6, characterized in that Before acquiring the first touch control information corresponding to the one or more microphones, the method further includes: Acquiring a first touch operation of a user on a mechanical transmission device included in the terminal device; According to the first touching operation, the mechanical transmission device is controlled to touch the one or more microphones.

8. The method according to any one of claims 1 to 7, characterized in that The first touch control information includes a second touch operation corresponding to the one or more microphones; The controlling the terminal device to perform the first operation according to the first touch control information includes: According to the second touch operation, the terminal device is controlled to perform the first operation.

9. The method according to any one of claims 1 to 7, characterized in that The first touch control information corresponds to a microphone, and the first touch control information represents energy differences between signals collected by the microphone at different times, and / or signal correlations between signals collected by the microphone at different times.

10. The method according to any one of claims 1 to 7, characterized in that The first touch control information corresponds to a plurality of microphones, and the first touch control information represents energy differences between signals collected by the plurality of microphones and / or signal correlations between signals collected by the plurality of microphones.

11. The method according to claim 10, characterized in that The plurality of microphones include a second microphone and a third microphone; Among them, the first operation is an operation corresponding to single-clicking the second microphone, and the first touch control information represents that within a first time period, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets a first preset condition.

12. The method according to claim 10, characterized in that The plurality of microphones include a second microphone and a third microphone; The first operation is an operation corresponding to continuously clicking or continuously touching the second microphone, and the first touch control information represents that in multiple time periods, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets the first preset condition.

13. The method according to claim 11 or 12, characterized in that The first touch control information indicates that energy of a signal collected by the second microphone is greater than or equal to a first preset threshold.

14. The method according to claim 10, characterized in that The plurality of microphones include a fourth microphone and a fifth microphone; Among them, the first operation is an operation corresponding to long pressing the fourth microphone, and the first touch control information represents that within the first frequency band, the energy of the signal collected by the fourth microphone is less than the energy of the signal collected by the fifth microphone, and the energy difference between the signal collected by the fourth microphone and the signal collected by the fifth microphone meets the second preset condition.

15. The method according to claim 14, characterized in that The first touch control information indicates that energy of the signal collected by the fourth microphone is less than or equal to a second preset threshold.

16. The method according to any one of claims 9 to 15, characterized in that The energy difference includes at least one of a difference in time domain amplitude, a difference in time domain energy, a difference in frequency domain sub-band amplitude, and a difference in frequency band energy.

17. A control device, characterized in that: include: an acquisition unit, configured to acquire first touch control information corresponding to one or more microphones, wherein the first touch control information indicates a first operation, the one or more microphones belong to a pre-set microphone array, and the microphones in the microphone array correspond to different spatial areas in the terminal device; A control unit is used to control the terminal device to perform the first operation according to the first touch control information.

18. The device according to claim 17, characterized in that The terminal device is a vehicle, the vehicle includes a cabin, the one or more microphones are located in a first area in the cabin, and the control unit is used to: Controlling cockpit equipment in the first area to perform the first operation.

19. The device according to claim 17, characterized in that The terminal device is a vehicle, the vehicle includes a cabin, the cabin includes a first area and a second area, the one or more microphones are located in the first area, and the control unit is used to: Controlling the cockpit equipment in the second area to perform the first operation.

20. The device according to claim 17, characterized in that The terminal device is a vehicle, the vehicle includes a cabin, the cabin includes multiple areas, the one or more microphones include a first microphone, and the first microphone corresponds to the multiple areas.

21. The device according to any one of claims 17 to 20, characterized in that The terminal device is a vehicle, and the first operation includes controlling the opening or closing of a window, adjusting the volume intensity of a sound device in the vehicle, switching multimedia files, controlling the locking or unlocking of a door, or controlling a display device in the vehicle to display a desktop.

22. The device according to any one of claims 17 to 21, characterized in that The microphone array is configured as a linear array, a circular array, a rectangular array or a multi-arm array.

23. The device according to any one of claims 17 to 22, characterized in that The terminal device also includes a mechanical transmission device, The acquisition unit is further configured to acquire a first touch operation of the user on the mechanical transmission device before acquiring the first touch control information corresponding to the one or more microphones; The control unit is further configured to control the mechanical transmission device to touch the one or more microphones according to the first touch operation.

24. The device according to any one of claims 17 to 23, characterized in that The first touch control information includes a second touch operation corresponding to the one or more microphones; The control unit is used to control the terminal device to perform the first operation according to the second touch operation.

25. The device according to any one of claims 17 to 23, characterized in that The first touch control information corresponds to a microphone, and the first touch control information represents energy differences between signals collected by the microphone at different times, and / or signal correlations between signals collected by the microphone at different times.

26. The device according to any one of claims 17 to 23, characterized in that The first touch control information corresponds to a plurality of microphones, and the first touch control information represents energy differences between signals collected by the plurality of microphones and / or signal correlations between signals collected by the plurality of microphones.

27. The device according to claim 26, characterized in that The plurality of microphones include a second microphone and a third microphone; Among them, the first operation is an operation corresponding to single-clicking the second microphone, and the first touch control information represents that within a first time period, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets a first preset condition.

28. The device according to claim 26, characterized in that The plurality of microphones include a second microphone and a third microphone; The first operation is an operation corresponding to continuously clicking or continuously touching the second microphone, and the first touch control information represents that in multiple time periods, the energy of the signal collected by the second microphone is greater than the energy of the signal collected by the third microphone, and the energy difference between the signal collected by the second microphone and the signal collected by the third microphone meets the first preset condition.

29. The device according to claim 27 or 28, characterized in that The first touch control information indicates that energy of a signal collected by the second microphone is greater than or equal to a first preset threshold.

30. The device according to claim 26, characterized in that The plurality of microphones include a fourth microphone and a fifth microphone; Among them, the first operation is an operation corresponding to long pressing the fourth microphone, and the first touch control information represents that within the first frequency band, the energy of the signal collected by the fourth microphone is less than the energy of the signal collected by the fifth microphone, and the energy difference between the signal collected by the fourth microphone and the signal collected by the fifth microphone meets the second preset condition.

31. The device according to claim 30, characterized in that The first touch control information indicates that energy of the signal collected by the fourth microphone is less than or equal to a second preset threshold.

32. The device according to any one of claims 25 to 31, characterized in that The energy difference includes at least one of a difference in time domain amplitude, a difference in time domain energy, a difference in frequency domain sub-band amplitude, and a difference in frequency band energy.

33. A control device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute a computer program stored in the memory so that the device performs the control method according to any one of claims 1 to 16.

34. A control system, characterized in that: It comprises a microphone array and a computing platform, wherein the computing platform comprises a control device as described in any one of claims 16 to 33.

35. A terminal device, characterized in that: It comprises a control device as claimed in any one of claims 16 to 33, or comprises a control system as claimed in claim 34.

36. The terminal device according to claim 35, characterized in that The terminal device is a vehicle.

37. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the control method according to any one of claims 1 to 16 is implemented.

38. A chip, characterized in that: The chip includes a circuit for executing the control method according to any one of claims 1 to 16.

Images