Display device and sound processing method of display device

Abstract

The embodiment of the invention belongs to the display technology, and provides a display device and a sound processing method of the display device, the display device comprises a separate microphone, a built-in microphone module and a sound control module, and the built-in microphone module is connected with the separate microphone and the sound control module. The separation microphone is configured to collect first sound signals of different frames within a preset first distance. The built-in microphone module is configured to receive first sound signals of different frames and collect second sound signals of different frames within a preset second distance. The sound control module is configured to process the first sound signal and the second sound signal, wherein the processing comprises sound amplification processing and echo cancellation processing. In the application, the sound signal collected by the microphone can be directly received and separated through the built-in microphone module of the display device, and a sound console device does not need to be independently configured, so that the tedious time consumption of switching the sound transmission path through the sound console device by manpower is reduced, and the sound acquisition efficiency and the intelligent degree are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to display technology, and more specifically, to a display device and a sound processing method of the display device. Background Art

[0002] With the rapid development of display devices, more and more display devices are used in the teaching field. Display devices in teaching scenarios are usually equipped with separate microphones and built-in microphones to achieve multi-channel pickup of user voices.

[0003] In the prior art, a mixer device is usually required to be separately configured in the display device. Depending on the type of microphone that picks up the sound (separate microphone, built-in microphone), the user manually switches the sound transmission path of the mixer device to the sound path corresponding to the microphone type. The picked up sound is then transmitted to the microphone module corresponding to the display device and the microphone type through the path, so that the corresponding microphone module controls the acquired sound.

[0004] However, the existing technology requires the display device to be separately equipped with a mixer device, and the method of obtaining sound is cumbersome and time-consuming, and the degree of intelligence is low. Summary of the invention

[0005] The embodiments of the present application provide a display device and a sound processing method for the display device, which can be used to solve the problem in the prior art that the display device needs to be separately configured with a mixing console device, the method of obtaining sound is cumbersome and time-consuming, and the degree of intelligence is low.

[0006] In a first aspect, an embodiment of the present application provides a display device, the display device comprising:

[0007] A separation microphone, wherein the separation microphone is configured to: collect first sound signals of different frames within a preset first distance;

[0008] a built-in microphone module connected to the separate microphone, the built-in microphone module being configured to: receive the first sound signals of the different frames, and collect the second sound signals of the different frames within a preset second distance, the preset first distance not exceeding the preset second distance;

[0009] A sound control module connected to the built-in microphone module is configured to process the first sound signal and the second sound signal, wherein the processing includes sound amplification processing and echo cancellation processing.

[0010] In the present application, the built-in microphone module of the display device can directly receive the sound signal collected by the separate microphone. At the same time, it can also directly collect the sound signal without the need to separately configure a mixing console device, thereby reducing the tedious and time-consuming manpower switching of the sound transmission path through the mixing console device, improving the efficiency and intelligence of sound acquisition, and enhancing the user experience.

[0011] In some embodiments of the present application, the separation microphone includes: a first separation microphone and a second separation microphone;

[0012] Accordingly, when the built-in microphone module is configured to receive the first sound signal of the different frames, the specific configuration is as follows:

[0013] If the separation microphone is the first separation microphone, receiving the first sound signal of the different frame through a preset first priority separation microphone channel;

[0014] If the separate microphone is the second separate microphone, the first sound signal of the different frame is received through a preset second priority separate microphone channel.

[0015] In the present application, the separate microphone includes a first separate microphone and a second separate microphone. Different separate microphones have corresponding respective paths. By setting paths with different priorities, important sound signals can be transmitted to the built-in microphone module in priority.

[0016] In some embodiments of the present application, the built-in microphone module is further configured as follows:

[0017] If the first sound signals of different frames are received through the preset first priority separation microphone path, the sound output mode of the built-in microphone module is determined according to the signal strength of the first sound signal within a continuous preset number of frames;

[0018] The sound output mode includes: a first separate microphone sound output mode and a built-in microphone sound output mode. The first separate microphone sound output mode requires amplification of the sound signal, while the built-in microphone sound output mode does not require amplification of the sound signal.

[0019] In the present application, if the first sound signal is received through a preset first priority separation microphone channel, the sound output mode of the built-in microphone module is determined to be more accurate based on the signal strength of the first sound signal within a continuous preset number of frames, thereby making the output of the first sound signal more accurate.

[0020] In some embodiments of the present application, when the built-in microphone module is configured to determine the sound output mode of the built-in microphone module according to the signal strength of the first sound signal within a continuous preset number of frames, the built-in microphone module is specifically configured as follows:

[0021] If the signal strength of the first sound signal within a first preset number of consecutive frames reaches a first preset signal strength threshold, determining that the sound output mode of the built-in microphone module is the first separated microphone sound output mode;

[0022] If the signal strength of the first sound signal within a second consecutive preset number of frames does not reach the first preset signal strength threshold, the sound output mode of the built-in microphone module is determined to be the built-in microphone sound output mode.

[0023] In the present application, if the signal strength of the first sound signal reaches the first preset signal strength threshold within the first consecutive preset number of frames, it is the first separated microphone sound output mode, and if the signal strength of the first sound signal does not reach the first preset signal strength threshold within the second consecutive preset number of frames, it is the built-in microphone sound output mode. This makes the sound control according to the determined sound output mode more accurate and convenient, and improves the degree of intelligence.

[0024] In some embodiments of the present application, the built-in microphone module is further configured as follows:

[0025] If the sound output mode of the built-in microphone module is the first separate microphone sound output mode, the first sound signal is sent to the sound control module, so that the sound control module performs sound amplification processing according to the first sound signal.

[0026] In the present application, if it is the first separate microphone sound output mode, after the built-in microphone module obtains the first sound signal, it needs to send it to the sound control module to amplify the sound, thereby improving the control efficiency and utilization rate of the first sound signal and enhancing the user experience.

[0027] In some embodiments of the present application, the built-in microphone module is further configured as follows:

[0028] If the first sound signals of different frames are received only through the preset second priority separation microphone path, determining the sound output mode of the built-in microphone module according to the signal strength of the first sound signal within a continuous preset number of frames;

[0029] The sound output mode includes: a second separate microphone sound output mode and a built-in microphone sound output mode, and both the second separate microphone sound output mode and the built-in microphone sound output mode do not require amplification of the sound signal.

[0030] In the present application, if the first sound signal is received only through the preset second priority separation microphone channel, the sound output mode of the built-in microphone module is determined to be more accurate based on the signal strength of the first sound signal within a continuous preset number of frames, thereby making the output of the first sound signal more accurate.

[0031] In some embodiments of the present application, when the built-in microphone module is configured to determine the sound output mode of the built-in microphone module according to the signal strength of the first sound signal within a continuous preset number of frames, the built-in microphone module is specifically configured as follows:

[0032] If the signal strength of the first sound signal within a first preset number of consecutive frames reaches a preset signal strength threshold, determining that the sound output mode of the built-in microphone module is the second separated microphone sound output mode;

[0033] If the signal strength of the first sound signal within a second consecutive preset number of frames does not reach the preset signal strength threshold, the sound output mode of the built-in microphone module is determined to be the built-in microphone sound output mode.

[0034] In the present application, if the signal strength of the first sound signal reaches the first preset signal strength threshold within the first consecutive preset number of frames, it is the second separated microphone sound output mode, and if the signal strength of the first sound signal does not reach the first preset signal strength threshold within the second consecutive preset number of frames, it is the built-in microphone sound output mode. This makes the sound control according to the determined sound output mode more accurate and convenient, and improves the degree of intelligence.

[0035] In some embodiments of the present application, the sound control module includes: a built-in sound amplification module and a separate sound amplification module;

[0036] The built-in sound amplification module and the separate sound amplification module are respectively configured to: amplify the first sound signal in the first separate microphone sound output mode;

[0037] The smart devices are also configured to: obtain a third sound signal transmitted by other smart devices, and amplify the third sound signal.

[0038] In the present application, the sound control module includes a built-in sound amplification module and a separate sound amplification module. The built-in sound amplification module amplifies the acquired sound signal, i.e., the first sound signal collected by the user and the third sound signal transmitted by other smart devices, thereby improving the processing efficiency and utilization rate of the sound signal and enhancing the user experience.

[0039] In some embodiments of the present application, the sound control module further includes: an echo cancellation module;

[0040] The echo cancellation module is configured to: obtain a first echo signal after the built-in amplification module amplifies the third sound signal and a second echo signal after the separate amplification module amplifies the third sound signal;

[0041] Determine a target echo signal corresponding to an upper limit value from the first echo signal and the second echo signal;

[0042] If the signal strength of the target echo signal reaches a preset second signal strength threshold, echo cancellation processing is performed according to the signal strength of the target echo signal.

[0043] In the present application, the echo generated after the sound signal is amplified is eliminated by the echo cancellation module, thereby improving the clarity and comfort of the sound heard by the user.

[0044] In a second aspect, the present application provides a sound processing method for a display device, the method comprising:

[0045] The separation microphone collects first sound signals of different frames within a preset first distance;

[0046] The built-in microphone module receives the first sound signals of different frames, and collects the second sound signals of different frames within a preset second distance, wherein the preset first distance does not exceed the preset second distance;

[0047] The sound control module processes the first sound signal and the second sound signal, and the processing includes sound amplification processing and echo cancellation processing.

[0048] In the present application, the built-in microphone module of the display device can directly receive the sound signal collected by the separate microphone. At the same time, it can also directly collect the sound signal without the need to separately configure a mixing console device, thereby reducing the tedious and time-consuming manpower switching of the sound transmission path through the mixing console device, improving the efficiency and intelligence of sound acquisition, and enhancing the user experience.

[0049] In a third aspect, the present application provides a computer-readable storage medium having computer execution instructions stored thereon. When the computer execution instructions are executed by a processor, the sound processing method of the display device described in the second aspect is implemented.

[0050] In the present application, the present application provides storage conditions for executing the sound processing method of the display device by providing a computer-readable storage medium.

[0051] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the sound processing method for the display device described in the second aspect.

[0052] In the present application, a computer program product is provided to provide an operating program for executing the sound processing method of the display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the implementation methods in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technology descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0054] Figure 1 A schematic diagram of an application scenario provided for this application;

[0055] Figure 2 A schematic diagram of the structure of a display device provided in an embodiment of the present application;

[0056] Figure 3 A schematic diagram of a built-in microphone module acquiring a sound signal provided in an embodiment of the present application;

[0057] Figure 4 A schematic diagram of the structure of a sound control module provided in an embodiment of the present application;

[0058] Figure 5 A functional schematic diagram of a built-in microphone module provided in an embodiment of the present application;

[0059] Figure 6 A flowchart of a method for performing echo cancellation provided in an embodiment of the present application;

[0060] Figure 7 A schematic diagram of performing echo cancellation provided in an embodiment of the present application;

[0061] Figure 8 A schematic diagram of a flow chart of a sound processing method for a display device provided in an embodiment of the present application;

[0062] Fig. 9 A sound processing device for a display device is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0064] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0065] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0066] With the rapid development of display technology, more and more display devices are used in the field of teaching. In remote teaching, due to the high demand for sound pickup, teachers and students are usually equipped with separate microphones, such as lavalier microphones and neck microphones, to obtain sound signals and send them to the display device. At the same time, the display device will also have a built-in microphone configured inside, so as to achieve multi-channel pickup of the user's voice.

[0067] In the prior art, a mixer device is usually required to be separately configured in the display device. Depending on the type of microphone that picks up the sound (separate microphone, built-in microphone), the user manually switches the sound transmission path of the mixer device to the sound path corresponding to the microphone type, and then transmits the picked up sound to the microphone module corresponding to the display device and the microphone type through this path.

[0068] For example, assuming that the microphone type for picking up sound is a separate microphone, the user needs to manually change the sound transmission path of the mixing console device to the sound path corresponding to the separate microphone, and then transmit the picked up sound to the separate microphone module of the display device through this path, so that the microphone module can control the acquired sound.

[0069] For another example, assuming that the microphone type for picking up sound is a built-in microphone, the user needs to manually change the sound transmission path of the mixing console device to the sound path corresponding to the built-in microphone, and then transmit the picked up sound to the built-in microphone module of the display device through this path, so that the microphone module can control the acquired sound.

[0070] The control of the sound may include amplifying the sound or performing echo cancellation processing on the sound.

[0071] However, the prior art requires that the display device be equipped with an additional mixer device, and the sound transmission path of the mixer device needs to be manually switched according to the type of microphone picking up the sound, resulting in the display device obtaining sound signals in a cumbersome and time-consuming manner, with a low degree of intelligence, which affects the user experience.

[0072] Therefore, in view of the above-mentioned technical problems in the prior art, the present application proposes a display device and a sound processing method for the display device, wherein the display device includes a separate microphone, a built-in microphone module and a sound control module, wherein the built-in microphone module is connected to the separate microphone and the sound control module respectively. The separate microphone is configured to collect first sound signals of different frames within a preset first distance. The built-in microphone module is configured to receive first sound signals of different frames, and collect second sound signals of different frames within a preset second distance, and the preset first distance does not exceed the preset second distance. The sound control module is configured to process the first sound signal and the second sound signal, and the processing includes amplification processing and echo cancellation processing. In the present application, the sound signal collected by the separate microphone can be directly received by the built-in microphone module of the display device, and at the same time, the sound signal can also be directly collected, which improves the efficiency and intelligence of the display device in obtaining sound and improves the user experience.

[0073] To facilitate understanding of the present application, an application scenario is provided below as an example: Figure 1 A schematic diagram of an application scenario provided for this application, such as Figure 1 As shown:

[0074] Taking the display device as an example, the smart education blackboard has remote teaching functions such as audio and video playback and touch writing. The smart education blackboard 10 includes a separate microphone 101, a built-in microphone module 102, and a sound control module 103.

[0075] The separate microphone 101 is used to collect the voice signal of the teacher or the student and send the voice signal to the built-in microphone module 102 .

[0076] The built-in microphone module 102 is used to receive the sound signal collected by the separate microphone 101, and can also directly collect the sound signal of the teacher, students or other smart terminals.

[0077] The sound control module 103 is used to receive sound signals and perform processing such as sound amplification or echo cancellation on the sound signals.

[0078] It is understandable that the above scenarios are only used for illustration, and the display device may also be other teaching devices. In the specific application of the solution, it can be set according to actual needs, and this application does not limit it.

[0079] In addition, the present application does not limit the number and type of the separate microphone 101, the built-in microphone module 102 and the sound control module 103.

[0080] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0081] Figure 2 A schematic diagram of a display device provided in an embodiment of the present application, wherein the display device has a sound acquisition function, such as Figure 2 As shown, the display device includes: a separate microphone 201, a built-in microphone module 202 and a sound control module 203.

[0082] The connection relationship is as follows: the separate microphone 201 is connected to the built-in microphone module 202 , and the built-in microphone module 202 is connected to the sound control module 203 .

[0083] Its functions are:

[0084] The separation microphone 201 is configured to collect first sound signals of different frames within a preset first distance.

[0085] The separate microphone 201 generally has a close-range sound pickup capability, and can pick up a single person's voice and transmit the picked-up sound signal to the built-in microphone module 202 of the display device.

[0086] The types of the separate microphone 201 include, but are not limited to, a handheld microphone, a gooseneck microphone, a lavalier microphone, a head-mounted microphone, a neck-mounted microphone, and the like.

[0087] In this embodiment, taking a display device used in the field of education as an example, the separate microphone 201 is used to collect voice signals of teachers or students.

[0088] The built-in microphone module 202 is configured to: receive first sound signals of different frames, and collect second sound signals of different frames within a preset second distance, where the preset first distance does not exceed the preset second distance.

[0089] The built-in microphone module 202 is composed of a multi-microphone array and an audio processor, and has the capabilities of long-distance sound pickup and audio signal processing.

[0090] In this embodiment, the built-in microphone module 202 can directly receive the user's voice signal collected by the separate microphone 201, or directly collect the sound within a preset distance of the space through the built-in microphone, for example, it can be the voice of the user without the separate microphone 201, the sound emitted by other smart devices, etc.

[0091] The sound control module 203 is configured to process the first sound signal and the second sound signal, wherein the processing includes sound amplification processing and echo cancellation processing.

[0092] For example,

[0093] If the first sound signal is the teacher's voice collected by the separate microphone 201, in order to enable students in the classroom to hear the teacher's teaching content clearly, the sound control module 203 needs to amplify the first sound signal.

[0094] If the second sound signal is the sound of other smart devices collected by the built-in microphone, in order to reduce the echo effect caused by the other smart devices playing the sound through speakers, etc., the sound control module is required to perform echo cancellation processing on the second sound signal.

[0095] It should be understood that the above examples are only used for illustration and do not limit the present application.

[0096] In the above embodiment of the present application, the display device includes a separate microphone 201, wherein the separate microphone 201 is used to collect first sound signals of different frames within a preset first distance, and also includes a built-in microphone module 202 connected to the separate microphone 201, the built-in microphone module 202 is used to receive the first sound signals of different frames, and collect the second sound signals of different frames within a preset second distance, wherein the preset first distance does not exceed the preset second distance. It also includes a sound control module 203 connected to the built-in microphone module 202, and the sound control module 203 is used to process the first sound signal and the second sound signal, and the processing includes amplification processing and echo cancellation processing. In this embodiment, the built-in microphone module 202 of the display device can directly receive the sound signal collected by the separate microphone 201, and can also directly collect the sound signal, without the need to configure a separate mixer device, which reduces the tedious and time-consuming manual switching of the sound transmission path, improves the efficiency and intelligence of sound acquisition, and enhances the user experience.

[0097] Furthermore, based on the above embodiments, the function of the built-in microphone module is further explained through the following embodiments.

[0098] In this embodiment, the separate microphone may include a first separate microphone and a second separate microphone. The first separate microphone is used to collect the voice of the teacher, and the second separate microphone is used to collect the voice of the student.

[0099] In this embodiment, different separate microphone paths are pre-set in the built-in microphone module and are prioritized. Since teachers play an important role in outputting content during teaching, the separate microphone path corresponding to the first separate microphone is set as the first priority separate microphone path, and the separate microphone path corresponding to the second separate microphone is set as the second priority separate microphone path.

[0100] When neither the first separate microphone nor the second separate microphone collects a sound signal, in order to ensure uninterrupted output of the display device, the channel corresponding to the built-in microphone is set as a default path, that is, a path with the lowest priority.

[0101] Therefore, the built-in microphone module can receive the sound signal collected by the first separate microphone, the sound signal collected by the second separate microphone, and the sound signal collected by the built-in microphone. Figure 3 As shown, Figure 3 A schematic diagram of a built-in microphone module acquiring a sound signal provided in an embodiment of the present application.

[0102] When only the first separation microphone collects the teacher's first sound signal, the built-in microphone module can receive the first sound signal of different frames through the preset first priority separation microphone channel. Furthermore, the built-in microphone module determines the sound output mode of the built-in microphone module according to the signal strength of the first sound signal within a continuous preset number of frames.

[0103] The sound output modes include: a first separate microphone sound output mode and a built-in microphone sound output mode. The first separate microphone sound output mode requires amplification of the sound signal, while the built-in microphone sound output mode does not require amplification of the sound signal.

[0104] One possible way to determine the sound output mode of the built-in microphone module is:

[0105] If the signal strength of the first sound signal reaches the first preset signal strength threshold within the first consecutive preset number of frames, the sound output mode of the built-in microphone module is determined to be the first separated microphone sound output mode. The built-in microphone module sends the first sound signal to the sound control module so that the sound control module performs sound amplification processing according to the first sound signal.

[0106] For example,

[0107] In this embodiment, every 20ms signal can be used as a detection frame. When the signal strength of the first sound signal within three consecutive detection frames (total duration 60ms) is higher than -30dBFS, the sound output mode of the built-in microphone module is determined to be the first separated microphone sound output mode. The built-in microphone module uses the teacher's voice signal as the total output and sends it to the sound control module for amplification.

[0108] Figure 4 A schematic diagram of the structure of a sound control module provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the sound control module includes: a built-in sound amplification module 401 and a separate sound amplification module 402.

[0109] Among them, the built-in sound amplification module 401 can be an internal sound amplification module composed of an amplifier and a speaker of the display device, and the separated sound amplification module 402 can be an external sound amplification module composed of a separate amplifier and a speaker, which has a stronger sound amplification capability than the built-in sound amplification module 401.

[0110] For example,

[0111] The built-in microphone module can send the acquired teacher's voice signal to the built-in sound amplification module 401 or the separate sound amplification module 402 to amplify the voice signal, so that students in the classroom can hear the teacher's teaching content more clearly.

[0112] The built-in microphone module continues to detect, and if the signal strength of the first sound signal within a second consecutive preset number of frames does not reach the first preset signal strength threshold, the sound output mode of the built-in microphone module is determined to be the built-in microphone sound output mode.

[0113] For example,

[0114] In this embodiment, every 20ms signal can also be used as a detection frame. When the signal strength of the first sound signal is lower than -30dBFS within 50 consecutive detection frames (total duration 1s), the first separated microphone sound output mode is exited, and the sound output mode of the built-in microphone module is determined to be the built-in microphone sound output mode, that is, the default mode. In the default mode, the sound signal is collected by the built-in microphone.

[0115] In this embodiment, the reason why the detection frame number for exiting the first separate microphone sound output mode is set to be longer is that the time-varying characteristics of the sound are taken into consideration to reduce the sound stuttering sensation generated during the mode switching process.

[0116] When the first separated microphone does not collect the first sound signal of the teacher, the first sound signal of the student collected by the second separated microphone is detected. When only the second separated microphone collects the first sound signal of the student, the built-in microphone module can receive the sound signal of different frames through the preset second priority separated microphone channel. Furthermore, the built-in microphone module determines the sound output mode of the built-in microphone module according to the signal strength of the first sound signal within a continuous preset number of frames.

[0117] The sound output modes include: a second separate microphone sound output mode and a built-in microphone sound output mode. Both the second separate microphone sound output mode and the built-in microphone sound output mode do not require amplification of the sound signal.

[0118] Another possible implementation to determine the sound output mode of the built-in microphone module is:

[0119] If the signal strength of the first sound signal within a first preset number of consecutive frames reaches a preset signal strength threshold, the sound output mode of the built-in microphone module is determined to be the second separate microphone sound output mode.

[0120] For example,

[0121] In this embodiment, every 20ms signal is also used as a detection frame. When the signal strength of the first sound signal is higher than -30dBFS within three consecutive detection frames (total duration 60ms), the sound output mode of the built-in microphone module is determined to be the second separated microphone sound output mode, and the built-in microphone module uses the student's sound signal as the total output.

[0122] It should be noted that the student's voice signal does not need to be amplified in this application, so the built-in microphone module does not need to send the voice signal to the built-in amplification module or the separate amplification module.

[0123] The built-in microphone module continues to detect, and if the signal strength of the first sound signal within a second consecutive preset number of frames does not reach the preset signal strength threshold, the sound output mode of the built-in microphone module is determined to be the built-in microphone sound output mode.

[0124] For example,

[0125] In this embodiment, every 20ms signal can also be used as a detection frame. When the signal strength of the first sound signal is lower than -30dBFS within 50 consecutive detection frames (total duration 1s), the second separate microphone sound output mode is exited, and the sound output mode of the built-in microphone module is determined to be the built-in microphone sound output mode, that is, the default mode. In the default mode, the sound signal is collected by the built-in microphone component.

[0126] It should be noted that if the built-in microphone module detects the sound signal collected by the first separate microphone and the sound signal collected by the second separate microphone at the same time, the built-in microphone module will switch from the current sound output mode to the first separate microphone sound output mode, and only output the sound signal collected by the first separate microphone, so that the teacher's voice has the first priority.

[0127] In the above embodiment of the present application, the built-in microphone module transmits the collected sound signal to the built-in microphone module through different priority paths according to the different separated microphones. And according to the relationship between the signal strength of the sound signal within a continuous preset number of frames and the preset signal strength threshold, the sound output mode of the built-in microphone module is determined, so that the determined sound output mode is more accurate and the degree of intelligence is improved.

[0128] In order to facilitate understanding of the contents of the above embodiments, Figure 5 Briefly explain the purpose of the built-in microphone module. Figure 5 A functional schematic diagram of a built-in microphone module provided in an embodiment of the present application is shown in FIG. Figure 5 As shown:

[0129] S501: Determine the sound output mode of the built-in microphone module according to the path of receiving the sound signal.

[0130] S502: If it is the first separate microphone sound output mode, determine whether the signal strength of the sound signal within 50 consecutive detection frames (total duration 1 second) is lower than -30 dBFS.

[0131] S503: If yes, the sound output mode of the built-in microphone module is switched to the built-in microphone sound output mode, ie, the default mode.

[0132] S504, if not, the sound output mode of the built-in microphone module is still the first separated microphone sound output mode, the teacher's voice signal is used as the total output, and sent to the sound control module (built-in amplification module, separated amplification module) for amplification.

[0133] S505: If it is the second separate microphone sound output mode, determine whether the signal strength of the sound signal within 50 consecutive detection frames (total duration 1 second) is lower than -30 dBFS.

[0134] S506: If yes, the sound output mode of the built-in microphone module is switched to the built-in microphone sound output mode, ie, the default mode.

[0135] S507: If not, the sound output mode of the built-in microphone module is still the second separate microphone sound output mode, and the student's sound signal is used as the total output.

[0136] S508: If the built-in microphone module output mode is the default mode, the sound signal is received through the preset first priority separation microphone channel, and it is determined whether the signal strength of the sound signal within three consecutive detection frames (total duration 60ms) is higher than -30dBFS.

[0137] S509: If yes, the sound output mode of the built-in microphone module is switched to the first separated microphone sound output mode, and the teacher's voice signal is used as the total output and sent to the sound control module (built-in amplification module, separated amplification module) for amplification.

[0138] S510: If not, receive the sound signal through the preset second priority separation microphone channel, and determine whether the signal strength of the sound signal within three consecutive detection frames (total duration 60 ms) is higher than -30 dBFS.

[0139] S511: If yes, the sound output mode of the built-in microphone module is switched to the second separate microphone sound output mode, and the student's sound signal is used as the total output.

[0140] S512: If not, the sound output mode of the built-in microphone module is switched to the built-in microphone sound output mode, ie, the default mode.

[0141] For the specific implementation process of the above steps, please refer to the above embodiments. To avoid redundancy, the description will not be repeated.

[0142] In the present application, the display device can also eliminate echoes, so the sound control module connected to the built-in microphone module also includes an echo cancellation module.

[0143] When the built-in sound amplification module detects that only the separate microphone is picking up sound, the separate microphone has a short sound pickup distance and basically will not receive the sound played by the speaker, etc., and many separate microphones need to be amplified at the same time after picking up voice (for example, teachers often need to simultaneously broadcast the sound to the entire classroom after picking up the separate microphone). The overlap between the voice signal and the playback signal is very high, so the need for echo cancellation is small. Therefore, when it is detected that only the separate microphone is picking up sound, echo cancellation processing is not required.

[0144] When other smart devices send sound signals to the display device of the present application, the sound amplification module of the display device, such as a speaker, obtains the third sound signal transmitted by the other smart devices and amplifies the third sound signal. If the built-in sound amplification module detects that only the built-in sound amplification module is picking up the sound signal, since the built-in microphone is far-field pickup, the pickup distance is, for example, more than 6m, and it is inevitable that the sound played by the speaker will be picked up. Therefore, there is a strong demand for the echo cancellation function, and echo cancellation is required.

[0145] Figure 6 A flow chart of a method for performing echo cancellation provided in an embodiment of the present application is shown as follows: Figure 6 As shown, the echo cancellation module is configured to perform the following steps:

[0146] S601: Obtain a first echo signal after a third sound signal is amplified by the built-in sound amplification module and a second echo signal after amplification by the separate sound amplification module.

[0147] In this embodiment, if the built-in sound amplification module and the separate sound amplification module respectively amplify the third sound signal, the corresponding echo signals are respectively obtained.

[0148] If only the built-in sound amplification module performs sound amplification according to the third sound signal, only the echo signal corresponding to the built-in sound amplification module is obtained.

[0149] Correspondingly, if only the separate sound amplification module performs sound amplification according to the third sound signal, only the echo signal corresponding to the separate sound amplification module is obtained.

[0150] S602: Determine a target echo signal corresponding to an upper limit value from the first echo signal and the second echo signal.

[0151] The first echo signal and the second echo signal are compared in terms of their signal strengths, and the echo signal with the largest signal strength value is determined as the target echo signal.

[0152] S603: If the signal strength of the target echo signal reaches a preset second signal strength threshold, perform echo cancellation processing according to the signal strength of the target echo signal.

[0153] If the signal strength of the target echo signal is greater than a preset second signal strength threshold, such as -52 dBFS, echo cancellation processing is performed according to the determined signal strength of the target echo signal.

[0154] If only the echo signal corresponding to the built-in sound amplification module is obtained, it is determined whether the signal strength of the echo signal is greater than -52dBFS. If so, echo cancellation processing is performed on the signal strength of the echo signal.

[0155] Correspondingly, if only the echo signal corresponding to the separate amplification module is obtained, it is determined whether the signal strength of the echo signal is greater than -52dBFS. If so, echo cancellation processing is performed on the signal strength of the echo signal.

[0156] For the specific method of echo cancellation, please refer to the patent publication CN113096681A, which will not be described repeatedly in this application.

[0157] In order to facilitate understanding of the above method in this application, below, Figure 7 The embodiment shown briefly illustrates the process of performing echo cancellation. Figure 7 A schematic diagram of performing echo cancellation is provided in an embodiment of the present application, such as Figure 7 As shown:

[0158] S701, the built-in microphone module determines whether only the separate microphone is picking up sound.

[0159] S702: If yes, the sound signal is not sent to the echo cancellation module, and the echo cancellation module does not perform echo cancellation.

[0160] S703: If not, determine whether the echo signal corresponding to the built-in sound amplification module is greater than the echo signal corresponding to the separate sound amplification module, and whether the signal strength of the echo signal corresponding to the built-in sound amplification module is greater than a preset signal strength threshold.

[0161] S704: If yes, perform echo cancellation processing according to the signal strength of the echo signal corresponding to the built-in sound amplification module.

[0162] S705: If not, perform echo cancellation processing according to the signal strength of the echo signal corresponding to the separate amplification module.

[0163] For the specific implementation process of the above steps, please refer to the above embodiments. To avoid redundancy, the description will not be repeated.

[0164] Figure 8 A flow chart of a sound processing method for a display device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the method comprises the following steps:

[0165] S801: A separate microphone collects first sound signals of different frames within a preset first distance.

[0166] S802: The built-in microphone module receives first sound signals of different frames, and collects second sound signals of different frames within a preset second distance, wherein the preset first distance does not exceed the preset second distance.

[0167] S803: The sound control module processes the first sound signal and the second sound signal, and the processing includes sound amplification processing and echo cancellation processing.

[0168] In the above embodiments of the present application, the specific implementation steps and technical effects please refer to the above embodiments. To avoid redundancy, this embodiment will not be repeated.

[0169] Fig. 9 An embodiment of the present application provides a sound processing device for a display device, and the device includes: a collection module 901, a receiving module 902 and a processing module 903.

[0170] A collection module 901 is used to collect first sound signals of different frames within a preset first distance;

[0171] A receiving module 902 is used to receive first sound signals of different frames, and collect second sound signals of different frames within a preset second distance, wherein the preset first distance does not exceed the preset second distance;

[0172] The processing module 903 is used to process the first sound signal and the second sound signal, and the processing includes sound amplification processing and echo cancellation processing.

[0173] The sound processing device for a display device provided in an embodiment of the present application can execute the sound processing method for a display device in the above method embodiment, and its implementation principle and technical effect are similar and will not be repeated here.

[0174] It should be noted that the above Fig. 9The division of the modules shown is only a schematic diagram, and the present application does not limit the division of the modules and the naming of the modules.

[0175] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores computer execution instructions, and the program instructions are used for the sound processing method of the display device in the above-mentioned embodiment.

[0176] The present application also provides a computer program product, which includes a computer program stored in a readable storage medium. At least one control module of the display device can read the execution instruction from the readable storage medium, and at least one control module executes the execution instruction so that the display device implements the sound processing method of the display device provided by the various embodiments described above.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0178] For ease of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are intended to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: The display device comprises: A separation microphone, wherein the separation microphone is configured to: collect first sound signals of different frames within a preset first distance; a built-in microphone module connected to the separate microphone, the built-in microphone module being configured to: receive the first sound signals of the different frames, and collect the second sound signals of the different frames within a preset second distance, the preset first distance not exceeding the preset second distance; A sound control module connected to the built-in microphone module is configured to process the first sound signal and the second sound signal, wherein the processing includes sound amplification processing and echo cancellation processing.

2. The display device according to claim 1, characterized in that The separation microphone comprises: a first separation microphone and a second separation microphone; Accordingly, when the built-in microphone module is configured to receive the first sound signal of the different frames, the specific configuration is as follows: If the separation microphone is the first separation microphone, receiving the first sound signal of the different frame through a preset first priority separation microphone channel; If the separate microphone is the second separate microphone, the first sound signal of the different frame is received through a preset second priority separate microphone channel.

3. The display device according to claim 2, characterized in that The built-in microphone module is also configured as follows: If the first sound signals of different frames are received through the preset first priority separation microphone path, the sound output mode of the built-in microphone module is determined according to the signal strength of the first sound signal within a continuous preset number of frames; The sound output mode includes: a first separate microphone sound output mode and a built-in microphone sound output mode. The first separate microphone sound output mode requires amplification of the sound signal, while the built-in microphone sound output mode does not require amplification of the sound signal.

4. The display device according to claim 3, characterized in that When the built-in microphone module is configured to determine the sound output mode of the built-in microphone module according to the signal strength of the first sound signal within a continuous preset number of frames, the built-in microphone module is specifically configured as follows: If the signal strength of the first sound signal within a first preset number of consecutive frames reaches a first preset signal strength threshold, determining that the sound output mode of the built-in microphone module is the first separated microphone sound output mode; If the signal strength of the first sound signal within a second consecutive preset number of frames does not reach the first preset signal strength threshold, the sound output mode of the built-in microphone module is determined to be the built-in microphone sound output mode.

5. The display device according to claim 4, characterized in that The built-in microphone module is also configured as follows: If the sound output mode of the built-in microphone module is the first separate microphone sound output mode, the first sound signal is sent to the sound control module, so that the sound control module performs sound amplification processing according to the first sound signal.

6. The display device according to claim 2, characterized in that The built-in microphone module is also configured as follows: If the first sound signals of different frames are received only through the preset second priority separation microphone path, determining the sound output mode of the built-in microphone module according to the signal strength of the first sound signal within a continuous preset number of frames; The sound output mode includes: a second separate microphone sound output mode and a built-in microphone sound output mode, and both the second separate microphone sound output mode and the built-in microphone sound output mode do not require amplification of the sound signal.

7. The display device according to claim 6, characterized in that When the built-in microphone module is configured to determine the sound output mode of the built-in microphone module according to the signal strength of the first sound signal within a continuous preset number of frames, the built-in microphone module is specifically configured as follows: If the signal strength of the first sound signal within a first preset number of consecutive frames reaches a preset signal strength threshold, determining that the sound output mode of the built-in microphone module is the second separated microphone sound output mode; If the signal strength of the first sound signal within a second consecutive preset number of frames does not reach the preset signal strength threshold, the sound output mode of the built-in microphone module is determined to be the built-in microphone sound output mode.

8. The display device according to claim 5 or 7, characterized in that: The sound control module includes: a built-in sound amplification module and a separate sound amplification module; The built-in sound amplification module and the separate sound amplification module are respectively configured to: amplify the first sound signal in a first separate microphone sound output mode; The smart devices are also configured to: obtain a third sound signal transmitted by other smart devices, and amplify the third sound signal.

9. The display device according to claim 8, characterized in that The sound control module also includes: an echo cancellation module; The echo cancellation module is configured to: obtain a first echo signal after the built-in amplification module amplifies the third sound signal and a second echo signal after the separate amplification module amplifies the third sound signal; Determine a target echo signal corresponding to an upper limit value from the first echo signal and the second echo signal; If the signal strength of the target echo signal reaches a preset second signal strength threshold, echo cancellation processing is performed according to the signal strength of the target echo signal.

10. A sound processing method for a display device, characterized in that: The method comprises: The separation microphone collects first sound signals of different frames within a preset first distance; The built-in microphone module receives the first sound signals of different frames, and collects the second sound signals of different frames within a preset second distance, wherein the preset first distance does not exceed the preset second distance; The sound control module processes the first sound signal and the second sound signal, and the processing includes sound amplification processing and echo cancellation processing.

Images