Portable intelligent display and directional sound system

Abstract

The invention provides a portable intelligent display and directional sound system, and belongs to the technical field of intelligent display and acoustics, the system is used for indoor film watching and outdoor speech and singing scenes, the space occupation is reduced by adopting a flexible OLED foldable screen, and the display effect is improved by adopting high-brightness high-contrast hardware performance and an image enhancement technology. A binocular vision system, a matrix microphone array and a multispectral environment sensor are used for realizing user positioning and environment perception, and an ultrasonic wave carrier wave modulation array (16-24 units) and an acoustic wave beam forming controller are used for realizing directional transmission and quality improvement of sound. The environmental adaptability, the intelligent level and the customization requirement of the product are improved, and interference to other people is reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent display and acoustic technologies, and particularly to a portable intelligent display and directional sound system for indoor movie viewing, outdoor speech and singing scenarios. Background Art

[0002] There are certain technical defects in current display and audio devices on the market: Traditional portable display devices lack the ability of ambient light adaption and have poor outdoor visibility. Ordinary audio systems cannot achieve sound field directional control and are prone to cause sound pollution. The display and audio systems are designed separately and lack intelligent collaborative interaction functions, resulting in poor visual and auditory experiences. Fixed directional speakers are bulky and difficult to meet the requirements of portable and mobile scenarios. Summary of the Invention

[0003] To solve the above technical problems, the present invention provides a portable intelligent display and directional sound system.

[0004] The technical solution of the present invention is as follows:

[0005] A portable intelligent display and directional sound system, first, a multimodal perception unit collects user information and environmental information, and transmits the collected data to an intelligent collaborative control system. The intelligent collaborative control system completes user identity recognition and position confirmation, analyzes the illumination of the working environment, controls the foldable display panel to adjust the brightness and contrast to adapt to the environmental illumination, and controls the beamforming acoustic array to output audio directionally to the user.

[0006] Among them, the intelligent collaborative control system includes an embedded AI chip, a dynamic resource manager, and a communication system. The foldable display panel includes a flexible OLED display module and a foldable aluminum alloy outer frame. The beamforming acoustic array includes an ultrasonic carrier modulation array and an acoustic beam forming controller.

[0007] Furthermore,

[0008] The multimodal perception unit includes visual, optical, and acoustic sensing modules.

[0009] First, the visual module uses a binocular autofocus camera, which can accurately identify the position of the target user, determine the distance between the user and the system, identify the user's face and clothing to achieve identity confirmation, and at the same time can identify the user's limb movements to achieve intelligent interaction control.

[0010] Second, the optical module uses a multispectral environmental sensor to verify the accuracy of the identified target through infrared sensing, analyze the ambient light, confirm the components and intensities of various spectra, and provide parameter basis for the output of the display module.

[0011] Finally, the acoustic sensing module uses a rectangular microphone array, which can assist in identity recognition through voiceprint recognition and assist in user positioning through sound source localization. At the same time, it can achieve noise reduction through noise recognition, improve speech recognition accuracy, and realize intelligent voice interaction.

[0012] All the above information is processed through the intelligent collaborative control system.

[0013] Furthermore,

[0014] The intelligent collaborative control system uses an embedded AI chip to achieve CNN acceleration and efficiently processes the information transmitted by the multimodal perception unit to achieve identity recognition, user location confirmation, identity feature recognition, intelligent voice recognition, and lighting environment information processing.

[0015] The dynamic resource manager automatically arranges the priorities of the currently received tasks, requests the required resources, and dynamically adjusts the resource usage requests according to the current cluster task situation to achieve the reasonable allocation and use of dynamic resources.

[0016] The communication system sends the tasks required after completing information analysis and processing to the foldable display panel and beamforming acoustic array, and activates the spatial sound-visual content linkage mode.

[0017] Furthermore,

[0018] The foldable display panel adopts a flexible OLED module and uses advanced display technology. On the one hand, its self-luminous characteristics can achieve pixel-level brightness adjustment, with a peak brightness of up to 1500nit, a contrast ratio expanded to 1000000:1, and a refresh rate of up to 240Hz, providing a more detailed and smooth display effect, and working with the beamforming acoustic array to achieve a more restored audio and video effect presentation.

[0019] Its aluminum alloy frame supports 0°-180° folding. On the one hand, it can provide support for the display panel and fix the driver board and system board during use. On the other hand, it can be folded and stored after use, which can both protect the display screen and save space.

[0020] The beamforming acoustic array uses a 16-24 unit ultrasonic carrier modulation array, an FPGA acoustic beamforming controller, combined with user location information, and a dynamic beamforming algorithm to achieve directional transmission of output sound.

[0021] Further, it also includes: at least 3 memories and 3 processors;

[0022] The memory is used for storing programs and local data of the drive control board of the foldable display panel, the intelligent collaborative control system, and the beamforming acoustic array controller;

[0023] The processors are respectively used for program calls of the foldable display panel, the intelligent collaboration control system, and the beamforming acoustic array. Description of the Drawings

[0024] Figure 1 It is a schematic diagram of the system structure of the present invention;

[0025] Figure 2 It is a schematic diagram of the working process of the present invention. Detailed Embodiments

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0027] The present invention provides a portable intelligent display and directional audio system. By adopting a flexible OLED foldable screen, the reduction of space occupation is achieved. The high-brightness and high-contrast hardware performance and image enhancement technology are used to improve the display effect. The binocular vision system, matrix microphone array, and multispectral environmental sensor are used to realize user positioning and environmental perception. The ultrasonic carrier modulation array (16 - 24 units) and the acoustic beam forming controller are adopted to realize the directional transmission and quality improvement of sound. The environmental adaptability, intelligent level, and customization requirements of the product are improved, and the interference to others is reduced.

[0028] Embodiment 1 (Outdoor speech scenario):

[0029] As Figure 1 shown in the intelligent display and directional audio system, during an outdoor speech, the system locks the position of the speaker through the binocular camera and tracks it in real time. The multispectral sensor obtains the current illumination information. The control of the foldable display panel is divided into two situations.

[0030] In one situation, the display panel is used as a teleprompter. The system adaptively adjusts the illumination intensity of pixel points in different regions according to the relative position and angle between the speaker and the display panel, eliminating the influence of illumination on the display panel. The acoustic module generates a manually adjustable (default 30°) conical sound beam to cover the audience area.

[0031] Another case is that the display panel faces the audience for information display. The binocular vision system analyzes the distribution of the audience in real time. The system adjusts the brightness of different areas of the display panel according to the lighting conditions, automatically switches to the HDR mode, and dynamically expands the contrast to 1,000,000:1. At the same time, a conical sound beam with automatic angle adjustment is formed to cover the audience area. The rectangular microphone array collects the on-site reverberation for feedback, and adjusts the angle, direction, and power of the sound wave beam through the beamforming acoustic array to eliminate the reverberation.

[0032] Embodiment 2 (Personal viewing mode):

[0033] As Figure 2 shown, the binocular vision system activates the privacy mode through user positioning, iris recognition, and voiceprint recognition, automatically generates a narrow beam of about 15° according to the user's position, synchronously generates spatial audio metadata for the displayed content, and starts the spatial audio-visual content linkage mode in combination with the acoustic array. At the same time, pixel-level brightness adjustment is achieved according to the indoor lighting conditions, the HDR mode is enabled, the contrast is increased, and a cinema-level visual experience is provided.

[0034] At the same time, the binocular vision system recognizes the hand movements of the user, and the rectangular microphone array recognizes the user's voice to achieve contactless playback control and intelligent voice interaction.

[0035] Embodiment 3 (Commercial promotion mode):

[0036] The binocular vision system determines the current best audience area, counts the number of people in this area, analyzes the gender ratio, age distribution, and occupational characteristics of the crowd. The multi-spectral environmental sensor obtains the current ambient lighting conditions. The above information is fed back to the intelligent collaborative control system for optimal target group selection. The intelligent collaborative control system calls the beamforming acoustic array to adjust the sound field into a sound wave beam facing the target group, controls the display panel to push advertising content targeted at the optimal target group, and adjusts the brightness to improve the display effect under the influence of strong light outdoors and in exhibition halls.

[0037] The above are only the preferred embodiments of the present invention, which are only used to illustrate the technical solutions of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A portable intelligent display and directional sound system, characterized in that: include: First, the multimodal perception unit collects user information and environmental information, and transmits the collected data to the intelligent collaborative control system; Intelligent collaborative control system to complete user identity and location confirmation, analyze the lighting conditions of the working environment, control the foldable display panel to adjust the brightness and contrast to adapt to the ambient lighting conditions, and control the beamforming acoustic array to output audio in a directional manner to the user; The beamforming acoustic array combines user location information and uses a dynamic beamforming algorithm to achieve directional transmission of output sound; The foldable display panel and its luminous characteristics can achieve pixel-level brightness adjustment, providing a more detailed and smoother display effect, and can be linked with the beamforming acoustic array to achieve a more restored audio and video effect presentation.

2. The system according to claim 1, characterized in that: The multimodal perception unit includes visual, optical, and acoustic sensing modules; First, the visual module uses a binocular autofocus camera to identify the location of the target user, determine the distance between the user and the system, and recognize the user's face and clothing to confirm the identity. At the same time, it can recognize the user's body movements and realize intelligent interactive control. Secondly, the optical module uses a multi-spectral environmental sensor to verify the accuracy of target recognition through infrared sensing, analyze the ambient light, confirm the composition and intensity of various spectra, and provide parameter basis for the output of the display module; Finally, the acoustic sensing module uses a rectangular microphone array to assist in identity recognition through voiceprint recognition and assist in user positioning through sound source localization. At the same time, it can achieve noise reduction through noise recognition, improve speech recognition accuracy, and realize intelligent voice interaction; All the above information is processed through the intelligent collaborative control system.

3. The system according to claim 1, characterized in that: The intelligent collaborative control system includes an embedded AI chip, a dynamic resource manager, and a communication system; Adopt embedded AI chip to realize CNN acceleration, process the information transmitted by multimodal perception unit, realize identity recognition, user location confirmation, identity feature recognition, intelligent voice recognition, and lighting environment information processing; The dynamic resource manager automatically arranges the priorities of the currently received tasks, requests the required resources, and dynamically adjusts the resource usage requests according to the current cluster task situation, so as to realize the reasonable allocation and use of dynamic resources; The communication system sends the tasks required after completing information analysis and processing to the foldable display panel and beamforming acoustic array, and activates the spatial sound-visual content linkage mode.

4. The system according to claim 1, characterized in that: The multimodal perception unit includes a binocular vision system, a multispectral environmental sensor, and a rectangular microphone array.

5. The system according to claim 1, characterized in that: The foldable display panel includes a flexible OLED display module and a foldable aluminum alloy frame, and the beamforming acoustic array includes an ultrasonic carrier modulation array and an acoustic beamforming controller.

6. The system according to claim 5, characterized in that: The aluminum alloy frame of the foldable display panel supports 0°-180° folding. On the one hand, it can provide support for the display panel during use and fix the driver board and system board. On the other hand, it can be folded away after use.

7. The system according to claim 1, characterized in that: The beamforming acoustic array uses a 16-24 unit ultrasonic carrier modulation array and an FPGA acoustic beamforming controller.

8. The system according to claim 1, characterized in that: Also includes at least 3 memories and 3 processors; The memory is used for storing programs and local data of the drive control board of the foldable display panel, the intelligent collaborative control system, and the beamforming acoustic array controller; The processors are respectively used for program calls of the foldable display panel, the intelligent collaborative control system, and the beamforming acoustic array.

Images