Exhibition hall interactive device and exhibition hall interactive system

Abstract

This application relates to an interactive exhibition hall device and system, including a display device, a virtual human driver, an interactive sensing device, and a processor. Both the interactive sensing device and the virtual human driver are connected to the processor, and the display device is connected to the virtual human driver. The interactive sensing device collects the interactive characteristics of visitors in the exhibition hall and transmits the interactive signals to the processor. The processor receives the interactive signals and outputs driving commands to the virtual human driver. The virtual human driver drives the display device to display a virtual human, and upon receiving a driving command, drives the display device to display the virtual human's posture corresponding to the driving command. By collecting the interactive characteristics of visitors and utilizing the flexibility of virtual human technology for feedback, real-time interaction with visitor behavior is achieved, enhancing the convenience and diversity of interactive experiences in the exhibition hall.

Description

Technical Field

[0001] This application relates to the field of interactive exhibition hall technology, and in particular to an interactive exhibition hall device and system. Background Technology

[0002] With the rapid development of digital technology, visitors have higher demands for the interactivity, immersion, and information acquisition efficiency of exhibition halls. Modern visitors are no longer satisfied with the traditional one-way information reception method, but expect to acquire knowledge, participate in interactions, and enjoy a more attractive exhibition experience through natural interaction, personalized experiences, and immersive environments.

[0003] Traditional large-screen systems in exhibition halls employ static display methods, which are increasingly unable to meet the interactive needs of visitors. Their content is often monotonous, lacking interactivity and engagement, failing to capture audience attention and thus failing to meet the diverse and intelligent display requirements of modern exhibition halls. Therefore, improving the interactivity and diversity of exhibition halls is an urgent problem to be solved. Utility Model Content

[0004] Therefore, it is necessary to provide an interactive exhibition hall device and system that can improve the convenience and diversity of interaction, addressing the aforementioned technical issues.

[0005] In a first aspect, this application provides an interactive exhibition hall device, which includes a display device, a virtual human driver, an interactive sensing device, and a processor. The interactive sensing device and the virtual human driver are both connected to the processor, and the display device is connected to the virtual human driver.

[0006] The interactive sensing device collects the interactive features of visitors in the exhibition hall and transmits the interactive signals to the processor. The processor receives the interactive signals and outputs driving instructions to the virtual human driver. The virtual human driver is used to drive the display device to display a virtual human, and when it receives the driving instructions, it drives the display device to display the virtual human posture corresponding to the driving instructions.

[0007] In one embodiment, the interactive sensing device includes an image collector and a sound collector, both of which are connected to the processor.

[0008] In one embodiment, the interactive sensing device further includes an infrared sensor connected to the processor.

[0009] In one embodiment, the interactive sensing device further includes an emotion recognition unit connected to the image acquisition unit and the processor.

[0010] In one embodiment, the interactive exhibition hall device further includes a speech synthesizer connected to the processor.

[0011] In one embodiment, the interactive exhibition hall device further includes a network communication device connected to the processor.

[0012] In one embodiment, the display device includes a display control unit and a display screen, the display screen being connected to the virtual human driver via the display control unit.

[0013] In one embodiment, the exhibition hall interactive device further includes a management terminal connected to the processor.

[0014] In one embodiment, the exhibition hall interactive device further includes a power management device, and the display device, the virtual human driver, the interactive sensing device, and the processor are all connected to the power management device.

[0015] Secondly, this application also provides an exhibition hall interactive system, which includes a motion capture device and an exhibition hall interactive device as described above. The motion capture device is connected to a virtual human driver of the exhibition hall interactive device to establish different virtual human poses.

[0016] The aforementioned interactive exhibition hall equipment and system includes a display device, a virtual human driver, an interactive sensing device, and a processor. Both the interactive sensing device and the virtual human driver are connected to the processor, and the display device is connected to the virtual human driver. The interactive sensing device collects the interactive characteristics of visitors in the exhibition hall and transmits the interactive signals to the processor. The processor receives the interactive signals and outputs driving commands to the virtual human driver. The virtual human driver drives the display device to show the virtual human, and upon receiving a driving command, drives the display device to show the virtual human's posture corresponding to the driving command. By collecting the interactive characteristics of visitors and utilizing the flexibility of virtual human technology for feedback, real-time interaction with visitor behavior is achieved, enhancing the convenience and diversity of the exhibition hall's interactive features. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of the interactive equipment in the exhibition hall in one embodiment;

[0018] Figure 2 This is a schematic diagram of the structure of the interactive equipment in the exhibition hall in another embodiment;

[0019] Figure 3 This is a schematic diagram of the structure of the interactive equipment in the exhibition hall in another embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0021] It is understood that the terms "first," "second," etc., used herein may be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, a first resistor may be referred to as a second resistor, and similarly, a second resistor may be referred to as a first resistor. Both the first resistor and the second resistor are resistors, but they are not the same resistor.

[0022] It is understood that the term "connection" in the following embodiments should be understood as "electrical connection," "communication connection," etc., if the connected circuits, modules, units, etc., have electrical signal or data transmission with each other.

[0023] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” or “having,” etc., specify the presence of the stated feature, whole, step, operation, component, part, or combination thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0025] The interactive exhibition hall equipment provided in this application can be applied to various exhibition hall environments, including but not limited to art exhibitions, museums, trade shows, cultural and tourism promotional exhibitions, and various scenarios requiring guided displays. In these scenarios, there is some professional knowledge or content that needs to be guided for the audience to understand, and exhibition halls are usually arranged to facilitate the audience's understanding. However, current exhibition halls only use large screens or promotional posters, banners, and other materials for display, which do not generate much interest among the audience and have certain limitations in display.

[0026] Therefore, the applicant is considering combining the exhibition hall with virtual human technology, using virtual human interaction to engage with visitors, provide knowledge explanations, and guide them through various activities, allowing visitors to easily and conveniently understand the content they need. Furthermore, because virtual human technology can flexibly respond to various interactions, after its functions are debugged, it can cover the needs of most visitors, thus offering interactive diversity.

[0027] Next, we will explain virtual human technology. Virtual human technology refers to virtual digital human technology. A virtual digital human is a digital human avatar created using technologies such as computer graphics (CG), artificial intelligence (AI), motion capture, and speech synthesis. It possesses lifelike appearance, interactive behavior, and intelligent decision-making capabilities. Its core goal is to achieve natural interaction with real humans. It typically includes the following steps: avatar design and modeling, model rigging, performance capture, driving and rendering, and generating interactive content.

[0028] (1) Character Design and Modeling: Based on the IP character or real-life idol, 2D original artwork is designed, and a topologically sound facial and body model is constructed using 3D modeling software (such as Maya and Blender). For example, during modeling, 52 facial key points (including brow ridges, cheekbones, corners of the mouth, etc.) and 24 body joints (conforming to human skeletal standards) can be marked. The density and distribution of key points directly affect the finesse of subsequent movements. These key points are like the "control switches" of the model; their distribution density and position directly determine the naturalness of subsequent movements.

[0029] (2) Modeling and Binding: After modeling is completed, in order to bring this static model to life, modeling and binding technology is used to perfectly combine the previously marked key points with the model, just like putting strings on a puppet. The predefined key points are mapped to the corresponding areas of the model. Binding accuracy is achieved through weight adjustment. For example, the weight optimization of eye micro-expressions is implemented, that is, the key points of the eyes need to be assigned higher weights to support micro-expression capture.

[0030] (3) Performance capture: Using professional motion capture equipment, smart cameras, optical sensors, or visual algorithms (such as iPhone Face ID), the actions of real actors are recorded in real time. From body posture to subtle facial expressions, even the movement of the eyes and the subtle movements of the fingers, everything is accurately captured, and the motion data of the real person is collected and converted into digital signals. Furthermore, deep learning (such as GAN, Transformer) can be used to automatically generate lip-sync and body language to improve the simulation level of the virtual human.

[0031] (4) Driving and rendering: The captured data is used to drive the virtual digital human to perform the same actions, just like giving the virtual digital human life. The driver processes this motion data in real time, and with the support of a high-performance rendering engine (that is, the virtual human driver), it ensures that every movement and every expression of the virtual digital human is smooth and natural.

[0032] (5) Generate content and interact: Finally, through the interaction of the audience, the processor can analyze the interaction feature signals and use mature emotion recognition algorithms to output corresponding driving instructions, so that the virtual human driver can adjust the posture of the virtual digital human based on the driving instructions, which is equivalent to the virtual digital human being able to directly interact and communicate with the audience.

[0033] Please note that the above-mentioned user interaction analysis, virtual digital human action / expression binding, and the correspondence between user interaction and virtual human action / expression binding are all based on mature virtual human programs currently on the market. Virtual humans include, but are not limited to, realistic virtual humans, cartoon-style virtual humans (such as virtual anchors), and abstract symbolic virtual humans; the interaction methods include, but are not limited to, preset scripts or AI-driven methods.

[0034] In one embodiment, such as Figure 1 As shown, this application provides an interactive exhibition hall device, which includes a display device 200, a virtual human driver 400, an interactive sensing device 600, and a processor 800. The interactive sensing device 600 and the virtual human driver 400 are both connected to the processor 800, and the display device 200 is connected to the virtual human driver 400.

[0035] The interactive sensing device collects the interactive features of visitors in the exhibition hall and transmits the interactive signals to the processor. The processor receives the interactive signals and outputs driving instructions to the virtual human driver. The virtual human driver is used to drive the display device to display the virtual human, and when it receives the driving instruction, it drives the display device to display the virtual human posture corresponding to the driving instruction.

[0036] Specifically, the interactive sensing device collects the interactive characteristics of visitors in the exhibition hall. For example, visitors may use specific actions or voice commands to wake up a virtual digital human, or they may interact with triggering devices within the exhibition hall, such as clicking a touchscreen. Interactive characteristics may also include one or more of the visitor's facial expressions, actions, or voice, representing their user needs. After collecting these interactive characteristics, the interactive sensing device can generate interactive signals; for example, different facial expressions, voice commands, or actions will generate different level signals / pulse signals / digital signals.

[0037] Alternatively, in one embodiment, such as Figure 2As shown, the interactive sensing device 600 includes an image acquisition unit 620 and a sound collector 640, both of which are connected to the processor 800. The image acquisition unit 620 can record the audience's movements and expressions, while the sound collector 640 can record the audience's voice.

[0038] Specifically, the image acquisition device can be a camera, capable of capturing the audience's movement routes, location, body language, and facial expressions. These can all be used as interactive features collected by the interactive sensing device, and corresponding interactive signals can be generated and transmitted to the processor. Similarly, the sound collector can collect the audience's voice, and through speech recognition, use the text information expressed by the audience as interactive features, generating corresponding interactive signals that are transmitted to the processor. Furthermore, the sound collector can be a microphone, and may include noise reduction components to reduce noise interference within the exhibition hall.

[0039] Furthermore, in order to more accurately collect the interactive characteristics of the audience, in one embodiment, such as Figure 2 As shown, the interactive sensing device also includes an infrared sensor 660, which is connected to the processor 800.

[0040] Infrared sensors are used to assist image acquisition devices in collecting interactive features of the audience. For example, in low light conditions, infrared sensors can use infrared images to calibrate the interactive features of the audience in the image acquisition device, such as movement routes, location, and body behavior, thereby making the obtained interactive features and signals accurate. This allows the subsequent processor to drive the virtual human accurately and improves the accuracy of the interaction.

[0041] Based on the aforementioned interaction principles of virtual digital humans, the interaction signals can be processed into emotional inclination signals that facilitate responses from the virtual digital human after they are received. In one embodiment, such as... Figure 3 As shown, the interactive sensing device 600 also includes an emotion recognition unit 680, which is connected to the image acquisition unit 620 and the processor 800. If an infrared sensor 660 is included, the emotion recognition unit 680 is also connected to the infrared sensor 660.

[0042] Specifically, the emotion recognition unit can further analyze the emotional tendencies of visitors in the exhibition hall based on the interaction characteristics collected by the image acquisition device and / or infrared sensor, and obtain emotional tendency signals. The emotion recognition principle of the emotion recognition unit can be based on artificial intelligence algorithms using deep learning technology to achieve facial expression recognition of visitors in the exhibition hall. For details, please refer to relevant technologies, which will not be elaborated here.

[0043] Optionally, the interactive signals can be processed into text information suitable for virtual digital humans. In one embodiment, the interactive sensing device may further include a speech recognition unit connected to a sound collector and a processor. The speech recognition unit can further analyze the speech of visitors in the exhibition hall collected by the sound collector and obtain the corresponding text information. The text recognition principle of the speech recognition unit can be based on artificial intelligence algorithms using deep learning technology, which will not be elaborated here.

[0044] For example, the audience's facial expressions can be smiling, angry, or surprised, the audience's actions can be waving upwards, waving downwards, or shaking their heads, and the audience's speech can include Mandarin or Cantonese, or sentences of varying lengths.

[0045] A processor is a computing chip with a certain computing power. Generally speaking, the higher the computing power, the more stable and reliable the operation of the interactive equipment in the exhibition hall. The processor pre-stores the correspondence between user interactions and the virtual human's actions / expressions. Therefore, after the interactive sensing device transmits the generated interactive signals to the processor, the processor can generate corresponding driving instructions based on the received interactive signals, emotional inclination signals, or text information, according to a preset processing logic. These driving instructions are used to drive the virtual human's driver to control the virtual human. The controller can process the input interactive signals according to a preset processing logic to generate structured driving instructions, ensuring that the virtual digital human can accurately respond to the audience's interactions.

[0046] A virtual human driver is a component that stores data related to the virtual human. In practice, when the processor's computing power is sufficient, the virtual human driver can be integrated with the processor, storing the virtual human-related data on the processor. This data includes, but is not limited to, the virtual human's model, preset animations, and rendering data. Upon receiving a driving command, the virtual human driver transmits the corresponding virtual human-related data to the display device, enabling the display device to show the virtual human. Depending on the driving command, different virtual human poses are displayed, such as the virtual human opening its mouth, raising its hand, or shaking its head. The specific driving principle can be found in the driver interaction between the virtual anchor's face and the person behind it; it will not be detailed here.

[0047] Furthermore, in addition to using virtual humans for on-screen interaction, the interactive exhibition hall device also allows for voice interaction. In one embodiment, the interactive exhibition hall device further includes a speech synthesizer connected to a processor.

[0048] Among them, the speech synthesizer can support natural speech synthesis of multiple languages. By receiving the driving instructions output by the processor, matching the corresponding speech of the driving instructions, and using AI technology or other speech synthesis technology, it realizes speech synthesis and output, and cooperates with the virtual human's image to realize natural language dialogue with the audience.

[0049] In one embodiment, the display device includes a display control unit and a display screen, the display screen being connected to a virtual human driver via the display control unit. Multiple display screens can be used, and they can collectively display images under the control of the display control unit. The display control unit can display a virtual human corresponding to the virtual human driver. Optionally, if the display control unit is connected to a processor, it can also be directly controlled by the processor to display other content or perform start / stop control, etc.

[0050] Optionally, when there are multiple displays, the display control unit can also work with a multi-screen collaboration unit to control multiple displays. The multi-screen collaboration unit can be connected to a processor. Specifically, the display control unit controls the display parameters of the large screens, such as resolution, brightness, and contrast, to ensure clear display of the virtual digital person and content. The multi-screen collaboration unit supports collaborative display between multiple large screens, achieving richer visual effects. For example, the display device can use a high-definition LED or LCD display screen to display the image and content of the virtual digital person.

[0051] In the above embodiments, the exhibition hall interactive equipment includes a display device, a virtual human driver, an interactive sensing device, and a processor. Both the interactive sensing device and the virtual human driver are connected to the processor, and the display device is connected to the virtual human driver. The interactive sensing device collects the interactive characteristics of visitors in the exhibition hall and transmits the interactive signals to the processor. The processor receives the interactive signals and outputs driving commands to the virtual human driver. The virtual human driver drives the display device to display a virtual human, and upon receiving a driving command, drives the display device to display the virtual human's posture corresponding to the driving command. By collecting the interactive characteristics of visitors and utilizing the flexibility of virtual human technology for feedback, real-time interaction with visitor behavior is achieved, enhancing the convenience and diversity of interaction in the exhibition hall.

[0052] In one embodiment, the interactive exhibition hall device also includes a network communication device connected to the processor. The network communication device supports 5G or Wi-Fi communication, enabling high-speed data transmission and remote control, allowing the processor to interact with the cloud or other terminals, facilitating adjustments or data input / output by staff.

[0053] For example, staff can interact with the processor to update the mapping between user interactions and virtual human actions / expressions stored within the processor. Staff can also interact with the processor to replace virtual human-related data in the virtual human driver connected to the processor. In this embodiment, the use of a network communication device enhances the convenience of the exhibition hall's interactive equipment, making the interaction process controllable, adjustable, and more intelligent.

[0054] In one embodiment, the interactive exhibition hall device further includes a management terminal connected to the processor. The management terminal has administrator privileges over the interactive exhibition hall device, provides an administrator interface, supports system configuration, content management, user management, and other functions for the interactive exhibition hall device, and supports fault diagnosis and maintenance by monitoring the system's operating status in real time and providing fault diagnosis and maintenance suggestions.

[0055] In one embodiment, the interactive exhibition hall device further includes a power management device, which connects to the display device, virtual human driver, interactive sensing device, and processor. The power management device provides stable power to the various components of the interactive exhibition hall device, and can supply power to one or more of the display device, virtual human driver, interactive sensing device, processor, speech synthesizer, network communication device, and management terminal. It can also switch to standby power when the components are not used for a preset period of time, supporting energy-saving mode.

[0056] Based on the same technical concept, this application also provides an exhibition hall interactive system, which includes a motion capture device and an exhibition hall interactive device as described in the above embodiments. The motion capture device is connected to the virtual human driver of the exhibition hall interactive device to establish different virtual human postures.

[0057] Among them, motion capture equipment can record the every move of the staff wearing it in real time, from body posture to subtle facial expressions, even the movement of the eyes and the minute movements of the fingers. All of these are accurately captured, collecting real-person motion data and converting it into digital signals. At this time, the exhibition hall interactive system can allow staff wearing motion capture equipment to interact with visitors in real time through virtual avatars, providing more flexible and convenient interactivity. At the same time, since real-time interaction is not limited by the virtual avatar's preset motion library, it increases the diversity of the exhibition hall interactive system.

[0058] Alternatively, staff wearing motion capture equipment can expand the virtual human's preset motion library by adding preset motions to the virtual human, so that the virtual human can interact with the audience more flexibly and richly.

[0059] To better understand the above solution, a detailed explanation will be provided below with reference to a specific embodiment.

[0060] In one embodiment, the exhibition hall interactive system includes motion capture equipment and exhibition hall interactive devices. The motion capture equipment can be used for real-time interaction or to enrich the virtual human's preset motion library. The exhibition hall interactive devices include a display device, a virtual human driver, a speech synthesizer, a network communication device, a management terminal, a power management device, an interactive sensing device, and a processor. The interactive sensing device includes an image acquisition device, a sound collector, an infrared sensor, and an emotion recognition unit. The display device includes a display control unit and a display screen, which is a high-definition LED or LCD display screen. The connection relationships have been described above and will not be repeated here.

[0061] In this embodiment, the processor uniformly processes multi-source inputs (actions, voice, facial expressions) to generate structured instructions, achieving millisecond-level response (measured latency <100ms). By collecting the interactive features of the audience and utilizing the flexibility of virtual human technology for feedback, real-time interaction with the audience's behavior is achieved, enhancing the interactive convenience and diversity of the exhibition hall.

[0062] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the utility model patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An interactive exhibition hall device, characterized in that, The interactive equipment in the exhibition hall includes a display device, a virtual human driver, an interactive sensing device, and a processor. The interactive sensing device and the virtual human driver are both connected to the processor, and the display device is connected to the virtual human driver. The interactive sensing device collects the interactive features of visitors in the exhibition hall and transmits the interactive signals to the processor. The processor receives the interactive signals and outputs driving instructions to the virtual human driver. The virtual human driver is used to drive the display device to display the virtual human, and when it receives the driving instructions, it drives the display device to display the virtual human posture corresponding to the driving instructions.

2. The interactive exhibition hall device according to claim 1, characterized in that, The interactive sensing device includes an image collector and a sound collector, both of which are connected to the processor.

3. The interactive exhibition hall device according to claim 2, characterized in that, The interactive sensing device also includes an infrared sensor, which is connected to the processor.

4. The interactive exhibition hall device according to any one of claims 2-3, characterized in that, The interactive sensing device also includes an emotion recognition unit, which is connected to the image acquisition unit and the processor.

5. The interactive exhibition hall device according to claim 1, characterized in that, The interactive equipment in the exhibition hall also includes a speech synthesizer, which is connected to the processor.

6. The interactive exhibition hall device according to claim 1, characterized in that, The interactive equipment in the exhibition hall also includes a network communication device, which is connected to the processor.

7. The interactive exhibition hall device according to claim 1, characterized in that, The display device includes a display control unit and a display screen, and the display screen is connected to the virtual human driver through the display control unit.

8. The interactive exhibition hall device according to claim 1, characterized in that, The interactive equipment in the exhibition hall also includes a management terminal, which is connected to the processor.

9. The interactive exhibition hall device according to claim 1, characterized in that, The interactive equipment in the exhibition hall also includes a power management device, and the display device, the virtual human driver, the interactive sensing device, and the processor are all connected to the power management device.

10. An interactive exhibition hall system, characterized in that, The exhibition hall interactive system includes a motion capture device and an exhibition hall interactive device as described in any one of claims 1-9, wherein the motion capture device is connected to the virtual human driver of the exhibition hall interactive device for establishing different virtual human poses.

Images