Intelligent XR multi-mode interactive exhibition stadium panoramic projection system

Abstract

The invention relates to the technical field of panoramic projection systems, in particular to an intelligent XR multi-mode interactive exhibition stadium panoramic projection system, which comprises an intelligent data terminal, a display interaction module in communication connection with the intelligent data terminal, a VR terminal comprising a user client, an action capture card and virtual reality glasses, an intelligent service background connected with the VR terminal through a network, and a display interaction module in communication connection with the intelligent data terminal. Comprising an action data processing module used for acquiring action and position data from an action capture card and decoding and processing the data; the virtual reality module is used for updating a virtual scene according to the action data; the data communication and control module is used for performing data communication with the VR terminal and controlling the display screen; the panoramic projection display system comprises a plurality of projectors and is used for forming a curved screen or a spherical screen in an exhibition hall; determining the content projected to the display screen according to the real-time action position of the user; the user experience is greatly improved, and the exhibition information transmission efficiency can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of panoramic projection systems, and in particular to an intelligent XR multi-modal interactive exhibition venue panoramic projection system. Background Art

[0002] With the rapid development of science and technology, the traditional exhibition and convention industry is facing tremendous changes. In recent years, the emergence of technologies such as virtual reality (VR), augmented reality (AR) and mixed reality (MR) has brought new possibilities to the exhibition industry. However, current exhibition technology still has many limitations and shortcomings.

[0003] Traditional exhibition methods usually rely on static displays and passive viewing, which makes it difficult to provide a deep interactive experience. Although some exhibition halls have begun to introduce touch screens or simple interactive devices, these methods still cannot meet the audience's growing demand for personalized and immersive experiences. On the other hand, although pure VR experience can provide a high sense of immersion, it often completely isolates users from the real environment, which is not conducive to group exhibition activities and social interaction.

[0004] Recently, some exhibition halls have tried to introduce holographic projection technology to enhance the display effect. However, these systems can usually only provide preset content in a fixed position and lack the ability to dynamically adjust according to the characteristics and behavior of the audience. In addition, the application of existing motion capture technology in exhibition environments also faces challenges, such as insufficient accuracy and high latency, which affect the smoothness and realism of the user experience.

[0005] When multiple people visit at the same time, existing technologies have difficulty in providing personalized content for each visitor, and cannot effectively coordinate the interaction between multiple users. This leads to the monotony of exhibition content, which cannot fully meet the needs of visitors of different backgrounds, ages and interests.

[0006] In addition, the current exhibition system often uses different technologies in isolation and lacks a unified platform to integrate multiple technologies such as VR, AR, motion capture, panoramic projection, etc. This not only increases the complexity and maintenance cost of the system, but also limits the full use of the advantages of various technologies.

[0007] In the face of these challenges, the industry urgently needs to integrate multiple advanced technologies to provide highly personalized, immersive and real-time interactive exhibition systems. Such a system should be able to intelligently identify user characteristics, track user behavior in real time, and dynamically adjust the display content and methods based on this information. At the same time, it also needs to allow users to stay connected with the real environment and other audiences while maintaining a high sense of immersion. Summary of the invention

[0008] The intelligent XR multi-modal interactive exhibition venue panoramic projection system of the present invention is designed to solve the above technical problems. The system realizes a new exhibition paradigm by innovatively integrating technologies such as virtual reality, motion capture, panoramic projection and artificial intelligence.

[0009] The present invention proposes an intelligent XR multi-modal interactive exhibition venue panoramic projection system, including:

[0010] Intelligent data terminal for:

[0011] Receive image information captured by the camera device in the venue;

[0012] Identify and analyze image information according to preset recognition algorithms;

[0013] Generate display instructions;

[0014] The display interaction module is connected to the intelligent data terminal for:

[0015] Responding to a display instruction generated by the intelligent data terminal;

[0016] Control what is displayed and how it is displayed;

[0017] VR terminal, including user client, motion capture card and virtual reality glasses, is used to:

[0018] Receive data transmitted by motion capture devices and visual trackers;

[0019] Display virtual scene data;

[0020] Provide visual rendering module data;

[0021] The intelligent service backend is connected to the VR terminal via a network, and includes:

[0022] The motion data processing module is used to obtain motion and position data from the motion capture card, and decode and process the data;

[0023] A virtual reality module, used to update the virtual scene according to the motion data;

[0024] Data communication and control module, used for data communication with VR terminal and controlling display screen;

[0025] A panoramic projection display system, comprising a plurality of projectors, is used for:

[0026] Forming a curved screen or a spherical screen in the exhibition hall;

[0027] Determine the content projected onto the display screen based on the user's real-time action position;

[0028] Among them, when the user is in the projection area, a camera device for capturing image information is arranged above the venue; the projection area is the booth area; the camera device is triggered to capture image information based on the user standing in the user identification area, and the boundary points of the user identification area are projected by the first projection device on the ceiling of the venue.

[0029] Preferably, the user identification area includes a first area and a second area, the line connecting the first area and the second area is a rectangular boundary line, the rectangular area includes a number of unit blocks, the first area is a unit block, the first area corresponds to the user's standing area, and the second area is a walkable area formed by splicing multiple unit blocks; the first area is located above the image recognition area; the image recognition area corresponds to the projection area of ​​the venue booth.

[0030] Preferably, when the user stands in the second area, a standing signal is generated by sensing the user's standing posture through a pressure sensor on the venue floor, the venue floor includes foot blocks, and the foot blocks cover the unit blocks of the second area; the standing signal is sent wirelessly to the camera device, and the camera device captures image information based on the standing signal.

[0031] Preferably, when the user moves from the first area to the second area, the camera device captures real-time image information. At least two projection areas are set in the venue, and the areas of the two projection areas cover all booths in the venue. The intelligent data terminal is equipped with a display area, and the display area is provided with a first area and a second area. When the user walks in an area between the first area and the second area, the user standing area and walking area projected by the first projection device also change accordingly, and the first projection device cooperates with the camera device to capture and process real-time image information.

[0032] Preferably, the display interaction module performs interactive control based on the display instructions generated by the intelligent data terminal, and the interactive control includes control of display content and display method. The display content and display method control different interactive results according to the analysis information received by the intelligent data terminal, and the interactive results include a first display method, a second display method, and a third display method; the display content is associated with the image information; the display content includes static display, stereoscopic dynamic display and interactive display, and the display methods include holographic projection, panoramic stereoscopic projection, video projection, naked-eye 3D image projection, and AR projection.

[0033] Preferably, a corresponding display method is selected according to the display content identified by the user; when the display content is a static display, the display method is selected from holographic projection and video projection; when the display content is a three-dimensional dynamic display or an interactive display, the display method is selected from panoramic stereoscopic projection, naked-eye 3D image projection, and AR projection; the display content is generated according to the analysis information, and the analysis information is calculated and analyzed according to the image information by a calculation module preset by the intelligent data terminal.

[0034] Preferably, the area of ​​the first area is smaller than that of the second area, the first area corresponds to the venue booth and the second area, the first area corresponds to the user standing area, and the second area corresponds to the user walking area; the size of the unit block is adjusted according to the venue booth to form a standing area corresponding to the first area and a walking area corresponding to the second area, and the unit block size and the unit block arrangement are projected by a first projection device in a preset splicing method; the venue booth includes the booth size and the booth spacing.

[0035] Preferably, the display area for users standing in the user walking area is different from the display area for users standing in the first area, and the display area for users standing in the second area is projected between the first area and the second area by a first projection device in the venue to indicate the separation between the first area and the second area, including static projection and dynamic projection methods; when the user stands in the second area, dynamic images are displayed in the second area by the first projection device in the venue; when the user walks between the first area and the second area, dynamic images are displayed in the second area by the first projection device in the venue, and the users standing in the walking area are represented by the display area moving in the second area.

[0036] Preferably, when the user is in the first area:

[0037] When the intelligent data terminal receives the image information, it calculates the user's height based on the image information and determines the user's display method. After determining the user's display method, it uses the venue booth corresponding to the user's standing position as the projection area, and obtains the user's exhibit information from the cloud based on the venue booth corresponding to the user.

[0038] The intelligent data terminal receives image information captured by the camera device in the venue, and identifies and analyzes the image information according to a preset recognition algorithm, including: obtaining an upper body image of a user standing in the image information, the upper body image being a complete image of the user, including the torso, upper body and neck, or including the torso, upper body, neck and face of the user, and identifying the gender, age and race of the user according to preset image features; identifying whether the position where the user stands is a standing area according to the preset image features, and identifying the venue exhibits corresponding to the area where the user stands according to the preset image features, and obtaining corresponding exhibit information;

[0039] The intelligent data terminal identifies and analyzes image information according to a preset recognition algorithm, and determines whether the user is standing according to the height of the user's head. If it is recognized that the user is standing, the height of the user's head is compared with the boundary of the standing area, and the display method is determined according to the different heights.

[0040] Preferably, the recognition algorithm comprises:

[0041] According to the image information obtained by the intelligent data terminal, determine whether the image information contains user information. If it contains user information, obtain the image features of the user to identify the user's gender, age, and ethnicity; according to the user's gender, age, ethnicity, and the height of the user's head and the height of the boundary of the standing area, determine the exhibit information on the venue booth corresponding to the user;

[0042] The intelligent data terminal recognizes and analyzes image information according to a preset recognition algorithm, and displays the boundary of the position through a second projection device in the venue based on a standing signal of the user's standing position, and the standing signal is obtained through a pressure sensor in the venue.

[0043] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0044] First, through the collaborative work of intelligent data terminals and display interactive modules, the system can analyze user characteristics and behaviors in real time and provide highly personalized display content. This not only greatly improves the user experience, but also effectively improves the efficiency of information transmission of the exhibition.

[0045] Secondly, the combination of the VR terminal and the intelligent service background of the present invention realizes the seamless integration of virtual and reality. Users can experience rich virtual content through virtual reality glasses in a real exhibition hall environment. At the same time, the system can also adjust the virtual scene according to the user's real-time position and action, creating an unprecedented sense of immersion.

[0046] In addition, the panoramic projection display system of the present invention solves the problem of single content and lack of interaction in traditional display methods. Through the collaborative work of multiple projectors, the system can create a continuous curved or spherical projection screen in the exhibition hall and adjust the projection content in real time according to the user's position. This dynamic panoramic projection not only enhances the visual impact of the exhibition, but also provides an ideal solution for multiple people to visit at the same time.

[0047] Another important contribution of the present invention is the realization of accurate user identification and tracking. Through the cleverly designed user identification area and dynamic projection boundary, the system can accurately capture the user's location and behavior, laying the foundation for providing personalized content. This design also solves the problem of insufficient accuracy of traditional motion capture systems in large open spaces.

[0048] It is worth mentioning that the system of the present invention performs well when multiple people visit at the same time. By intelligently allocating different display areas and personalized content, the system can meet the needs of multiple users at the same time, while also promoting interaction and communication between users. This not only increases the participation in the exhibition, but also enhances its social attributes.

[0049] From the perspective of technical implementation, the present invention successfully integrates a variety of complex technologies through modular design and advanced data processing algorithms. This integration not only improves the scalability and maintainability of the system, but also achieves the synergistic effect of various technical advantages. For example, real-time processing of motion capture data enhances the realism of virtual reality, while panoramic projection provides a wider display space for virtual content.

[0050] In general, the intelligent XR multimodal interactive exhibition venue panoramic projection system of the present invention successfully solves many challenges facing current exhibition technology through innovative technology integration and clever interactive design. It not only brings unprecedented immersive and personalized experience to the audience, but also points the way for the digital transformation of the exhibition industry. The emergence of this system marks the beginning of a new era for exhibition technology, which is expected to completely change the way people visit exhibitions and bring revolutionary changes to the fields of cultural communication, scientific and technological display, and commercial promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a logic block diagram of the entire system of the present invention.

[0052] Figure 2 It is a logic block diagram of the intelligent data terminal of the present invention.

[0053] Figure 3 It is a logic block diagram showing the interactive module of the present invention.

[0054] Figure 4 This is a logic block diagram of the VR terminal of the present invention.

[0055] Figure 5 This is a logic block diagram of the intelligent service background of the present invention.

[0056] Figure 6 It is a logic block diagram of the panoramic projection display system of the present invention. DETAILED DESCRIPTION

[0057] See also Figure 1-6 The present invention provides an intelligent XR multi-modal interactive panoramic projection system for exhibition venues. The system aims to provide a highly interactive and immersive experience environment for exhibition venues. By combining advanced technologies such as virtual reality, motion capture and panoramic projection, it realizes the intelligent presentation and personalized interaction of exhibition content.

[0058] Specifically, the system of the present invention includes an intelligent data terminal 1, a display interaction module 2, a VR terminal 3, an intelligent service background 4 and a panoramic projection display system 5. These modules work together to create an interactive exhibition space that integrates reality and virtuality for users.

[0059] First, the intelligent data terminal 1 is the core processing unit of the system. It is mainly responsible for receiving the image information captured by the camera device 6 in the venue, and analyzing and identifying the information using a preset recognition algorithm. Based on the analysis results, the intelligent data terminal 1 generates corresponding display instructions. For example, when the camera device 6 captures a visitor standing in front of an exhibit, the intelligent data terminal 1 may identify the visitor's age range, gender and other characteristics, and generate instructions for display content and methods suitable for the visitor based on these characteristics.

[0060] Secondly, the display interaction module 2 maintains a communication connection with the intelligent data terminal 1. Its main function is to respond to the display instructions generated by the intelligent data terminal 1 and control the display content and display method. For example, according to the instructions of the intelligent data terminal 1, the display interaction module 2 may choose to use holographic projection technology to display a 3D model of a work of art, or start a video introduction to explain the principle of a certain scientific invention.

[0061] Again, VR terminal 3 is a key component in the system to realize virtual reality experience. It includes user client 31, motion capture card 32 and virtual reality glasses 33. VR terminal 3 can receive data transmitted by motion capture device 7 and visual tracker 8, which include user's motion information and position information. Based on this data, VR terminal 3 can update and display virtual scene data in real time to provide users with an immersive virtual experience. At the same time, VR terminal 3 is also responsible for providing visual rendering module data to ensure high-quality presentation of virtual scenes.

[0062] The intelligent service background 4 is connected to the VR terminal 3 through the network and is the data processing and control center of the entire system. It includes three core submodules: a motion data processing module 41, a virtual reality module 42, and a data communication and control module 43. The motion data processing module 41 is responsible for obtaining motion and position data from the motion capture card 32, and decoding and processing these data. For example, it may use a Kalman filter algorithm to smooth the motion data and reduce the effects of jitter and noise. The processed data is transmitted to the virtual reality module 42, which updates the virtual scene in real time based on these data. The data communication and control module 43 is responsible for data communication with the VR terminal 3 and controlling the content output of the display screen 9.

[0063] The panoramic projection display system 5 is another important component of the present invention. It includes a plurality of projectors 51, which are carefully arranged to form a continuous curved screen or spherical screen in the exhibition hall. Preferably, the system uses 8-10 single projectors, which are seamlessly spliced ​​by edge fusion technology. The uniqueness of the panoramic projection display system 5 is that it can dynamically adjust the content projected on the display screen 9 according to the real-time actions and positions of the users. This means that when the user moves in the exhibition hall, the projected content will change accordingly, creating an immersive feeling for the user.

[0064] A key innovation of the present invention lies in its user identification and interaction mechanism. When a user is in the projection area, the camera device 6 installed above the venue will be triggered to capture the user's image information. The projection area here usually corresponds to the booth area. In order to accurately identify the user's position, the system installs a first projection device on the ceiling of the venue to project the boundary points of the user identification area. This design not only improves the recognition accuracy of the system, but also provides users with intuitive visual feedback, letting them know when they have entered the interaction area.

[0065] Furthermore, the user identification area of ​​the present invention is cleverly divided into a first area and a second area. The line connecting the first area and the second area forms a rectangular boundary line, and the rectangular area contains a number of unit blocks. Among them, the first area corresponds to a unit block, which serves as a standing area for the user. The second area is composed of multiple unit blocks to form a walkable area. The advantage of this design is that it can meet the needs of static display and dynamic interaction at the same time. For example, when the user stands in the first area, the system may trigger a detailed introduction to the exhibits; when the user moves in the second area, the system may provide a more comprehensive exhibition overview or guidance information.

[0066] It is worth noting that the first area is deliberately set above the image recognition area. The image recognition area here corresponds to the projection area of ​​the venue booth. This layout design ensures that the system can accurately capture the full body image of the standing user, thereby performing more accurate recognition and analysis.

[0067] In order to further improve the interaction accuracy of the system, the present invention installs pressure sensors on the ground of the second area. When the user stands in the second area, these pressure sensors can sense the user's standing posture and generate a standing signal. Specifically, foot blocks are provided on the ground of the venue, and these foot blocks cover the unit blocks of the second area. When the user steps on these foot blocks, the pressure sensors will be activated. Subsequently, the standing signal is sent to the camera device 6 wirelessly to trigger the capture of image information. This design not only improves the response speed of the system, but also enhances the accuracy of the interaction.

[0068] For example, suppose we set the trigger threshold of the pressure sensor to 50N (approximately equal to the pressure of an adult standing on one foot). When the pressure exceeding this threshold is detected, the system will think that a user is standing in the area. This threshold is selected based on ergonomic research and can effectively distinguish between real human standing and other possible interference factors.

[0069] In general, the intelligent XR multimodal interactive exhibition venue panoramic projection system of the present invention realizes a highly personalized and interactive exhibition experience by comprehensively applying a variety of advanced technologies. It can not only accurately identify and track the user's position and movement, but also dynamically adjust the display content and method according to the user's characteristics and behavior. This innovative interactive method provides a new display paradigm for modern exhibition venues, which is expected to significantly improve the participation and satisfaction of visitors. The system of the present invention further optimizes the user's mobile experience in the exhibition hall. When the user walks from the first area through the second area, the camera device 6 captures real-time image information. This real-time tracking ensures that the system can continuously respond to the user's movement, thereby providing a seamless interactive experience.

[0070] Preferably, the system of the present invention sets at least two projection areas in the venue, and the areas of these projection areas cover all the booths in the venue. This design ensures that users can enjoy a comprehensive visual experience no matter where they are in the venue. For example, in a 1,000 square meter exhibition hall, two projection areas of 500 square meters each may be set up, covering the front half and the back half of the venue respectively.

[0071] A notable feature of the present invention is that the intelligent data terminal 1 is configured with a display area. In this display area, a first area and a second area are also set. When a user walks in an area between the two areas, the first projection device dynamically adjusts the projected user standing area and walking area. This dynamic adjustment enables the system to accurately track the user's movement trajectory and provide the user with more personalized display content.

[0072] For example, suppose a user moves from exhibit A to exhibit B, a distance of about 10 meters. During this process, the first projection device may adjust the projection area at a frequency of once per second to ensure that the user is always in the best interactive position. This high-frequency update can create a smooth movement experience for users, as if the entire exhibition hall is changing with their footsteps.

[0073] It is worth noting that the first projection device and the camera device 6 work closely together to achieve real-time image information capture and processing. This collaborative working mode greatly improves the response speed and accuracy of the system.

[0074] The system of the present invention is also unique in terms of display interaction. The display interaction module 2 performs interactive control based on the display instructions generated by the intelligent data terminal 1, and this control includes the adjustment of the display content and display mode. The system will control different interactive results according to the analysis information received by the intelligent data terminal 1, and these results include the first display mode, the second display mode and the third display mode.

[0075] Specifically, the display content is closely related to the image information. The system of the present invention supports multiple display content types, including static display, stereoscopic dynamic display and interactive display. Correspondingly, the display methods are also diverse, including holographic projection, panoramic stereoscopic projection, video projection, naked eye 3D image projection and AR projection.

[0076] Such diverse display content and methods enable the system of the present invention to adapt to various exhibition needs. For example, for a static artwork, the system may choose to use holographic projection to display its three-dimensional structure; while for a highly interactive technological exhibit, the system may use AR projection to allow users to explore various parts of the exhibit through gestures.

[0077] In a preferred embodiment of the present invention, the system selects a corresponding display method according to the result of user recognition. Specifically, when the display content is a static display, the system selects between holographic projection and video projection; when the display content is a stereoscopic dynamic display or an interactive display, the system selects between panoramic stereoscopic projection, naked eye 3D image projection and AR projection.

[0078] This intelligent display method selection greatly improves the viewing experience and interactivity of the exhibition. For example, for a young audience aged between 20 and 30, the system may tend to choose a more interactive and technological AR projection; while for an older audience, the system may choose a more intuitive and easy-to-understand video projection.

[0079] It is worth mentioning that the display content is generated based on the analysis information, which is obtained by the calculation module preset by the intelligent data terminal 1 based on the image information. This process involves complex image recognition and machine learning algorithms, ensuring the high personalization and adaptability of the display content.

[0080] In another embodiment of the present invention, the area of ​​the first area is smaller than that of the second area. This design is based on an in-depth analysis of user behavior patterns. The first area corresponds to the exhibition booths and user standing areas, while the second area corresponds to the user walking area. This layout fully takes into account the characteristics of user activities in the exhibition hall, that is, users usually stand in front of an exhibit for a while and then move to the next exhibit.

[0081] The system will adjust the size of the unit block according to the specific conditions of the venue booth to form a standing area corresponding to the first area and a walking area corresponding to the second area. The size and arrangement of the unit blocks here adopt a preset splicing method and are projected through the first projection device. For example, in a typical setting, the unit block in the first area may be 2 meters by 2 meters in size, while the unit block in the second area may be 1 meter by 1 meter in size. This differentiated design can better adapt to the user's activity needs in different areas.

[0082] The system of the present invention also takes into account the characteristics of the exhibition booths in the venue, including the size of the booths and the distance between the booths. For example, for a large exhibit, the system may allocate a larger first area; and for the aisles between the booths, the system will adjust the size and shape of the second area accordingly. This flexible layout adjustment ensures that the system can adapt to various exhibition hall layouts and provide the best user experience.

[0083] In general, the intelligent XR multimodal interactive exhibition venue panoramic projection system of the present invention provides a new interactive paradigm for modern exhibitions through its advanced user tracking, intelligent display selection and flexible space layout design. It can not only accurately capture user behavior and preferences, but also dynamically adjust the display content and methods based on this information, thereby creating a highly personalized and immersive exhibition experience. This innovative exhibition method is expected to significantly improve the participation and satisfaction of visitors, and bring revolutionary changes to the exhibition industry. The system of the present invention has made further innovations in the user interaction experience. Specifically, the system has carefully designed the display effects of users in different areas to enhance the user's spatial perception and interactive experience.

[0084] In a preferred embodiment of the present invention, the display area of ​​the user standing in the user walking area (i.e., the second area) is significantly different from the display area of ​​the user standing in the first area. This differentiated design is intended to provide users with a clear sense of spatial positioning and also to guide users to focus on specific display content.

[0085] In order to achieve visual separation between the first area and the second area, the system of the present invention cleverly utilizes the first projection device in the venue. Specifically, the first projection device projects the display area of ​​the standing user in the second area, and this projection can be static or dynamic. For example, when the user stands in the second area, the system may project a glowing circle under the user's feet, and the color of the circle may change over time, or change size according to the user's movement speed. This dynamic visual feedback can not only enhance the user's sense of immersion, but also provide the user with intuitive location information.

[0086] When the user moves in the second area, the system of the present invention will display dynamic images in the second area through the first projection device. This dynamic display can reflect the user's movement trajectory in real time, creating an "interactive carpet" effect for the user. For example, the system may leave a series of luminous footprints on the user's moving path, and these footprints may gradually disappear over time, creating a dynamically changing visual effect.

[0087] Furthermore, when the user moves from the first area to the second area, or walks between the two areas, the system will display dynamic images in the second area through the first projection device. This display method can clearly mark the user's walking path and provide useful spatial information to other visitors. For example, the system may project a colored light strip on the user's walking path, and the color of the light strip may change according to the user's moving speed or stay time.

[0088] The system of the present invention has also been deeply optimized in terms of user identification and information processing. When the user is in the first area, the intelligent data terminal 1 will first receive image information, then calculate the user's height based on this information, and determine the user's display method. This process involves complex image processing and machine learning algorithms, which can quickly and accurately identify user characteristics and make corresponding display decisions.

[0089] For example, suppose the system detects an adult male user who is about 180cm tall standing in the first area. Based on this information, the system may determine that the user is suitable for a more in-depth introduction to the exhibits. The system will then use the venue booth corresponding to the user's standing position as the projection area and obtain the corresponding exhibit information from the cloud. This approach ensures that the display content is highly relevant and personalized.

[0090] The intelligent data terminal 1 uses advanced recognition algorithms when processing image information. This algorithm can extract the upper body image of the user from the image, including the torso, upper body and neck, and sometimes the face. Based on this information, the system can identify the user's gender, age, and ethnicity. For example, the system may use a deep learning model to analyze the user's facial features to estimate the user's age range and gender. This accurate user portrait can help the system provide more personalized display content.

[0091] In addition to user feature recognition, the system of the present invention can also recognize whether the location where the user is standing is a standing area, and the venue exhibits corresponding to the area. This spatial perception capability enables the system to provide users with exhibit information that is highly relevant to their location.

[0092] In terms of judging the user's posture, the system of the present invention adopts a judgment method based on the head height. Specifically, the system compares the user's head height with the boundary height of the standing area. If the user's head height is significantly higher than the boundary height, the system will judge that the user is in a standing posture. The advantages of this method are its simplicity and reliability, and it can quickly judge the user's posture, so as to adjust the display method in time.

[0093] The system of the present invention also includes a set of complex recognition algorithms. This algorithm first determines whether the image information contains user information. If a user is detected, the algorithm further extracts the user's image features and identifies the user's gender, age and ethnicity. Subsequently, the algorithm combines these user features and the comparison result of the user's head height and the standing area boundary height to determine the exhibit information on the venue booth corresponding to the user.

[0094] For example, if the system identifies a female user between the ages of 30 and 40, approximately 165cm tall, standing in front of a booth in the art exhibition area, the system may choose to play an audio introduction about the background of the creation of the artwork, while projecting an animation of the creation process of the artwork on the ground in front of the user.

[0095] Finally, the system of the present invention also uses the pressure sensor in the venue to obtain the user's standing signal. This signal triggers the second projection device in the venue to display the boundary of the user's location. This method not only improves the response speed of the system, but also provides the user with clear spatial positioning feedback.

[0096] In general, the intelligent XR multimodal interactive exhibition venue panoramic projection system of the present invention creates a highly intelligent and personalized exhibition experience by comprehensively using advanced image processing, machine learning, projection technology and sensor technology. The system can accurately identify user characteristics, track user locations in real time, and dynamically adjust the display content and methods based on this information. This innovative exhibition model can not only significantly improve the user experience, but also provide new possibilities for the digital transformation of the exhibition industry.

[0097] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. Intelligent XR multi-modal interactive exhibition venue panoramic projection system, characterized by: include: Intelligent data terminal for: Receive image information captured by the camera device in the venue; Identify and analyze image information according to preset recognition algorithms; Generate display instructions; The display interaction module is connected to the intelligent data terminal for: Responding to a display instruction generated by the intelligent data terminal; Control what is displayed and how it is displayed; VR terminal, including user client, motion capture card and virtual reality glasses, is used to: Receive data transmitted by motion capture devices and visual trackers; Display virtual scene data; Provide visual rendering module data; The intelligent service backend is connected to the VR terminal via a network, and includes: The motion data processing module is used to obtain motion and position data from the motion capture card, and decode and process the data; A virtual reality module, used to update the virtual scene according to the motion data; Data communication and control module, used for data communication with VR terminal and controlling display screen; A panoramic projection display system, comprising a plurality of projectors, is used for: Forming curved or spherical screens in the exhibition hall; Determine the content projected onto the display screen based on the user's real-time action position; Among them, when the user is in the projection area, a camera device for capturing image information is arranged above the venue; the projection area is the booth area; the camera device is triggered to capture image information based on the user standing in the user identification area, and the boundary points of the user identification area are projected by the first projection device on the ceiling of the venue.

2. The system according to claim 1, characterized in that The user identification area includes a first area and a second area, the line connecting the first area and the second area is a rectangular boundary line, the rectangular area includes a plurality of unit blocks, the first area is a unit block, the first area corresponds to a standing area for the user, and the second area is a walkable area formed by splicing a plurality of unit blocks; the first area is located above the image identification area; the image identification area corresponds to the projection area of ​​the venue booth.

3. The system according to claim 2, characterized in that When the user stands in the second area, a standing signal is generated by sensing the user's standing posture through a pressure sensor on the venue floor. The venue floor includes foot blocks, and the foot blocks cover the unit blocks of the second area. The standing signal is sent wirelessly to the camera device, and the camera device captures image information based on the standing signal.

4. The system according to claim 2, characterized in that When the user moves from the first area to the second area, the camera device captures real-time image information. At least two projection areas are set in the venue, and the areas of the two projection areas cover all the booths in the venue; the intelligent data terminal is equipped with a display area, and the display area is provided with a first area and a second area. When the user walks in an area between the first area and the second area, the user standing area and walking area projected by the first projection device also change accordingly, and the first projection device cooperates with the camera device to capture and process real-time image information.

5. The system according to claim 1, characterized in that The display interaction module performs interactive control based on the display instruction generated by the intelligent data terminal, and the interactive control includes display content and display mode control. The display content and display mode control different interactive results according to the analysis information received by the intelligent data terminal. The interactive results include a first display mode, a second display mode, and a third display mode; The display content is associated with the image information; the display content includes static display, stereoscopic dynamic display and interactive display, and the display methods include holographic projection, panoramic stereoscopic projection, video projection, naked-eye 3D image projection, and AR projection.

6. The system according to claim 5, characterized in that Select a corresponding display method according to the display content identified by the user; when the display content is a static display, the display method is selected from holographic projection and video projection; when the display content is a three-dimensional dynamic display or an interactive display, the display method is selected from panoramic stereoscopic projection, naked eye 3D image projection, and AR projection; The display content is generated based on the analysis information, and the analysis information is calculated and analyzed by a calculation module preset in the intelligent data terminal based on the image information.

7. The system according to claim 2, characterized in that The area of ​​the first area is smaller than that of the second area. The first area corresponds to the venue booth and the second area. The first area corresponds to the user standing area, and the second area corresponds to the user walking area. The size of the unit block is adjusted according to the venue booth to form a standing area corresponding to the first area and a walking area corresponding to the second area. The unit block size and the unit block arrangement method are both projected by a first projection device in a preset splicing method. The venue booth includes the booth size and the booth spacing.

8. The system according to claim 1, characterized in that The display area for users standing in the user walking area is different from the display area for users standing in the first area. Between the first area and the second area, the display area for users standing in the second area is projected by a first projection device in the venue to indicate the separation between the first area and the second area, including static projection and dynamic projection methods. When a user stands in the second area, a dynamic image is displayed in the second area by the first projection device in the venue. When a user walks between the first area and the second area, a dynamic image is displayed in the second area by the first projection device in the venue, and the users standing in the walking area are represented by the display area moving in the second area.

9. The system according to any one of claims 1 to 8, characterized in that: When the user is in the first area: When the intelligent data terminal receives the image information, it calculates the user's height based on the image information and determines the user's display method. After determining the user's display method, it uses the venue booth corresponding to the user's standing position as the projection area, and obtains the user's exhibit information from the cloud based on the venue booth corresponding to the user. The intelligent data terminal receives image information captured by the camera device in the venue, and identifies and analyzes the image information according to a preset recognition algorithm, including: obtaining an upper body image of a user standing in the image information, the upper body image being a complete image of the user, including the torso, upper body and neck, or including the torso, upper body, neck and face of the user, and identifying the gender, age and race of the user according to preset image features; identifying whether the position where the user stands is a standing area according to the preset image features, and identifying the venue exhibits corresponding to the area where the user stands according to the preset image features, and obtaining corresponding exhibit information; The intelligent data terminal identifies and analyzes image information according to a preset recognition algorithm, and determines whether the user is standing according to the height of the user's head. If it is recognized that the user is standing, the height of the user's head is compared with the boundary of the standing area, and the display method is determined according to the different heights.

10. The system according to any one of claims 1 to 8, characterized in that: The recognition algorithm includes: According to the image information obtained by the intelligent data terminal, determine whether the image information contains user information. If it contains user information, obtain the image features of the user to identify the user's gender, age, and ethnicity; according to the user's gender, age, ethnicity, and the height of the user's head and the height of the boundary of the standing area, determine the exhibit information on the venue booth corresponding to the user; The intelligent data terminal recognizes and analyzes image information according to a preset recognition algorithm, and displays the boundary of the position through a second projection device in the venue based on a standing signal of the user's standing position, and the standing signal is obtained through a pressure sensor in the venue.

Images