Digital exhibition interaction system and method

Abstract

The invention relates to the technical field of digital exhibition, in particular to a digital exhibition interaction system and method, and the system comprises an augmented reality module which is used for generating augmented reality content related to exhibition content, and pushing AR exhibition content to an exhibition terminal; the intelligent display module is used for identifying exhibits according to the images acquired by the image acquisition device and superposing relevant information and interactive elements on a display terminal; the interaction module is also used for capturing interaction actions of the exhibitors on the AR content of the exhibit and adjusting the AR display content according to the interaction actions; the intelligent positioning module is used for analyzing the distance between the exhibitors and the exhibits according to the collected images and adjusting the size and precision of the AR display content according to the distance; the intelligent guide module is used for guiding according to the visiting route in combination with the collected image; the system is also used for adjusting the basic explanation content according to the residence time and the interaction action of the exhibitors. According to the technical scheme, personalized digital exhibition interaction can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of digital exhibition, and in particular to a digital exhibition interactive system and method. Background Art

[0002] With the rapid development of science and technology, digital exhibitions have become an important part of the modern exhibition industry. Traditional exhibition methods are often limited to static display materials, such as pictures, texts and videos, which make it difficult to provide rich interactive experiences and in-depth exhibit information. In order to overcome this limitation, digital exhibition interactive systems have emerged in recent years, aiming to bring exhibitors a more vivid, intuitive and immersive exhibition experience through advanced technical means.

[0003] In the existing digital exhibition interactive system, augmented reality (AR) technology is widely used. AR technology can superimpose virtual information into the real environment, providing exhibitors with dynamic displays closely related to the exhibition content. However, many existing AR exhibition systems simply push pre-made AR content to the display terminal, lacking real-time interaction and personalized display with exhibitors, resulting in a relatively monotonous exhibition experience and lack of depth. Summary of the invention

[0004] The purpose of the present invention is to provide a digital exhibition interaction system and method, which can realize personalized digital exhibition interaction.

[0005] To achieve the above objectives, in a first aspect, an embodiment of the present disclosure provides a digital exhibition interactive system, including: The augmented reality module is used to generate augmented reality content related to the exhibition content and push the AR display content to the display terminal; The intelligent display module is used to identify exhibits through images captured by the image acquisition device and to overlay relevant information and interactive elements on the display terminal; it is also used to capture the interactive actions of the exhibitors on the AR content of the exhibits and adjust the AR display content according to the interactive actions; Intelligent positioning module, used to analyze the distance between the exhibitor and the exhibits based on the collected images, and adjust the size and accuracy of the AR display content according to the distance; The intelligent navigation module is used to provide guidance based on the tour route and the collected images; it is also used to adjust the basic explanation content based on the exhibitors' stay time and interactive actions.

[0006] Beneficial effects of the basic solution: The augmented reality module of this technical solution generates AR display content that is closely related to the exhibition content, and pushes this content to the display terminal in real time. As the basis of digital exhibition, it enriches the expression form of the exhibition and enhances the immersion and fun of the exhibition.

[0007] The intelligent display module uses an image acquisition device to identify exhibits and overlays relevant information and interactive elements on the display terminal to ensure that exhibitors can quickly obtain accurate information related to the exhibits. The intelligent positioning module adjusts the size and accuracy of the AR display content according to the distance between the exhibitor and the exhibit, ensuring that exhibitors can obtain clear and easy-to-read AR display content regardless of their position. The intelligent guide module provides guidance based on the tour route combined with the collected images, which avoids the guide from getting lost or missing important exhibits, and can also improve the efficiency of viewing the exhibition. At the same time, the AR display module is used for guidance, with better technical uniformity, more intuitive and accurate. In addition, it can also analyze the interests of exhibitors, provide personalized explanations, and improve the exhibition experience.

[0008] In addition, the above technical features are all implemented using image analysis technology, which reduces hardware costs and facilitates the deployment of the system.

[0009] As an implementable preferred solution, the intelligent display module collects exhibit images through the image acquisition device on the display terminal, compares them with the exhibit image database pre-stored in the cloud platform to identify the exhibits, and superimposes AR display content related to the exhibits after successful identification.

[0010] As an implementable preferred solution, the intelligent positioning module analyzes the distance between the exhibitor and the exhibits based on the images captured by the image acquisition device, and adjusts the size and accuracy of the AR display content.

[0011] As an implementable preferred solution, the intelligent positioning module generates the exhibitor's tour route based on the zoom record and sequence of the exhibits' corresponding display content and the exhibits' locations stored in the cloud platform; The intelligent positioning module takes the exhibit closest to the participant when they last stopped as the target point. Other exhibits zoomed in during the same period are only used to correct the moving route from the previous target point to the next target point. Based on the participant's movement records of the exhibits, points of interest are marked on the visiting route.

[0012] As an implementable preferred solution, the intelligent guide module uses a neural network model to learn the trajectories of a large number of exhibitors in the trajectory generation module, analyzes the exhibits closest to the exhibitors through the images collected by the image acquisition module, and determines the direction of the exhibitors based on other exhibits in the image, and provides directional guidance on the display interface of the display terminal according to the visiting route.

[0013] As an implementable preferred solution, 6. A digital exhibition interactive system according to claim 1, characterized in that: the intelligent guide module uses a generative large model to adjust the basic explanation content, each exhibit is provided with basic explanation content, and the generative large model adjusts the basic explanation content according to the exhibitor's stay time and interactive actions; The intelligent guide module is also used to collect text or voice input by exhibitors, and use the generative big model to answer the exhibitors' questions, or remotely connect with a human tour guide for online explanations.

[0014] As an implementable preferred solution, the smart guide module is also used to track the exhibitors' attention points based on an eye tracking device or a front camera, mark the attention points as points of interest, and expand the basic explanation content based on the points of interest.

[0015] As an implementable preferred solution, the intelligent navigation module is also used to analyze the parts of the current exhibit that are related to the previous explanation content, and use the related parts as requirements for generating explanation content for the current exhibit for content linkage.

[0016] As an implementable preferred solution, the intelligent guide module is also used to plan the guide route, analyze the crowd density around the exhibits based on the images obtained by the image acquisition device, identify the congestion points on the guide route, and analyze the congestion degree and congestion duration of the congestion points in different time periods based on the historical visit data stored on the cloud platform.

[0017] In a second aspect, the disclosed embodiments further provide a digital exhibition interaction method, which utilizes a digital exhibition interaction system as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a schematic diagram of the architecture of a digital exhibition interactive system.

[0019] Figure 2 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the technical solution and advantages of the present application clearer, the technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings. It will be understood that the specific embodiments described herein are only partial embodiments of the present invention, which are only used to explain the present application, rather than to limit the present application. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered isolated, and they can be combined with each other to achieve better technical effects. The same reference numerals appearing in the drawings of the following embodiments represent the same features or components, which can be applied to different embodiments.

[0021] In addition, unless otherwise defined, the technical terms or scientific terms used in the description of the present invention should have the common meanings understood by those skilled in the art in the art to which the present invention belongs.

[0022] Reference numerals: electronic device 500 , processor 501 , communication interface 502 , memory 503 , bus 504 .

[0023] The present invention is further described in detail below in conjunction with the accompanying drawings: Reference Figure 1 , a digital exhibition interactive system, comprising: The augmented reality module includes a content generation submodule and a content push submodule.

[0024] The content generation submodule is used to generate AR content related to the exhibition content, including three-dimensional models, videos, and images, etc., including steps such as data collection, model building, rendering, and publishing.

[0025] In the data collection phase, physical information of the exhibits, such as size, shape, and color, is obtained through scanning and photography. Then, the model is constructed, and a 3D model of the exhibits is constructed based on the collected data using 3D modeling software. In the rendering phase, the system performs coloring and mapping on the 3D model. Finally, the rendered AR content is published to the cloud platform.

[0026] The content push submodule is used to push AR display content to display terminals (such as smartphones, tablets, smart glasses, etc.). When visitors enter the exhibition area, their display terminals will connect to the cloud platform to receive the corresponding AR content.

[0027] The intelligent display module includes an exhibit display submodule and a gesture recognition submodule.

[0028] The exhibit display submodule is used to identify exhibits and overlay relevant information and interactive elements on the display terminal. The image of the exhibit is collected by the image acquisition device on the display terminal, and compared with the exhibit image database pre-stored in the cloud platform to identify the exhibit. After successful identification, AR display content related to the exhibit is overlaid on the visitor's device.

[0029] The gesture recognition submodule is used to capture the exhibitor's interactive actions on the AR content of the exhibits, including zooming and rotating. The exhibit display submodule adjusts the AR display content accordingly based on the captured action instructions. For example, if the exhibitor performs a two-finger zoom operation on the screen, the exhibit display submodule will enlarge the three-dimensional model of the exhibit to show more details; or, if the exhibitor wears smart glasses and raises his hand to rotate the AR display content of the exhibit in front of him, the exhibitor's interactive actions are captured by the image acquisition device of the smart glasses, and the exhibit display submodule will adjust the AR content of the exhibit according to the amplitude and direction of the exhibitor's rotation.

[0030] The intelligent positioning module includes a distance analysis submodule and a trajectory generation submodule.

[0031] The distance analysis submodule is used to analyze the distance between the exhibitor and the exhibit according to the image captured by the image acquisition device, so as to adjust the size and accuracy of the AR display content. That is, when the image captured by the image acquisition device contains multiple exhibits, the size of the exhibit is adjusted according to the distance between the exhibitor and the exhibitor. The exhibits farther away from the exhibitors have smaller AR display content and lower accuracy, and the exhibits closer to the exhibitors have larger AR display content and higher accuracy; thereby achieving a well-arranged distribution of AR display content and highlighting the key points. In addition, it is also convenient for exhibitors to know the locations of other exhibits that can have AR display content. This technical solution does not need to adopt indoor positioning technology, has lower hardware requirements, and is convenient for the popularization of this system. At the same time, due to the dense distribution of indoor exhibits, indoor positioning technology is sometimes disturbed and not accurate enough. This technical solution can also solve this problem. The image captured by the image acquisition device can synchronize the position of the exhibitor. When the reference person is close to the exhibit, the size of the AR display content corresponding to the exhibit and the accuracy of the exhibit will be gradually increased. When the reference person is far away from the exhibit, the size and accuracy of the AR display content corresponding to the exhibit will be gradually reduced.

[0032] The trajectory generation submodule is used to generate the exhibitor's tour route. This technical solution does not rely on indoor positioning technology. According to the zoom records and sequence of the corresponding display content of the exhibits, combined with the exhibits' locations stored in the cloud platform, the exhibitor's tour route is generated. During the movement of the exhibitors, there may be multiple exhibits that are zoomed at the same time. In this case, the exhibit closest to the exhibitor when the exhibitor last stopped is used as the target point. Other exhibits zoomed in the same period are only used to correct the movement route from the previous target point to the next target point, avoiding the tour route being too complicated or circling in place, reducing interference when the intelligent guide module is learning. In addition, according to the exhibitors' records of exhibitor movements, points of interest are marked on the tour route. By generating the exhibitor's tour route and marking points of interest, it is possible to analyze the overall preferences of the exhibitors, set recommended tour routes, or automatically generate recommended tour routes based on the tour routes that overlap with most exhibitors, so that the display content of the exhibits that the exhibitors are interested in can be further enriched, and it is convenient to adjust and optimize the overly dense points of interest to avoid congestion in the tour route.

[0033] The intelligent navigation module includes a path navigation sub-module and an intelligent explanation sub-module.

[0034] The path navigation submodule uses a neural network model to learn the trajectories of a large number of exhibitors in the trajectory generation module, which is used to guide visitors based on the recommended tour routes stored in the cloud platform and the collected images. The image collected by the image acquisition module analyzes the exhibits closest to the exhibitors, and determines the direction of the exhibitors based on other exhibits in the image. Direction guidance is provided on the display interface of the display terminal based on the recommended tour route. Direction guidance can be provided in the form of arrows or text. Since the trajectory generation submodule records the tour routes of a large number of visitors, it can guide exhibitors with different directions to the next target point of the recommended tour route. When the route is congested and the exhibitors deviate from the path in the recommended tour route, they can also be guided to the next target point, thereby improving the flexibility and ease of use of the navigation.

[0035] The intelligent explanation submodule uses the generative large model to adjust the basic explanation content. Each exhibit is set with basic explanation content. The generative large model expands, reduces, associates and adjusts the basic explanation content according to the exhibitor's stay time, interactive actions, etc. When the exhibitor stays at the current exhibit for longer than the average stay time corresponding to the exhibit, the intelligent explanation submodule will use the generative large model to expand the basic explanation content. When the exhibitor starts to move to other exhibits (that is, the AR display content corresponding to the current explanation starts to shrink, and the AR display content of other exhibits starts to enlarge), the generative large model will be used to make a brief summary and end the current explanation. When the exhibitor interacts with the current exhibit, such as zooming in on a detail of the exhibit or rotating to a specific angle of the exhibit, the intelligent explanation submodule will use the generative large model to expand the basic explanation content, enrich the explanation content, and satisfy the exhibitor's interest.

[0036] In one embodiment, the smart guide module also includes an interest analysis submodule, which is used to track the exhibitors' attention points based on the eye tracking device on the smart glasses, the front camera on the mobile phone or tablet, and the algorithm, and mark the attention points as the exhibitors' points of interest. The smart explanation submodule will expand the basic explanation content based on the points of interest.

[0037] In one embodiment, the intelligent tour guide module also includes an intelligent question-and-answer submodule, which is used to collect text or voice input by exhibitors, and use the generative big model to answer the exhibitors' questions, and answer the exhibitors' questions in a timely manner. If the exhibitors' questions are not answered, the intelligent question-and-answer submodule can also remotely connect with the human tour guide to check the exhibitors' questions and explain them online, thereby improving the efficiency of the explanation.

[0038] In one embodiment, the intelligent navigation module also includes an associative analysis submodule for analyzing the parts of the current exhibit that are related to the previous explanation content, the related parts including exhibits, features and points of interest related to the current exhibit, and using the related parts as requirements for generating explanation content for the current exhibit for content linkage, thereby improving the logic and depth of the explanation and helping exhibitors to quickly establish a knowledge system of the exhibits. Since the previous explanation content was originally generated based on the exhibitors' interests, the associated explanation content can be closer to the content that the exhibitors want to know, thereby improving the effect of the explanation.

[0039] Embodiment 2 The technical feature that distinguishes this embodiment from the above-mentioned embodiments is that the intelligent navigation module is also used to plan the navigation route, analyze the crowd density around the exhibits based on the images of the image acquisition device (or directly use the data of the exhibition hall monitoring equipment), identify the congestion points of the navigation route, and analyze the congestion degree and congestion duration of the congestion points in different time periods based on the historical visit data stored in the cloud platform. Rearrange the order of visiting exhibits, the steps are as follows: Use the currently selected tour route as the initial plan , set the initial temperature , cooling rate and termination temperature The initial temperature is dynamically adjusted according to the remaining distance of the current tour route. The longer the distance, the higher the initial temperature T, so more iterations are performed and the cooling rate is Dynamic adjustment based on the congestion level of the next exhibit on the current tour route. The higher the congestion level, the faster the cooling rate. The bigger.

[0040] Setting the fitness function , the formula is as follows:

[0041] in, represents the path distance from the Kth exhibit to the k+1th exhibit; Indicates the movement speed of the exhibitors; represents the crowd density of the k+1th exhibit; Indicates the congestion level of the path from the kth exhibit to the k+1th exhibit; It represents the total waiting time for an exhibit, estimated based on the crowd density and capacity of the exhibit; The fatigue factor is a comprehensive assessment based on the total length of the visiting path, the difficulty of moving between exhibits, and other factors; , , , is the weight coefficient, which will be adjusted according to the actual situation.

[0042] Randomly perturb the current solution Generate a neighboring solution . Calculate the performance of the new solution , if the performance of the solution is better, the new solution is accepted; if the performance is worse, the new solution is accepted with a certain probability. The formula is as follows:

[0043] in, is a constant, Indicates the current temperature. It represents the degree of acceptance of the direction guidance by the exhibitors. A high degree of acceptance means a higher probability of accepting the new plan and a lower probability of rejection. This can reduce the number of reminders for changing the exhibitors' routes in actual experience. If the exhibitors themselves do not accept the direction guidance and the navigation system repeatedly reminds the users, it will result in a bad experience for the exhibitors.

[0044] Update temperature parameters .

[0045] The iteration is repeated until the termination temperature is reached, and the weight allocation scheme with the highest performance is selected as the final resource allocation scheme.

[0046] During the exhibitors' visit, the intelligent guide module will continuously detect changes in crowd density and congestion, and adjust the guide route in real time to ensure that exhibitors can visit along the optimal route. The disclosed embodiment also provides a digital exhibition interactive method, which utilizes a digital exhibition interactive system.

[0047] The embodiment of the present disclosure further provides a storage medium, in which a computer program is stored. When the computer program is executed by a processor, all steps of the above-mentioned digital exhibition interaction method can be implemented.

[0048] Those of ordinary skill in the art can understand that the implementation of all or part of the process in a digital exhibition interactive method can be completed by instructing related hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the process of each embodiment of a digital exhibition interactive method. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0049] The embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned digital exhibition interactive method when executing the program. In the embodiment of the present application, the processor is the control center of the computer system, which can be a processor of a physical machine or a processor of a virtual machine.

[0050] Reference Figure 2 The electronic device 500 includes: at least one processor 501, at least one communication interface 502, at least one memory 503 and at least one bus 504. The bus 504 is used to realize the connection and communication between these components, the communication interface 502 is used to communicate with other node devices for signaling or data, and the memory 503 stores machine-readable instructions executable by the processor 501. When the electronic device 500 is running, the processor 501 communicates with the memory 503 through the bus 504, and when the machine-readable instructions are called by the processor 501, the steps of the digital exhibition interactive method as described above are executed.

[0051] The above contents are only embodiments of the present invention. The common sense such as the known specific structures and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for the ordinary technicians in the relevant field to implement this application. It should be pointed out that for the technicians in this field, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to explain the content of the claims.

Claims

1. A digital exhibition interactive system, characterized by: include: The augmented reality module is used to generate augmented reality content related to the exhibition content and push the AR display content to the display terminal; The intelligent display module is used to identify exhibits through images captured by the image acquisition device and to overlay relevant information and interactive elements on the display terminal; it is also used to capture the interactive actions of the exhibitors on the AR content of the exhibits and adjust the AR display content according to the interactive actions; Intelligent positioning module, used to analyze the distance between the exhibitor and the exhibits based on the collected images, and adjust the size and accuracy of the AR display content according to the distance; The intelligent navigation module is used to provide guidance based on the tour route and the collected images; it is also used to adjust the basic explanation content based on the exhibitors' stay time and interactive actions.

2. A digital exhibition interactive system according to claim 1, characterized in that: The intelligent display module collects exhibit images through the image acquisition device on the display terminal, compares them with the exhibit image database pre-stored in the cloud platform to identify the exhibits, and superimposes AR display content related to the exhibits after successful identification.

3. A digital exhibition interactive system according to claim 2, characterized in that: The intelligent positioning module analyzes the distance between the exhibitors and the exhibits based on the images collected by the image acquisition device, and adjusts the size and accuracy of the AR display content.

4. A digital exhibition interactive system according to claim 1, characterized in that: The intelligent positioning module generates the exhibitor's tour route based on the zoom record and sequence of the exhibits' corresponding display content and the exhibits' locations stored in the cloud platform; The intelligent positioning module takes the exhibit closest to the participant when they last stopped as the target point. Other exhibits zoomed in during the same period are only used to correct the moving route from the previous target point to the next target point. Based on the participant's movement records of the exhibits, points of interest are marked on the visiting route.

5. A digital exhibition interactive system according to claim 1, characterized in that: The intelligent navigation module uses a neural network model to learn the trajectories of a large number of exhibitors in the trajectory generation module, analyzes the exhibits closest to the exhibitors through the images collected by the image acquisition module, and determines the direction of the exhibitors based on other exhibits in the image, and provides directional guidance on the display interface of the display terminal according to the visiting route.

6. A digital exhibition interactive system according to claim 1, characterized in that: The intelligent guide module uses the generative big model to adjust the basic explanation content. Each exhibit is equipped with basic explanation content. The generative big model adjusts the basic explanation content according to the exhibitors' stay time and interactive actions. The intelligent guide module is also used to collect text or voice input by exhibitors, and use the generative big model to answer the exhibitors' questions, or remotely connect with a human tour guide for online explanations.

7. A digital exhibition interactive system according to claim 1, characterized in that: The intelligent navigation module is also used to track the exhibitors' attention points based on the eye tracking device or the front camera, mark the attention points as points of interest, and expand the basic explanation content based on the points of interest.

8. A digital exhibition interactive system according to claim 1, characterized in that: The intelligent navigation module is also used to analyze the parts of the current exhibit that are related to the previous explanation content, and use the related parts as the requirements for generating explanation content for the current exhibit to perform content linkage.

9. A digital exhibition interactive system according to claim 1, characterized in that: The intelligent tour guide module is also used to plan tour routes, analyze the crowd density around the exhibits based on the images from the image acquisition device, identify congestion points on the tour routes, and analyze the congestion degree and duration of congestion points in different time periods based on the historical visit data stored on the cloud platform.

10. A digital exhibition interactive method, characterized in that: A digital exhibition interactive system as described in any one of claims 1 to 9 is used.

Images