Exhibition hall interest point prediction method and prediction system based on multi-modal data

Abstract

The invention discloses an exhibition hall interest point prediction method and prediction system based on multi-modal data. The method comprises the following steps: acquiring sensing data acquired by a sensor; the sensing data comprises voice data, gesture data and track data generated by the target user in the target exhibition hall and interaction data of the target user on a plurality of browsed exhibits in the target exhibition hall; extracting features of the sensing data, and fusing the features of the sensing data to obtain multi-modal fusion features; determining interest values of the target user to all exhibits in the target exhibition hall based on the multi-modal fusion features; according to the interest values of all the exhibits in the target exhibition hall, screening out candidate interest exhibits from the non-browsed exhibits as candidate interest points; and on the basis of the layout information of the target exhibition hall and the candidate interest points, the interest point prediction result of the target user in the target exhibition hall is generated, the exhibition hall interest point prediction result can be generated through multi-modal data dynamic weighted fusion in combination with the exhibition layout, personalized navigation is realized, and then exhibition hall operation is optimized.

Description

Technical Field

[0001] This application relates to the field of intelligent exhibition hall navigation technology, and in particular to an exhibition hall interest point prediction method and prediction system based on multimodal data. Background Technology

[0002] As important venues for cultural dissemination and product display, exhibition halls directly impact visitor experience and exhibition effectiveness through the quality of their guided tour services. With the development of intelligent technologies, traditional passive guided tours can no longer meet users' personalized information needs. There is an urgent need to collect user behavior data through intelligent devices, accurately predict points of interest, and achieve personalized guided tour recommendations. Simultaneously, this provides data support for optimizing exhibition hall operations, improving visitor efficiency and the scientific basis of operational decisions.

[0003] Existing exhibition hall guided tours rely on human guides and static displays, failing to provide personalized services. Furthermore, electronic guide devices often only support basic navigation and exhibit descriptions, with limited interaction methods and fixed functions. Current methods for predicting points of interest (POIs) in exhibition halls collect single-modal user data, relying on superficial data such as dwell time. This makes them susceptible to environmental interference and prone to misjudgments. Their limited analytical dimensions result in insufficient accuracy in POI predictions, hindering the provision of precise personalized guided tours and failing to provide a scientific basis for optimizing exhibition hall operations. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0005] The main objective of this disclosure is to propose a method and system for predicting exhibition hall points of interest based on multimodal data. This method can generate predicted results of exhibition hall points of interest by dynamically weighting and fusing multimodal data and combining them with the exhibition layout, thereby achieving personalized tour guidance and optimizing exhibition hall operations.

[0006] A first aspect of this application provides a method for predicting points of interest in an exhibition hall based on multimodal data, the method comprising: Acquire sensor data collected by sensors; the sensor data includes voice data, gesture data, trajectory data generated by the target user in the target exhibition hall, and interaction data of the target user with multiple viewed exhibits in the target exhibition hall; Extract the features from the sensor data and fuse the features of the sensor data to obtain multimodal fusion features; Based on the multimodal fusion features, the interest value of the target user for all exhibits in the target exhibition hall is determined; all exhibits in the target exhibition hall include both viewed and unviewed exhibits. Based on the interest values ​​of all exhibits in the target exhibition hall, candidate exhibits of interest are selected from the unviewed exhibits and used as candidate points of interest. Based on the layout information of the target exhibition hall and the candidate points of interest, a prediction result of the target user's points of interest in the target exhibition hall is generated.

[0007] In some embodiments of this application, the step of extracting features from the sensing data and fusing the features of the sensing data to obtain multimodal fusion features includes: The voice data, gesture data, trajectory data, and interaction data are preprocessed respectively to obtain processed voice data, processed gesture data, processed trajectory data, and processed interaction data. The processed speech data is encoded using a pre-trained language model to obtain a text feature vector; Long Short-Term Memory (LSTM) networks are used to mine temporal features from the processed trajectory data to obtain trajectory feature vectors. A preset convolutional neural network is used to extract spatial features from the processed gesture data and the processed interaction data respectively, so as to obtain the gesture feature vector corresponding to the processed gesture data and the operation feature vector corresponding to the processed interaction data. The multimodal fusion features are calculated based on the text feature vector, the trajectory feature vector, the gesture feature vector, and the operation feature vector.

[0008] In some embodiments of this application, calculating the multimodal fusion features based on the text feature vector, the trajectory feature vector, the gesture feature vector, and the operation feature vector includes: Construct attention query vectors; The correlation between the attention query vector and the text feature vector, the trajectory feature vector, the gesture feature vector, and the operation feature vector are calculated respectively to obtain the weights of the text feature vector, the trajectory feature vector, the gesture feature vector, and the operation feature vector. The text feature vector and its weight, the trajectory feature vector and its weight, the gesture feature vector and its weight, and the operation feature vector and its weight are weighted respectively to obtain the multimodal fusion feature.

[0009] In some embodiments of this application, determining the target user's interest value for all exhibits in the target exhibition hall based on the multimodal fusion features includes: The multimodal fusion features are input into the interest value scoring model to obtain the interest value of the target user for all exhibits in the target exhibition hall, which is output by the interest value scoring model; the interest value scoring model includes a gradient boosting decision tree model.

[0010] In some embodiments of this application, after inputting the multimodal fusion features into the interest value scoring model to obtain the interest values ​​of the target user for all exhibits in the target exhibition hall output by the interest value scoring model, the method further includes: When the time interval between the current moment and the time of collection of the sensor data is greater than a preset time interval, second data is acquired. The second data is the voice data, gesture data, trajectory data and interaction data generated by the target user in the target exhibition hall during the time interval. Based on the second data and the target user's interest values ​​for all exhibits in the target exhibition hall, the interest value scoring model is trained to obtain an updated exhibition hall interest value scoring model.

[0011] In some embodiments of this application, the step of selecting candidate exhibits of interest from the unviewed exhibits based on the interest values ​​of all exhibits in the target exhibition hall, as candidate points of interest, includes: Extract the interest values ​​of unviewed exhibits from the interest values ​​of all exhibits in the target exhibition hall, and form a ranking of the interest values ​​of unviewed exhibits; Based on the sorting of interest values ​​of the unviewed exhibits, the unviewed exhibits with interest values ​​greater than a preset interest value threshold are identified as the candidate interest exhibits. The location of the exhibit corresponding to the candidate exhibit of interest is taken as the candidate point of interest.

[0012] In some embodiments of this application, after generating the target user's predicted interest points in the target exhibition hall based on the layout information of the target exhibition hall and the location points of the candidate interest points, the method further includes: Based on the sensor data, the real-time popularity ranking of all exhibits in the target exhibition hall is determined to generate the real-time popular areas and real-time trajectory density of the target exhibition hall; Based on the real-time popularity ranking of all exhibits in the target exhibition hall, exhibits with popularity values ​​lower than a preset popularity threshold in the real-time popularity ranking are identified as low-attention exhibits. Based on the real-time popular areas and real-time trajectory density of the target exhibition hall, an optimization scheme for the low-attention exhibits in the target exhibition hall is generated.

[0013] The first aspect of this application provides a method for predicting exhibition hall points of interest based on multimodal data. This method acquires sensor data, including voice data, gesture data, trajectory data generated by a target user in a target exhibition hall, and interaction data of the target user with multiple viewed exhibits in the target exhibition hall. Features of the sensor data are extracted and fused to obtain multimodal fusion features. Based on the multimodal fusion features, the target user's interest value for all exhibits in the target exhibition hall is determined. Based on the interest values ​​of all exhibits in the target exhibition hall, candidate interest exhibits are selected from unviewed exhibits as candidate points of interest. Based on the layout information of the target exhibition hall and the candidate points of interest, a prediction result of the target user's points of interest in the target exhibition hall is generated. This method enables personalized tour guidance by dynamically weighting and fusing multimodal data, combined with the exhibition layout, thereby optimizing exhibition hall operations.

[0014] To achieve the above objectives, a second aspect of this application provides a showroom point of interest prediction system based on multimodal data, the system comprising: The acquisition module is used to acquire sensor data collected by the sensor; the sensor data includes voice data, gesture data, trajectory data generated by the target user in the target exhibition hall, and interaction data of the target user with multiple viewed exhibits in the target exhibition hall; An extraction module is used to extract features from the sensing data and fuse the features of the sensing data to obtain multimodal fusion features; The determination module is used to determine the interest value of the target user for all exhibits in the target exhibition hall based on the multimodal fusion features; all exhibits in the target exhibition hall include viewed exhibits and unviewed exhibits; The filtering module is used to filter out candidate interest exhibits from the unviewed exhibits based on the interest values ​​of all exhibits in the target exhibition hall, and to use them as candidate interest points. The generation module is used to generate a prediction result of the target user's points of interest in the target exhibition hall based on the layout information of the target exhibition hall and the location points of the candidate points of interest.

[0015] To achieve the above objectives, a third aspect of this application provides an electronic device, including: at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform the above-described method for predicting exhibition hall points of interest based on multimodal data.

[0016] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described method for predicting points of interest in exhibition halls based on multimodal data.

[0017] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating a method for predicting points of interest in an exhibition hall based on multimodal data, provided in an embodiment of this application. Figure 2 This is a schematic diagram of the overall architecture process provided in the embodiments of this application; Figure 3 This is a detailed architecture diagram of the multimodal data fusion model provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of an exhibition hall interest point prediction system based on multimodal data provided in an embodiment of this application; Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0020] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0021] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0022] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0023] The existing exhibition hall tour guide model has obvious technical limitations: traditional manual explanations and static display boards cannot provide personalized services; electronic tour guide devices mostly only support basic navigation and exhibit introductions, with limited interaction methods and fixed functions; although some intelligent tour guide systems can collect user data, they have problems such as limited data modalities and limited analytical dimensions.

[0024] For example, judging user interests solely based on dwell time is susceptible to environmental interference (such as misjudgments due to exhibits being near rest areas), and lacks in-depth data integration and analysis of user questions and exhibit interactions, resulting in insufficient accuracy in predicting points of interest. These shortcomings make it difficult for existing systems to provide accurate personalized tour guide services, and also fail to provide a scientific basis for optimizing exhibition hall operations.

[0025] Based on this, this application provides a method and system for predicting exhibition hall points of interest based on multimodal data. The aim is to generate exhibition hall point of interest prediction results by dynamically weighting and fusing multimodal data and combining it with a scoring network, thereby achieving personalized tours and optimizing exhibition hall operations.

[0026] The exhibition hall interest point prediction method and prediction system based on multimodal data provided in this application are specifically described through the following embodiments. First, the exhibition hall interest point prediction method based on multimodal data in this application embodiment is described.

[0027] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0028] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0029] The exhibition hall interest point prediction method based on multimodal data provided in this application relates to the field of intelligent exhibition hall navigation technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the exhibition hall interest point prediction method based on multimodal data, but is not limited to the above forms.

[0030] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0031] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0032] Therefore, referring to Figure 1This application provides a method for predicting points of interest in an exhibition hall based on multimodal data. This method is applied to a central controller, which can be a server, an electronic device, or a mobile terminal, etc. There are no specific limitations here. The method includes the following steps S110 to S150.

[0033] Step S110: Acquire sensor data collected by the sensor; the sensor data includes voice data, gesture data, trajectory data generated by the target user in the target exhibition hall, and interaction data of the target user with multiple exhibits viewed in the target exhibition hall; Step S120: Extract the features of the sensor data and fuse the features of the sensor data to obtain multimodal fusion features; Step S130: Based on multimodal fusion features, determine the target user's interest value for all exhibits in the target exhibition hall; all exhibits in the target exhibition hall include both viewed and unviewed exhibits; Step S140: Based on the interest values ​​of all exhibits in the target exhibition hall, select candidate interest exhibits from the unviewed exhibits as candidate interest points; Step S150: Based on the layout information and candidate points of interest of the target exhibition hall, generate the prediction results of the target user's points of interest in the target exhibition hall.

[0034] In this step, the sensor data collected by the sensors is acquired first, which lays a targeted data foundation for subsequent prediction of interest in unviewed exhibits. The sensor data includes voice data, gesture data, trajectory data generated by the target user in the target exhibition hall, and interaction data of the target user with multiple viewed exhibits in the target exhibition hall.

[0035] Preferably, the sensor data is collected through a combination of multiple sensors adapted to the exhibition hall scenario, covering four core dimensions: user semantic needs, operational behavior, spatial movement, and deep interaction. Voice data is captured by audio acquisition components with noise reduction capabilities (such as multi-microphone arrays in AR glasses), including user questions, voice commands (such as "introduce this exhibit"), and interaction duration during the visit. This data is then converted into structured text through preliminary speech recognition, facilitating subsequent semantic feature mining. Gesture data is collected through optical sensors (such as TOF depth sensors), capturing user gestures such as clicking, zooming, and swiping on viewed exhibits. This data is then parsed by algorithms and converted into a data structure including gesture type, operation object (corresponding to the specific viewed exhibit), and... Structured data on interaction duration accurately reflects users' operational preferences for viewed exhibits; trajectory data is spatiotemporal behavioral data that integrates the time dimension, generated by indoor high-precision positioning components (such as UWB modules with positioning accuracy ±10cm) combined with timestamps, covering users' real-time location coordinates, travel routes, and dwell time in each viewed exhibit area, intuitively reflecting the user's spatial attention to viewed exhibits; interaction data of viewed exhibits is collected through IoT interfaces, including user operation types such as touch-sensor triggering and infrared-sensor triggering, as well as interaction frequency, content browsing progress (such as whether the complete introduction of viewed exhibits has been viewed), directly depicting the depth of user interaction with viewed exhibits.

[0036] Furthermore, through feature extraction and multimodal fusion, fragmented raw behavioral data is transformed into high-dimensional fused features that accurately represent user interests. Specifically, targeted optimizations are first carried out based on the characteristics of four types of raw sensor data: for voice data, environmental noise is filtered out, invalid segments are removed, and text format is standardized; for gesture data, false trigger trajectories are removed, and gesture type encoding is standardized; for trajectory data, positioning deviations are corrected through Kalman filtering, and timestamps are standardized to ensure temporal continuity; for interactive data of viewed exhibits, duplicate reports and false trigger data are removed, operation type encoding is unified, and browsing progress is quantified.

[0037] Furthermore, targeted feature extraction is performed on the four types of processed sensor data, and an appropriate model is selected based on the characteristics of different modalities. Specifically, a pre-trained language model (such as BERT, fine-tuned with exhibition hall scene corpus) is used to encode the structured text-based speech data to obtain text feature vectors that reflect users' semantic needs; a Long Short-Term Memory (LSTM) network is used to mine temporal features of trajectory data, capturing the correlation between user movement patterns and dwell time on viewed exhibits to obtain trajectory feature vectors; a convolutional neural network (CNN) with 3 convolutional layers and 2 pooling layers is used to extract spatial features from the gesture data and interaction data corresponding to viewed exhibits, respectively, to obtain gesture feature vectors and interaction feature vectors that reflect operation preferences and interaction depth.

[0038] Furthermore, a cross-modal attention mechanism is employed for multimodal fusion to achieve dynamic weighted fusion. Specifically, firstly, an attention query vector adapted to the exhibition hall scenario is constructed (trained with historical user data to accurately measure the correlation strength between each modality feature and user interests). Then, the correlation degree between this vector and the four types of feature vectors is calculated and normalized using a normalized exponential function (softmax) to obtain the weights corresponding to each modality (weights range from 0 to 1, summing to 1; modalities with higher correlation to user interests have larger weights). Subsequently, each feature vector is weighted with its corresponding weight and then horizontally concatenated to form a multimodal fusion feature that integrates multi-dimensional effective information, preserving the core value of each modality's data.

[0039] Furthermore, based on the interest value scoring model, the interest representation capabilities of multimodal fusion features are transferred to all exhibits, achieving a comprehensive quantitative assessment of interest in both viewed and unviewed exhibits. The interest value scoring model is a Gradient Boosting Decision Tree (GBDT) model specifically optimized for the exhibition hall scenario. The model architecture and parameters are adapted to the exhibition hall interest prediction requirements: it comprises an ensemble learning framework of 100-200 decision trees, with each decision tree limited to a depth of 8-12 layers. This ensures both the full exploitation of the non-linear correlation between fusion features and user interests and the suppression of overfitting through L1 regularization. A small learning rate of 0.01-0.05, combined with a gradient descent strategy, is used to optimize parameters, ensuring the model's generalization ability.

[0040] In the training phase of the interest value scoring model, the multimodal fusion features of historical users are used as input, and the actual interaction behavior of users with the exhibits they have viewed (such as the duration of deep browsing and the number of repeated interactions) is used as the real interest label. Training and validation sets are constructed, and the prediction bias is calculated through the mean squared error loss function and the decision tree splitting rules and weight allocation of the model are optimized in reverse, so that the model can accurately grasp the mapping logic of "user behavior features - interest in the exhibits viewed".

[0041] Specifically, after inputting the multimodal fusion features into the interest value scoring model, the model first verifies and outputs the interest values ​​of the viewed exhibits based on the mapping logic learned during training. Then, it matches the user's interest features on the viewed exhibits with the attribute features of the unviewed exhibits (such as theme, type, related exhibits, etc., which are pre-stored in the exhibition hall database). Through an ensemble learning mechanism, it mines the interest association between the two (for example, if a user has a high interest in the viewed "ancient porcelain" exhibit, the interest value scoring model will identify a strong association between the unviewed "ancient pottery" exhibit and this interest). Finally, it outputs the target user's interest value for all exhibits in the exhibition hall (viewed + unviewed). This interest value is presented in the form of probability (range 0-1). The higher the value, the stronger the user's potential interest in the corresponding exhibit. This not only quantifies the existing interests but also achieves accurate prediction of the interests in unviewed exhibits, providing a quantitative basis for the full range of exhibits for subsequent selection of candidate interest points.

[0042] Furthermore, by filtering candidate exhibits that users are potentially highly interested in based on interest values, the system avoids recommending content that has already been viewed, thus enhancing the novelty and practicality of the guided tour. The filtering process follows a dual logic of "range limitation + threshold filtering." First, from all exhibit interest values, the interest value data of viewed exhibits is extracted, retaining only a subset of interest values ​​for unviewed exhibits. Then, based on the exhibition hall size, the user's estimated visit duration, and historical recommendation data, a reasonable preset threshold for interest values ​​is set (typically 0.6-0.7; this can be appropriately increased for smaller exhibition halls to reduce the number of candidates, and appropriately decreased for larger exhibition halls to ensure recommendation diversity) for threshold filtering.

[0043] Specifically, the interest values ​​of unviewed exhibits are first sorted in descending order. Then, the sorting results are traversed, and unviewed exhibits with interest values ​​higher than a preset threshold are identified as candidate interest exhibits, i.e., candidate points of interest. Unviewed exhibits with interest values ​​lower than or equal to the threshold are directly eliminated. This achieves both accurate identification of users' core interest in unviewed exhibits and ensures a reasonable number of candidate points of interest, providing a clear and efficient filtering range for generating practical guide results by combining spatial information.

[0044] Furthermore, by combining the exhibition hall's spatial layout information, candidate points of interest (unviewed high-interest exhibits) are transformed into personalized prediction results that align with users' actual visit needs, balancing interest matching and practicality. Specifically, the exhibition hall point of interest prediction results can include two core categories: a recommended list of unviewed candidate points of interest and the optimal visit route. The personalized recommendation list is based on the selected unviewed candidate points of interest, sorted in descending order of interest value, and supplemented with core information for each candidate exhibit (such as theme category, estimated visit duration, and connections with already viewed high-interest exhibits), allowing users to quickly understand the core value of unviewed high-interest exhibits; it also marks the approximate location of candidate exhibits, providing a basis for users to understand the path; the optimal visit route is based on the layout information of the target exhibition hall (such as passageway direction, obstacle distribution, and entrance / exit locations), combined with the specific coordinates of the unviewed candidate points of interest and the user's current real-time location, and calculated using an improved Dijkstra algorithm.

[0045] Specifically, the planning principle of the optimal visitor route is to prioritize interests and minimize detours. Specifically, it prioritizes connecting the exhibits with the highest interest value among the unvisited candidate points of interest, while avoiding areas where already viewed exhibits are located (reducing repeated travel), taking into account the efficiency of passageways, avoiding crossing areas with congested crowds, and minimizing the travel time from the user's current location to each unvisited candidate point of interest, ensuring a smooth and continuous route.

[0046] Furthermore, the predicted points of interest (POIs) in the exhibition hall can be visualized using portable navigation devices (such as AR glasses). Unvisited candidate POIs are highlighted in AR markers at their corresponding locations, and the optimal viewing path is dynamically overlaid on the actual exhibition hall layout. This allows users to intuitively identify the location and direction of unvisited, highly interesting exhibits, improving the user experience and information acquisition efficiency.

[0047] In one embodiment, such as Figure 2 As shown, a cloud-edge collaborative exhibition hall point of interest prediction system is constructed using a multimodal data-based method. This system comprises a data acquisition layer, an edge processing layer, a cloud processing layer, and an application output layer. The data acquisition layer sequentially displays the data streams from the voice interaction acquisition module, gesture recognition acquisition module, spatiotemporal positioning module, and exhibit interaction acquisition module. The edge processing layer presents the data cleaning, compression, and prioritization processes. The cloud processing layer includes a logical relationship diagram of the AI ​​question-answering engine, multimodal fusion algorithm, and point of interest prediction model. The application output layer displays the bidirectional output path between user-side navigation recommendations and operational-side data analysis.

[0048] In this embodiment, the cloud-edge collaborative exhibition hall point of interest prediction system consists of a portable tour guide device (AR glasses) and a cloud server. The edge layer includes an AR glasses integrated with a voice interaction module, a gesture recognition module, a positioning module, and an edge computing unit, responsible for real-time collection of user behavior data and preliminary preprocessing for data cleaning, compression, and prioritization. The cloud layer includes an AI question-answering engine, a multimodal fusion algorithm, a point of interest prediction model, and a knowledge base, undertaking in-depth data analysis and decision output functions. The edge layer and the cloud layer achieve low-latency data interaction through a wireless communication protocol, forming a collaborative mechanism of "lightweight edge processing - in-depth cloud analysis".

[0049] Specifically, the cloud-edge collaborative exhibition hall point of interest prediction system collects data through a data acquisition layer. This layer includes a voice interaction acquisition module for voice interaction, a gesture recognition acquisition module for gesture interaction, a spatiotemporal positioning module for spatiotemporal behavior data, and an exhibit interaction acquisition module for exhibit interaction data. Preferably, user voice commands are collected via a high-sensitivity microphone built into the AR glasses. After noise reduction, the voice recognition engine converts the commands into structured text data. User gesture trajectories are captured using optical sensors, and gesture recognition algorithms analyze the intentions (such as clicking to select, zooming, or swiping to switch) to generate behavior records containing gesture type, target object, and interaction duration. Real-time user location coordinates are obtained using an indoor high-precision positioning system (UWB / Bluetooth beacon), and combined with timestamps, route trajectory data is generated, simultaneously recording parameters such as dwell time and movement speed in each exhibit area. Finally, user interaction records with smart exhibits are collected via an IoT interface, including quantitative indicators such as operation type (e.g., touch sensing, infrared triggering), interaction frequency, and content browsing progress.

[0050] Furthermore, the glasses perform a cleaning process on the collected raw data, removing noisy data (such as accidentally triggered gestures or invalid voice segments) using outlier detection algorithms, and simplifying redundant information using data compression techniques (such as lossy compression) to reduce transmission bandwidth consumption. The pre-processed data is then transmitted to the cloud server via Wi-Fi 6 or 5G edge networks, employing a breakpoint resumption mechanism to ensure data integrity, and prioritizing the transmission of critical data (such as user questions) to ensure timely response. Additionally, the cloud uses a distributed database (such as MongoDB) to store multimodal data, indexed by user ID, timestamp, and data type, supporting high-concurrency read / write operations and historical data backtracking queries.

[0051] Furthermore, such as Figure 3As shown, feature extraction and fusion are performed using the multimodal data fusion algorithm module in this embodiment. The multimodal data fusion algorithm module architecture mainly includes a feature extraction layer, a cross-modal attention module, a feature concatenation layer, and an interest point prediction layer. Specifically, the four types of processed modal data (text vectors, spatiotemporal features, gesture features, and exhibit operation features) are first input into their respective encoding networks (BERT encoding, LSTM network, and CNN network) to map the original data of different dimensions into high-dimensional feature vectors.

[0052] The multimodal data fusion algorithm module processes text vectors (question content) through a BERT encoder to obtain 768-dimensional features; processes spatiotemporal features (trajectory + dwell time) through an LSTM network to obtain 256-dimensional features; processes gesture features through a CNN network to obtain 256-dimensional features; and processes exhibit operation features through a CNN network to obtain 256-dimensional features. Then, a cross-modal attention module calculates and dynamically weights the corresponding features of each modality, and the weighted feature vectors are concatenated at a feature concatenation layer to obtain a 1536-dimensional fused vector. Finally, this fused vector is input into the GBDT model to output the final interest point prediction result.

[0053] Specifically, a semantic understanding module is built based on a BERT pre-trained model to perform word segmentation, entity recognition, and intent classification on user-submitted questions. The classification knowledge base uses a knowledge graph structure to store exhibit information (including attributes, relationships, and multimedia resources), and achieves accurate matching through vector retrieval algorithms. Then, a structured answer is output through an answer generation module combined with Natural Language Generation (NLG) technology, and displayed via an AR glasses visualization interface (supporting multiple formats such as text, voice, and 3D models).

[0054] Specifically, regarding the extraction of text features, let the voice interaction text data be... ( (Words after segmentation), processed by the BERT encoder as follows: ,in The pre-trained language model mapping function outputs a 768-dimensional text feature vector; for the extraction of spatiotemporal features, the user's spatiotemporal trajectory data is assumed to be... ( (Location coordinates and timestamp), processed by an LSTM network as follows: The LSTM hidden layer has a dimension of 256. Let be the mapping function for the Long Short-Term Memory network; for gesture feature extraction, let the gesture trajectory data be... ( (These are the coordinates of the sampling points), processed by a CNN network as follows: ,in A convolutional neural network with 3 convolutional layers and 2 pooling layers outputs a 256-dimensional gesture feature vector; for extracting the operation features of exhibits, let the exhibit interaction data be... ( (Encoded as operation type), processed by a CNN network as follows: ,in It is a convolutional neural network with the same structure as the gesture processing, which outputs a 256-dimensional operational feature vector.

[0055] Furthermore, define the attention query vector. ,in, The dimension of the query vector (in this embodiment) ),at the same time It also serves as a scaling factor. Since the feature vectors of different modalities have inconsistent dimensions, they must first be projected onto a common dimensional space. The calculation process for the weights of each modal feature is as follows: First, calculate the association score between each modal feature and the query vector: ; ; ; ; in, Scoring is given for the association between text features and attention query vectors. Scoring the association between spatiotemporal features and attention query vectors. Scoring is given for the association between gesture features and attention query vectors. Scoring is applied to the correlation between exhibit operational features and attention query vectors. For attention query vectors, For the dimension of the attention query vector, For text features, Spatiotemporal characteristics, Gesture characteristics, For the operational characteristics of the exhibits, These are the dimensional projection matrices for text features, spatiotemporal features, gesture features, and exhibit operation features, respectively, used to map each original feature vector to a dimensional projection matrix. .

[0056] Furthermore, the four association scores are normalized using the Softmax function to obtain the final feature weights, calculated as follows: ; ; ; ; in, The feature weights for text features. The feature weights are the spatiotemporal features. The feature weights are the gesture features. The feature weights for the operational features of the exhibits. Let these be the attention weights corresponding to each modality, and satisfy the following conditions: , Scoring the association between text features and query vectors. Scoring the association between spatiotemporal features and query vectors. Scoring is applied to the association between gesture features and query vectors. The association score between exhibit operation features and query vectors is calculated. This calculation method ensures that features of different dimensions (such as 768-dimensional text features and 256-dimensional gesture features) can be evaluated for importance in a unified metric space, avoiding dimensional conflicts caused by direct calculation, and achieving reasonable normalization of weight allocation.

[0057] Furthermore, by fusing the features from each modality and concatenating the weighted feature vectors from each modality, the expression is as follows: ; in, The vectors are concatenated horizontally to obtain a 1536-dimensional fused feature vector. The feature weights for text features. The feature weights are the spatiotemporal features. The feature weights are the gesture features. The feature weights for the operational features of the exhibits. For text features, Spatiotemporal characteristics, Gesture characteristics, The characteristics of the exhibits' operation.

[0058] Specifically, the raw data from each modality are mapped into high-dimensional feature space vectors through corresponding feature extraction functions. After weights are dynamically allocated through an attention mechanism, cross-modal information fusion is achieved through vector concatenation. As input to the GBDT model: ( (Predict the probability vector for the point of interest).

[0059] Furthermore, based on the fused 1536-dimensional feature vector, an interest scoring model is constructed using Gradient Boosting Tree (GBDT), with the GBDT tree depth limited to 10, to generate interest probability values ​​for each exhibit in the exhibition hall. Then, combined with the user's current location and the exhibition hall layout, a personalized recommended route is generated through a path planning algorithm and pushed to the AR glasses navigation interface.

[0060] Furthermore, the model parameters are iterated every 5 minutes based on newly collected behavioral data, dynamically optimizing the list of points of interest recommendations. This dynamic feedback loop transforms the system from a static "predictor" into a dynamic "adaptive learning system." For example, a user's interest might shift immediately after listening to an explanation of an exhibit; the 5-minute iteration cycle can handle such scenarios.

[0061] Furthermore, the cloud-edge collaborative exhibition hall interest prediction system also includes an application output layer, which comprises user-end navigation recommendations and operational-end data analysis. Specifically, for exhibition hall operation optimization, a user interest analysis dashboard can be built to visually display indicators such as exhibit popularity rankings, interest area distribution, and pedestrian flow density; then, intelligent decision-making suggestions can be provided, such as generating layout adjustment schemes for low-attention exhibits and recommending exhibit combination display strategies based on interest relevance. In addition, A / B testing functionality is supported to compare user behavior data of different display schemes and quantitatively evaluate the optimization effect.

[0062] Furthermore, for the cloud-edge collaborative exhibition hall interest point prediction system, a user interest point annotation database can be established, a semi-supervised learning method can be used to expand the training samples, and the prediction model parameters can be fine-tuned periodically (e.g., weekly) using the newly added data. The model accuracy can be evaluated through cross-validation, and a full retraining can be triggered when the accuracy falls below a threshold (e.g., 85%). In addition, knowledge gaps can be identified based on frequently asked user questions, and pushed to the administrator for content supplementation, thereby achieving automatic updates to the knowledge base.

[0063] In one implementation, taking AR glasses and a cloud system as an example, the AR glasses (edge ​​device) are preferably equipped with a Qualcomm Snapdragon XR2 Gen2 chip processor, supporting edge AI computing; a 6-microphone array (noise reduction), a TOF depth sensor (gesture recognition), and a UWB positioning module (positioning accuracy ±10cm); a MicroOLED screen with a resolution of 2560×1440 and a refresh rate of 90Hz; a 5000mAh battery, supporting 6 hours of continuous operation, and equipped with a magnetic fast charging interface; the software deployment is preferably based on a customized XR-specific system based on Android 13, supporting multi-task parallel processing; it integrates iFlytek's speech recognition SDK (recognition accuracy of 98%) and a self-developed gesture recognition algorithm (supporting 12 common gestures); a lightweight processing script is written in Python, using regular expressions to filter invalid speech (such as silence and noise), and Kalman filtering to optimize the positioning trajectory; the MQTT protocol is used to achieve asynchronous communication with the cloud, and QoS=1 is set to ensure reliable message transmission.

[0064] Furthermore, the cloud system server is configured with an 8-node GPU server cluster (each node contains 2×NVIDIA A100), 100TB SSD storage capacity, and a Kubernetes containerized platform. AI models are trained using the TensorFlow framework, and a full-text search engine is built using Elasticsearch. The training parameters for the multimodal fusion model include BERT-based text encoder, 256-dimensional LSTM hidden layers, 3×3 kernel size in the CNN network, 8 attention heads, 32 training batches, and 100 iterations. The training parameters for the interest point prediction model include a GBDT tree depth limit of 10, a learning rate of 0.01, a regularization coefficient of 0.001, and 5-fold cross-validation to optimize hyperparameters.

[0065] Furthermore, in this implementation, the user activates the device by wearing AR glasses. The system automatically calibrates the positioning module and synchronizes the exhibition hall map data, while also collecting multimodal behavioral data in real time. After edge layer preprocessing, the data is pushed to the cloud, and the AI ​​question-answering agent has a response latency of ≤1.5 seconds. A list of points of interest is generated every 30 seconds, and the AR glasses highlight recommended exhibits using AR annotations. The optimal visitor route is planned simultaneously. Additionally, data cleaning and model iteration are automatically performed every morning at midnight, generating an operational analysis report which is then pushed to the administrator's terminal.

[0066] On the other hand, this implementation method supports interface with existing IoT devices in the exhibition hall (such as RFID exhibit tags and infrared sensors), and can be extended to multiple terminals such as mobile APP through API interface, and is compatible with Android 10.0 and above, iOS 14.0 and above operating systems.

[0067] In some embodiments, features of the sensing data are extracted in step S120 and fused to obtain multimodal fusion features, including the following steps S210 to S250: Step S210: Preprocess the voice data, gesture data, trajectory data and interaction data respectively to obtain processed voice data, processed gesture data, processed trajectory data and processed interaction data. Step S220: Encode the processed speech data using a pre-trained language model to obtain text feature vectors; Step S230: Use a long short-term memory network to perform temporal feature mining on the processed trajectory data to obtain trajectory feature vectors; Step S240: Using a preset convolutional neural network, spatial features are extracted from the processed gesture data and the processed interaction data respectively to obtain the gesture feature vector corresponding to the processed gesture data and the operation feature vector corresponding to the processed interaction data. Step S250: Calculate multimodal fusion features based on text feature vectors, trajectory feature vectors, gesture feature vectors, and operation feature vectors.

[0068] In this embodiment, targeted preprocessing is carried out based on the characteristics of the four types of multimodal raw data. Through optimization in two dimensions of denoising and simplification, the data quality is improved, laying the foundation for subsequent accurate feature extraction. Specifically, for voice data, noise reduction algorithms are used to filter out environmental noise and device interference, and regular expressions are used to filter out silent segments, meaningless syllables, and other invalid data. At the same time, the text data converted from speech recognition is standardized into structured text in a unified format to ensure the integrity of semantic information. For gesture data, outlier detection algorithms are used to identify and remove erroneously triggered gesture trajectories (such as unintentional waving or device mis-sensing), retaining only valid trajectory records with clear operational intentions (such as clicking or zooming), and simultaneously standardizing gesture type encoding. For trajectory data, Kalman filtering algorithms are used to correct positional deviations generated during indoor positioning, and abnormal coordinate points caused by signal interruption are removed. At the same time, timestamp formats are standardized to ensure the continuity and consistency of time-series information such as user routes and dwell times. For interaction data, valid interaction records are filtered, duplicate reports and erroneously triggered interaction data are removed, and encoding standards for operation types such as touch sensing and infrared triggering are unified, and key indicators such as content browsing progress are quantified.

[0069] Furthermore, pre-trained language models transform structured text into text feature vectors that accurately represent user needs. Specifically, pre-trained language models (such as BERT) perform operations such as word segmentation, word embedding, and contextual semantic fusion to map discrete text information into fixed-dimensional high-dimensional text feature vectors. This not only captures the semantic information of individual words but also deeply mines the contextual relationships within the text (such as the semantic relationship between a user's question "the historical background of an exhibit" and "the creation date of the exhibit"), accurately reflecting the user's semantic needs (such as focusing on different dimensions like history, technology, and art), providing more valuable textual dimensional information for subsequent multimodal fusion.

[0070] Furthermore, leveraging the temporal processing advantages of Long Short-Term Memory (LSTM) networks, deep correlations between user spatial movement and time allocation are mined from trajectory data to generate trajectory feature vectors. The processed trajectory data includes temporal information such as the user's real-time location coordinates, travel route, and dwell time in each exhibit area. The LSTM network, through its gating mechanism (input gate, forget gate, output gate), can effectively capture the long-short-term dependencies in the temporal data, thereby outputting a fixed-dimensional trajectory feature vector that accurately represents key information such as the user's spatial movement preferences and core areas of interest, providing crucial spatiotemporal support for subsequent interest prediction.

[0071] For example, it can identify the user's travel route pattern of "first visiting exhibit A and then going to exhibit B", capture the time allocation characteristics of "staying in exhibit C for 10 minutes without in-depth interaction", and correlate the changes in movement speed in different time periods (such as slowing down when approaching exhibits of high interest).

[0072] Furthermore, a pre-structured Convolutional Neural Network (CNN) is used to extract spatial correlation features from gesture data and interaction data respectively, generating gesture feature vectors and operation feature vectors with consistent dimensions, providing adapted spatial dimension information for subsequent multimodal fusion. The pre-structured CNN uses a classic structure of 3 convolutional layers and 2 pooling layers, and is trained and optimized using exhibition hall gesture and interaction data to adapt to the spatial feature characteristics of both types of data.

[0073] Specifically, for the processed gesture data, CNN captures spatial features such as the spatial shape of the gesture trajectory (e.g., the accuracy of the click, the magnitude of the zoom) and the rhythm of the movement through convolutional operations. Then, it reduces the dimensionality and simplifies the data through pooling layers, and finally outputs a gesture feature vector that reflects the user's operation preferences (e.g., preference for zooming to view details, clicking to quickly switch). For the processed interaction data, CNN focuses on the spatial correlation features of the user's interaction with the exhibit, such as the distribution pattern of the operation position and the combination pattern of different operation types (e.g., touch sensing first → infrared triggering). It extracts features and reduces the dimensionality through the same convolution-pooling process to generate operation feature vectors.

[0074] Furthermore, a cross-modal attention mechanism is used to dynamically weight and fuse the four types of feature vectors, integrating multi-dimensional effective information to generate multi-modal fusion features that comprehensively represent user interests. Specifically, the fusion process includes dynamic weight allocation and feature vector integration.

[0075] The dynamic weight allocation process includes constructing an attention query vector adapted to the exhibition hall scenario (this vector is trained with a large amount of exhibition hall user data and can accurately measure the correlation strength between each modality feature and user interests), calculating the correlation between the text feature vector, trajectory feature vector, gesture feature vector, and operation feature vector and the query vector respectively, and then normalizing it through a normalized exponential function to convert the correlation into the weights corresponding to each modality (the weight values ​​range from 0 to 1, and the sum of the four weights is 1).

[0076] Feature vector integration involves weighting each modality feature vector with its corresponding weight to strengthen the representation of highly correlated features and weaken secondary information. Then, the four weighted feature vectors are horizontally concatenated to form a multimodal fusion feature with higher dimensions and more comprehensive information. This retains the core value of each modality data (textual semantics, temporal trajectory, gesture operation, exhibit interaction) and solves the problems of fixed weights and insufficient information utilization in traditional feature fusion through dynamic weighting. It achieves the complementarity and enhancement of multi-dimensional information and provides a high-information-density core input for subsequent interest value calculation.

[0077] In some embodiments, in step S250, multimodal fusion features are calculated based on text feature vectors, trajectory feature vectors, gesture feature vectors, and operation feature vectors, including the following steps S310 to S330: Step S310: Construct the attention query vector; Step S320: Calculate the correlation between the attention query vector and the text feature vector, trajectory feature vector, gesture feature vector and operation feature vector respectively, and obtain the weights of the text feature vector, trajectory feature vector, gesture feature vector and operation feature vector. Step S330: The text feature vector, trajectory feature vector, gesture feature vector and operation feature vector are weighted and fused with the weights of the text feature vector, trajectory feature vector, gesture feature vector and operation feature vector respectively to obtain multimodal fusion features.

[0078] In this embodiment, an attention query vector adapted to the exhibition hall scenario is constructed to provide a core reference benchmark for accurately measuring the correlation strength between various modal features and user interests. Specifically, the attention query vector is an interest-oriented vector trained based on the specific needs of the exhibition hall scenario, using a large amount of historical multimodal data of exhibition hall users and corresponding interest tags.

[0079] During the training of the attention query vector, actual user interaction behaviors on exhibits (such as deep browsing and repeated interactions) are used as real interest tags to optimize the dimensionality and numerical distribution of the query vector, ensuring that it can accurately capture the correlation logic between "user behavior characteristics and interest tendencies". At the same time, the dimensions of the query vector are strictly set to be consistent with those of the text feature vector, trajectory feature vector, gesture feature vector, and operation feature vector (e.g., all 768-dimensional or 256-dimensional) to avoid subsequent correlation calculation failures due to dimensional mismatch.

[0080] Furthermore, by calculating the correlation between the query vector and each modal feature vector, the dynamic allocation of modal feature weights is achieved. Specifically, the correlation between the attention query vector and each modal feature vector is calculated using vector dot product operation. The larger the dot product result, the higher the similarity between the modal feature vector and the query vector (i.e., the user interest representation), and the greater the value of the corresponding modal data in reflecting user interests. Then, based on the correlation of each modality, the correlation is converted into weights with values ​​between 0 and 1 through normalization exponential function normalization processing, and it is ensured that the sum of the weights of text feature vectors, trajectory feature vectors, gesture feature vectors, and operation feature vectors is 1, thus achieving reasonable weight allocation.

[0081] Furthermore, through weighted operations and vector integration, the core effective information of each modality feature is fused into a unified multimodal fusion feature. Specifically, firstly, the text feature vector, trajectory feature vector, gesture feature vector, and operation feature vector are multiplied element-wise with their corresponding weights to amplify the effective information related to user interests in high-weight modal features (such as semantic demand information in high-weight text features) while suppressing secondary information in low-weight modal features (such as unintentional operation traces in low-weight gesture features). This ensures that the fused modal features better reflect the user's true interests. Then, the weighted four types of feature vectors are horizontally concatenated (i.e., stacked dimensionally; for example, if each type of feature vector is 256-dimensional, the concatenation results in a 1024-dimensional fusion feature vector), fully preserving the effective information of each modality after weight optimization.

[0082] Furthermore, by weighted fusion, multimodal fusion features are obtained, realizing the core value of integrating four types of data: voice, trajectory, gesture, and operation. This can comprehensively and accurately represent user interest tendencies, providing high-quality core input for the subsequent interest value scoring model in the exhibition hall for quantitative evaluation.

[0083] In some embodiments, determining the target user's interest value for all exhibits in the target exhibition hall based on multimodal fusion features in step S140 includes the following step S410: Step S410: Input the multimodal fusion features into the interest value scoring model to obtain the interest value of the target user on all exhibits in the target exhibition hall, as output by the interest value scoring model; the interest value scoring model includes a gradient boosting decision tree model.

[0084] In this embodiment, an interest value scoring model (gradient boosting decision tree model) adapted to the exhibition hall scenario is used to transform multi-dimensional, high-information-density multimodal fusion features into quantified interest values, accurately assessing the target user's level of interest in all exhibits within the target exhibition hall. The input to the interest value scoring model is a multimodal fusion feature vector that has undergone preprocessing, feature extraction, and cross-modal attention weighted fusion. The output is the target user's interest value for each exhibit in the exhibition hall, presented in probabilistic form (range 0-1). Higher values ​​indicate a stronger potential interest and interaction intention from the user towards the corresponding exhibit.

[0085] Specifically, the Gradient Boosting Decision Tree (GBDT) model has been optimized for exhibition hall scenarios. Its model is configured with 100-200 decision trees to form an ensemble learning framework, with the depth of a single decision tree limited to 8-12 layers, thereby fully exploring the nonlinear and complex correlation between multimodal fusion features and user interests. Its parameter optimization adopts a small learning rate of 0.01-0.05 combined with a gradient descent strategy to iteratively optimize the model parameters. At the same time, an L1 regularization mechanism is introduced to suppress overfitting by constraining the absolute value of parameters and improve the model's generalization ability.

[0086] Furthermore, the model training phase has been adapted based on a large amount of historical exhibition hall data: using the multimodal fusion features of historical users as input, and the actual interaction behavior of users with exhibits (such as deep browsing time, number of repeated interactions, whether they actively query details, etc.) as real interest labels, training sets and validation sets are constructed. The deviation between the predicted value and the real label is calculated through the mean squared error loss function. Backpropagation optimizes the decision tree splitting rules and weight allocation of the model to ensure that the model can accurately match the mapping logic of "user behavior characteristics - interest tendencies" in the exhibition hall scenario.

[0087] Furthermore, during the prediction process, after the multimodal fusion feature vector is input into the interest value scoring model, the interest value is calculated through an ensemble learning mechanism to obtain the target user's interest value for each exhibit. Specifically, each decision tree judges the input features based on its own feature splitting rules and outputs its corresponding local interest prediction value; then, the interest value scoring model performs a weighted ensemble of the local prediction values ​​of all decision trees (the weights are determined by the prediction accuracy of each decision tree during the training phase, with higher accuracy resulting in higher weights), and then normalizes the results using a normalized exponential function to finally output the target user's interest value for each exhibit.

[0088] In some embodiments, after inputting the multimodal fusion features into the interest value scoring model in step S410 to obtain the interest values ​​of the target user for all exhibits in the target exhibition hall, the following steps S510 to S520 are further included: Step S510: When the time interval between the current moment and the time of sensor data acquisition is greater than a preset time interval, acquire the second data. The second data is the voice data, gesture data, trajectory data and interaction data generated by the target user in the target exhibition hall within the time interval. Step S520: Based on the second data and the target user's interest values ​​for all exhibits in the target exhibition hall, train the interest value scoring model to obtain the updated exhibition hall interest value scoring model.

[0089] In this embodiment, when the time interval between the current moment and the moment the sensor data was collected is greater than a preset time interval, voice data, gesture data, trajectory data, and interaction data generated by the target user in the target exhibition hall within that time interval are acquired. The preset time interval is designed to fully adapt to the exhibition hall scenario, specifically considering the average pace of visitor experience, the frequency of user behavior changes, and the real-time requirements of the model; it is typically set to 5 minutes (but can be flexibly adjusted based on the size of the exhibition hall and the density of exhibits). The second data is the complete multimodal behavioral data generated by the user in the target exhibition hall within the aforementioned time interval, with the same data types as the sensor data (voice data, gesture data, trajectory data, and interaction data), ensuring the continuity and compatibility of the data dimensions.

[0090] Specifically, the data collection subject can be a combination of multiple sensors adapted to the exhibition hall scene (such as the noise-canceling microphone array of AR glasses, TOF optical sensor, UWB high-precision positioning module and IoT interface of exhibits), and the collected content can accurately cover the new behaviors of users within the interval time period.

[0091] Furthermore, by combining new data and historical prediction feedback, the interest score model is dynamically optimized to continuously adapt to shifts in user interests and improve the accuracy of subsequent interest predictions. Specifically, the second set of data and the target user's interest values ​​for all exhibits in the target exhibition hall are used as incremental training data for the interest score model, and incremental fine-tuning is employed to train the model.

[0092] The correction process uses the updated multimodal fusion features as input features and the actual user interaction behavior corresponding to the second data (such as whether they visit historically highly interested exhibits or have in-depth interactions with new exhibits) as the true label. At the same time, it combines the deviation between historical interest values ​​and actual behavior (i.e., prediction error) to correct the model's decision logic in reverse. Then, while retaining the effective parameters of the original model, it adopts a gradient descent strategy with a small learning rate to fine-tune the splitting rules and weight allocation of the decision tree. At the same time, it introduces an L1 regularization mechanism to suppress overfitting, thereby optimizing the model parameters and ensuring that the updated model not only adapts to the latest user interests but also has good generalization ability.

[0093] In some embodiments, in step S140, candidate exhibits of interest are selected from the unviewed exhibits based on the interest values ​​of all exhibits in the target exhibition hall, and are used as candidate points of interest, including the following steps S610 to S620: Step S610: Extract the interest values ​​of unviewed exhibits from all exhibits in the target exhibition hall, and form a ranking of the interest values ​​of unviewed exhibits. Step S620: Sort the unviewed exhibits according to their interest values, and determine the unviewed exhibits with interest values ​​greater than the preset interest value threshold as candidate interest exhibits; Step S630: Select the locations of the exhibits corresponding to the candidate exhibits of interest as candidate points of interest.

[0094] In this embodiment, the data of unviewed exhibits is precisely extracted and sorted from the interest values ​​of all exhibits, clarifying the interest priority of unviewed exhibits and providing a focused and orderly basis for subsequent screening. Specifically, firstly, the interest values ​​corresponding to unviewed exhibits are extracted from the interest values ​​of all exhibits in the target exhibition hall, forming a dedicated dataset of unviewed exhibit interest values. Then, this dataset is sorted in descending order to obtain the sorting result of the unviewed exhibit interest values. Among them, the unviewed exhibit with the highest interest value (in probability form, ranging from 0 to 1) is placed first, and then decreases in descending order, intuitively presenting the potential interest level of users for unviewed exhibits.

[0095] Furthermore, by setting a preset interest threshold, the range of unviewed exhibits is further refined, filtering out candidate exhibits that users are potentially highly interested in, thus balancing recommendation accuracy with the user's selection burden. Specifically, firstly, the preset interest threshold is optimized based on the characteristics of the exhibition hall scene, user visiting needs, and historical data. Then, it is dynamically adjusted according to the current exhibition hall size (for example, in small exhibition halls with fewer exhibits, the threshold can be set to 0.7 for further simplification; in large exhibition halls, it can be set to 0.6 to ensure recommendation diversity), ensuring that the threshold can both filter out exhibits with genuine high potential interest and not miss potentially valuable unviewed exhibits.

[0096] Specifically, the screening process is based on the ranking of unviewed exhibits by interest value. It iterates through the ranking list and identifies unviewed exhibits with interest values ​​greater than a preset threshold as candidate exhibits of interest. Unviewed exhibits with interest values ​​lower than or equal to the threshold are directly eliminated. This threshold-based screening accurately focuses on the core potential interests of users and avoids user decision fatigue caused by an excessive number of candidate exhibits.

[0097] Furthermore, the selected candidate exhibits of interest are transformed into candidate points of interest in a spatial dimension, which are then adapted to the subsequent path generation based on the exhibition hall layout. The exhibit locations are derived from the basic data of the target exhibition hall (pre-set in the exhibition hall database), specifically the precise spatial coordinates of each exhibit (such as exhibition hall zoning, specific directional coordinates), and are compatible with the format of exhibition hall layout information (passage direction, area division, etc.).

[0098] Specifically, each candidate exhibit of interest can be bound to its corresponding precise spatial location, and these spatial locations can be directly defined as candidate points of interest. This transforms abstract interest values ​​into concrete spatial points. Subsequently, based on the spatial coordinates of these candidate points of interest, combined with the exhibition hall layout, visitor paths can be planned to ensure that the recommendation results can be translated into spatial guidance that users can perceive and follow. This provides key spatial data support for the final generation of practical point of interest prediction results.

[0099] In some embodiments, after generating the target user's point of interest prediction result in the target exhibition hall based on the layout information of the target exhibition hall and the location points of candidate points of interest in step S150, the following steps S710 to S730 are included: Step S710: Based on the sensor data, determine the real-time popularity ranking of all exhibits in the target exhibition hall to generate the real-time popular areas and real-time trajectory density of the target exhibition hall. Step S720: Based on the real-time popularity ranking of all exhibits in the target exhibition hall, identify exhibits with popularity values ​​lower than the preset popularity threshold as low-attention exhibits. Step S730: Based on the real-time popular areas and real-time trajectory density of the target exhibition hall, generate an optimization plan for low-attention exhibits in the target exhibition hall.

[0100] In this embodiment, real-time popularity ranking is determined by multi-dimensional user behavior indicators from comprehensive sensor data. Specifically, taking an individual exhibit as a unit, the frequency of voice questions, the density of gesture interactions (such as zooming and clicking), dwell time, the frequency of exhibit operations (such as touch and infrared triggering), and the depth of browsing progress of the target user and other exhibit users in the same period are statistically analyzed. The real-time popularity value of each exhibit is calculated through a weighted scoring model, and then the exhibits are sorted in descending order of popularity value to obtain the real-time popularity ranking.

[0101] Furthermore, by using algorithms to identify physical areas of the exhibition hall with high user density and concentrated interactive behaviors, and combining the overall popularity value of the exhibits in the area, the scope of popular areas is delineated. At the same time, the core attractive exhibits in the area (exhibits ranked at the top of the popularity list) are marked, thus clarifying the core area of ​​the flow of people in the exhibition hall and generating real-time popular areas.

[0102] Furthermore, based on users' real-time location data and movement trajectories, the throughput of each channel and the dwell density around the exhibits are statistically analyzed per unit time (e.g., 5 minutes). This data is then converted into trajectory density indicators through heatmap quantification logic to obtain real-time trajectory density, thereby clearly presenting the flow of people, potential congestion areas, and sparsely populated areas.

[0103] Furthermore, by combining historical operational data of the exhibition hall with the minimum popularity value required for effective exposure of exhibits, a preset popularity threshold is determined. Then, exhibits with popularity values ​​below the preset popularity threshold in the real-time popularity ranking are identified as low-attention exhibits. Subsequently, based on the real-time pedestrian flow distribution characteristics of the exhibition hall (popular areas, trajectory density), a scientific and feasible layout optimization plan is formulated for low-attention exhibits to improve their exposure rate and user interaction probability, thereby optimizing the overall operational efficiency of the exhibition hall.

[0104] Specifically, based on real-time popular areas, it is recommended to move low-attention exhibits to the vicinity of popular areas (such as at the entrance of popular areas or next to core exhibits), or to display them in a thematically related manner with high-attention exhibits in popular areas (such as moving low-attention "ancient farming tools" exhibits to the side of the popular "agricultural civilization" themed exhibition area), thereby leveraging the foot traffic in popular areas to increase the exposure of low-attention exhibits; if space is limited in popular areas, it is recommended to add lightweight display terminals (such as electronic information screens) for low-attention exhibits in popular areas to indirectly increase exposure.

[0105] Furthermore, for high-traffic, high-density passageways (such as main exhibition hall corridors and entrance / exit connecting corridors), it is recommended to move low-attention exhibits to prominent positions beside these passageways, utilizing natural pedestrian flow to achieve passive exposure. For sparse areas with low traffic density, if low-attention exhibits exist, it is recommended to relocate them and add interactive experience projects (such as VR demonstrations) to attract foot traffic. This, combined with related displays of low-attention exhibits, will achieve increased exposure through projects that drive up exhibit visibility. In addition, the plan will also consider the overall rationality of the exhibition hall layout, such as ensuring smooth and uncongested passageways and logical coherence of exhibit themes after adjustments. It will also provide multiple alternative plans and expected performance evaluation indicators (such as expected increase in exposure and interaction frequency targets) to facilitate exhibition hall operators in choosing implementation based on actual conditions, ultimately achieving the core objectives of enhancing the value of low-attention exhibits and optimizing the utilization efficiency of exhibition hall space.

[0106] like Figure 4 As shown in some embodiments of this application, a system for predicting points of interest in an exhibition hall based on multimodal data is provided. The system includes an acquisition module 410, an extraction module 420, a determination module 430, a filtering module 440, and a generation module 450. Specifically: The acquisition module 410 is used to acquire the sensing data collected by the sensor; the sensing data includes voice data, gesture data, trajectory data generated by the target user in the target exhibition hall, and interaction data of the target user with multiple exhibits that have been viewed in the target exhibition hall; The extraction module 420 is used to extract features from the sensor data and fuse the features of the sensor data to obtain multimodal fusion features; The determination module 430 is used to determine the interest value of the target user for all exhibits in the target exhibition hall based on multimodal fusion features; all exhibits in the target exhibition hall include both viewed exhibits and unviewed exhibits; The filtering module 440 is used to filter out candidate interest exhibits from the unvisited exhibits based on the interest values ​​of all exhibits in the target exhibition hall, and to use them as candidate interest points. The generation module 450 is used to generate prediction results of the target user's points of interest in the target exhibition hall based on the layout information of the target exhibition hall and the location points of candidate points of interest.

[0107] It should be noted that the exhibition hall interest point prediction system based on multimodal data provided in this embodiment is based on the same inventive concept as the exhibition hall interest point prediction method based on multimodal data described above. Therefore, the relevant content of the exhibition hall interest point prediction method based on multimodal data described above is also applicable to the content of the exhibition hall interest point prediction system based on multimodal data. Therefore, it will not be repeated here.

[0108] Therefore, the system acquires sensor data, including voice data, gesture data, trajectory data, and interaction data between the target user and multiple viewed exhibits in the target exhibition hall. Features are extracted from and fused to obtain multimodal fusion features. Based on these features, the system determines the target user's interest in all exhibits in the target exhibition hall. Based on these interest values, candidate exhibits are selected from unviewed exhibits as candidate points of interest. Finally, based on the layout information of the target exhibition hall and the candidate points of interest, a prediction result of the target user's points of interest in the target exhibition hall is generated. This allows for dynamic weighted fusion of multimodal data, combined with the exhibition layout, to generate predicted points of interest in the exhibition hall, enabling personalized navigation and optimizing exhibition hall operations.

[0109] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described exhibition hall interest point prediction method based on multimodal data.

[0110] like Figure 5 , Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes: At least one battery; At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the above-described method for predicting exhibition hall points of interest based on multimodal data, as described in this disclosure.

[0111] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0112] The electronic devices according to embodiments of this application will now be described in detail.

[0113] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to perform a method for predicting exhibition hall points of interest based on multimodal data according to an embodiment of this disclosure.

[0114] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0115] This disclosure also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described method for predicting points of interest in exhibition halls based on multimodal data.

[0116] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0117] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0118] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0119] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0121] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any related variations, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0122] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0123] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0124] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0125] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0126] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0127] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

[0128] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A method for predicting points of interest in exhibition halls based on multimodal data, characterized in that, The method includes: Acquire sensor data collected by sensors; the sensor data includes voice data, gesture data, trajectory data generated by the target user in the target exhibition hall, and interaction data of the target user with multiple viewed exhibits in the target exhibition hall; Extract the features from the sensor data and fuse the features of the sensor data to obtain multimodal fusion features; Based on the multimodal fusion features, the interest value of the target user for all exhibits in the target exhibition hall is determined; all exhibits in the target exhibition hall include both viewed and unviewed exhibits. Based on the interest values ​​of all exhibits in the target exhibition hall, candidate exhibits of interest are selected from the unviewed exhibits and used as candidate points of interest. Based on the layout information of the target exhibition hall and the candidate points of interest, a prediction result of the target user's points of interest in the target exhibition hall is generated.

2. The exhibition hall interest point prediction method based on multimodal data according to claim 1, characterized in that, The step of extracting features from the sensing data and fusing the features to obtain multimodal fusion features includes: The voice data, gesture data, trajectory data, and interaction data are preprocessed respectively to obtain processed voice data, processed gesture data, processed trajectory data, and processed interaction data. The processed speech data is encoded using a pre-trained language model to obtain a text feature vector; Long Short-Term Memory (LSTM) networks are used to mine temporal features from the processed trajectory data to obtain trajectory feature vectors. A preset convolutional neural network is used to extract spatial features from the processed gesture data and the processed interaction data respectively, so as to obtain the gesture feature vector corresponding to the processed gesture data and the operation feature vector corresponding to the processed interaction data. The multimodal fusion features are calculated based on the text feature vector, the trajectory feature vector, the gesture feature vector, and the operation feature vector.

3. The exhibition hall interest point prediction method based on multimodal data according to claim 2, characterized in that, The calculation of the multimodal fusion features based on the text feature vector, the trajectory feature vector, the gesture feature vector, and the operation feature vector includes: Construct attention query vectors; The correlation between the attention query vector and the text feature vector, the trajectory feature vector, the gesture feature vector, and the operation feature vector are calculated respectively to obtain the weights of the text feature vector, the trajectory feature vector, the gesture feature vector, and the operation feature vector. The text feature vector and its weight, the trajectory feature vector and its weight, the gesture feature vector and its weight, and the operation feature vector and its weight are weighted respectively to obtain the multimodal fusion feature.

4. The exhibition hall interest point prediction method based on multimodal data according to claim 1, characterized in that, The step of determining the target user's interest value for all exhibits in the target exhibition hall based on the multimodal fusion features includes: The multimodal fusion features are input into the interest value scoring model to obtain the interest value of the target user for all exhibits in the target exhibition hall, which is output by the interest value scoring model; the interest value scoring model includes a gradient boosting decision tree model.

5. The exhibition hall interest point prediction method based on multimodal data according to claim 4, characterized in that, After inputting the multimodal fusion features into the interest value scoring model to obtain the interest values ​​of the target user for all exhibits in the target exhibition hall, as output by the interest value scoring model, the method further includes: When the time interval between the current moment and the time of collection of the sensor data is greater than a preset time interval, second data is acquired. The second data is the voice data, gesture data, trajectory data and interaction data generated by the target user in the target exhibition hall during the time interval. Based on the second data and the target user's interest values ​​for all exhibits in the target exhibition hall, the interest value scoring model is trained to obtain an updated exhibition hall interest value scoring model.

6. The exhibition hall interest point prediction method based on multimodal data according to claim 1, characterized in that, The step of selecting candidate exhibits of interest from the unviewed exhibits based on the interest values ​​of all exhibits in the target exhibition hall, as candidate points of interest, includes: Extract the interest values ​​of unviewed exhibits from the interest values ​​of all exhibits in the target exhibition hall, and form a ranking of the interest values ​​of unviewed exhibits. Based on the sorting of interest values ​​of the unviewed exhibits, the unviewed exhibits with interest values ​​greater than a preset interest value threshold are identified as the candidate interest exhibits. The location of the exhibit corresponding to the candidate exhibit of interest is taken as the candidate point of interest.

7. The exhibition hall interest point prediction method based on multimodal data according to claim 1, characterized in that, After generating the target user's predicted interest points in the target exhibition hall based on the layout information of the target exhibition hall and the location points of the candidate interest points, the method further includes: Based on the sensor data, the real-time popularity ranking of all exhibits in the target exhibition hall is determined to generate the real-time popular areas and real-time trajectory density of the target exhibition hall; Based on the real-time popularity ranking of all exhibits in the target exhibition hall, exhibits with popularity values ​​lower than a preset popularity threshold in the real-time popularity ranking are identified as low-attention exhibits. Based on the real-time popular areas and real-time trajectory density of the target exhibition hall, an optimization scheme for the low-attention exhibits in the target exhibition hall is generated.

8. A system for predicting points of interest in exhibition halls based on multimodal data, characterized in that, The system includes: The acquisition module is used to acquire sensor data collected by the sensor; the sensor data includes voice data, gesture data, trajectory data generated by the target user in the target exhibition hall, and interaction data of the target user with multiple viewed exhibits in the target exhibition hall; An extraction module is used to extract features from the sensing data and fuse the features of the sensing data to obtain multimodal fusion features; The determination module is used to determine the interest value of the target user for all exhibits in the target exhibition hall based on the multimodal fusion features; all exhibits in the target exhibition hall include viewed exhibits and unviewed exhibits; The filtering module is used to filter out candidate interest exhibits from the unviewed exhibits based on the interest values ​​of all exhibits in the target exhibition hall, and to use them as candidate interest points. The generation module is used to generate a prediction result of the target user's points of interest in the target exhibition hall based on the layout information of the target exhibition hall and the location points of the candidate points of interest.

9. An electronic device, characterized in that, It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enables the at least one control processor to perform a method for predicting points of interest in an exhibition hall based on multimodal data as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a method for predicting points of interest in an exhibition hall based on multimodal data, as described in any one of claims 1 to 7.

Images