A perception-based exhibition hall self-adaptive guide and atmosphere regulation method and system

Abstract

The application relates to the technical field of intelligent exhibition halls, and discloses an exhibition hall self-adaptive tour and atmosphere regulation method and system based on sensing. The method collects visual behaviors, physiological signals, interactive behaviors and environmental state data of audiences through a distributed sensor network, adopts a feature extraction and self-adaptive fusion method to obtain a comprehensive state feature vector, adopts a multi-task learning model and a time sequence prediction method to evaluate the interest degree, fatigue degree, emotional state and cognitive load degree of the audiences, adopts a reinforcement learning and multi-objective optimization method to generate a personalized self-adaptive tour strategy, adopts a situation matching and cooperative regulation method to generate an exhibition hall atmosphere regulation strategy, executes services and continuously optimizes through a real-time feedback and dynamic optimization mechanism, and adopts an online learning and model updating method to continuously improve the system performance. The application can realize the intelligentization, personalization and precision of exhibition hall services, and improve the audience visiting experience quality and the exhibition hall service capability.

Description

Technical Field

[0001] This invention relates to the field of intelligent exhibition hall technology, and more specifically, to a perception-based adaptive tour guide and atmosphere control method and system for exhibition halls. Background Technology

[0002] With the development of the cultural industry and the increasing demands of the public, the number of visitors to museums, art galleries, and other exhibition halls continues to grow, and visitors' needs for personalized and intelligent services are increasing. Traditional services rely on fixed guided tour routes and uniform environmental settings, making it difficult to meet differentiated needs. Although intelligent tour guide and environmental control technologies have been gradually applied, existing technologies still have significant shortcomings.

[0003] Existing tour guide systems mostly rely on preset rules or simple user profiles for push notifications, lacking the ability to perceive and respond to the real-time status of visitors. These systems cannot dynamically adjust tour guide strategies based on visitor fatigue levels, changes in interest, and cognitive load, leading to some visitors experiencing excessive fatigue, decreased interest, or information overload, thus negatively impacting their visit experience. Furthermore, current technologies have a limited perspective on visitor status, primarily relying on location tracking or simple interaction records, lacking comprehensive data collection on visitor emotional states, physiological indicators, and behavioral patterns, making it difficult to accurately assess the true needs of visitors.

[0004] Existing atmosphere control systems are mostly based on fixed environmental parameters or time-period adjustments, lacking sufficient correlation with visitor status and exhibit characteristics. Adjustments to environmental parameters such as lighting, sound effects, temperature, and humidity fail to adequately consider individual visitor differences and real-time changes, nor do they harmonize with the cultural connotations and emotional tone of the exhibits, resulting in unsatisfactory atmosphere creation. Furthermore, there is a lack of collaborative optimization mechanisms between guided tours and atmosphere control; the two often operate independently, failing to form an integrated intelligent service system and limiting further improvement in overall service quality. Summary of the Invention

[0005] This invention provides a perception-based adaptive tour guide and atmosphere control method and system for exhibition halls, solving the technical problems of incomplete audience state perception, lack of adaptability in tour guide services, and disconnect between atmosphere control and audience needs in related technologies.

[0006] This invention provides a perception-based adaptive tour guide and atmosphere control method for exhibition halls, comprising the following steps:

[0007] S1: Collect raw perception data of visitors in the exhibition hall, and use distributed sensor networks and data preprocessing methods to obtain a multi-source heterogeneous perception dataset;

[0008] S2 receives a multi-source heterogeneous sensing dataset and uses feature extraction and adaptive fusion methods to obtain a comprehensive state feature vector;

[0009] S3 receives the comprehensive state feature vector, and uses a multi-task learning model and time-series prediction method to obtain the audience's multi-dimensional state evaluation results and evolution trend prediction.

[0010] S4 receives the multi-dimensional state evaluation results of the audience and uses a multi-objective optimization method to obtain a personalized adaptive tour guide strategy.

[0011] S5 receives multi-dimensional state assessment results from the audience and uses context matching and collaborative control methods to obtain personalized exhibition hall atmosphere control strategies.

[0012] S6 receives tour guide strategies and atmosphere control strategies, and uses a real-time feedback and dynamic optimization mechanism to obtain continuously optimized service execution results and feedback data.

[0013] S7 receives service execution results and feedback data, and uses online learning and model update methods to obtain an intelligent service system with continuously improving performance.

[0014] In a preferred embodiment, S1 includes:

[0015] Deploy distributed visual perception devices in the exhibition hall, adopt a multi-angle coverage strategy to collect audience behavior image data, perform quality detection on the collected data and mark quality labels to obtain a visual behavior image dataset;

[0016] Deploy non-contact physiological sensing devices to collect respiratory signals, heartbeat signals, and facial temperature data through millimeter-wave radar sensors and thermal imaging sensors, and fuse wearable device data for cross-validation to obtain a physiological signal dataset;

[0017] Deploy intelligent interactive terminal devices to record touch screen interaction events, voice query records, and mobile application usage records. By linking the interaction records of different terminals through the unique identifier of the audience, an interactive behavior dataset is obtained.

[0018] Deploy environmental monitoring sensors in the exhibition hall to collect data on temperature, humidity, illuminance, color temperature, noise, and crowd density to obtain an environmental status dataset;

[0019] The data is format-standardized, cleaned, synchronized and aligned, and quality-assessed from multiple sources. Based on audience identifiers and timestamps, the multimodal data is organized into associated data packets to obtain a multi-source heterogeneous sensing dataset.

[0020] In a preferred embodiment, S2 includes:

[0021] A pre-trained residual network is used to extract facial expression features, which are then input into an emotion classifier to output the probability distribution of emotions. A key point detection algorithm is used to identify body postures, and the emotional state, behavioral patterns, and attention distribution features are concatenated to obtain a visual emotion and behavioral feature vector.

[0022] Time-domain feature calculation and frequency-domain analysis were performed on the heart rate signal to extract respiratory features and facial temperature features. The physiological features were then input into the physiological calculation model to calculate the fatigue index and stress level index, thus obtaining the physiological signal feature vector.

[0023] The number of times and duration of visitors browsing various exhibits are statistically analyzed to calculate preference weights. Sequence pattern mining algorithms are used to identify access patterns, and graph embedding methods are used to construct interest graphs to obtain interactive behavior feature vectors.

[0024] The environmental state feature vector was obtained by using thermal comfort model, visual comfort score, acoustic environment quality score and space crowding index;

[0025] An adaptive weighted fusion method based on attention mechanism is adopted. The fusion weight is dynamically calculated according to the data quality scores of each modality. Feature mapping is performed through a feature compression network with an autoencoder structure. Missing data is imputed with historical statistical features or similar audience features to obtain a comprehensive state feature vector.

[0026] In a preferred embodiment, S3 includes:

[0027] A multi-task learning model is constructed, which adopts an architecture of shared underlying feature extractor and multiple task-specific prediction heads. Each prediction head outputs evaluation values ​​of interest, fatigue, emotional state and cognitive load. The model is trained using a multi-task loss function to obtain the evaluation results of the audience's current state.

[0028] The Monte Carlo Dropout method is used to quantify the prediction uncertainty of the model. Multiple forward propagations are performed on the same input, and the mean and standard deviation of the predicted values ​​are calculated. The standard deviation is normalized to obtain the confidence score, and the prediction confidence of each state index is obtained.

[0029] Based on the historical state sequence of the audience, a long short-term memory network is used for time series prediction to predict the values ​​and rates of change of each state indicator within the future time window. The prediction results are then tested for rationality and monotonicity constraints to obtain the multidimensional state assessment results and evolution trend prediction of the audience.

[0030] In a preferred embodiment, S4 includes:

[0031] By integrating audience state characteristics, exhibit attribute characteristics, spatial layout characteristics, and time constraint characteristics, a guided tour decision state space is constructed, resulting in a standardized guided tour decision state representation.

[0032] A hierarchical decision-making architecture is adopted, with high-level decision-making determining recommended exhibits, mid-level decision-making using path planning algorithms to calculate the optimal path, and low-level decision-making dynamically adjusting the explanation mode and content detail based on cognitive load and fatigue, thus obtaining action plans for exhibit selection, path planning, and pace control.

[0033] Define a reward function to evaluate the effectiveness of guided tours, taking into account audience satisfaction, knowledge acquisition efficiency, experience comfort, and resource utilization, and obtain a comprehensive reward signal by weighting and combining the scores of each objective;

[0034] A deep Q-network algorithm is used to train the tour guide decision model. Experience replay and target network techniques are used to improve training stability. Hard and soft constraint mechanisms are introduced to ensure policy security, resulting in a personalized adaptive tour guide strategy.

[0035] In a preferred embodiment, S5 includes:

[0036] Cultural characteristics are labeled for the exhibits in the exhibition hall, including historical period, artistic style, theme category, emotional tone and atmosphere requirements, to obtain a cultural characteristic database of the exhibits;

[0037] Atmosphere control rules were designed based on emotional state, fatigue level, and cognitive load. Fuzzy logic control method was used to handle nonlinear relationships, and a multi-input multi-output fuzzy control system was established to obtain the target parameters for atmosphere control based on audience perception.

[0038] By integrating the cultural characteristics of the exhibits and the state of the audience, the optimal illuminance, color temperature and volume parameters are solved using a weighted optimization method, resulting in a comprehensive atmosphere control strategy.

[0039] Based on the spatial layout, atmosphere control areas are divided. The audience needs in each area are clustered and analyzed to determine the dominant needs. Boundary transition areas are designed to achieve a gradual transition of parameters between adjacent areas, resulting in a personalized exhibition hall atmosphere control strategy.

[0040] In a preferred embodiment, S6 includes:

[0041] Guided tour services are pushed to visitors through mobile applications, smart voice assistants, AR glasses, and exhibition area information screens. Each guided tour terminal keeps data synchronized and records visitor response behavior as a feedback signal of service effectiveness.

[0042] The atmosphere control strategy is converted into equipment control commands for lighting, sound, and temperature and humidity control systems. Gradual adjustment is used to achieve smooth parameter transitions, and an equipment status monitoring mechanism is established to detect faults and activate emergency plans.

[0043] In a preferred embodiment, S6 further includes:

[0044] Establish a real-time monitoring mechanism for service execution, adopt evaluation indicators for tour guide effectiveness and atmosphere control effectiveness, calculate the overall service quality score, and obtain a quantitative evaluation of service effectiveness;

[0045] Establish a condition judgment mechanism to trigger optimization. When the audience status index changes suddenly, the service effect index falls below the preset threshold, or an abnormal environmental event occurs, the dynamic optimization process is initiated, the audience status assessment is re-executed, and a new tour guide strategy and atmosphere strategy are generated.

[0046] Establish an optimization effect tracking and verification mechanism. After implementing optimization adjustments, strengthen the monitoring of audience status and service effect indicators, verify the effectiveness of optimization measures, record optimization adjustment events to form an optimization log, and obtain continuously optimized service execution effects and feedback data.

[0047] In a preferred embodiment, S7 includes:

[0048] A data collection and annotation mechanism was constructed, and an active learning strategy was adopted to prioritize manual annotation of samples with high model prediction uncertainty. A data quality control mechanism was established to perform consistency checks and anomaly detection to obtain a high-quality training dataset.

[0049] The decision model is updated using incremental learning, and the transfer learning method is used to freeze the low-level parameters of the model and update only the high-level parameters. Regularization constraints are introduced to prevent overfitting, and a model version management mechanism is established to obtain a decision model with continuously improving performance.

[0050] Establish a system operation monitoring platform, design a hierarchical alarm mechanism, implement root cause analysis of anomalies, and establish an automatic recovery mechanism to obtain an intelligent service system with continuously improving performance.

[0051] In a preferred embodiment, a perception-based adaptive tour guide and atmosphere control system for exhibition halls is used to perform the steps in the aforementioned perception-based adaptive tour guide and atmosphere control method for exhibition halls, including:

[0052] The multi-source sensing data acquisition module is used to collect raw sensing data from visitors in the exhibition hall. It uses a distributed sensor network and data preprocessing methods to obtain a multi-source heterogeneous sensing dataset.

[0053] The feature extraction and fusion module is used to receive multi-source heterogeneous sensing datasets and obtain a comprehensive state feature vector by using feature extraction and adaptive fusion methods.

[0054] The state assessment and prediction module is used to receive the comprehensive state feature vector, and use a multi-task learning model and time series prediction method to obtain the multi-dimensional state assessment results and evolution trend prediction of the audience.

[0055] The adaptive tour guide decision-making module receives multi-dimensional status evaluation results from visitors and uses a multi-objective optimization method to obtain a personalized adaptive tour guide strategy.

[0056] The atmosphere control strategy module is used to receive multi-dimensional state evaluation results of the audience and use context matching and collaborative control methods to obtain personalized exhibition hall atmosphere control strategies.

[0057] The service execution and feedback module is used to receive tour guide strategy plans and atmosphere control strategies, and adopts a real-time feedback and dynamic optimization mechanism to obtain continuously optimized service execution results and feedback data.

[0058] The online learning and update module is used to receive service execution results and feedback data, and adopts online learning and model update methods to obtain an intelligent service system with continuously improving performance.

[0059] The beneficial effects of this invention are as follows:

[0060] By constructing a multimodal perception system, this invention collects data on audience visual behavior, physiological signals, interactive behaviors, and environmental states. An adaptive weighted fusion method is used to integrate multi-source heterogeneous data, achieving comprehensive perception and accurate assessment of audience states. A multi-task learning model is employed to simultaneously evaluate interest, fatigue, emotional state, and cognitive load, while a temporal prediction method forecasts state evolution trends, providing a reliable basis for adaptive service decisions. Compared to existing methods that rely solely on single-dimensional perception, this invention can more accurately identify the audience's true needs and state changes, improving the accuracy and timeliness of service decisions.

[0061] An adaptive guided tour decision-making model is constructed using reinforcement learning and multi-objective optimization methods. This model dynamically adjusts recommended exhibits, tour routes, and narration pace based on the real-time status of visitors, achieving personalized and intelligent guided tour services. Atmosphere control strategies are generated using context matching and collaborative regulation methods, integrating the cultural characteristics of exhibits with the perceptual needs of visitors, achieving a synergistic match between atmosphere creation and visitor status and exhibit content. Through real-time feedback and dynamic optimization mechanisms, as well as online learning methods, the system can continuously optimize service strategies and adapt to environmental changes, improving the overall quality of exhibition hall services and visitor satisfaction. Attached Figure Description

[0062] Figure 1 This is a flowchart of a perception-based adaptive tour guide and atmosphere control method for exhibition halls according to the present invention;

[0063] Figure 2 This is a flowchart of a perception-based adaptive tour guide and atmosphere control method for exhibition halls according to the present invention. Detailed Implementation

[0064] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0065] At least one embodiment of the present invention discloses a perception-based adaptive tour guide and atmosphere control method for exhibition halls, such as... Figures 1 to 2 As shown, it includes the following steps:

[0066] S1: Collect raw perception data of visitors in the exhibition hall, and use distributed sensor networks and data preprocessing methods to obtain a multi-source heterogeneous perception dataset;

[0067] This step addresses the problems of limited dimensions and unstable data quality in existing technologies for audience state perception. It constructs a multimodal perception system covering visual behavior, physiological signals, interaction patterns, and environmental states, providing a comprehensive and reliable data foundation for subsequent state assessment. Specifically, it includes the following steps:

[0068] S11, deploy distributed visual perception equipment in the exhibition hall, and use a multi-angle coverage strategy to collect audience behavior image data;

[0069] High-definition cameras and depth sensors are deployed at the exhibition hall entrance, main passageways, key exhibit areas, and rest areas. The cameras use wide-angle lenses with a field of view covering 120 degrees horizontally and 90 degrees vertically, with 20% to 30% overlap between adjacent devices to ensure no blind spots. The depth sensors are deployed in pairs, using structured light or time-of-flight technology to acquire 3D spatial information, with a ranging range of 0.5 to 8 meters and an accuracy of ±2 centimeters.

[0070] The vision devices capture facial images, body postures, and spatial positions of the audience at 25 frames per second. All devices are connected to the edge computing node via Ethernet and synchronized using Network Time Protocol (NTP), with timestamp errors controlled within 10 milliseconds.

[0071] The collected data underwent quality checks, including brightness detection (range 50-200), occlusion detection (face occlusion ratio <50%), and depth hole detection (invalid pixel ratio <30%). The detection results were labeled as high quality, medium quality, or low quality, resulting in a visual behavior image dataset with quality labels.

[0072] S12, deploy non-contact physiological sensing devices and use multi-parameter monitoring methods to collect audience physiological signal data;

[0073] Millimeter-wave radar sensors and thermal imaging sensors were deployed in key exhibit areas and near rest seating areas. The millimeter-wave radar operates in the 60 GHz band with a sampling frequency of 100 Hz. It extracts respiratory signals (0.2-0.5 Hz frequency components) and heartbeat signals (1-2 Hz frequency components) using phase difference technology, and continuously collects data for 30 seconds to obtain respiratory rate and heart rate data.

[0074] The thermal imaging sensor samples at a frequency of 10 frames per second, with a resolution of 320×240 pixels, a temperature measurement range of 30 to 45 degrees Celsius, and an accuracy of ±0.1 degrees Celsius. It extracts the average temperature and temperature gradient of key areas such as the forehead, tip of the nose, and around the eyes. Under pressure, the forehead temperature rises by 0.5 to 1.5 degrees Celsius, while under fatigue, the temperature around the eyes decreases by 0.3 to 0.8 degrees Celsius.

[0075] At the entrance of the exhibition hall, smart wristbands are provided for visitors to wear voluntarily. They are equipped with a photoplethysmography (PPG) sensor (100Hz sampling, measurement range 40-200 beats / minute) and a skin conductance sensor (20Hz sampling, measurement range 0.1-20 micro Siemens) to monitor heart rate, heart rate variability and skin conductance signals.

[0076] Data from contactless and wearable devices is integrated: when both types of data are acquired simultaneously, wearable data is used as the primary source for cross-validation, and any deviation exceeding 10% is marked as low quality; when only contactless data is available, accuracy is compensated for through model calibration. This results in a physiological signal dataset containing respiratory rate, heart rate, heart rate variability, and facial temperature distribution.

[0077] S13, Deploy intelligent interactive terminal devices and use multi-dimensional recording methods to collect audience interaction behavior data;

[0078] Touchscreen information displays (32-inch, 1920×1080 resolution, 10-point touch) and voice interaction terminals (equipped with a 4-microphone array, voice recognition accuracy >95%) are deployed in each exhibit area. The touchscreens display detailed exhibit information, background stories, high-definition images and videos, while the voice terminals support natural language dialogue, providing exhibit information inquiry, route navigation, and explanation services.

[0079] The touchscreen records all interaction events: timestamps (milliseconds), touch location coordinates, touch type (single, double, long press, swipe, zoom), page navigation, dwell time, video playback duration, and favorites. The voice terminal records query text content, topic categories (exhibit introductions, route navigation, time information, facility locations, etc.), dialogue rounds, and satisfaction levels.

[0080] The mobile application provides electronic maps, audio guides, AR-enhanced displays, and personalized recommendations. It records launch time, feature usage type, browsed exhibit list, duration of guided tours, AR usage counts, and preference settings. Data is cached locally and automatically uploaded when connected to the exhibition hall's Wi-Fi.

[0081] Interaction records from different terminals are linked by a unique viewer identifier (anonymized ID), and behavioral trajectory sequences are constructed by chronological order. This yields an interactive behavior dataset containing interaction event types, interaction frequencies, content preferences, and chronological patterns.

[0082] S14, Deploy environmental monitoring sensors in the exhibition hall and collect environmental status data using a zoned sampling method;

[0083] Based on the exhibition hall's spatial layout, the exhibition hall is divided into several monitoring units (each unit is 50 to 100 square meters), and a comprehensive environmental monitoring station is deployed in the center, integrating sensors for temperature, humidity, light, noise, and air quality.

[0084] Temperature and humidity sensor measurement range: temperature -10 to 50℃ (accuracy ±0.3℃), humidity 0 to 100%RH (accuracy ±3%RH), sampling period 1 minute. Illuminance sensor measures illuminance 0 to 20000 lux and color temperature 2000 to 10000 Kelvin, sampling period 10 seconds. Noise sensor measurement range 30 to 120 dB, conforming to A-weighting standard, sampling period 1 second, recording instantaneous noise and 1 minute of equivalent continuous sound level. People counter uses stereo vision technology to count the number of people entering and exiting, with a counting accuracy of over 95%, sampling period 10 seconds.

[0085] Data from each monitoring unit was aggregated into the exhibition hall environmental database and spatiotemporally annotated. This resulted in an environmental status dataset containing temperature, humidity, illuminance, color temperature, noise level, and crowd density.

[0086] S15. Preprocess the multi-source heterogeneous data, and use data cleaning and quality assessment methods to obtain a standardized multimodal perception dataset.

[0087] The visual behavior, physiological signals, interactive behavior, and environmental status data collected in steps S11 to S14 are aggregated into the exhibition hall big data platform for systematic preprocessing.

[0088] Data format is standardized; it is converted to a unified JSON format, which includes data type, data source (device identifier), audience identifier (anonymous ID generated by SHA-256 hash algorithm), timestamp (Unix millisecond timestamp), data content, and quality label (high / medium / low).

[0089] Data cleaning; removal of low-quality image frames; removal of abnormal physiological data (heart rate fluctuations >20 beats / minute, respiratory rate exceeding 10-30 breaths / minute, heart rate exceeding 40-200 beats / minute) using sliding window detection; removal of falsely triggered interactive events (click duration <50 milliseconds); removal of sensor fault data (measured values ​​exceeding the device's range, or more than 10 consecutive identical readings).

[0090] Data synchronization and alignment; high-frequency data is downsampled or aggregated using a 1-minute baseline time window. Statistical characteristics (mean, variance, maximum, and minimum) are calculated for visual data (25 frames / second) and physiological data (100Hz sampling) within 1 minute; interactive data is used to statistically analyze the total number of events and the distribution of events of each type within 1 minute.

[0091] Data quality assessment: A comprehensive quality score (out of 100) is calculated based on completeness (missing rate <10%), reliability (outlier rate <5%), timeliness (delay <5 seconds), and consistency (conflict rate <10%). A score of 80 or above is considered high quality, 60-80 is medium quality, and below 60 is low quality. Low-quality data is marked and retained, but its weight is reduced or imputation methods are used to repair it in subsequent processing.

[0092] Data association and integration: Based on audience identifiers and timestamps, multimodal data from the same audience within the same time period (1 minute) are organized into associated data packets. A structured multimodal perception dataset is obtained, indexed by audience and sequenced by time, containing multidimensional features and quality labels of four modalities: visual behavior image data, physiological signal data, interactive behavior data, and environmental state data, which serve as input for feature extraction in step S2.

[0093] S2 receives a multi-source heterogeneous sensing dataset and uses feature extraction and adaptive fusion methods to obtain a comprehensive state feature vector;

[0094] This step receives the multimodal perception dataset output from step S1, including visual behavior image data, physiological signal data, interaction behavior data, and environmental state data as input. Addressing the challenges of high heterogeneity, high feature dimensionality, and inconsistent quality in multimodal data, a targeted feature extraction strategy and a robust fusion mechanism are designed to transform the scattered multi-source data into a compact, unified representation. Through visual behavior feature extraction, physiological signal feature extraction, interaction behavior feature extraction, and environmental state feature extraction, a comprehensive state feature vector is obtained through fusion. Specifically, the steps include:

[0095] S21. Extract visual behavioral features and use a deep learning model to recognize facial expressions and body postures to obtain the audience's visual emotional features and behavioral pattern features.

[0096] The system receives visual behavior image data from the multimodal perception dataset output in step S1 and extracts facial expression features using a pre-trained 50-layer residual network. It then detects face bounding boxes and keypoints using a multi-task cascaded convolutional network, crops and scales the face region to 224×224 pixels, extracts facial feature vectors, and inputs them into an emotion classifier to output the probability distributions of seven basic emotions (pleasure, surprise, sadness, anger, fear, disgust, and calmness). Within a one-minute time window, the frequency of each emotion category is counted, and the dominant emotion and its frequency of change are calculated.

[0097] A keypoint detection algorithm was used to identify body postures, detecting 17 key points on the human body. Posture features were calculated based on these key points: the angle between the head and torso determined the viewing posture, and changes in the distance between key points determined the movement state (standing viewing is defined as head-torso angle ±15 degrees, and movement state is defined as speed >0.5 m / s). The duration of different postures within one minute was statistically analyzed.

[0098] In close-range interaction scenarios, eye-tracking algorithms are used to calculate the gaze point on the exhibit by combining head posture and eye orientation, and the duration of gaze in different areas is counted to identify the focus of interest.

[0099] By concatenating emotional states, behavioral patterns, and attention distribution features, a visual emotion and behavior feature vector is formed, which includes emotional probability distribution, dominant emotional category, frequency of emotional changes, proportion of posture duration, movement speed, distribution of gaze area, and other visual behavioral parameters.

[0100] S22, extract physiological signal features, and use time-frequency analysis and physiological calculation models to obtain the fatigue and stress level characteristics of the audience;

[0101] Receive physiological signal data from the multimodal sensing dataset output in step S1, including heart rate, heart rate variability, respiratory rate and facial temperature data, and extract physiological state features.

[0102] Heart rate time-domain characteristics: Calculate the mean heart rate, standard deviation, maximum and minimum heart rate within a 5-minute window. Heart rate variability frequency domain analysis: Calculate low-frequency power (0.04-0.15Hz) and high-frequency power (0.15-0.4Hz) using Fast Fourier Transform, and calculate the low-frequency / high-frequency ratio. An increased ratio indicates stress or tension, while a decreased ratio indicates relaxation or fatigue.

[0103] Respiratory characteristics: The mean and coefficient of variation of respiratory cycles were calculated within a 3-minute window, with a normal resting state of 12-20 breaths / minute. Facial temperature characteristics: The average temperature and temperature difference of the forehead, tip of the nose, and periorbital area were extracted. The forehead temperature increased under stress, while the periorbital temperature decreased under fatigue.

[0104] Physiological characteristics are input into a physiological computational model, and fatigue and stress levels (0-100) are calculated through weighted combination. Fatigue is comprehensively considered in terms of decreased heart rate variability, slowed breathing, and decreased periorbital temperature; stress levels are comprehensively considered in terms of increased heart rate, increased low-frequency / high-frequency ratio, and increased forehead temperature. By comparing the deviation of the audience's current parameters from their personal baseline level (initial calm state during the visit), changes in state can be more accurately assessed.

[0105] The physiological signal feature vector is obtained, which includes heart rate time domain features, heart rate variability frequency domain features, respiratory features, facial temperature features, fatigue index, stress level index and other physiological parameters.

[0106] S23, extract interactive behavior features, and use sequence pattern mining and interest modeling algorithms to obtain audience interest preference features and participation features;

[0107] Receive interactive behavior data from the multimodal perception dataset output in step S1, including touchscreen interaction records, voice query records, and mobile application usage records, and mine audience interests and participation levels.

[0108] Touchscreen interaction analysis; exhibits are labeled by historical period, category, and theme tag. The number of times and duration of visitor views of various exhibits are statistically analyzed, and preference weights (normalized browsing time percentage) are calculated. The level of content detail is analyzed, and the proportion of visits to each level of content (introduction, detailed introduction, images, videos) is statistically analyzed, calculating a content depth preference index.

[0109] Voice query analysis: Queries are categorized into information retrieval, navigation, and interaction types, and the frequency percentage of each type is calculated to determine the interaction activity index. Mobile application analysis: The frequency and duration of use of functions such as voice explanation, AR augmented reality, and social sharing are statistically analyzed, and audience types (knowledge acquisition, technology experience, and social sharing) are identified based on function usage patterns.

[0110] Sequence pattern mining: Visitor visits to exhibits are arranged in chronological order to form a sequence. A sequence pattern mining algorithm is used to identify high-frequency access patterns. The actual browsing path is matched with the high-frequency patterns, and the path similarity is calculated.

[0111] Interest graph construction: A directed weighted graph is constructed using exhibits and themes as nodes and browsing behavior as edges. Edge weights represent the frequency or duration of exhibit transitions. A graph embedding method is used to compress the high-dimensional graph structure into a low-dimensional vector.

[0112] The interaction behavior feature vector is formed by embedding and splicing exhibit category preferences, content depth preferences, interaction activity, function usage patterns, path similarity, and interest graphs. This vector includes exhibit category preference weights, content depth preference indices, interaction activity indices, function usage frequency, path similarity, interest graph embedding vectors, and other interaction features.

[0113] S24. Extract environmental state features and use spatiotemporal correlation analysis to obtain the environmental comfort features and crowding features of the exhibition hall.

[0114] Receive environmental status data from the multimodal sensing dataset output in step S1, including temperature, humidity, illuminance, color temperature, noise, and crowd density data, and assess the environmental quality of the exhibition hall.

[0115] Thermal comfort: Calculated using the Predicted Mean Thermal Sensation Index (PMV) model, which comprehensively considers air temperature, humidity, airflow velocity, radiation temperature, metabolic rate, and clothing thermal resistance, outputting values ​​from -3 (cold) to +3 (hot), and mapping PMV to a comfort score of 0-100.

[0116] Visual comfort: Set lighting requirements (50-200 lux for cultural relics, 200-500 lux for artworks, and 500-1000 lux for interactive exhibits) and color temperature range (2700-3000K for warm colors, 4000-4500K for neutral colors, and 5000-6500K for cool colors) according to the type of exhibit, and calculate the lighting suitability score.

[0117] Acoustic environment quality: The ideal noise level in the exhibition hall is 40-50 decibels. A score of 100 is given within the ideal range, and 10 points are deducted for every 5 decibels deviating from the ideal range. Space crowding: The comfortable crowd density is 0.3-0.5 people / square meter. The crowding index is calculated as crowd density / upper limit of comfortable density. An index <1 indicates a relaxed atmosphere, and >1 indicates a crowded atmosphere.

[0118] Thermal comfort, visual comfort, noise comfort, and congestion are integrated into an environmental quality assessment vector. This yields an environmental state feature vector, including thermal comfort score, visual comfort score, noise comfort score, congestion index, current location environmental parameters, and path environmental parameters.

[0119] S25, by fusing multimodal features and employing an adaptive weighted attention mechanism, obtains the comprehensive state feature vector of the audience;

[0120] The system receives the 64-dimensional visual features output from step S21, the 32-dimensional physiological features output from step S22, the 48-dimensional interaction features output from step S23, and the 16-dimensional environmental features output from step S24, and fuses them to construct a comprehensive state representation.

[0121] An adaptive weighted fusion method based on an attention mechanism is adopted, and the fusion weights are dynamically calculated according to the data quality scores of each modality and the requirements of the evaluation task. First, the feature vectors of each modality are standardized, transforming each feature dimension into a distribution with a mean of 0 and a standard deviation of 1.

[0122] An attention weight calculation module (shallow neural network) is constructed, which takes the feature vectors of each modality and the quality score as input and outputs the attention weights. These weights are mapped to attention scores through a fully connected layer and then normalized to weight coefficients (summing to 1) using softmax. The fusion strategy is adjusted according to the evaluation objective: physiological features are given higher weights when evaluating fatigue, while interactive and visual attention features are given higher weights when evaluating interest.

[0123] Weighted fusion is performed by multiplying the feature vectors of each modality with attention weights and then concatenating or summing them using a hybrid approach. The fused features are mapped to a compact representation through a feature compression network with an autoencoder structure. The autoencoder consists of an encoder (a 3-layer fully connected network using ReLU activation) and a decoder (symmetric structure), trained by minimizing the reconstruction error (mean squared error loss, Adam optimizer, learning rate 0.001).

[0124] Data missing compensation: When a modality is completely missing, its weight is set to 0, and the weights of other modalities are increased proportionally so that the sum of the weights remains 1; when some modalities are missing or the quality is too low (quality score < 60), historical statistical features (average features of the audience over the past 10 minutes) or average features of similar audiences (mean features of audience groups with the same age and visit type) are used for imputation, while the weight of that modality is reduced to 50% of its original weight. A robust comprehensive state feature vector is obtained.

[0125] S3 receives the comprehensive state feature vector, and uses a multi-task learning model and time-series prediction method to obtain the audience's multi-dimensional state evaluation results and evolution trend prediction.

[0126] This step receives the comprehensive state feature vector output from step S2 as input. Considering the multidimensionality and dynamism of the audience's state, it constructs an intelligent model capable of simultaneously evaluating multiple state indicators and predicting future trends, providing a decision-making basis for adaptive guidance and atmosphere control. Specifically, it includes the following steps:

[0127] S31, Construct an audience status assessment model, using a multi-task deep neural network to obtain the current assessment values ​​of interest, fatigue, emotion, and cognitive load;

[0128] The comprehensive state feature vector obtained in step S2 is used as input to construct a multi-task learning model to simultaneously predict four state indicators of the audience: interest, fatigue, emotion, and cognitive load.

[0129] The multi-task learning model employs an architecture that shares a low-level feature extractor and multiple task-specific prediction heads. The low-level feature extractor consists of three fully connected neural network layers, each with 256 neurons, using the ReLU activation function to extract higher-level abstract representations. Above the low-level feature extractor, a dedicated prediction head is designed for each task, each containing two fully connected network layers with 128 neurons.

[0130] The output layers of each prediction head are designed according to task characteristics: interest and fatigue levels use regression outputs (0-100); sentiment levels use multi-classification outputs (7 basic sentiment probability distributions); and cognitive load levels use regression outputs (0-100). Model training employs a multi-task loss function, with the total loss being the weighted sum of the losses from each task. Mean squared error is used for interest and fatigue levels, while cross-entropy is used for sentiment levels. Training data is derived from labeled historical audience data (questionnaires, real-time feedback, and expert annotations).

[0131] Interest level (0-100) reflects the audience's level of attention to the current exhibit. High interest is characterized by longer dwell time, frequent interaction, and focused gaze. Fatigue level (0-100) reflects the degree of accumulated physiological and cognitive fatigue, manifested as decreased heart rate variability, slowed breathing, relaxed posture, and reduced interaction frequency. Fatigue level 0-30 indicates high energy, 30-60 indicates mild fatigue, 60-80 indicates moderate fatigue, and 80-100 indicates severe fatigue. Emotion level outputs the probability distribution of 7 basic emotions, with the category with the highest probability being the dominant emotion. Cognitive load level (0-100) reflects the cognitive pressure of information processing, manifested as increased forehead temperature, increased heart rate, and frequent switching of content. Cognitive load level 0-40 indicates easy cognitive processing, 40-70 indicates moderate cognitive processing, and 70-100 indicates cognitive overload.

[0132] The comprehensive state feature vector is input into the trained multi-task model, and the forward propagation calculates the output values ​​of four state indicators. The current state assessment of the audience is obtained, including interest level (0-100), fatigue level (0-100), dominant emotion category and probability distribution, and cognitive load level (0-100), which serve as inputs for confidence assessment in step S32 and trend prediction in step S33.

[0133] S32, Construct a state confidence assessment module and use uncertainty quantification method to obtain the prediction confidence of each state index;

[0134] The state assessment results output in step S31 are received. Since the audience state assessment relies on sensor data and model inference, there are uncertainties such as data noise, model error and individual differences. Therefore, the credibility of the assessment results needs to be quantified.

[0135] The Monte Carlo Dropout method is used to quantify the model's predictive uncertainty. During the model inference phase, the Dropout layer is kept active, and 20 forward propagations are performed on the same input. Each propagation produces a different output due to the randomness of Dropout. For regression tasks, the mean of the 20 predicted values ​​is calculated as the final predicted value, and the standard deviation is calculated as a measure of uncertainty. For classification tasks, the number of times each category is predicted with the highest probability is counted, and the consistency of category predictions is calculated.

[0136] A smaller standard deviation of uncertainty indicates a more stable model prediction and a higher confidence level. The standard deviation is normalized to the range of 0 to 1, and the confidence score is calculated by subtracting the normalized standard deviation from the confidence score. A confidence score of 0.8 to 1 indicates high confidence, 0.6 to 0.8 indicates medium confidence, and 0 to 0.6 indicates low confidence.

[0137] For assessment results with low confidence, analyze the sources of uncertainty. Examine the quality scores of each modality in the input feature vector; if a modality has a low quality score, that modality may be the main source of uncertainty.

[0138] The confidence level information is output along with the state assessment results. An assessment report is generated, containing predicted values ​​and confidence scores for each state indicator, including interest level and its confidence level, fatigue level and its confidence level, emotional state and its confidence level, and cognitive load level and its confidence level, providing an uncertainty reference for subsequent decision-making. In cases of low confidence, the decision-making module should adopt a conservative strategy or wait for more data to accumulate before making a decision.

[0139] S33, Construct a state evolution trend prediction model and use a time series deep learning network to obtain the state change trend of future time windows;

[0140] The system receives the current state assessment results and historical state sequence data output in step S31. The state of the visitors evolves dynamically as the visit progresses, and it is necessary to predict the future trend of the visitors' state in order to make forward-looking service adjustments.

[0141] Based on the audience's historical state sequence, a Long Short-Term Memory (LSTM) network is used for time series prediction. The time series is composed of the comprehensive state feature vector of the audience for each minute in the past 10 minutes (continuously output by step S2) and the state evaluation results (continuously output by step S31, with interest, fatigue, and cognitive load each having 1 dimension, for a total of 3 dimensions), for a total of 10 time steps. The LSTM network contains 2 layers, each with 128 hidden units, which can capture long-term dependencies and change patterns in the state sequence.

[0142] The output layer of the LSTM predicts the changing trend of the audience's state over the next 5 minutes. The prediction targets include the values ​​of interest, fatigue, and cognitive load at the next 5 time points, for a total of 15 output values ​​(3 indicators × 5 time points), as well as the rate of change of these indicators (increasing, stable, or decreasing). The rate of change is obtained by calculating the difference between the predicted value and the current value; a difference > 5 indicates an upward trend, and a difference < -5 indicates a downward trend.

[0143] The model training adopts a sequence-to-sequence learning paradigm. The training data is a complete sequence of historical visitors. The first part of the sequence is used as input and the second part is used as the prediction target. The model is trained by minimizing the mean square error between the predicted value and the true value.

[0144] The prediction results are then validated for reasonableness. The predicted state values ​​are checked to ensure they fall within a reasonable range (0-100) and to identify any non-physiological abrupt changes (single-step changes >30). If the predictions are unreasonable, smoothing filters or a regression to linear extrapolation are employed. For cumulative indicators such as fatigue, the trend is checked to ensure it conforms to monotonicity constraints. Fatigue generally exhibits a monotonically increasing trend; a decrease in fatigue is only permitted after an audience member has entered the rest area and remained there for more than 5 minutes.

[0145] Based on state trend predictions, potential risks are identified. If the prediction indicates that fatigue levels will exceed 60 within the next 5 minutes (entering moderate fatigue), the system should take proactive intervention measures, such as recommending visitors to rest areas or reducing the density of guided information. If the prediction indicates a continued decline in interest, the system should consider adjusting recommended content and introducing more attractive exhibits.

[0146] The current state assessment value, state confidence level, and future trend prediction are integrated into a complete state analysis report. This yields a comprehensive assessment of the audience's multidimensional state, including the current interest level (0-100) and its trend over the next 5 minutes, the current fatigue level (0-100) and its trend over the next 5 minutes, emotional state (7 types of emotional probability distribution) and its changing tendencies, the current cognitive load level (0-100) and its trend over the next 5 minutes, and the prediction confidence level (0-1) for each indicator. This state assessment result serves as the input for step S4 adaptive tour guidance decision-making and step S5 atmosphere control decision-making.

[0147] S4 receives the multi-dimensional state evaluation results of the audience and uses a multi-objective optimization method to obtain a personalized adaptive tour guide strategy.

[0148] This step receives the audience's multi-dimensional state assessment results (including interest, fatigue, emotional state, cognitive load, and their changing trends) from step S3 as input. Addressing the issues of significant individual differences among audience members, dynamic state changes, and complex multi-objective constraints, it constructs an intelligent tour guide decision-making system capable of responding to audience states in real time and balancing multiple objectives. Specifically, it includes the following steps:

[0149] S41, construct the navigation decision state space, and use feature engineering methods to obtain the input state representation of the navigation decision;

[0150] The multi-dimensional state evaluation results of the audience output in step S3 are received as input. The tour guide decision takes into account the audience state, exhibit attributes, spatial layout and time constraints, and integrates them into the state space of the decision model.

[0151] Audience status characteristics; including interest level, fatigue level, emotional state, cognitive load and trend of change obtained in step S3, basic audience attributes (age group, visit type), and historical behavioral characteristics (list of exhibits visited, cumulative duration, frequency of interaction).

[0152] Exhibit attribute characteristics are extracted from the exhibit knowledge graph, including basic information (name, category, era), cultural characteristics (theme tags, emotional tone, difficulty level), and related information (related exhibits, background stories). The extracted attribute vectors include category coding, era coding, multi-hot theme tag coding, difficulty level, and estimated visit duration.

[0153] Spatial layout characteristics; calculate the physical distance and path length from the visitor's current location to each candidate exhibit. Obtain the real-time crowd density in each exhibit area, and reduce the recommendation priority of areas with excessively high density. Construct a topology map of the exhibition hall, where nodes represent exhibits or locations, edges represent walkable paths, and edge weights represent the combined cost of path length and congestion level.

[0154] Time constraints include remaining visitor time and exhibition hall opening hours. The number and pace of subsequent exhibits should be allocated reasonably based on the remaining time.

[0155] The state vector is constructed by splicing and integrating data, including audience status, audience attributes and history, candidate exhibit attributes, spatial layout, time constraints, and other auxiliary information. Normalization ensures that each feature dimension is within a similar numerical range, resulting in a standardized representation of the tour guide decision-making state, which serves as the input for the action space construction in step S42.

[0156] S42 constructs a guided tour decision-making action space, adopts a hierarchical decision-making architecture, and obtains action plans for exhibit selection, path planning, and rhythm control.

[0157] The guide decision state representation output from step S41 is received as input, and a hierarchical decision architecture is used to decompose complex decisions into manageable sub-tasks.

[0158] High-level decision-making: Determine the next recommended exhibit and the overall visitor route. The action space is the set of all exhibits in the exhibition hall. Calculate the recommendation score for each candidate exhibit based on the state vector, comprehensively considering: interest matching (calculated through collaborative filtering and content matching), state suitability (adjusted according to fatigue, emotional state, and cognitive load; recommending short, highly interactive exhibits when fatigue is high; recommending exhibits with positive themes when emotions are negative; and recommending exhibits with low difficulty when cognitive load is high), and accessibility (considering both spatial distance and time cost). A weighted sum is obtained to obtain the comprehensive recommendation score, and the exhibit with the highest score is selected as the recommendation target.

[0159] Mid-level decision-making: Planning the visitor route from the current location to recommended exhibits. The optimal route is calculated using the A* path planning algorithm, with the path cost function comprehensively considering path length, congestion, and the attractiveness of exhibits along the way. A sequence of navigation instructions is generated and presented via text, voice, or AR arrows through the guide terminal.

[0160] Low-level decision-making; controlling the pace and level of detail in the guided tour presentation. Dynamically adjusting based on cognitive load and fatigue: In-depth explanation mode (low cognitive load and fatigue, detailed background and artistic value, 5-8 minutes), Standard explanation mode (moderate cognitive load or fatigue, basic introduction and core points, 2-4 minutes), Simplified explanation mode (high cognitive load or fatigue, names and obvious features, 1-2 minutes or a break recommended). Selecting language style and multimedia methods based on emotional state.

[0161] The integrated guide strategy output includes: recommended exhibit labels, navigation path instruction sequences, explanation mode tags (in-depth / standard / simplified), language style tags (interactive / narrative / concise), multimedia presentation schemes, and estimated guide duration. This guide strategy serves as input for the reward function evaluation in step S43 and the reinforcement learning training in step S44, and ultimately as input for service execution in step S6.

[0162] S43, construct the tour guide decision reward function, and use a multi-objective weighted method to obtain the comprehensive reward signal for evaluating the tour guide effect;

[0163] Receive the tour guide strategy and audience feedback data output from step S42, define a reward function to evaluate the tour guide effectiveness, and comprehensively consider multiple objectives:

[0164] Audience satisfaction is inferred from explicit feedback (exhibit ratings, satisfaction evaluations, and collection / sharing behaviors) and implicit behaviors (moderate dwell time, high path following, and high interactive participation), normalized to the 0-1 range, and weighted to obtain a satisfaction score.

[0165] Knowledge acquisition efficiency was evaluated using alternative metrics, including the proportion of in-depth content viewed, the diversity of exhibits visited (covering different categories and themes), and the relevance of knowledge points (knowledge connections exist between exhibits). The knowledge acquisition index was calculated and divided by the cumulative visit duration.

[0166] Experience comfort; physiological comfort is reflected by fatigue level (maintaining a low level or increasing slowly); psychological comfort is reflected by emotional state (maintaining a positive attitude) and cognitive load (moderate).

[0167] Resource utilization; avoid concentrating visitors on a few popular exhibits (load balancing); avoid excessively circuitous paths (efficient and compact); avoid overuse of high-cost multimedia resources.

[0168] The scores of each objective are weighted and combined to obtain a comprehensive reward signal (0-100). The weights are set according to the exhibition hall management objectives and the characteristics of the audience. The Pareto optimization approach is used to find a balance point among multiple objectives. The obtained comprehensive reward signal serves as the feedback signal for training the reinforcement learning model in step S44.

[0169] S44, construct a reinforcement learning-based tour guide decision-making model and use a deep Q-network algorithm to obtain an optimized tour guide strategy generator;

[0170] The system receives the state space output from step S41, the action space output from step S42, and the reward function output from step S43, and trains the navigation decision model using reinforcement learning. Reinforcement learning learns which actions to take in different states to maximize long-term cumulative rewards through interaction between the agent and the environment.

[0171] The core algorithm employs a Deep Q-Network (DQN). DQN takes a state vector as input and outputs the Q-value (expected cumulative reward) of each candidate action. The network consists of 5 fully connected layers, each with 256 neurons, and uses the ReLU activation function. By learning historical state-action-reward data, a mapping from state to optimal action is established.

[0172] Training employs experience replay and target network techniques to improve stability. Experience replay stores interactive experiences (states, actions, rewards, next state) in a replay buffer, and randomly samples small batches of experience during training to break temporal correlation. The target network is a copy of the Q-network used to calculate the target Q-value, updating parameters at regular intervals to stabilize the training objective.

[0173] Training data comes from a combination of simulated and real-world environments. In the initial deployment phase, virtual audiences and scenarios are generated using a simulator, and the initial strategy is obtained through large-scale offline training in the simulated environment. After deployment, the model is fine-tuned online in a real-world environment, collecting real audience interaction data and feedback to gradually optimize the model and adapt it to the real-world scenario.

[0174] A constraint mechanism is introduced to ensure the safety and acceptability of the strategy. Hard constraints: If fatigue level > 80, a rest session must be recommended; exhibits that are closed or under maintenance should not be recommended; and path planning must avoid obstacles. Soft constraints are implemented through a reward function penalty: recommending exhibits that are significantly inconsistent with the audience's interests is penalized, as is a guided tour pace that is too fast or too slow. When fatigue level reaches 60-80 (moderate fatigue), the system should prioritize recommending short, highly interactive exhibits or suggest rest sessions.

[0175] After training convergence, an optimized tour guide decision-making model is obtained. In practical applications, the model quickly calculates the Q-value of each candidate action based on real-time state input, selects the action with the highest Q-value as the tour guide decision, and generates a complete plan for exhibit recommendation, path planning, and pace control. The model can dynamically adjust decisions according to changes in audience state, achieving adaptive tour guiding. The obtained tour guide strategy plan serves as the input for service execution in step S6.

[0176] S5 receives multi-dimensional state assessment results from the audience and uses context matching and collaborative control methods to obtain personalized exhibition hall atmosphere control strategies.

[0177] This step receives the audience multi-dimensional state assessment results output from step S3 as input, and simultaneously acquires exhibit cultural characteristic information. Addressing the issue of the lack of correlation between exhibition hall atmosphere control and audience state and exhibit connotation, it constructs an intelligent atmosphere control system capable of dynamically matching audience perceptual needs with exhibit cultural characteristics; specifically, it includes the following steps:

[0178] S51, construct a cultural feature database of exhibits, and use knowledge extraction and semantic annotation methods to obtain the emotional tone and atmosphere requirements of the exhibits;

[0179] Cultural characteristics were labeled for all exhibits in the exhibition hall, and a database of cultural characteristics of the exhibits was established. The labeling content includes: historical period (labeling the era to which the exhibit belongs), artistic style (labeling the artistic expression), theme category (labeling the theme of the content), emotional tone (labeling the emotional tendency, such as solemn and dignified, warm and harmonious, exciting and uplifting, calm and restrained, tragic and solemn, relaxed and joyful, etc.), and atmosphere requirements (describing the appropriate range of environmental parameters, including recommended illuminance range, recommended color temperature range, recommended background sound effect type, and recommended volume level).

[0180] The annotation method combines expert knowledge with automated approaches. Initial annotations are performed by cultural relics experts and curators to ensure accuracy. For exhibits with rich textual descriptions, natural language processing technology is used to automatically extract keywords and semantic tags from the explanatory text, and sentiment dictionaries and sentiment analysis algorithms are used to identify sentiment trends.

[0181] The annotation results are stored in the exhibit cultural feature database, with the exhibit identifier as the primary key, and include fields such as exhibit name, location, historical period, artistic style, theme category, emotional tone, recommended illumination, recommended color temperature, recommended sound effect type, and recommended volume. This results in a structured exhibit cultural feature library, which serves as the input for the atmosphere strategy fusion in step S53.

[0182] S52, Construct a matching model between atmosphere parameters and audience state, and use a fuzzy logic control algorithm to obtain the atmosphere control target based on audience perception.

[0183] The audience state assessment results obtained in step S3 are used as input to establish a correlation model between atmosphere parameters and audience state.

[0184] The rules for regulating emotional states are designed as follows: maintain or enhance positive elements when you are happy (bright illumination + warm color temperature + upbeat music); create a tranquil environment when you are calm (soft illumination + neutral color temperature + natural sound effects or low volume); alleviate discomfort when you are tired or negative (appropriately increase illumination + gentle color temperature + soothing music).

[0185] A fatigue-based control strategy is designed: when fatigue is low, support in-depth visits (standard or slightly higher illumination + rich sound effects); when fatigue is moderate, reduce the intensity of stimulation (soft illumination + lower volume); when fatigue is high, guide rest (increased illumination in rest areas + relaxing music).

[0186] Adjustment strategies are designed to address cognitive load: when cognitive load is low, increase information richness (diversified lighting changes + layered sound effects); when cognitive load is moderate, maintain a simple atmosphere (stable lighting + uniform sound effects); when cognitive load is high, simplify stimuli (single soft lighting + silent or minimalist sound effects).

[0187] Fuzzy logic control is employed to handle nonlinear relationships. Fuzzification converts precise numerical values ​​of audience states into fuzzy sets (e.g., fatigue levels are mapped to low, medium, and high). Rule-based reasoning establishes a fuzzy rule base based on expert knowledge, and a fuzzy reasoning engine calculates the activation degree of each rule and aggregates the output. Defuzzification uses the centroid method to calculate the centroid of the output fuzzy set as the final control variable.

[0188] A multi-input multi-output fuzzy control system is established, with fatigue level, emotional level, and cognitive load level as inputs, and illuminance adjustment, color temperature adjustment, and volume adjustment as outputs. Atmosphere control target parameters based on audience perception are obtained, including target illuminance range, target color temperature range, and target volume range, which serve as inputs for atmosphere strategy fusion in step S53.

[0189] S53, by integrating exhibit characteristics and visitor status, and using weighted combination and constraint optimization methods, a comprehensive atmosphere control strategy is obtained;

[0190] The exhibit cultural characteristics output from step S51 and the audience state-based atmosphere control target parameters output from step S52 are received as inputs. Atmosphere control must simultaneously meet the exhibit characteristics requirements and the audience state requirements, and coordinate any possible conflicts.

[0191] Extract the current exhibit atmosphere requirement features from the exhibit cultural feature database, and obtain the atmosphere target parameters based on the audience status from step S52. Compare the two; if the illuminance ranges overlap, select the overlapping range; otherwise, weigh the options.

[0192] Design priority principles: The primary principle is to ensure the protection of exhibits (the upper limit of illumination for cultural relics cannot be exceeded, a hard constraint); the secondary principle is to meet the bottom line of visitor comfort (visitor comfort takes priority when fatigue or discomfort reaches the threshold); the third principle is to optimize the display effect and aesthetic experience.

[0193] A weighted optimization method is used to solve the problem. An optimization objective function is established to maximize the weighted sum of the exhibit display effect score and the audience comfort score, with constraints including exhibit protection requirements and equipment adjustment range. The optimal illuminance, color temperature, and volume parameters are obtained using the Lagrange multiplier method or sequential quadratic programming algorithm.

[0194] Lighting control: Calculate target illuminance and color temperature. Adjust illuminance by controlling the current intensity of LED lamps, and adjust color temperature by controlling the mixing ratio of warm and cool LEDs. Achieve zoned lighting, with main lighting for key exhibits (high illuminance to highlight details) and auxiliary lighting for the background (low illuminance to create atmosphere).

[0195] Sound Effects Control: Based on the emotional tone of the exhibits and the emotional state of the audience, matching background music or environmental sound effects are selected from the sound effects library. A content matching algorithm is used to calculate the similarity between exhibit feature tags and sound effect tags. Playback parameters are adjusted according to the audience's emotional state. Considering acoustic characteristics, the volume is adjusted based on the space size and reverberation time.

[0196] Temperature and humidity control: Based on the exhibit protection standards (temperature 18-24℃, humidity 40%-60%, hard constraints), adjustments are made within permissible ranges to bring the humidity into the comfort zone for visitors. A predictive control strategy is employed, adjusting the temperature and humidity of the target area in advance based on visitor routes and expected arrival times.

[0197] A comprehensive atmosphere control strategy is integrated, including target illuminance distribution, color temperature configuration, dynamic lighting effects, background sound effect selection, volume level, sound effect switching timing, and temperature and humidity settings for each exhibition area. A time-series arrangement is established to ensure a smooth and gradual atmosphere transition (illuminance and color temperature change over 5-10 seconds, with sound effects fading in and out). This time-series atmosphere control strategy serves as input for step S54 (zonal collaborative control) and step S6 (service execution).

[0198] S54, implement zoned collaborative control of the exhibition hall atmosphere, and adopt regional decoupling and boundary coordination methods to obtain a multi-region collaborative control scheme;

[0199] The atmosphere control strategy output from step S53 is received as input. The exhibition hall contains multiple exhibition areas, each with different visitors and varying atmosphere requirements. A zoned control strategy is adopted to improve personalization and reduce the impact of conflicts.

[0200] The atmosphere control zones are divided according to the spatial layout and exhibit distribution. The principles of division are: exhibits within the same zone have similar themes and consistent styles; zones are physically separated or have sufficient spatial distance; and the effects of lighting and sound effects do not interfere with each other. Each zone is equipped with independent lighting, sound, and environmental control equipment.

[0201] The number and status distribution of audience members are statistically analyzed within each area. A clustering algorithm is used to group audience atmosphere needs within each area and identify the dominant demand types. Based on these dominant demands, an atmosphere control strategy is determined for that area to satisfy the needs of the majority of audience members.

[0202] For a small number of visitors whose needs differ significantly from the dominant needs, personalized compensation is provided: wearable personal audio devices are offered (to listen to personalized explanations and music through headphones); seating areas with different illuminance and color temperatures are set up in the rest area; and personalized recommendations are provided through the tour guide system to guide visitors to exhibition areas that better match their state.

[0203] Address the issue of boundary coordination between adjacent areas. Design a boundary transition zone (3-5 meters of movement distance), with ambient parameters using a gradual transition between the two sides. Illumination and color temperature exhibit smooth changes, and sound effects utilize two-way mixing to gradually adjust volume.

[0204] Establish an inter-regional coordination and optimization mechanism. When resource competition arises due to multi-regional regulation needs (such as limited total air conditioning cooling capacity), a priority scheduling algorithm is used to allocate resources. Priorities are determined comprehensively based on the number of visitors in each region, the urgency of the visitors' situation, and the importance of the exhibits.

[0205] A multi-regional coordinated atmosphere control scheme is obtained, including independent control strategies for each region, parameter configurations for boundary transition zones, inter-regional coordination rules, and resource allocation schemes. This atmosphere control strategy serves as the input for the service execution in step S6.

[0206] S6 receives tour guide strategies and atmosphere control strategies, and uses a real-time feedback and dynamic optimization mechanism to obtain continuously optimized service execution results and feedback data.

[0207] This step receives the tour guide strategy plan output from step S4 and the atmosphere control strategy output from step S5 as input, transforms the tour guide strategy and atmosphere control strategy into actual execution actions, continuously monitors audience feedback during execution, dynamically adjusts the service plan, and outputs service execution effect data and audience feedback data; specifically, it includes the following steps:

[0208] S61 transforms the tour guide strategy into execution instructions and uses a multi-channel push method to obtain the actual presentation of the tour guide service;

[0209] The system receives the guided tour strategy scheme generated in step S4 as input and provides services to visitors through various guided tour terminals: mobile applications, smart voice assistants, AR glasses, and exhibition area information screens.

[0210] Mobile application: Pushes exhibit information, navigation paths, and multimedia content. Selects the appropriate content version based on the explanation mode (in-depth / standard / simplified); the navigation map marks the current and target locations, draws recommended routes, and provides turning prompts; provides video and audio explanation resources. Employs a responsive design, adapting to network conditions.

[0211] Intelligent voice assistant: Deployed at fixed locations within the exhibition area or on mobile guide robots. Supports natural language interaction, allowing users to inquire about exhibit information, route navigation, and facility locations. It proactively recommends exhibits based on the guide strategy, automatically providing a brief introduction when visitors approach them. The speech synthesis adjusts tone and speed according to the visitor's emotional state.

[0212] AR glasses: Provide immersive guided tours. They identify exhibits viewed by the audience and overlay a virtual information layer (3D annotations, historical scene reconstructions, interactive prompts) onto the viewer's field of vision. Navigation guidance is presented as virtual arrows or path lines overlaid on the real-world scene. The complexity of the AR content is adjusted according to cognitive load.

[0213] Exhibition area information screens: Deployed next to each exhibit. They automatically activate when a visitor stops, and the content is customized based on the visitor's interests, preferences, and cognitive level. Touch interaction is supported for in-depth browsing.

[0214] All guided tour terminals maintain data synchronization to ensure service continuity and consistency. While providing guided tours, the system records visitor responses (whether they accept recommendations, duration of stay, viewing in-depth content, and interaction) as feedback signals on service effectiveness, outputting guided tour execution data.

[0215] S62 transforms the atmosphere strategy into equipment control commands, and uses gradual regulation and predictive control methods to achieve smooth execution of atmosphere regulation;

[0216] The system receives the atmosphere control strategy generated in step S5 as input and adjusts the exhibition hall environmental parameters through the intelligent control system.

[0217] The lighting system converts target illuminance and color temperature into control parameters for each luminaire. It employs a distributed intelligent lighting system, with each luminaire equipped with an independent dimming and color temperature controller that receives commands via a wireless network. A gradual adjustment method is used, with an adjustment rate of 10%–20% of the current value per second and a transition time of 3–8 seconds. Closed-loop control is implemented via light sensor feedback, stopping adjustment when the error between the actual and target values ​​is less than 5%.

[0218] The audio system performs sound effect switching and volume adjustment. It employs a zoned speaker layout, with each control zone equipped with an independent audio channel. Sound effect switching uses crossfade-in / fade-out technology with a crossover duration of 5-8 seconds. Volume is dynamically compensated based on ambient noise levels.

[0219] Temperature and humidity control system; adjusted via air conditioning and humidification / dehumidification equipment. Employing a predictive control strategy, it anticipates visitor arrival times based on their path and speed, initiating environmental adjustments in advance. Temperature control uses a PID control algorithm, calculating cooling and heating power based on deviation, rate of change of deviation, and accumulated deviation. Humidity control employs segmented control.

[0220] Establish an equipment status monitoring mechanism to monitor the online status of equipment, the delivery of commands, and whether the actual output meets expectations in real time. When a fault or abnormality is detected, trigger an alarm and activate the emergency plan (switch to backup equipment, adjust the output compensation of adjacent equipment, and notify maintenance personnel).

[0221] Record execution data (target parameters, actual parameters, adjustment commands, and equipment responses at each moment) to form an execution log for post-event analysis of control effects, diagnosis of equipment performance, and optimization of control algorithms. Output atmospheric control execution data.

[0222] S63, Establish a real-time monitoring mechanism for service execution and adopt a multi-dimensional indicator evaluation method to obtain a quantitative evaluation of service effectiveness;

[0223] The system receives the guided tour service execution data output in step S61 and the atmosphere control execution data output in step S62 as inputs. During the guided tour and atmosphere control execution, it continuously monitors audience feedback and status changes to evaluate the actual service effect.

[0224] Continue collecting multimodal perception data from the audience, using the method in step S1 to acquire real-time data on visual behavior, physiological signals, interactive behavior, and environmental status. Compare the changes in the audience's state before and after the service execution.

[0225] Evaluation metrics for guided tour effectiveness: Interest response (duration of stay and frequency of interaction; longer stay and more active interaction indicate successful recommendation); Behavioral compliance (similarity between actual movement path and recommended path; following the recommended path indicates that the recommendation is reasonable); Satisfaction feedback (explicit evaluation and implicit behavior).

[0226] Evaluation indicators for the effect of atmosphere control: changes in physiological comfort (comparing fatigue and stress levels before and after adjustment; a decrease in fatigue or a slower rate of increase indicates improved physiological comfort due to improved atmosphere); improvement in emotional state (negative emotions turning into positive emotions or increased emotional intensity indicates that the atmosphere effectively enhances psychological experience); environmental parameter compliance rate (the degree to which actual parameters meet target parameters, calculating the deviation rates of illuminance, color temperature, volume, temperature, and humidity).

[0227] Based on the combined effects of guided tours and atmosphere, a weighted scoring method is used to calculate the overall service quality score (0-100). Each evaluation indicator is assigned a weight according to its importance, and the weighted sum is used to obtain the comprehensive score. The score result is compared with the preset service quality target; if it falls below the target threshold, a service optimization process is triggered. Quantitative evaluation data of service effectiveness is output as input for dynamic optimization and adjustment in step S64.

[0228] S64, implement dynamic optimization and adjustment of services, and adopt a closed-loop feedback control method to obtain a real-time optimized service solution;

[0229] The system receives the quantitative evaluation data of service effectiveness output from step S63 as input, and dynamically adjusts the tour guide and atmosphere strategies when it detects that the service quality is not up to expectations or the audience's state changes.

[0230] Establish a condition-based judgment mechanism to trigger optimization. Trigger conditions include: sudden changes in audience status indicators (rapid increase in fatigue in a short period of time, change of emotion from positive to negative); low values ​​in service effectiveness indicators (consistent compliance below the threshold, decline in satisfaction feedback scores); and abnormal environmental events (sudden surge in crowds in a certain area leading to congestion, equipment malfunction affecting the atmosphere). The dynamic optimization process is initiated when any of the trigger conditions are met.

[0231] Re-evaluate the audience's state, updating interest, fatigue, emotional state, and cognitive load using the method in step S3. Analyze the reasons for changes in state: if it is a natural result of accumulated fatigue, adjust the tour guide strategy to recommend rest or reduce the intensity of the visit; if it is due to mismatched exhibit content, recalculate the recommended exhibit scores and switch to more suitable exhibits; if it is due to an uncomfortable atmosphere, regenerate the atmosphere control strategy and quickly implement the adjustment.

[0232] Dynamic adjustment of tour guide strategy: Re-execute the tour guide decision-making process in step S4, input the updated visitor status and current context, and generate new recommended exhibits and route options. Push the new options through the tour guide terminal, with the push method selected based on urgency.

[0233] Atmosphere Strategy Dynamic Adjustment: The atmosphere matching and optimization process from step S5 is re-executed, new environmental parameter target values ​​are calculated, adjustment instructions are generated and executed. Response speed depends on the parameter type (lighting and sound effects are adjusted within seconds, while temperature and humidity require several minutes for gradual adjustment). The adjustment process maintains a smooth transition to avoid frequent changes that could cause perceptual confusion.

[0234] Establish a mechanism for tracking and verifying the effectiveness of optimization. Within 3-5 minutes of implementing optimization adjustments, intensify monitoring of audience status and service performance indicators to verify whether the optimization measures have effectively improved service quality. If the indicators show significant improvement, confirm the optimization is successful and continue implementing the new strategy; if the indicators do not improve, analyze the reasons for failure and consider reverting to the pre-optimization strategy or trying other solutions.

[0235] Record all optimization and adjustment events and effect data (triggering reasons, adjustment content, execution process, and effect verification results) to form an optimization log. The optimization log is used for offline analysis of service strategy effectiveness, identification of common problem patterns, and training and updating of the decision model. This results in continuously optimized service execution effects, enabling adaptive and intelligent exhibition hall services through a closed-loop feedback mechanism. The service execution data, audience feedback data, and optimization logs accumulated during system operation serve as input for continuous learning and optimization in step S7.

[0236] S7 receives service execution results and feedback data, and uses online learning and model update methods to obtain an intelligent service system with continuously improving performance;

[0237] This step receives the service execution effect data and audience feedback data output from step S6 as input. By collecting data and feedback accumulated during system operation, it continuously optimizes various decision models and control strategies to improve the long-term performance of the system and outputs an updated optimized model. Specifically, it includes the following steps:

[0238] S71, construct a data collection and annotation mechanism, and use active learning and crowdsourced annotation methods to obtain a high-quality training dataset;

[0239] The system receives the service execution data, audience feedback data, and optimization logs output from step S6 as input. During operation, the system continuously accumulates audience perception data, behavioral data, service decision data, and feedback data. This data is stored on a big data platform to establish a data warehouse, employing a layered storage architecture.

[0240] Training the state assessment model requires real-world labels of the audience's state. Multiple methods are used to obtain this label data: at the end of the visit, a short questionnaire is sent via mobile application to inquire about the audience's fatigue levels, interest levels, and satisfaction at different times during the visit; feedback terminals are set up at the exhibition hall exit, where visitors can voluntarily participate in a detailed experience survey; for key research samples, visitors are invited to participate in laboratory labeling, watching their own visit videos and labeling their state of mind at each moment.

[0241] An active learning strategy is adopted to improve annotation efficiency. The system automatically identifies samples with high uncertainty in model prediction, as these samples have the greatest value for model training. High-value samples are prioritized for manual annotation, while common samples that the model can already accurately predict are directly used as pseudo-labels, reducing the burden of manual annotation.

[0242] Training the decision-making model requires reward signals for the decision-making effects. These reward signals are obtained through the service effectiveness evaluation in step S6. Each guide and atmosphere decision is correlated with its subsequent effectiveness evaluation results to form decision-reward sample pairs.

[0243] Establish a data quality control mechanism. Perform consistency checks and anomaly detection on the collected data, removing obviously erroneous data. Conduct multi-person cross-validation on the labeled data; when there are discrepancies in the results of different annotators, determine the final label through expert arbitration. Regularly evaluate the representativeness and balance of the training data; if it is found that the samples for certain categories or scenarios are too few, supplement the corresponding data accordingly. Obtain a high-quality labeled training dataset, including audience status labels, decision effect labels, and service quality labels, providing a data foundation for model updates and serving as input for the incremental learning update in step S72.

[0244] S72, Implement incremental learning and updating of the model, and use transfer learning and parameter regularization methods to obtain a decision model with continuously improved performance;

[0245] The system receives the high-quality labeled training dataset output from step S71 as input and periodically updates and optimizes each decision model. Model updates employ incremental learning, incorporating information from new data while retaining previously learned knowledge, thus avoiding catastrophic forgetting.

[0246] For the audience state assessment model, incremental updates are performed weekly. Newly accumulated labeled data from the current week is mixed with historical training data to form an updated training set. A transfer learning approach is used, freezing the parameters of the model's underlying feature extractors and updating only the parameters of the top-level task-specific prediction head. The top-level parameters are updated using a small learning rate, fine-tuned over several training epochs on new data. L2 regularization is introduced to limit the magnitude of parameter updates and prevent the model from overfitting to new data.

[0247] For the tour guidance decision-making model, the strategy is updated every two weeks. Newly accumulated decision-making experience (state-action-reward sequence) is added to the experience replay buffer, and the model is trained using an online update method of reinforcement learning. A soft update strategy is adopted, where the parameters of the new model are obtained by weighted averaging of the current model parameters and the updated parameters obtained from training (weight coefficients 0.9 and 0.1), allowing the model to evolve gradually. Before the updated model is deployed online, A / B testing is conducted in a simulated environment and a small-scale real-world scenario to compare the performance of the old and new models. The model is only officially deployed if the new model outperforms the old model.

[0248] For the atmosphere matching model, the fuzzy rules and weight parameters are adjusted based on audience feedback. Audience comfort ratings are statistically analyzed for different atmosphere configurations to identify high-performing and low-performing configurations. For high-performing configurations, their weights in the rule base are increased or they are fixed as new rules; for low-performing configurations, their weights are decreased or the corresponding rules are deleted. Genetic algorithms or particle swarm optimization algorithms are used to globally optimize the parameters of the fuzzy system.

[0249] Establish a model version management mechanism. Each model update generates a new version, recording the version number, update time, training data size, performance metrics, and change descriptions. Retain historical model files so that a quick revert to a stable older version is possible if a new version encounters issues. Establish a model performance monitoring dashboard to track the accuracy, response time, and resource consumption of each model in real-world applications in real time.

[0250] Through continuous incremental learning and model updates, the system's decision-making capabilities improve with data accumulation and experience growth, gradually adapting to changes in audience characteristics, updates to exhibit content, and adjustments to the exhibition hall environment, thus achieving the long-term evolution of the intelligent service system. Updated audience status assessment models, tour guidance decision-making models, and atmosphere matching models are output. These optimized models will be applied to subsequent service processes, replacing older models and continuously improving system performance.

[0251] S73 implements system operation monitoring and fault diagnosis, and adopts anomaly detection and root cause analysis methods to obtain highly reliable system operation and maintenance guarantee;

[0252] To ensure the stable operation of the system, a comprehensive monitoring and diagnostic mechanism should be established.

[0253] A system monitoring platform is deployed to collect real-time operational status data from each module. Monitoring content includes the online rate and data quality of the sensor network, CPU and memory utilization of computing nodes, inference latency and throughput of the decision model, response status and control accuracy of execution devices, and bandwidth usage and packet loss rate of network communication. Each monitoring indicator is compared with preset normal range thresholds, and an alarm is triggered when an indicator exceeds the normal range.

[0254] Design a tiered alarm mechanism. Based on the severity of the anomaly, it is divided into three levels: Warning, Error, and Critical. A Warning level indicates a potential problem but the system can still operate normally; an Error level indicates that some functions are affected but core services are available; a Critical level indicates that core functions have failed and urgent handling is required. Different notification methods are used for different alarm levels.

[0255] Perform root cause analysis of the anomaly. When an anomaly is detected, the system automatically traces the timeline and impact chain of the anomaly. By analyzing the temporal changes in monitoring data, the system identifies the earliest location of the anomaly and its possible triggering causes. Once the root cause is located, targeted remedial measures are taken, such as isolating faulty sensors, cleaning up invalid data, and restarting relevant services.

[0256] Establish an automatic recovery mechanism. For common failure modes, pre-design automatic recovery scripts. When specific failure characteristics are detected, the system automatically executes the recovery scripts to attempt self-repair. Common recovery actions include restarting the failed service, switching to a backup device, downgrading to basic function mode, and clearing cache and temporary files. If automatic recovery is successful, the system resumes normal operation and records the failure event; if automatic recovery fails, it escalates to manual handling.

[0257] Regularly conduct system health checks and stress tests. During exhibition hall closures, perform a comprehensive system check, including sensor calibration, device function testing, network connectivity checks, and data backup verification. Simulate high-concurrency scenarios for stress testing to verify the system's stability and performance under extreme loads, identify potential bottlenecks and vulnerabilities, and proactively optimize and harden the system.

[0258] Establish an operations and maintenance (O&M) knowledge base to record the phenomena, causes, handling processes, and preventative measures for all failure events. The O&M knowledge base supports keyword search and similarity matching, allowing O&M personnel to quickly search for similar historical cases and refer to successful handling experiences when new failures occur, thus shortening troubleshooting time. Through systematic O&M support, ensure the high availability and reliability of the exhibition hall's intelligent service system.

[0259] This invention was applied to a large, comprehensive museum, which includes historical artifact exhibition halls, art treasure exhibition halls, and interactive science and technology exhibition halls, receiving approximately 3,000 visitors daily. After system deployment, the visitor experience and the museum's service quality were improved.

[0260] This invention focuses on the application of intelligent tour guiding and atmosphere control in large-scale comprehensive museums. These museums are characterized by diverse exhibits, highly heterogeneous visitor groups, and varying visit durations, thus requiring high levels of personalized service and dynamic adaptability. The system needs to simultaneously serve the in-depth research needs of professional researchers, the cultural experience needs of general visitors, the educational needs of student groups, and the interactive entertainment needs of families, achieving overall optimization of service quality under multi-objective constraints.

[0261] During the initial phase of operation, the system collected visitor data from several typical visitors as a baseline sample. Table 1 shows the status data of five different types of visitors during the initial visit:

[0262] Table 1. Visitor Status Data at the Initial Stage of the Visit

[0263]

[0264] The system generates personalized tour strategies for each visitor based on initial status data. Visitor V001, interested in art treasures and in good spirits, is recommended a deep-visit mode, with a route covering the main works in the art gallery and a professional narrative style for the explanation. Visitor V004, a teenager with an interest focused on interactive technology, is recommended a relaxed interactive mode, prioritizing interactive technology exhibits and using a fun Q&A style for the explanation.

[0265] The system's workflow is illustrated using the visit of visitor V001 as an example. Visitor V001 views an Impressionist oil painting in the art gallery. The system monitors their status in real time: facial expression recognition shows sustained focus and a smile; eye tracking shows their gaze is concentrated on the details of the painting's light and shadow; their viewing time exceeds the average at 4 minutes; and touchscreen interaction records show they viewed the work's background and detailed descriptions of its techniques. The status assessment model calculates an interest score of 85, a fatigue score of 18, an emotional state of pleasure, and a cognitive load of 45. The system determines that the visitor is highly interested in the work and in a good state, maintaining an in-depth explanation mode and recommending other works by the same artist as the next visit target.

[0266] In terms of atmosphere control, the system identified the emotional tone of the Impressionist work as light and cheerful, recommending a warm color temperature of 3000 Kelvin and a medium illuminance of 400 lux to highlight the colors of the painting. Soft classical music was chosen as the background sound to create an artistic atmosphere. Combining this with the pleasant emotional state of viewer V001, the system confirmed a good match between the current atmosphere and the viewer's state, maintaining the atmosphere parameters unchanged. When viewer V001 moved to the next artwork, the system pre-adjusted the lighting in that area, using a 5-second gradual transition to the new color temperature and illuminance to ensure a smooth environmental change for the viewer.

[0267] At the 60-minute mark of the tour, the system detected changes in the physiological signals of visitor V001: heart rate increased from an initial 72 beats / min to 78 beats / min, and the low-frequency to high-frequency ratio of heart rate variability increased, indicating enhanced sympathetic nerve activity; facial expression recognition showed a decrease in the frequency of smiling and a slight decrease in the temperature around the eyes; interaction frequency decreased from 3 times per 10 minutes to 1 time per 10 minutes. The status assessment model updated and calculated that the fatigue score increased to 55 and the cognitive load increased to 62. The system determined that the visitor was experiencing mild fatigue and high cognitive load, triggering a dynamic adjustment of the tour guide strategy.

[0268] The system generated optimization suggestions: Change the explanation mode from in-depth mode to standard mode, reduce technical jargon and detailed background information, and highlight the core highlights of the works; recommend that visitors take a short break in the exhibition hall's rest area, or suggest visiting smaller sculptures with shorter durations and lighter content as a transition. Simultaneously, adjust the atmosphere strategy: increase the illumination in the rest area to 600 lux to create a refreshing environment, play light music to help visitors recover, and appropriately lower the temperature by 1 degree Celsius to enhance comfort. Visitor V001 accepted the system's rest suggestions. After staying in the rest area for 8 minutes, their fatigue score decreased to 42, their emotional state returned to pleasant, and they continued their visit.

[0269] At the end of the visit, the system recorded complete visit data for visitor V001: total visit duration 105 minutes, 27 exhibits viewed, 12 exhibits viewed in depth, 35 interactive operations, average interest rating 78, final fatigue rating 68, and overall satisfaction rating 92. Upon leaving the museum, visitor V001 submitted feedback via a mobile application, stating that the guided tour recommendations perfectly matched their interests, the atmosphere enhanced the artistic experience, the rest stop reminders were thoughtful, and overall, they were very satisfied with the experience. The system added visitor V001's data and feedback to the training dataset for subsequent model optimization.

[0270] Through the implementation of this invention, the museum achieved an intelligent upgrade of its exhibition hall services. The average visitor satisfaction rating increased from 75 before implementation to 88 after implementation, the average visit duration increased from 65 minutes to 95 minutes, and the willingness to revisit increased from 42 to 67. The system can serve hundreds of visitors simultaneously, providing personalized guidance and atmosphere control based on each visitor's real-time status, thus improving the quality of the visitor experience and the museum's service capabilities, and providing an innovative technical solution for the intelligent construction of exhibition halls.

[0271] This invention solves the technical problems of incomplete audience state perception, lack of adaptability of guided tour services, and disconnect between atmosphere control and audience needs in the prior art by constructing a multimodal perception system, establishing a precise audience state assessment model, designing an adaptive guided tour decision-making mechanism, implementing a contextualized atmosphere control strategy, and establishing a collaborative optimization system for guided tours and atmosphere. It realizes intelligent, personalized, and precise exhibition hall services and has certain technological progress and application value.

[0272] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A perception-based adaptive tour guide and atmosphere control method for exhibition halls, characterized in that, Includes the following steps: S1: Collect raw perception data of visitors in the exhibition hall, and use distributed sensor networks and data preprocessing methods to obtain a multi-source heterogeneous perception dataset; S2 receives a multi-source heterogeneous sensing dataset and uses feature extraction and adaptive fusion methods to obtain a comprehensive state feature vector; S3 receives the comprehensive state feature vector, and uses a multi-task learning model and time-series prediction method to obtain the audience's multi-dimensional state evaluation results and evolution trend prediction. S4 receives the multi-dimensional state evaluation results of the audience and uses a multi-objective optimization method to obtain a personalized adaptive tour guide strategy. S5 receives multi-dimensional state assessment results from the audience and uses context matching and collaborative control methods to obtain personalized exhibition hall atmosphere control strategies. S6 receives tour guide strategies and atmosphere control strategies, and uses a real-time feedback and dynamic optimization mechanism to obtain continuously optimized service execution results and feedback data. S7 receives service execution results and feedback data, and uses online learning and model update methods to obtain an intelligent service system with continuously improving performance.

2. The perception-based adaptive tour guide and atmosphere control method for exhibition halls according to claim 1, characterized in that, S1 includes: Deploy distributed visual perception devices in the exhibition hall, adopt a multi-angle coverage strategy to collect audience behavior image data, perform quality detection on the collected data and mark quality labels to obtain a visual behavior image dataset; Deploy non-contact physiological sensing devices to collect respiratory signals, heartbeat signals, and facial temperature data through millimeter-wave radar sensors and thermal imaging sensors, and fuse wearable device data for cross-validation to obtain a physiological signal dataset; Deploy intelligent interactive terminal devices to record touch screen interaction events, voice query records, and mobile application usage records. By linking the interaction records of different terminals through the unique identifier of the audience, an interactive behavior dataset is obtained. Deploy environmental monitoring sensors in the exhibition hall to collect data on temperature, humidity, illuminance, color temperature, noise, and crowd density to obtain an environmental status dataset; The data is formatted, cleaned, synchronized, and quality-assessed from multiple sources and heterogeneous data. Based on audience identifiers and timestamps, the multimodal data is organized into associated data packets to obtain a multi-source heterogeneous sensing dataset.

3. The perception-based adaptive tour guide and atmosphere control method for exhibition halls according to claim 1, characterized in that, S2 includes: A pre-trained residual network is used to extract facial expression features, which are then input into an emotion classifier to output the probability distribution of emotions. A key point detection algorithm is used to identify body postures, and the emotional state, behavioral patterns, and attention distribution features are concatenated to obtain a visual emotion and behavioral feature vector. Time-domain feature calculation and frequency-domain analysis were performed on the heart rate signal to extract respiratory features and facial temperature features. The physiological features were then input into the physiological calculation model to calculate the fatigue index and stress level index, thus obtaining the physiological signal feature vector. The number of times and duration of visitors browsing various exhibits are statistically analyzed to calculate preference weights. Sequence pattern mining algorithms are used to identify access patterns, and graph embedding methods are used to construct interest graphs to obtain interactive behavior feature vectors. The environmental state feature vector was obtained by using thermal comfort model, visual comfort score, acoustic environment quality score and space crowding index; An adaptive weighted fusion method based on attention mechanism is adopted. The fusion weight is dynamically calculated according to the data quality scores of each modality. Feature mapping is performed through a feature compression network with an autoencoder structure. Missing data is imputed with historical statistical features or similar audience features to obtain a comprehensive state feature vector.

4. The perception-based adaptive tour guide and atmosphere control method for exhibition halls according to claim 1, characterized in that, S3 includes: A multi-task learning model is constructed, which adopts an architecture of shared underlying feature extractor and multiple task-specific prediction heads. Each prediction head outputs evaluation values ​​of interest, fatigue, emotional state and cognitive load. The model is trained using a multi-task loss function to obtain the evaluation results of the audience's current state. The Monte Carlo Dropout method is used to quantify the prediction uncertainty of the model. Multiple forward propagations are performed on the same input, and the mean and standard deviation of the predicted values ​​are calculated. The standard deviation is normalized to obtain the confidence score, and the prediction confidence of each state index is obtained. Based on the historical state sequence of the audience, a long short-term memory network is used for time series prediction to predict the values ​​and rates of change of each state indicator within the future time window. The prediction results are tested for rationality and monotonicity constraints to obtain the multidimensional state assessment results and evolution trend prediction of the audience.

5. The perception-based adaptive tour guide and atmosphere control method for exhibition halls according to claim 1, characterized in that, S4 includes: By integrating audience state characteristics, exhibit attribute characteristics, spatial layout characteristics, and time constraint characteristics, a guided tour decision state space is constructed, resulting in a standardized guided tour decision state representation. A hierarchical decision-making architecture is adopted, with high-level decision-making determining recommended exhibits, mid-level decision-making using path planning algorithms to calculate the optimal path, and low-level decision-making dynamically adjusting the explanation mode and content detail based on cognitive load and fatigue, thus obtaining action plans for exhibit selection, path planning, and pace control. Define a reward function to evaluate the effectiveness of guided tours, taking into account audience satisfaction, knowledge acquisition efficiency, experience comfort, and resource utilization, and obtain a comprehensive reward signal by weighting and combining the scores of each objective; A deep Q-network algorithm is used to train the tour guide decision model. Experience replay and target network techniques are used to improve training stability. Hard and soft constraint mechanisms are introduced to ensure policy security, resulting in a personalized adaptive tour guide strategy.

6. The perception-based adaptive tour guide and atmosphere control method for exhibition halls according to claim 1, characterized in that, S5 includes: Cultural characteristics are labeled on the exhibits in the exhibition hall, including historical period, artistic style, theme category, emotional tone and atmosphere requirements, to obtain a cultural characteristic database of the exhibits; Atmosphere control rules were designed based on emotional state, fatigue level, and cognitive load. Fuzzy logic control method was used to handle nonlinear relationships, and a multi-input multi-output fuzzy control system was established to obtain the target parameters for atmosphere control based on audience perception. By integrating the cultural characteristics of the exhibits and the state of the audience, the optimal illuminance, color temperature and volume parameters are solved using a weighted optimization method, resulting in a comprehensive atmosphere control strategy. Based on the spatial layout, atmosphere control areas are divided. The audience needs in each area are clustered and analyzed to determine the dominant needs. Boundary transition areas are designed to achieve a gradual transition of parameters between adjacent areas, resulting in a personalized exhibition hall atmosphere control strategy.

7. The perception-based adaptive tour guide and atmosphere control method for exhibition halls according to claim 1, characterized in that, S6 includes: Guided tour services are pushed to visitors through mobile applications, smart voice assistants, AR glasses, and exhibition area information screens. Each guided tour terminal keeps data synchronized and records visitor response behavior as a feedback signal of service effectiveness. The atmosphere control strategy is converted into equipment control commands for lighting, sound, and temperature and humidity control systems. Gradual adjustment is used to achieve smooth parameter transitions, and an equipment status monitoring mechanism is established to detect faults and activate emergency plans.

8. The perception-based adaptive tour guide and atmosphere control method for exhibition halls according to claim 1, characterized in that, S6 further includes: Establish a real-time monitoring mechanism for service execution, adopt evaluation indicators for tour guide effectiveness and atmosphere control effectiveness, calculate the overall service quality score, and obtain a quantitative evaluation of service effectiveness; Establish a condition judgment mechanism to trigger optimization. When the audience status index changes suddenly, the service effect index falls below the preset threshold, or an abnormal environmental event occurs, the dynamic optimization process is initiated, the audience status assessment is re-executed, and a new tour guide strategy and atmosphere strategy are generated. Establish an optimization effect tracking and verification mechanism. After implementing optimization adjustments, strengthen the monitoring of audience status and service effect indicators, verify the effectiveness of optimization measures, record optimization adjustment events to form an optimization log, and obtain continuous optimization service execution effects and feedback data.

9. The perception-based adaptive tour guide and atmosphere control method for exhibition halls according to claim 1, characterized in that, S7 includes: A data collection and annotation mechanism was constructed, and an active learning strategy was adopted to prioritize manual annotation of samples with high model prediction uncertainty. A data quality control mechanism was established to perform consistency checks and anomaly detection to obtain a high-quality training dataset. The decision model is updated using incremental learning, and the transfer learning method is used to freeze the low-level parameters of the model and only update the high-level parameters. Regularization constraints are introduced to prevent overfitting, and a model version management mechanism is established to obtain a decision model with continuously improving performance. Establish a system operation monitoring platform, design a hierarchical alarm mechanism, implement root cause analysis of anomalies, and establish an automatic recovery mechanism to obtain an intelligent service system with continuously improving performance.

10. A perception-based adaptive tour guide and atmosphere control system for exhibition halls, used to execute the steps of the perception-based adaptive tour guide and atmosphere control method for exhibition halls as described in any one of claims 1-9, characterized in that, include: The multi-source sensing data acquisition module is used to collect raw sensing data from visitors in the exhibition hall. It uses a distributed sensor network and data preprocessing methods to obtain a multi-source heterogeneous sensing dataset. The feature extraction and fusion module is used to receive multi-source heterogeneous sensing datasets and obtain a comprehensive state feature vector by using feature extraction and adaptive fusion methods. The state assessment and prediction module is used to receive the comprehensive state feature vector, and use a multi-task learning model and time series prediction method to obtain the multi-dimensional state assessment results and evolution trend prediction of the audience. The adaptive tour guide decision-making module receives multi-dimensional status evaluation results from visitors and uses a multi-objective optimization method to obtain a personalized adaptive tour guide strategy. The atmosphere control strategy module is used to receive multi-dimensional state evaluation results of the audience and use context matching and collaborative control methods to obtain personalized exhibition hall atmosphere control strategies. The service execution and feedback module is used to receive tour guide strategy plans and atmosphere control strategies, and adopts a real-time feedback and dynamic optimization mechanism to obtain continuously optimized service execution results and feedback data. The online learning and update module is used to receive service execution results and feedback data, and adopts online learning and model update methods to obtain an intelligent service system with continuously improving performance.

Images