Digital exhibition hall multimedia equipment intelligent interaction control method and system

Abstract

The invention relates to the technical field of multimedia interaction, in particular to an intelligent interaction control method and system for digital exhibition hall multimedia equipment, and the method comprises the following steps: S1, carrying out the arrangement of a sensor, and carrying out the intelligent sensing and data collection; s2, generating an intelligent interaction strategy; s3, equipment cooperative control and execution; and S4, performing real-time feedback and system optimization. According to the method, user behaviors and environment data are collected in real time through the multi-modal sensor, user requirements are accurately speculated in combination with a deep learning algorithm, the optimal interaction strategy and task priority are generated, individuation and interaction efficiency of user experience are remarkably improved, multi-device cooperation is optimized through distributed task scheduling, and user experience is improved. According to the method, display content and environment parameters are dynamically adjusted, self-optimization of the system is realized by utilizing user feedback, real-time performance, smoothness and reliability of interactive experience of the exhibition hall are ensured, rapid fault response and accurate maintenance demand prediction are realized through anomaly detection and equipment log analysis, and the stability and sustainability of operation of the exhibition hall system are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of multimedia interaction technology, and in particular to an intelligent interactive control method and system for multimedia equipment in a digital exhibition hall. Background Art

[0002] As an important place for cultural communication, technology display and commercial promotion, digital exhibition halls have been widely used in museums, science and technology museums, exhibition centers and brand showrooms. Through the combination of multimedia equipment and intelligent interactive technology, digital exhibition halls can provide visitors with immersive content experience and efficient interactive methods. However, with the expansion of exhibition halls and the increase in technical complexity, traditional exhibition hall management and interactive methods can no longer meet the growing demand for personalized experience and intelligent management.

[0003] Traditional interactive control systems for digital exhibition halls mainly rely on preset interactive modes and manually operated equipment management. This approach has major limitations: on the one hand, it lacks the ability to perceive and respond to user behavior and the environment in real time, and cannot provide an accurate and personalized interactive experience; on the other hand, the collaboration efficiency between multiple devices is low, and equipment management relies on manual maintenance, which can easily lead to untimely troubleshooting and affect the overall operation of the exhibition hall. At the same time, the display content and interactive strategies usually lack a dynamic optimization mechanism, making it difficult to make real-time adjustments based on user needs or scene changes, reducing the attractiveness and operational efficiency of the exhibition hall. In order to solve the above problems, we provide a method and system for intelligent interactive control of multimedia equipment in a digital exhibition hall. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the first object of the present invention is to provide a method and system for intelligent interactive control of multimedia equipment in a digital exhibition hall to solve the problems in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: A method and system for intelligent interactive control of multimedia equipment in a digital exhibition hall, comprising the following steps: S1, sensor layout intelligent perception and data collection; S2, intelligent interaction strategy generation; S3, equipment collaborative control and execution; S4. Real-time feedback and system optimization.

[0006] The present invention is further configured as follows: in the step S1, sensor arrangement intelligent perception and data collection: S1.1. Multimodal sensors are deployed in the exhibition hall, including FLIR Lepton 3.5 high-precision infrared sensor (resolution 160x120), 360-degree surround view camera Raspberry Pi HQ Camera panoramic camera, ReSpeaker 4-MicArray microphone array and pressure-sensitive Tekscan MatScan system floor. These sensors are distributed at the entrance of the exhibition hall, key display areas and crowded areas to collect users' thermal imaging, position changes, audio signals and trampling behavior data. The experimental environment needs to be equipped with a Wi-Fi 6 IoT network with sufficient bandwidth to ensure that sensor data is uploaded to the edge computing node in real time; S1.2. Use the YOLOv8 deep learning algorithm combined with high-performance hardware graphics cards to perform real-time analysis on the video stream captured by the camera, identify the user's posture, gesture, and gait characteristics, and use the microphone array and speech recognition engine to identify the voice commands or specific keywords issued by the user. The floor sensor data is used to determine the user's stay area and the distribution of the number of people. The experimental environment needs to include a variety of OpenPose pre-trained model user scenario data sets to facilitate rapid adjustment of the system's behavior monitoring sensitivity; S1.3. Use SCD40 multifunctional environmental sensors for CO2 concentration and temperature and humidity detection to be arranged at different locations in the exhibition hall. By adjusting the detection frequency of the environmental sensors (once per minute), dynamically changing data including light intensity, temperature, humidity and noise level can be captured. The experimental environment needs to be fixed with calibrated sensors, and a climate simulation box should be used to verify the accuracy of environmental data collection. S1.4. After data collection, TensorFlow Lite is run on the edge computing device Raspberry Pi 4B for real-time data compression and denoising. Low-frequency data of temperature, humidity and light are sampled at intervals of 5 seconds, while high-frequency behavior data are sampled every 0.1 seconds. The cleaned data is uploaded to the cloud server (AWS IoT Core) via the MQTT protocol for subsequent processing. In the experiment, a network simulation tool (Packet Tracer) needs to be set up to evaluate data transmission delay and network stability.

[0007] The present invention is further configured as follows: in the step S2, generating the intelligent interaction strategy: S2.1. Integrate user behavior (gaze direction, gesture), voice input, and environmental parameters through a deep learning model (Transformer), and use edge computing hardware for model inference. The experimental environment needs to collect AV-HuBERT multimodal datasets containing action, voice, and environmental parameters for training and evaluating the effect of the fusion model. S2.2. Use the BERT-based natural language processing model and behavior analysis algorithm to infer the user's current needs (content switching, environment adjustment). By simulating real exhibition hall scenes, test the accuracy of the inference model under different user behavior combinations (the goal is higher than 90%). The experiment needs to use multimodal experimental data of speech-action matching (MOSEI dataset); S2.3. Based on the inferred intention, the predefined display content module (3D model) is called, and the exhibition hall interactive path is generated through the dynamic programming algorithm (A* algorithm). The hardware needs to be equipped with high-performance display equipment (4K projector and LED wall). The user scenario simulation software (Unreal Engine) is used in the experiment to test the running efficiency of dynamic programming and the delay time of content loading; S2.4. Prioritize multiple tasks through the Sugeno-type fuzzy inference system algorithm, give priority to user interaction instructions, and adjust tasks in the secondary execution environment. The experiment needs to simulate multiple users interacting at the same time to test the correctness of priority sorting and response time (the goal is within 100 milliseconds).

[0008] The present invention is further configured as follows: in the step S3, in the coordinated control and execution of the devices: S3.1. Use the unified ZigBee device management protocol to synchronize the status data of multiple devices in the exhibition hall (projector brightness, audio volume) in real time, integrate the device status through the edge gateway (AWS Greengrass) and upload it to the cloud. In the experiment, the projector Epson EH-TW7100 is used to simulate the exhibition hall equipment to test the synchronization delay and consistency; S3.2. Through the distributed scheduling method task scheduling algorithm based on deep reinforcement learning, interactive tasks are assigned to specific devices and the efficiency of task allocation is optimized. The scheduling system needs to run in the local area network and complete task transmission through the gateway (MQTT Broker). In the experiment, a small exhibition hall scene is built to simulate different tasks (switching screens, adjusting sound effects) and test the delay of task distribution (the target is less than 200 milliseconds); S3.3. Use DMX512 lighting control protocol and AV-over-IP technology to coordinate multiple devices and adjust lighting color, sound effects and display screen content simultaneously. The equipment includes digital audio (JBL EON ONE MK2) and lighting system (Chauvet DJ SlimPAR Pro). The experiment needs to use Lightjams lighting arrangement software to verify the smoothness of dynamic scene switching. S3.4. Monitor the equipment operating status (temperature, workload) in real time and use the isolated forest anomaly detection algorithm to detect faults. Ensure the operation of the exhibition hall through the backup equipment switching mechanism (dual projection system). Use the network injection tool to simulate equipment failure in the experiment and test the response speed of system fault switching (the target is less than 1 second).

[0009] The present invention is further configured as follows: in the step S4, real-time feedback and system optimization: S4.1. Use the Elo 22-inch touch screen and Intel RealSense D415 emotion recognition camera in the display area to record users’ feedback on the display content (satisfaction rating, emotional changes) in real time. Set up a real user test group in the experiment to analyze the comprehensiveness and real-time nature of feedback collection. S4.2. Combine the user stay time and content switching frequency indicators, and use the Tableau interactive data analysis tool to evaluate the interactive effect in real time. The experiment needs to preset multiple display content scenarios (static and dynamic content combination) and evaluate the changes in user engagement; S4.3. Based on real-time feedback data, use the PPO deep reinforcement learning algorithm to update the weights of the user intention inference and strategy generation model, and use the Ray RLlib automated training framework to achieve continuous optimization of the model. The convergence speed of the adaptive optimization needs to be verified in the experiment (the target convergence time is less than 1000 iterations). S4.4. Use the ELK Stack log analysis tool to record the equipment operation log, including the number of failures, working hours, and data throughput indicators. Combined with the machine learning algorithm of the time series prediction algorithm ARIMA, predict the maintenance needs of the equipment and generate early warnings. In the experiment, the collected log data is replayed and analyzed to evaluate the accuracy of the maintenance recommendations.

[0010] The present invention is further configured to include: a human-machine symbiotic perception network module for collecting data and guiding user behavior through a dynamically variable perception network to establish a feedback perception control system, a scene-guided interaction strategy module for shaping user behavior based on user intention generation strategies to guide them to experience different exhibition hall theme contents, an autonomous adaptive device linkage module for passive response strategies and autonomously optimizing linkage methods by learning user interaction habits and exhibition hall environment, and a data behavior bidirectional feedback module for breaking through one-way feedback, optimizing content design in real time using user behavior, and designing new interaction methods based on generative algorithms; The human-machine symbiotic perception network module includes a dynamic perception layout adjustment module for dynamically expanding the strategy for autonomously adjusting the perception range, a behavior trajectory mapping prediction module for generating a user behavior trajectory heat map in combination with real-time perception and predicting the next user behavior path through a time series prediction model, an environment-driven perception optimization module for the environment perception device to automatically adjust the sampling frequency according to environmental changes, and a virtual reality joint perception module for introducing an AR device to record the user's gaze point and gestures to achieve synchronous perception of virtual and real information; The scene-guided interaction strategy module includes a theme scene preset switching module for realizing seamless switching of multiple theme scenes through a dynamic projector and a lighting system, a reverse reasoning intention recognition module for inferring that a user may have ignored a point of interest based on an area or function not triggered by the user, a multi-path interaction planning module for simulating the optimal interaction path between the user and the exhibition hall using an interaction path planning method based on game theory, and an emotion-driven content adjustment module for adjusting content display in real time using emotion recognition technology; The autonomous adaptive equipment linkage module includes an equipment behavior learning module for using federated learning technology to allow different exhibition hall equipment to share experience data, a perception-driven scene adaptation module for autonomously selecting adaptive actions based on perception data, an equipment active task negotiation module for introducing negotiation protocols between equipment to form a "equipment symbiotic network", and a panoramic content dynamic reconstruction module for autonomously generating content display strategies through real-time monitoring of venue equipment layout; The data-behavior bidirectional feedback module includes a behavior feedback data correction module for adjusting the priority of the perception and strategy generation model according to the user's actual operating habits, a generation experience content design module for using a generative adversarial network to generate new display content in real time, a user feedback reverse guidance module for allowing users to participate in the exhibition hall interaction design through voice feedback or touch operation, and an interaction data precipitation evolution module for regularly summarizing the exhibition hall's interaction effect data to form a data-driven update strategy.

[0011] The present invention is further configured as follows: the behavior feedback data correction module includes a user behavior sampling adjustment module for dynamically adjusting the behavior sampling time interval and accuracy by real-time monitoring the user's operation frequency and type, a user preference feature extraction module for analyzing the user's behavior trajectory and the stay area to extract personalized preference features, an adaptive feedback determination module for judging whether the current feedback data can accurately reflect the user's needs according to the user's behavior changes, a dynamic model weight update module for dynamically adjusting the prediction model weight according to the user's actual behavior data, a behavior anomaly determination module for detecting abnormal behaviors in user operations to judge whether the system feedback strategy needs to be adjusted, a user intention error correction module for correcting the prediction error and updating the model by comparing the user's intention inferred by the system with the actual interactive behavior, a content presentation real-time optimization module for adjusting the content presentation form according to the user's behavior, and a user behavior data precipitation module for archiving the user's long-term behavior data to extract key patterns for overall optimization of the exhibition hall content and interaction methods; The adaptive feedback determination module includes a behavior feedback correlation calculation module for analyzing the correlation between user behavior and system feedback to further determine whether the feedback is effective, and an interaction response analysis module for detecting the timeliness of feedback response to determine whether the interaction affects the user experience due to delay; The behavior anomaly determination module includes a multi-level behavior monitoring module for refining the monitoring scope of user behavior, subdividing behavior anomalies into slight deviations, significant deviations and extreme deviations, and a user behavior environment association module for combining environmental data to determine whether the behavior anomaly is affected by external factors.

[0012] The present invention is further configured as follows: the user behavior sampling and adjustment module dynamically collects behavior data and inputs the optimized data into the user preference feature extraction module for refining personalized preferences; the user preference feature extraction module analyzes user behavior characteristics and passes the results to the adaptive feedback determination module to determine whether the feedback data accurately reflects user needs; the adaptive feedback determination module provides an optimization signal according to feedback accuracy and passes it to the dynamic model weight update module to adjust the prediction model weight; the dynamic model weight update module optimizes the prediction model weight and provides the optimized model to the behavior anomaly determination module to more accurately identify the user's abnormal behavior; the behavior anomaly determination module identifies the user's abnormal behavior and passes the abnormal result to the user intention error correction module to correct the intention prediction deviation; the user intention error correction module corrects the prediction deviation and inputs the calibrated user intention into the content presentation real-time optimization module to adjust the display form; the content presentation real-time optimization module dynamically adjusts the content display form and stores the display adjustment record into the user behavior data precipitation module to extract the long-term optimization mode.

[0013] The present invention is further configured as follows: the dynamic perception layout adjustment module expands the perception range and provides data to the behavior trajectory mapping prediction module to generate a user behavior trajectory heat map; the behavior trajectory mapping prediction module predicts the user behavior path and transmits the result to the environment-driven perception optimization module for adjusting the sampling frequency of the perception device; the environment-driven perception optimization module dynamically adjusts the sampling frequency and provides the virtual reality joint perception module with environmental perception data to synchronize virtual and real information; the virtual reality joint perception module combines the user's gaze point and gesture data and transmits the virtual and real information back to the dynamic perception layout adjustment module to optimize the perception layout; The theme scene preset switching module realizes scene switching through dynamic projection and passes the switching data to the reverse reasoning intention recognition module to infer the points of interest that the user may overlook; the reverse reasoning intention recognition module identifies the area not triggered by the user and passes the point of interest data to the multi-path interaction planning module to optimize the interaction path; the multi-path interaction planning module simulates the interaction path between the user and the exhibition hall and passes the path optimization result to the emotion-driven content adjustment module to adjust the content display; the emotion-driven content adjustment module adjusts the display content in real time and transmits the adjusted emotional response data back to the theme scene preset switching module to optimize the scene switching logic.

[0014] The present invention is further configured as follows: the device behavior learning module shares experience data through federated learning and passes the optimized device behavior to the perception-driven scene adaptation module to guide device action selection; the perception-driven scene adaptation module selects adaptive actions and passes the selection results to the device active task negotiation module to optimize task allocation between devices; the device active task negotiation module allocates tasks based on the negotiation protocol and provides the task allocation results to the panoramic content dynamic reconstruction module to generate a content display strategy; The panoramic content dynamic reconstruction module optimizes the content display according to the device layout and transmits the layout adjustment data back to the device behavior learning module to improve the device learning efficiency; the behavior feedback data correction module adjusts the model priority according to the user operation and provides the optimized priority data to the generated experience content design module to generate new display content in real time; the generated experience content design module generates new display content and passes the generated data to the user feedback reverse guidance module for users to participate in content design; the user feedback reverse guidance module optimizes the content design according to user feedback and passes the feedback results to the interactive data precipitation evolution module for forming an update strategy; the interactive data precipitation evolution module summarizes the interactive data and generates an update strategy and provides the update strategy to the behavior feedback data correction module to further optimize the model. Beneficial Effects

[0015] Compared with the known public technology, the technical solution provided by the present invention has the following beneficial effects: 1. The present invention uses multimodal sensors to collect user behavior and environmental data in real time, combines deep learning algorithms to accurately predict user needs, generates optimal interaction strategies and task priorities, significantly improves the personalization of user experience and interaction efficiency, optimizes multi-device collaboration through distributed task scheduling, dynamically adjusts display content and environmental parameters, and uses user feedback to achieve system self-optimization, ensuring the real-time, smoothness and reliability of the exhibition hall's interactive experience. Through anomaly detection and equipment log analysis, it achieves rapid response to faults and accurate maintenance demand prediction, greatly improving the stability and sustainability of the exhibition hall system operation; 2. The present invention realizes accurate perception and personalized guidance of user behavior, dynamically adjusts display content and optimizes interaction paths through the coordinated work of four major modules: human-machine symbiotic perception network, scenario-guided interaction strategy, autonomous adaptive device linkage, and data-behavior two-way feedback. It enhances the immersion and attractiveness of the exhibition hall's content display, and multi-device collaboration ensures the efficiency of task execution and the smoothness of display scene switching. The two-way feedback mechanism combined with the generative adversarial network further optimizes the content design and enhances the interactive innovation ability of the exhibition hall. On the whole, the system significantly improves the intelligent management level and user satisfaction of the exhibition hall through continuous strategy updates and fault response capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A method flow diagram of a method and system for intelligent interactive control of multimedia equipment in a digital exhibition hall according to the present invention; Figure 2 A schematic diagram of system modules of a method and system for intelligent interactive control of multimedia equipment in a digital exhibition hall according to the present invention; Figure 3 The present invention is a schematic diagram of a behavior feedback data correction module of a digital exhibition hall multimedia equipment intelligent interactive control method and system. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] The present invention will be further described below in conjunction with the embodiments. Example 1

[0019] like Figure 1 As shown, the present invention provides a solution: a method for intelligent interactive control of multimedia equipment in a digital exhibition hall, comprising the following steps: S1, sensor layout intelligent perception and data collection; S2, intelligent interaction strategy generation; S3, equipment collaborative control and execution; S4, real-time feedback and system optimization; In the step S1, sensor arrangement intelligent perception and data collection: S1.1. Multimodal sensors are deployed in the exhibition hall, including FLIR Lepton 3.5 high-precision infrared sensor (resolution 160x120), 360-degree surround view camera Raspberry Pi HQ Camera panoramic camera, ReSpeaker 4-MicArray microphone array and pressure-sensitive Tekscan MatScan system floor. These sensors are distributed at the entrance of the exhibition hall, key display areas and crowded areas to collect users' thermal imaging, position changes, audio signals and trampling behavior data. The experimental environment needs to be equipped with a Wi-Fi 6 IoT network with sufficient bandwidth to ensure that sensor data is uploaded to the edge computing node in real time; S1.2. Use the YOLOv8 deep learning algorithm combined with high-performance hardware graphics cards to perform real-time analysis on the video stream captured by the camera, identify the user's posture, gesture, and gait characteristics, and use the microphone array and speech recognition engine to identify the voice commands or specific keywords issued by the user. The floor sensor data is used to determine the user's stay area and the distribution of the number of people. The experimental environment needs to include a variety of OpenPose pre-trained model user scenario data sets to facilitate rapid adjustment of the system's behavior monitoring sensitivity; S1.3. Use SCD40 multifunctional environmental sensors for CO2 concentration and temperature and humidity detection to be arranged at different locations in the exhibition hall. By adjusting the detection frequency of the environmental sensors (once per minute), dynamically changing data including light intensity, temperature, humidity and noise level can be captured. The experimental environment needs to be fixed with calibrated sensors, and a climate simulation box should be used to verify the accuracy of environmental data collection. S1.4. After data collection, the edge computing device Raspberry Pi 4B runs TensorFlow Lite for real-time data compression and denoising. The low-frequency data of temperature, humidity and light are sampled at intervals of 5 seconds, while the high-frequency behavior data are sampled every 0.1 seconds. The cleaned data is uploaded to the cloud server (AWS IoT Core) through the MQTT protocol for subsequent processing. In the experiment, a network simulation tool (Packet Tracer) needs to be set up to evaluate the data transmission delay and network stability. In the step S2, generating the intelligent interaction strategy: S2.1. Integrate user behavior (gaze direction, gesture), voice input, and environmental parameters through a deep learning model (Transformer), and use edge computing hardware for model inference. The experimental environment needs to collect AV-HuBERT multimodal datasets containing action, voice, and environmental parameters for training and evaluating the effect of the fusion model. S2.2. Use the BERT-based natural language processing model and behavior analysis algorithm to infer the user's current needs (content switching, environment adjustment). By simulating real exhibition hall scenes, test the accuracy of the inference model under different user behavior combinations (the goal is higher than 90%). The experiment needs to use multimodal experimental data of speech-action matching (MOSEI dataset); S2.3. Based on the inferred intention, the predefined display content module (3D model) is called, and the exhibition hall interactive path is generated through the dynamic programming algorithm (A* algorithm). The hardware needs to be equipped with high-performance display equipment (4K projector and LED wall). The user scenario simulation software (Unreal Engine) is used in the experiment to test the running efficiency of dynamic programming and the delay time of content loading; S2.4. Prioritize multiple tasks through the Sugeno-type fuzzy inference system algorithm, prioritize user interaction instructions, and adjust tasks in the secondary execution environment. The experiment needs to simulate multiple users interacting at the same time to test the correctness of priority sorting and response time (the goal is within 100 milliseconds); In the step S3, equipment collaborative control and execution: S3.1. Use the unified ZigBee device management protocol to synchronize the status data of multiple devices in the exhibition hall (projector brightness, audio volume) in real time, integrate the device status through the edge gateway (AWS Greengrass) and upload it to the cloud. In the experiment, the projector Epson EH-TW7100 is used to simulate the exhibition hall equipment to test the synchronization delay and consistency; S3.2. Through the distributed scheduling method task scheduling algorithm based on deep reinforcement learning, interactive tasks are assigned to specific devices and the efficiency of task allocation is optimized. The scheduling system needs to run in the local area network and complete task transmission through the gateway (MQTT Broker). In the experiment, a small exhibition hall scene is built to simulate different tasks (switching screens, adjusting sound effects) and test the delay of task distribution (the target is less than 200 milliseconds); S3.3. Use DMX512 lighting control protocol and AV-over-IP technology to coordinate multiple devices and adjust lighting color, sound effects and display screen content simultaneously. The equipment includes digital audio (JBL EON ONE MK2) and lighting system (Chauvet DJ SlimPAR Pro). The experiment needs to use Lightjams lighting arrangement software to verify the smoothness of dynamic scene switching. S3.4. Monitor the equipment operation status (temperature, workload) in real time and use the isolated forest anomaly detection algorithm to find faults. Ensure the operation of the exhibition hall through the backup equipment switching mechanism (dual projection system). Use the network injection tool to simulate equipment failures in the experiment and test the response speed of system fault switching (the target is less than 1 second). In the step S4, real-time feedback and system optimization: S4.1. Use the Elo 22-inch touch screen and Intel RealSense D415 emotion recognition camera in the display area to record users’ feedback on the display content (satisfaction rating, emotional changes) in real time. Set up a real user test group in the experiment to analyze the comprehensiveness and real-time nature of feedback collection. S4.2. Combine the user stay time and content switching frequency indicators, and use the Tableau interactive data analysis tool to evaluate the interactive effect in real time. The experiment needs to preset multiple display content scenarios (static and dynamic content combination) and evaluate the changes in user engagement; S4.3. Based on real-time feedback data, use the PPO deep reinforcement learning algorithm to update the weights of the user intention inference and strategy generation model, and use the Ray RLlib automated training framework to achieve continuous optimization of the model. The convergence speed of the adaptive optimization needs to be verified in the experiment (the target convergence time is less than 1000 iterations). S4.4. Use the ELK Stack log analysis tool to record the equipment operation log, including the number of failures, working hours, and data throughput indicators. Combined with the machine learning algorithm of the time series prediction algorithm ARIMA, predict the maintenance needs of the equipment and generate early warnings. In the experiment, the collected log data is replayed and analyzed to evaluate the accuracy of the maintenance recommendations.

[0020] In this embodiment, first, multimodal sensors (infrared, panoramic camera, microphone array and pressure-sensitive floor) are used to collect user behavior data and environmental information, and the data is compressed and cleaned in real time through edge computing devices; then, based on the YOLOv8 deep learning model, user behavior and environmental parameters are integrated to infer user needs and generate interaction strategies, and the optimal exhibition hall path and multi-task priority sorting are achieved through dynamic planning and fuzzy reasoning; then, relying on the device management protocol and distributed scheduling algorithm, tasks are efficiently allocated to specific equipment (projectors, audio, lighting systems), and content display and environmental adjustments are completed in a coordinated manner, while the device status is monitored and faults are handled in a timely manner using anomaly detection algorithms; finally, user feedback is recorded through emotion recognition and touch devices, and the strategy generation model is updated in combination with deep reinforcement learning. At the same time, equipment maintenance needs are predicted based on log analysis tools to ensure continuous optimization and efficient operation of the exhibition hall system. Example 2

[0021] like Figure 1-3 As shown, the present invention provides a solution: an intelligent interactive control system for multimedia equipment in a digital exhibition hall, including a human-machine symbiotic perception network module for collecting data and guiding user behavior through a dynamically variable perception network to establish a feedback perception control system, a scene-guided interaction strategy module for shaping user behavior based on user intention generation strategies to guide them to experience different exhibition hall theme contents, an autonomous adaptive device linkage module for passive response strategies and autonomously optimizing linkage methods by learning user interaction habits and exhibition hall environment, and a data behavior bidirectional feedback module for breaking through one-way feedback, optimizing content design in real time using user behavior, and designing new interaction methods based on generative algorithms; The human-machine symbiotic perception network module includes a dynamic perception layout adjustment module for dynamically expanding the strategy for autonomously adjusting the perception range, a behavior trajectory mapping prediction module for generating a user behavior trajectory heat map in combination with real-time perception and predicting the next user behavior path through a time series prediction model, an environment-driven perception optimization module for the environment perception device to automatically adjust the sampling frequency according to environmental changes, and a virtual reality joint perception module for introducing an AR device to record the user's gaze point and gestures to achieve synchronous perception of virtual and real information; The scene-guided interaction strategy module includes a theme scene preset switching module for realizing seamless switching of multiple theme scenes through a dynamic projector and a lighting system, a reverse reasoning intention recognition module for inferring that a user may have ignored a point of interest based on an area or function not triggered by the user, a multi-path interaction planning module for simulating the optimal interaction path between the user and the exhibition hall using an interaction path planning method based on game theory, and an emotion-driven content adjustment module for adjusting content display in real time using emotion recognition technology; The autonomous adaptive equipment linkage module includes an equipment behavior learning module for using federated learning technology to allow different exhibition hall equipment to share experience data, a perception-driven scene adaptation module for autonomously selecting adaptive actions based on perception data, an equipment active task negotiation module for introducing negotiation protocols between equipment to form a "equipment symbiotic network", and a panoramic content dynamic reconstruction module for autonomously generating content display strategies through real-time monitoring of venue equipment layout; The data-behavior two-way feedback module includes a behavior feedback data correction module for adjusting the priority of the perception and strategy generation model according to the actual operation habits of the user, a generation experience content design module for using a generative adversarial network to generate new display content in real time, a user feedback reverse guidance module for allowing users to participate in the exhibition hall interaction design through voice feedback or touch operation, and an interaction data precipitation evolution module for regularly summarizing the interaction effect data of the exhibition hall to form a data-driven update strategy; The behavior feedback data correction module includes a user behavior sampling adjustment module for dynamically adjusting the behavior sampling time interval and accuracy by real-time monitoring the user's operation frequency and type, a user preference feature extraction module for analyzing the user's behavior trajectory and stay area to extract personalized preference features, an adaptive feedback determination module for judging whether the current feedback data can accurately reflect the user's needs according to the user's behavior changes, a dynamic model weight update module for dynamically adjusting the prediction model weight according to the user's actual behavior data, a behavior anomaly determination module for detecting abnormal behavior in the user's operation to judge whether the system feedback strategy needs to be adjusted, a user intention error correction module for correcting the prediction error and updating the model by comparing the user's intention inferred by the system with the actual interaction behavior, a content presentation real-time optimization module for adjusting the content presentation form according to the user's behavior, and a user behavior data precipitation module for archiving the user's long-term behavior data to extract key patterns for overall optimization of the exhibition hall content and interaction methods; The adaptive feedback determination module includes a behavior feedback correlation calculation module for analyzing the correlation between user behavior and system feedback to further determine whether the feedback is effective, and an interaction response analysis module for detecting the timeliness of feedback response to determine whether the interaction affects the user experience due to delay; The behavior anomaly determination module includes a multi-level behavior monitoring module for refining the monitoring scope of user behavior and subdividing behavior anomalies into slight deviations, significant deviations, and extreme deviations, and a user behavior environment association module for combining environmental data to determine whether the behavior anomaly is affected by external factors; The user behavior sampling and adjustment module dynamically collects behavior data and inputs the optimized data into the user preference feature extraction module for refining personalized preferences; the user preference feature extraction module analyzes user behavior characteristics and passes the results to the adaptive feedback determination module to determine whether the feedback data accurately reflects user needs; the adaptive feedback determination module provides an optimization signal according to the feedback accuracy and passes it to the dynamic model weight update module to adjust the prediction model weight; the dynamic model weight update module optimizes the prediction model weight and provides the optimized model to the behavior anomaly determination module to more accurately identify user abnormal behavior; the behavior anomaly determination module identifies the user's abnormal behavior and passes the abnormal result to the user intention error correction module to correct the intention prediction deviation; the user intention error correction module corrects the prediction deviation and inputs the calibrated user intention into the content presentation real-time optimization module to adjust the display form; the content presentation real-time optimization module dynamically adjusts the content display form and stores the display adjustment record in the user behavior data precipitation module to extract the long-term optimization mode; The dynamic perception layout adjustment module expands the perception range and provides data to the behavior trajectory mapping prediction module to generate a user behavior trajectory heat map; the behavior trajectory mapping prediction module predicts the user behavior path and passes the result to the environment-driven perception optimization module for adjusting the sampling frequency of the perception device; the environment-driven perception optimization module dynamically adjusts the sampling frequency and provides the virtual reality joint perception module with environmental perception data to synchronize virtual and real information; the virtual reality joint perception module combines the user's gaze point and gesture data and passes the virtual and real information back to the dynamic perception layout adjustment module to optimize the perception layout; The theme scene preset switching module realizes scene switching through dynamic projection and passes the switching data to the reverse reasoning intention recognition module to infer the points of interest that the user may ignore; the reverse reasoning intention recognition module identifies the area that the user has not triggered and passes the point of interest data to the multi-path interaction planning module to optimize the interaction path; the multi-path interaction planning module simulates the interaction path between the user and the exhibition hall and passes the path optimization result to the emotion-driven content adjustment module to adjust the content display; the emotion-driven content adjustment module adjusts the display content in real time and passes the adjusted emotional response data back to the theme scene preset switching module to optimize the scene switching logic; The device behavior learning module shares experience data through federated learning and passes the optimized device behavior to the perception-driven scene adaptation module to guide device action selection; the perception-driven scene adaptation module selects adaptive actions and passes the selection results to the device active task negotiation module to optimize task allocation between devices; the device active task negotiation module allocates tasks based on the negotiation protocol and provides the task allocation results to the panoramic content dynamic reconstruction module to generate content display strategies; The panoramic content dynamic reconstruction module optimizes the content display according to the device layout and transmits the layout adjustment data back to the device behavior learning module to improve the device learning efficiency; the behavior feedback data correction module adjusts the model priority according to the user operation and provides the optimized priority data to the generated experience content design module to generate new display content in real time; the generated experience content design module generates new display content and passes the generated data to the user feedback reverse guidance module for users to participate in content design; the user feedback reverse guidance module optimizes the content design according to user feedback and passes the feedback results to the interactive data precipitation evolution module for forming an update strategy; the interactive data precipitation evolution module summarizes the interactive data and generates an update strategy and provides the update strategy to the behavior feedback data correction module to further optimize the model.

[0022] In this embodiment, multimodal sensor data is collected through the human-machine symbiotic perception network module to generate user behavior trajectories and environmental perception information, providing real-time prediction and layout optimization support for user behavior; the scene-guided interaction strategy module dynamically switches theme scenes and plans interaction paths according to user intentions, and adjusts the display content in real time through emotional drive; the autonomous adaptive device linkage module uses federated learning to share device data, optimizes task allocation through negotiation, and dynamically reconstructs content display strategies; the data-behavior two-way feedback module adjusts model priorities through user behavior, generates new display content in combination with a generative adversarial network, and optimizes display design through user feedback; in this process, the adaptive feedback judgment module evaluates the matching degree and response timeliness of the feedback, and the behavior anomaly judgment module refines the abnormal behavior category and analyzes abnormal external causes in combination with environmental data, providing support for real-time optimization and fault decision-making of the overall system.

[0023] Among them: the behavior feedback association calculation module analyzes the matching degree of the feedback and passes the matching result to the interactive response analysis module to further evaluate the timeliness of the feedback; the multi-level behavior monitoring module refines the user behavior abnormality category and provides the classification result to the user behavior environment association module for analyzing whether the abnormality is affected by the external environment.

[0024] Wherein: the user behavior environment association module analyzes the external causes of abnormal behavior in combination with environmental data and feeds back the analysis results to the behavior anomaly determination module to assist in abnormal handling decisions.

[0025] like Figure 1-3 As shown, in actual use, the present invention first collects user behavior data and environmental information in real time by arranging multimodal sensors (infrared sensors, panoramic cameras, microphone arrays and pressure-sensitive floors), and uses edge computing devices to compress and clean data to ensure low latency and high reliability of data transmission; then, based on the YOLOv8 deep learning model, it integrates user behavior characteristics, voice commands and environmental parameters to infer user needs, and generates the best interaction path and task priority sorting through dynamic planning and fuzzy reasoning; then, it adopts a distributed task scheduling algorithm to efficiently allocate tasks to specific equipment such as projectors, audio and lighting systems, and collaboratively complete content display and environmental adjustment, while monitoring the operating status of the equipment and handling equipment failures in a timely manner through anomaly detection algorithms; finally, it collects user feedback in real time through emotion recognition devices and user touch screens, updates interaction strategies in combination with deep reinforcement learning models, and predicts equipment maintenance needs through log analysis to achieve continuous optimization of the system.

[0026] The human-machine symbiotic perception network module achieves synchronous perception of user behavior and exhibition hall environment by dynamically adjusting the perception range, predicting behavior trajectories, and optimizing environmental perception; the scene-guided interaction strategy module uses dynamic projection and emotion-driven technology to switch theme scenes according to user intentions, optimize interaction paths, and adjust content display in real time; the autonomous adaptive device linkage module uses federated learning technology to share device experience data, negotiate task allocation, and dynamically reconstruct content display strategies to achieve multi-device collaboration; the data-behavior bidirectional feedback module optimizes model priorities through behavioral data, generates new content using generative adversarial networks, and iteratively improves display design and interaction strategies based on user feedback. In addition, the adaptive feedback judgment module optimizes the interaction effect by analyzing the feedback matching degree and response timeliness, and the behavior anomaly judgment module refines the anomaly category based on environmental data to provide auxiliary decision-making for fault handling.

[0027] In actual applications, the system first collects user behavior and environmental data through a multimodal perception network, and infers user needs through a deep learning model to generate the optimal interaction strategy; then, the system coordinates the multimedia equipment in the exhibition hall to perform content display and environmental adjustment, while monitoring the status of equipment in real time to ensure efficient collaboration and abnormal response; finally, the interaction model is updated through real-time user feedback and behavior data, and generative adversarial networks and deep reinforcement learning are used to continuously optimize the display content and interactive experience, while log analysis is combined to predict system maintenance needs, thereby achieving dynamic adaptive optimization and long-term iterative improvement of the exhibition hall's interactive experience.

[0028] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent interactive control of multimedia equipment in a digital exhibition hall, characterized in that: The following steps are involved: S1. Arrangement of sensors for intelligent perception, using SCD40 data acquisition and detection multifunctional environmental sensors to be arranged at the exhibition hall, and then perform real-time data compression and denoising; S2, using edge computing hardware for model reasoning, intelligent interaction strategy generation to prioritize user interaction instructions, and secondary execution environment adjustment tasks; S3, equipment collaborative control and execution, real-time monitoring of equipment operating status and use of the Isolation Forest Anomaly Detection Algorithm to detect faults; S4. Real-time feedback and system optimization: the touch screen and emotion recognition camera in the display area provide feedback on the display content, and combine the machine learning algorithm of the time series prediction algorithm ARIMA to predict the maintenance needs of equipment and generate early warnings.

2. According to claim 1, a method for intelligent interactive control of multimedia equipment in a digital exhibition hall is characterized in that: In step S1, the sensor is arranged in intelligent perception and data collection: S1.

1. Arrange multimodal sensors in the exhibition hall, including high-precision infrared sensors, panoramic cameras, microphone arrays and pressure sensing system floors; S1.2, use YOLOv8 deep learning algorithm combined with high-performance hardware graphics card to perform real-time analysis on the video stream captured by the camera; S1.

3. Use SCD40 multifunctional environmental sensors for CO2 concentration and temperature and humidity detection to be arranged at different locations in the exhibition hall; S1.

4. After data collection, TensorFlow Lite is run on the edge computing device Raspberry Pi 4B for real-time data compression and denoising.

3. According to claim 1, a method for intelligent interactive control of multimedia equipment in a digital exhibition hall is characterized in that: In the step S2, the intelligent interaction strategy is generated: S2.1, integrate user behavior, voice input and environmental parameters through deep learning models, and use edge computing hardware for model reasoning; S2.2, using BERT-based natural language processing model and behavior analysis algorithm to infer users’ current needs; S2.3, based on the inferred intention, call the predefined display content module and generate the exhibition hall interaction path through the dynamic programming algorithm; S2.

4. Prioritize multiple tasks through Sugeno-type fuzzy inference system algorithm, give priority to user interaction instructions, and secondary execution environment adjustment tasks.

4. According to claim 1, a method for intelligent interactive control of multimedia equipment in a digital exhibition hall is characterized in that: In the step S3, equipment collaborative control and execution: S3.

1. Use the unified ZigBee device management protocol to synchronize the status data of multiple devices in the exhibition hall in real time, integrate the device status through the edge gateway and upload it to the cloud; S3.

2. Assign interactive tasks to specific devices through a distributed scheduling method task scheduling algorithm based on deep reinforcement learning S3.3, using DMX512 lighting control protocol and AV-over-IP technology multi-device collaboration technology to adjust lighting color, sound effects and display content at the same time; S3.

4. Monitor the equipment operation status in real time and use the isolated forest anomaly detection algorithm to detect faults, and ensure the operation of the exhibition hall through the backup equipment switching mechanism.

5. According to claim 1, a method for intelligent interactive control of multimedia equipment in a digital exhibition hall is characterized in that: In the step S4, real-time feedback and system optimization: S4.

1. Record users’ feedback on the displayed content in real time through the touch screen and emotion recognition camera in the display area; S4.

2. Combine the user stay time and content switching frequency indicators to evaluate the interaction effect in real time through Tableau interactive data analysis tool; S4.

3. Based on real-time feedback data, use the PPO deep reinforcement learning algorithm to update the weights of the user intention inference and strategy generation model, and use the Ray RLlib automated training framework to achieve continuous optimization of the model. S4.

4. Use the ELK Stack log analysis tool to record the equipment operation log, including the number of failures, working hours, and data throughput indicators. Combined with the machine learning algorithm of the time series prediction algorithm ARIMA, predict the maintenance needs of the equipment and generate early warnings.

6. According to any one of claims 1 to 5, a method for intelligent interactive control of multimedia equipment in a digital exhibition hall, the control system is characterized in that: It includes a human-machine symbiotic perception network module, a scenario-guided interaction strategy module, an autonomous adaptive equipment linkage module, and a data-behavior two-way feedback module.

7. The intelligent interactive control system for multimedia equipment in a digital exhibition hall according to claim 6 is characterized by: The human-machine symbiotic perception network module includes a dynamic perception arrangement adjustment module, a behavior trajectory mapping prediction module, an environment-driven perception optimization module, and a virtual reality joint perception module; The scenario-guided interaction strategy module includes a theme scenario preset switching module, a reverse reasoning intention recognition module, a multi-path interaction planning module, and an emotion-driven content adjustment module; The autonomous adaptive device linkage module includes a device behavior learning module, a perception-driven scene adaptation module, a device active task negotiation module, and a panoramic content dynamic reconstruction module; The data-behavior two-way feedback module includes a behavior feedback data correction module, a generated experience content design module, a user feedback reverse guidance module, and an interactive data precipitation evolution module.

8. The intelligent interactive control system for multimedia equipment in a digital exhibition hall according to claim 7 is characterized by: The behavior feedback data correction module includes a user behavior sampling adjustment module, a user preference feature extraction module, an adaptive feedback determination module, a dynamic model weight update module, a behavior anomaly determination module, a user intention error correction module, a content presentation real-time optimization module and a user behavior data precipitation module; The adaptive feedback determination module includes a behavior feedback association calculation module and an interactive response analysis module; The abnormal behavior determination module includes a multi-level behavior monitoring module and a user behavior environment association module.

Images