Multimedia exhibition hall AI intelligent interaction control system based on Internet of Things

Abstract

The invention discloses a multimedia exhibition hall AI intelligent interaction control system based on the Internet of Things, and belongs to the technical field of multimedia exhibition halls, and the system comprises an environment data collection and analysis module which is used for collecting real-time environment data of a multimedia exhibition hall, analyzing the real-time environment data of the multimedia exhibition hall, and determining an environment analysis result of the multimedia exhibition hall; the exhibition hall environment intelligent regulation and control module is used for carrying out intelligent regulation and control on the multimedia exhibition hall environment; the exhibition hall intelligent guide service module is used for recommending a customized route for the user and providing intelligent guide service; and the AI intelligent interaction control module is used for performing AI intelligent interaction based on voice recognition and gesture control so as to ensure immersive experience of the user. The problems that an existing multimedia exhibition hall is low in interactivity, exhibition effect and management efficiency, and intelligent operation of the multimedia exhibition hall cannot be effectively achieved are solved. According to the invention, the interactivity, the display effect and the management efficiency of the multimedia exhibition hall can be improved, and the intelligent operation of the multimedia exhibition hall can be effectively realized.

Description

Technical Field

[0001] This invention relates to the field of multimedia exhibition hall technology, specifically to an AI-powered intelligent interactive control system for multimedia exhibition halls based on the Internet of Things. Background Technology

[0002] Multimedia exhibition halls are dynamic display platforms built around 3D digital imaging technology, combined with hardware such as projectors, touchscreens, and sensors. They integrate technologies such as panoramic cinemas, digital sand tables, and interactive projection, using sound, light, and electricity fusion and immersive experiences to replace traditional static exhibition models. Widely used in museums, science and technology museums, corporate showrooms, and urban planning museums, they fulfill functions such as cultural dissemination, image display, and public interaction, becoming an important vehicle for international exchange.

[0003] Existing multimedia exhibition halls suffer from low interactivity, poor display effects, and low management efficiency, failing to effectively achieve intelligent operation. Summary of the Invention

[0004] The purpose of this invention is to provide an AI-powered intelligent interactive control system for multimedia exhibition halls based on the Internet of Things (IoT), which can improve the interactivity, display effect, and management efficiency of multimedia exhibition halls, effectively realize the intelligent operation of multimedia exhibition halls, and solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] The IoT-based AI-powered intelligent interactive control system for multimedia exhibition halls includes:

[0007] The environmental data acquisition and analysis module is used to collect real-time environmental data of the multimedia exhibition hall, analyze the real-time environmental data of the multimedia exhibition hall, and determine the environmental analysis results of the multimedia exhibition hall.

[0008] The exhibition hall environment intelligent control module is used to intelligently control the multimedia exhibition hall environment based on the analysis results of the multimedia exhibition hall environment;

[0009] The exhibition hall's intelligent navigation service module is used to recommend customized routes and provide intelligent navigation services to users, thereby optimizing the user's visitor experience.

[0010] The AI ​​intelligent interaction control module is used for AI intelligent interaction based on voice recognition and gesture control to ensure an immersive interactive experience for users.

[0011] Preferably, when a user engages in AI-powered intelligent interaction based on voice recognition, the system monitors the user's voice needs in real time, automatically retrieves the voice library based on the user's voice needs, matches the voice interaction information that meets the user's voice needs from the voice library, and outputs and displays the matched voice interaction information. The user then engages in another voice interaction with the multimedia exhibition hall based on the voice interaction information until the user no longer engages in voice interaction.

[0012] Preferably, when a user engages in AI-powered intelligent interaction based on speech recognition, the system monitors the user's speech needs in real time, automatically searches the speech library based on these needs, and matches speech interaction information from the speech library that meets the user's speech needs, including:

[0013] Obtain the user's current speech text and the speech text from the previous preset number of times. Perform intent vector feature recognition on the current speech text and the speech text from the previous preset number of times, and unify them to obtain the current intent vector and adjacent intent vectors.

[0014] The current intent vector is optimized based on the reference intent vector to obtain the target intent vector, and multiple voice interaction information that can meet the user's voice needs are matched from the voice library based on the target intent vector.

[0015] Preferably, when a user performs AI intelligent interaction based on gesture control, the system monitors the user's gesture needs in real time, automatically searches the gesture library based on the user's gesture needs, matches gesture interaction information that can meet the user's gesture needs from the gesture library, and outputs and displays the matched gesture interaction information. In this case, the user's gestures can be used to flip pages in the multimedia exhibition hall and provide an immersive interactive experience for the user.

[0016] Preferably, AI intelligent interaction is based on voice recognition and gesture control, and the following operations are performed:

[0017] During the visit, users can interact with AI intelligently based on voice recognition and gesture control. The system also monitors users' behavior and dwell time in real time and generates user interest heatmaps based on AI visual analysis of user behavior and dwell time.

[0018] Based on user interest heatmaps, the recommended content for user visits is adjusted in real time, and related extended information is automatically pushed, including other exhibits from the same period, the restoration process of the exhibit, and related research results.

[0019] During the process of pushing tour content, user feedback is obtained in real time, and the recommended tour content is flexibly adjusted based on the user feedback, forming a closed-loop management of the recommended tour content.

[0020] Preferably, collect real-time environmental data from the multimedia exhibition hall and perform the following operations:

[0021] Based on IoT technology, temperature, humidity, light and crowd density sensors are deployed in key areas of the multimedia exhibition hall, and connected to lighting and air conditioning equipment through multi-protocol gateways to form a full-area monitoring network for the multimedia exhibition hall, providing a data foundation for intelligent environmental control.

[0022] Temperature and humidity sensors are used to monitor the temperature and humidity in the multimedia exhibition hall in real time and collect temperature and humidity data of the multimedia exhibition hall;

[0023] A light sensor is used to monitor the light intensity in the multimedia exhibition hall in real time and collect illuminance data of the multimedia exhibition hall.

[0024] A crowd density sensor is used to monitor the crowd density in the multimedia exhibition hall in real time and collect crowd density data in the multimedia exhibition hall;

[0025] Among them, the real-time environmental data of the multimedia exhibition hall is determined based on the temperature and humidity data, illuminance data, and crowd density data of the multimedia exhibition hall.

[0026] Preferably, the real-time environmental data of the multimedia exhibition hall is analyzed, and the following operations are performed:

[0027] The real-time environmental data of the multimedia exhibition hall is compared and analyzed with the preset standard environmental data of the multimedia exhibition hall to evaluate the consistency between the real-time environmental data and the standard environmental data of the multimedia exhibition hall, determine whether the real-time environmental data of the multimedia exhibition hall is within the range of the standard environmental data of the multimedia exhibition hall, and determine the environmental analysis results of the multimedia exhibition hall.

[0028] When the real-time environmental data of the multimedia exhibition hall is within the range of the standard environmental data of the multimedia exhibition hall, the environmental analysis result of the multimedia exhibition hall is that the multimedia exhibition hall environment is normal and there is no need to perform intelligent control of the multimedia exhibition hall environment.

[0029] When the real-time environmental data of the multimedia exhibition hall is outside the range of the standard environmental data of the multimedia exhibition hall, the environmental analysis result of the multimedia exhibition hall is that the multimedia exhibition hall environment is abnormal, and intelligent control of the multimedia exhibition hall environment is required.

[0030] Preferably, the multimedia exhibition hall environment is intelligently controlled, and the following operations are performed:

[0031] The lighting equipment is automatically adjusted based on real-time monitoring of the illuminance data in the multimedia exhibition hall. When the illuminance data in the multimedia exhibition hall is lower than the standard data, the brightness of the lighting equipment is automatically increased; when the illuminance data in the multimedia exhibition hall is higher than the standard data, the brightness of the lighting equipment is automatically decreased.

[0032] The air conditioning equipment is automatically adjusted based on real-time monitoring of temperature and humidity data and crowd density data in the multimedia exhibition hall. When the temperature and humidity data and crowd density data in the multimedia exhibition hall are higher than the standard data, the temperature and fan speed of the air conditioning equipment are dynamically adjusted to lower the temperature and increase the fan speed, and crowd control measures are implemented in the multimedia exhibition hall.

[0033] Preferably, the system recommends customized routes and provides intelligent navigation services to users, optimizing their visitor experience by performing the following operations:

[0034] Based on machine vision technology, high-resolution cameras are used to monitor users entering the multimedia exhibition hall in real time, obtain user behavior data, and combine this with questionnaire surveys to quickly understand user interests and preferences.

[0035] Based on the layout and exhibit distribution of the multimedia exhibition hall, and combined with user behavior data and interests, a customized tour route is generated for the user, and the user is intelligently guided to the tour based on the customized tour route.

[0036] For users interested in history and culture, historical artifact exhibition halls are recommended first, and intelligent explanations are provided to the users of these exhibition halls. After the users confirm the historical artifact exhibition halls, an intelligent route is planned to connect the relevant exhibits, ensuring that the users do not miss important artifacts and optimizing the visiting experience.

[0037] For users interested in science and technology culture, the system prioritizes recommending science and technology culture exhibition halls and provides intelligent explanations of these halls. Once users confirm their selection of a science and technology culture exhibition hall, the system intelligently plans a route connecting related exhibits, guiding them to the science and technology experience area and rationally arranging the order of visits to improve visit efficiency and experience.

[0038] Preferably, the gesture library is automatically searched based on the user's gesture requirements, and gesture interaction information that meets the user's gesture requirements is matched from the gesture library, including:

[0039] The user's gestures are identified by recognizing hand contour features, hand key point features, and geometric features to obtain static gesture features, and the user's gestures are identified by recognizing hand movement sequences to obtain dynamic gesture features;

[0040] After standardizing the static and dynamic gesture features, they are concatenated into a joint feature vector. Based on the joint and other positive vectors, a preliminary matching is performed with the gesture library to obtain reference gesture interaction information.

[0041] Obtain common interaction information, related interaction information, and conflicting interaction information of reference gesture interaction information. Based on the correspondence between common interaction information, related interaction information, and conflicting interaction information and feature vectors, configure corresponding basic weights for each feature vector. Obtain a weighted joint feature vector based on the basic weights.

[0042] Based on the weighted joint feature vector and the gesture library, multiple candidate gesture interaction information are obtained. Voice information and exhibit status information synchronized with the user's gesture are obtained. The first associated feature is determined based on the voice information, and the second associated feature is determined based on the exhibit status information.

[0043] Based on the multimedia exhibition hall environment, the first weight of the first associated feature and the second weight of the second associated feature are determined, and an association evaluation vector is established based on the first associated feature, the second associated feature, the first weight and the second weight.

[0044] The gesture interaction information with the highest matching degree with the associated evaluation vector is selected from multiple candidate gesture interaction information as the final gesture interaction information that meets the user's gesture needs.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. This invention collects real-time environmental data of a multimedia exhibition hall by monitoring temperature, humidity, light, and crowd density in real time. It then compares and analyzes the real-time environmental data of the multimedia exhibition hall with preset standard environmental data to determine the environmental analysis results. Based on the environmental analysis results, the invention intelligently regulates the environment of the multimedia exhibition hall to achieve adaptive adjustment of the multimedia exhibition hall environment.

[0047] 2. This invention acquires user behavior data through high-resolution cameras and quickly understands user interests and preferences by combining questionnaire surveys. Based on the layout of the multimedia exhibition hall and the distribution of exhibits, it generates customized tour routes for users by combining user behavior data and interests and preferences. The customized tour routes intelligently guide users to visit the exhibition, rationally arrange the order of visits, and improve the efficiency and experience of visiting the exhibition.

[0048] 3. This invention utilizes voice recognition and gesture control for AI-powered intelligent interaction, and monitors user behavior and dwell time in real time to generate a user interest heatmap. Based on this heatmap, it adjusts recommended content for user visits in real time and automatically pushes related extended information. During the content push process, it obtains user feedback in real time and flexibly adjusts recommended content based on this feedback, forming a closed-loop management system for recommended content. This improves the interactivity, display effect, and management efficiency of multimedia exhibition halls, and effectively realizes the intelligent operation of multimedia exhibition halls. Attached Figure Description

[0049] Figure 1 This is a block diagram of the IoT-based AI intelligent interactive control system for multimedia exhibition halls according to the present invention.

[0050] Figure 2 This is a flowchart of the IoT-based AI intelligent interactive control system for multimedia exhibition halls according to the present invention. Detailed Implementation

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0052] To address the issues of low interactivity, poor display quality, and low management efficiency in existing multimedia exhibition halls, which hinder the effective implementation of intelligent operation, please refer to [link to relevant documentation]. Figure 1-Figure 2 This embodiment provides the following technical solution:

[0053] The IoT-based multimedia exhibition hall AI intelligent interactive control system includes: an environmental data acquisition and analysis module, an exhibition hall environment intelligent control module, an exhibition hall intelligent guide service module, and an AI intelligent interactive control module.

[0054] Specifically, through the interaction between the environmental data collection and analysis module, the exhibition hall environment intelligent control module, the exhibition hall intelligent guide service module, and the AI ​​intelligent interactive control module, the interactivity, display effect, and management efficiency of the multimedia exhibition hall can be improved, and the intelligent operation of the multimedia exhibition hall can be effectively realized.

[0055] The environmental data acquisition and analysis module is used to collect real-time environmental data of the multimedia exhibition hall, analyze the real-time environmental data of the multimedia exhibition hall, and determine the environmental analysis results of the multimedia exhibition hall.

[0056] In this embodiment, real-time environmental data of the multimedia exhibition hall is collected, and the following operations are performed:

[0057] Based on IoT technology, temperature, humidity, light and crowd density sensors are deployed in key areas of the multimedia exhibition hall, and connected to lighting and air conditioning equipment through multi-protocol gateways to form a full-area monitoring network for the multimedia exhibition hall, providing a data foundation for intelligent environmental control.

[0058] Temperature and humidity sensors are used to monitor the temperature and humidity in the multimedia exhibition hall in real time and collect temperature and humidity data; light sensors are used to monitor the light intensity in the multimedia exhibition hall in real time and collect illuminance data; and people density sensors are used to monitor the people density in the multimedia exhibition hall in real time and collect people density data.

[0059] Among them, the real-time environmental data of the multimedia exhibition hall is determined based on the temperature and humidity data, illuminance data, and crowd density data of the multimedia exhibition hall.

[0060] It should be noted that by monitoring the temperature, humidity, lighting, and crowd density in the multimedia exhibition hall in real time, real-time environmental data of the multimedia exhibition hall is collected, providing a data foundation for subsequent intelligent control of the multimedia exhibition hall environment.

[0061] In this embodiment, the real-time environmental data of the multimedia exhibition hall is analyzed, and the following operations are performed:

[0062] The real-time environmental data of the multimedia exhibition hall is compared and analyzed with the preset standard environmental data of the multimedia exhibition hall to evaluate the consistency between the real-time environmental data and the standard environmental data of the multimedia exhibition hall, determine whether the real-time environmental data of the multimedia exhibition hall is within the range of the standard environmental data of the multimedia exhibition hall, and determine the environmental analysis results of the multimedia exhibition hall.

[0063] When the real-time environmental data of the multimedia exhibition hall is within the range of the standard environmental data of the multimedia exhibition hall, the environmental analysis result of the multimedia exhibition hall is that the multimedia exhibition hall environment is normal and there is no need to perform intelligent control of the multimedia exhibition hall environment.

[0064] When the real-time environmental data of the multimedia exhibition hall is outside the range of the standard environmental data of the multimedia exhibition hall, the environmental analysis result of the multimedia exhibition hall is that the multimedia exhibition hall environment is abnormal, and intelligent control of the multimedia exhibition hall environment is required.

[0065] Specifically, the real-time environmental data of the multimedia exhibition hall was compared and analyzed based on the preset standard environmental data of the multimedia exhibition hall. The results of the multimedia exhibition hall environmental analysis are shown in Table 1:

[0066] Table 1: Environmental Analysis Results of Multimedia Exhibition Hall

[0067]

[0068] Therefore, by comparing and analyzing the real-time environmental data of the multimedia exhibition hall with the preset standard environmental data of the multimedia exhibition hall, the environmental analysis results of the multimedia exhibition hall can be determined. Based on the environmental analysis results, the environment of the multimedia exhibition hall can be intelligently controlled, thereby realizing the adaptive adjustment of the multimedia exhibition hall environment.

[0069] Among them, the intelligent control module for the exhibition hall environment is used to intelligently control the environment of the multimedia exhibition hall based on the analysis results of the multimedia exhibition hall environment.

[0070] In this embodiment, the multimedia exhibition hall environment is intelligently controlled by performing the following operations:

[0071] The lighting equipment is automatically adjusted based on real-time monitoring of the illuminance data in the multimedia exhibition hall. When the illuminance data in the multimedia exhibition hall is lower than the standard data, the brightness of the lighting equipment is automatically increased; when the illuminance data in the multimedia exhibition hall is higher than the standard data, the brightness of the lighting equipment is automatically decreased.

[0072] The air conditioning equipment is automatically adjusted based on real-time monitoring of temperature and humidity data and crowd density data in the multimedia exhibition hall. When the temperature and humidity data and crowd density data in the multimedia exhibition hall are higher than the standard data, the temperature and fan speed of the air conditioning equipment are dynamically adjusted to lower the temperature and increase the fan speed, and crowd control measures are implemented in the multimedia exhibition hall.

[0073] The exhibition hall intelligent tour guide service module is used to recommend customized routes and provide intelligent tour guide services to users, thereby optimizing the user's visitor experience.

[0074] In this embodiment, customized routes are recommended to users and intelligent tour guide services are provided to optimize the user's visit experience. The following operations are performed:

[0075] Based on machine vision technology, high-resolution cameras are used to monitor users entering the multimedia exhibition hall in real time, obtain user behavior data, and combine this with questionnaire surveys to quickly understand user interests and preferences.

[0076] Based on the layout and exhibit distribution of the multimedia exhibition hall, and combined with user behavior data and interests, a customized tour route is generated for the user, and the user is intelligently guided to the tour based on the customized tour route.

[0077] For users interested in history and culture, historical artifact exhibition halls are recommended first, and intelligent explanations are provided to the users of these exhibition halls. After the users confirm the historical artifact exhibition halls, an intelligent route is planned to connect the relevant exhibits, ensuring that the users do not miss important artifacts and optimizing the visiting experience.

[0078] For users interested in science and technology culture, the system prioritizes recommending science and technology culture exhibition halls and provides intelligent explanations of these halls. Once users confirm their selection of a science and technology culture exhibition hall, the system intelligently plans a route connecting related exhibits, guiding them to the science and technology experience area and rationally arranging the order of visits to improve visit efficiency and experience.

[0079] Among them, the AI ​​intelligent interaction control module is used for AI intelligent interaction based on voice recognition and gesture control to ensure an immersive interactive experience for users.

[0080] In this embodiment, AI intelligent interaction is performed based on voice recognition and gesture control to perform the following operations:

[0081] During the visit, users can interact with AI intelligently based on voice recognition and gesture control. The system also monitors users' behavior and dwell time in real time and generates user interest heatmaps based on AI visual analysis of user behavior and dwell time.

[0082] Based on user interest heatmaps, the recommended content for user visits is adjusted in real time, and related extended information is automatically pushed, including other exhibits from the same period, the restoration process of the exhibit, and related research results.

[0083] During the process of pushing tour content, user feedback is obtained in real time, and the recommended tour content is flexibly adjusted based on the user feedback, forming a closed-loop management of the recommended tour content.

[0084] In this embodiment, when a user performs AI intelligent interaction based on voice recognition, the system monitors the user's voice needs in real time, automatically searches the voice library based on the user's voice needs, matches voice interaction information that can meet the user's voice needs from the voice library, and outputs and displays the matched voice interaction information. The user then interacts with the multimedia exhibition hall again based on the voice interaction information until the user no longer needs to interact with the voice.

[0085] In one embodiment, when a user engages in AI-powered intelligent interaction based on speech recognition, the system monitors the user's speech needs in real time, automatically searches a speech library based on these needs, and matches speech interaction information from the speech library that meets the user's speech needs, including:

[0086] Obtain the user's current speech text and the speech text from the previous preset number of times. Perform intent vector feature recognition on the current speech text and the speech text from the previous preset number of times, and unify them to obtain the current intent vector and adjacent intent vectors.

[0087] Determine the reference intent vector based on all adjacent intent vectors;

[0088]

[0089] Among them, H R The reference intent vector is represented by n, the number of adjacent intent vectors is represented by l0, and the length of the current intent vector is represented by l. i β represents the length of the i-th adjacent intent vector. iγ represents the semantic repetition between the i-th neighboring intent vector and the current intent vector. i σ represents the semantic relevance between the i-th neighboring intent vector and the current intent vector. i H represents the distance weight between the i-th neighboring intent vector and the current intent vector. i This represents the i-th adjacent intent vector;

[0090] The current intent vector is optimized based on the reference intent vector to obtain the target intent vector, and multiple voice interaction information that can meet the user's voice needs are matched from the voice library based on the target intent vector.

[0091] The matching formula is as follows:

[0092]

[0093] Where Sim(D,T) represents the matching degree between the target intent vector D and the information T retrieved from the speech database, E(D,T) represents the minimum edit distance between the target intent vector D and the information T retrieved from the speech database, L(D) represents the path length of the target intent vector D, L(T) represents the path length of the information T retrieved from the speech database, and ρ represents the edit distance weight, with a value of 0.6. PY (D,T) represents the cosine similarity between the target intent vector D and the pinyin sequence of the information T retrieved from the speech database;

[0094] Information retrieved with a matching degree greater than a preset matching degree is selected as multiple voice interaction information to meet the user's voice needs;

[0095] The evaluation criteria include contextual logical relevance, user preference, and exhibit status adaptability. Multiple voice interaction messages are evaluated, and the voice interaction message with the highest evaluation value is selected as the final voice interaction message.

[0096] In this embodiment, minimum edit distance refers to the minimum number of operations required to transform one string into another string through the three basic operations of insertion, deletion, and replacement.

[0097] In this embodiment, path length refers to the total number of steps in the specific operation sequence involved in the conversion of two strings.

[0098] In this embodiment, the smaller the interval between the questions asked by the adjacent intent vector and the current intent vector, the greater the corresponding distance weight.

[0099] In this embodiment, n represents the number of adjacent intent vectors, which is usually 3, i.e., the three most recent question texts before obtaining the voice text.

[0100] In this embodiment, the current intent vector and the adjacent intent vectors have the same number of elements, but the length of each element is different.

[0101] The beneficial effects of the above design scheme are as follows: By acquiring the current voice text and the voice text from the previous preset number of times, the user's real-time needs are associated with historical interaction information, avoiding the isolated interpretation of single-sentence voice. Through intent vector feature recognition and unification, voice text from different rounds is converted into a vector space of a unified dimension, eliminating feature fragmentation caused by differences in expression forms. Furthermore, by combining the optimization of the current intent vector with the reference intent vector, semantic deviations in the user's ambiguous expression are further corrected, making the target intent vector closer to the user's real needs. The matching formula integrates three core indicators: minimum edit distance, path length, and cosine similarity of the pinyin sequence, avoiding the omission of semantically equivalent information due to a single indicator, improving the accuracy of retrieval, ensuring the accuracy of the final obtained voice interaction information, meeting user needs, and improving user experience satisfaction.

[0102] In this embodiment, when a user performs AI intelligent interaction based on gesture control, the system monitors the user's gesture needs in real time, automatically searches the gesture library based on the user's gesture needs, matches gesture interaction information that can meet the user's gesture needs from the gesture library, and outputs and displays the matched gesture interaction information. In this case, the user's gestures are used to flip pages in the multimedia exhibition hall and provide an immersive interactive experience.

[0103] In one embodiment, a gesture library is automatically retrieved based on the user's gesture requirements, and gesture interaction information that meets the user's gesture requirements is matched from the gesture library, including:

[0104] The user's gestures are identified by recognizing hand contour features, hand key point features, and geometric features to obtain static gesture features, and the user's gestures are identified by recognizing hand movement sequences to obtain dynamic gesture features;

[0105] After standardizing the static and dynamic gesture features, they are concatenated into a joint feature vector. Based on the joint and other positive vectors, a preliminary matching is performed with the gesture library to obtain reference gesture interaction information.

[0106] Obtain common interaction information, related interaction information, and conflicting interaction information of reference gesture interaction information. Based on the correspondence between common interaction information, related interaction information, and conflicting interaction information and feature vectors, configure corresponding basic weights for each feature vector. Obtain a weighted joint feature vector based on the basic weights.

[0107] Based on the weighted joint feature vector and the gesture library, multiple candidate gesture interaction information are obtained. Voice information and exhibit status information synchronized with the user's gesture are obtained. The first associated feature is determined based on the voice information, and the second associated feature is determined based on the exhibit status information.

[0108] Based on the multimedia exhibition hall environment, the first weight of the first associated feature and the second weight of the second associated feature are determined, and an association evaluation vector is established based on the first associated feature, the second associated feature, the first weight and the second weight.

[0109] The gesture interaction information with the highest matching degree with the associated evaluation vector is selected from multiple candidate gesture interaction information as the final gesture interaction information that meets the user's gesture needs.

[0110] In this embodiment, based on the correspondence between common interaction information, related interaction information, and conflicting interaction information and feature vectors, a corresponding basic weight is configured for each feature vector. Specifically, the common interaction information, related interaction information, and conflicting interaction information are configured in ascending order of weight, so that the resulting weighted joint feature vector pays more attention to features that cannot be accurately matched, thereby improving the subsequent matching accuracy.

[0111] In this embodiment, the first associated feature and the second associated feature are features related to voice and exhibit status, respectively, and gesture. In the multimedia exhibition hall environment, the more important the exhibit status, the greater the corresponding second weight, and vice versa.

[0112] The beneficial effects of the above design scheme are as follows: By multi-dimensional feature fusion, dynamic weight optimization, multi-modal information assistance, and environment adaptive matching, the accuracy, robustness, and scene adaptability of gesture interaction in multimedia exhibition halls are comprehensively improved. Specifically, static and dynamic features of user gestures are extracted simultaneously and standardized and concatenated into a joint feature vector, breaking through the limitations of single feature recognition. Corresponding basic weights are configured for each feature vector. Specifically, common interaction information, related interaction information, and conflicting interaction information are configured according to the weight from smallest to largest, so that the resulting weighted joint feature vector pays more attention to features that cannot be accurately matched, thereby improving the subsequent matching accuracy. Voice information and exhibit status information synchronized with gestures are introduced to improve the scene adaptability of matching and significantly enhance the intelligent interactive experience of multimedia exhibition halls.

[0113] In summary, by using voice recognition and gesture control for AI-powered intelligent interaction, and by monitoring user behavior and dwell time in real time to generate user interest heatmaps, the system can adjust recommended content for user visits in real time based on these heatmaps and automatically push relevant extended information. During the content push process, real-time user feedback is collected, and recommended content is flexibly adjusted based on this feedback, forming a closed-loop management system for recommended content. This approach can improve the interactivity, display effects, and management efficiency of multimedia exhibition halls, effectively achieving intelligent operation of multimedia exhibition halls.

[0114] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0115] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An IoT-based AI intelligent interactive control system for multimedia exhibition halls, characterized in that: include: The environmental data acquisition and analysis module is used to collect real-time environmental data of the multimedia exhibition hall, analyze the real-time environmental data of the multimedia exhibition hall, and determine the environmental analysis results of the multimedia exhibition hall. The exhibition hall environment intelligent control module is used to intelligently control the multimedia exhibition hall environment based on the analysis results of the multimedia exhibition hall environment; The exhibition hall's intelligent navigation service module is used to recommend customized routes and provide intelligent navigation services to users, thereby optimizing the user's visitor experience. The AI ​​intelligent interaction control module is used for AI intelligent interaction based on voice recognition and gesture control to ensure an immersive interactive experience for users. When users engage in AI-powered intelligent interaction based on voice recognition, the evaluation criteria include contextual logical relevance, user preference, and exhibit status adaptability. Multiple voice interaction messages are evaluated, and the voice interaction message with the highest evaluation value is selected as the final voice interaction message.

2. The IoT-based AI intelligent interactive control system for multimedia exhibition halls according to claim 1, characterized in that, When users engage in AI-powered intelligent interaction based on voice recognition, the system monitors their voice needs in real time, automatically searches the voice library based on those needs, matches voice interaction information that meets the user's voice requirements, and outputs and displays the matched voice interaction information. Users then engage in further voice interaction with the multimedia exhibition hall based on this information until the user ceases voice interaction.

3. The IoT-based AI intelligent interactive control system for multimedia exhibition halls according to claim 2, characterized in that, When users engage in AI-powered intelligent interaction based on speech recognition, the system monitors their voice requests in real time, automatically searches the voice library based on these requests, and matches voice interaction information that meets the user's voice needs, including: Obtain the user's current speech text and the speech text from the previous preset number of times. Perform intent vector feature recognition on the current speech text and the speech text from the previous preset number of times, and unify them to obtain the current intent vector and adjacent intent vectors. The current intent vector is optimized based on the reference intent vector to obtain the target intent vector, and multiple voice interaction information that can meet the user's voice needs are matched from the voice library based on the target intent vector. The evaluation criteria include contextual logical relevance, user preference, and exhibit status adaptability. Multiple voice interaction messages are evaluated, and the voice interaction message with the highest evaluation value is selected as the final voice interaction message.

4. The IoT-based AI intelligent interactive control system for multimedia exhibition halls according to claim 2, characterized in that, When users interact with AI using gesture control, the system monitors their gesture needs in real time, automatically searches the gesture library based on those needs, matches the gesture interaction information that meets the user's needs, and outputs and displays the matched gesture interaction information. This includes using user gestures to flip pages in the multimedia exhibition hall and provide an immersive interactive experience.

5. The IoT-based AI intelligent interactive control system for multimedia exhibition halls according to claim 4, characterized in that, AI-powered intelligent interaction based on voice recognition and gesture control performs the following operations: During the visit, users can interact with AI intelligently based on voice recognition and gesture control. The system also monitors users' behavior and dwell time in real time and generates user interest heatmaps based on AI visual analysis of user behavior and dwell time. Based on user interest heatmaps, the recommended content for user visits is adjusted in real time, and related extended information is automatically pushed, including other exhibits from the same period, the restoration process of the exhibit, and related research results. During the process of pushing tour content, user feedback is obtained in real time, and the recommended tour content is flexibly adjusted based on the user feedback, forming a closed-loop management of the recommended tour content.

6. The IoT-based AI intelligent interactive control system for multimedia exhibition halls according to claim 5, characterized in that, Collect real-time environmental data from the multimedia exhibition hall and perform the following operations: Based on IoT technology, temperature, humidity, light and crowd density sensors are deployed in key areas of the multimedia exhibition hall, and connected to lighting and air conditioning equipment through multi-protocol gateways to form a full-area monitoring network for the multimedia exhibition hall, providing a data foundation for intelligent environmental control. Temperature and humidity sensors are used to monitor the temperature and humidity in the multimedia exhibition hall in real time and collect temperature and humidity data of the multimedia exhibition hall; A light sensor is used to monitor the light intensity in the multimedia exhibition hall in real time and collect illuminance data of the multimedia exhibition hall. A crowd density sensor is used to monitor the crowd density in the multimedia exhibition hall in real time and collect crowd density data in the multimedia exhibition hall; Among them, the real-time environmental data of the multimedia exhibition hall is determined based on the temperature and humidity data, illuminance data, and crowd density data of the multimedia exhibition hall.

7. The IoT-based AI intelligent interactive control system for multimedia exhibition halls according to claim 6, characterized in that, Analyze the real-time environmental data of the multimedia exhibition hall and perform the following operations: The real-time environmental data of the multimedia exhibition hall is compared and analyzed with the preset standard environmental data of the multimedia exhibition hall to evaluate the consistency between the real-time environmental data and the standard environmental data of the multimedia exhibition hall, determine whether the real-time environmental data of the multimedia exhibition hall is within the range of the standard environmental data of the multimedia exhibition hall, and determine the environmental analysis results of the multimedia exhibition hall. When the real-time environmental data of the multimedia exhibition hall is within the range of the standard environmental data of the multimedia exhibition hall, the environmental analysis result of the multimedia exhibition hall is that the multimedia exhibition hall environment is normal and there is no need to perform intelligent control of the multimedia exhibition hall environment. When the real-time environmental data of the multimedia exhibition hall is outside the range of the standard environmental data of the multimedia exhibition hall, the environmental analysis result of the multimedia exhibition hall is that the multimedia exhibition hall environment is abnormal, and intelligent control of the multimedia exhibition hall environment is required.

8. The IoT-based AI intelligent interactive control system for multimedia exhibition halls according to claim 7, characterized in that, To intelligently control the multimedia exhibition hall environment, perform the following operations: The lighting equipment is automatically adjusted based on real-time monitoring of the illuminance data in the multimedia exhibition hall. When the illuminance data in the multimedia exhibition hall is lower than the standard data, the brightness of the lighting equipment is automatically increased; when the illuminance data in the multimedia exhibition hall is higher than the standard data, the brightness of the lighting equipment is automatically decreased. The air conditioning equipment is automatically adjusted based on real-time monitoring of temperature and humidity data and crowd density data in the multimedia exhibition hall. When the temperature and humidity data and crowd density data in the multimedia exhibition hall are higher than the standard data, the temperature and fan speed of the air conditioning equipment are dynamically adjusted to lower the temperature and increase the fan speed, and crowd control measures are implemented in the multimedia exhibition hall.

9. The IoT-based AI intelligent interactive control system for multimedia exhibition halls according to claim 8, characterized in that, To recommend customized routes and provide intelligent navigation services to users, thereby optimizing their visitor experience, the following actions are performed: Based on machine vision technology, high-resolution cameras are used to monitor users entering the multimedia exhibition hall in real time, obtain user behavior data, and combine this with questionnaire surveys to quickly understand user interests and preferences. Based on the layout and exhibit distribution of the multimedia exhibition hall, and combined with user behavior data and interests, a customized tour route is generated for the user, and the user is intelligently guided to the tour based on the customized tour route. For users interested in history and culture, historical artifact exhibition halls are recommended first, and intelligent explanations are provided to the users of these exhibition halls. After the users confirm the historical artifact exhibition halls, an intelligent route is planned to connect the relevant exhibits, ensuring that the users do not miss important artifacts and optimizing the visiting experience. For users interested in science and technology culture, the system prioritizes recommending science and technology culture exhibition halls and provides intelligent explanations of these halls. Once users confirm their selection of a science and technology culture exhibition hall, the system intelligently plans a route connecting related exhibits, guiding them to the science and technology experience area and rationally arranging the order of visits to improve visit efficiency and experience.

10. The IoT-based AI intelligent interactive control system for multimedia exhibition halls according to claim 4, characterized in that, The system automatically searches the gesture library based on user gesture requirements and matches gesture interaction information that meets those requirements, including: The user's gestures are identified by recognizing hand contour features, hand key point features, and geometric features to obtain static gesture features, and the user's gestures are identified by recognizing hand movement sequences to obtain dynamic gesture features; After standardizing the static and dynamic gesture features, they are concatenated into a joint feature vector. Based on the joint and other positive vectors, a preliminary matching is performed with the gesture library to obtain reference gesture interaction information. Obtain common interaction information, related interaction information, and conflicting interaction information of reference gesture interaction information. Based on the correspondence between common interaction information, related interaction information, and conflicting interaction information and feature vectors, configure corresponding basic weights for each feature vector. Obtain a weighted joint feature vector based on the basic weights. Based on the weighted joint feature vector and the gesture library, multiple candidate gesture interaction information are obtained. Voice information and exhibit status information synchronized with the user's gesture are obtained. The first associated feature is determined based on the voice information, and the second associated feature is determined based on the exhibit status information. Based on the multimedia exhibition hall environment, the first weight of the first associated feature and the second weight of the second associated feature are determined, and an association evaluation vector is established based on the first associated feature, the second associated feature, the first weight and the second weight. The gesture interaction information with the highest matching degree with the associated evaluation vector is selected from multiple candidate gesture interaction information as the final gesture interaction information that meets the user's gesture needs.

Images