Intelligent interaction system for induction screen of intelligent exhibition hall

Abstract

The invention discloses an intelligent interaction system for induction screens of an intelligent exhibition hall, which belongs to the field of intelligent interaction and comprises a plurality of induction screens and a central server, and the central server is provided with a personalized recommendation module. Each induction screen is provided with a touch recognition module, a voice recognition module and a facial recognition module, and is used for recognizing behaviors of a user according to touch, voice input and facial expression of the user and collecting interaction data of the user; the central server is connected with the induction screen, receives and analyzes interaction data of a user in real time through a network, and transmits the interaction data to the personalized recommendation module; and the personalized recommendation module analyzes to obtain interest data of the user according to the interaction data of the user, and stores the data in the central server. According to the method, the exhibit information can be dynamically adjusted, so that each visitor can obtain the exhibition content which better conforms to the personal interest, and the immersion and the participation degree are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent interaction technology, and more specifically, to an intelligent interactive system for an intelligent exhibition hall sensing screen. Background Art

[0002] Current exhibition halls, museums, and digital exhibition halls mostly adopt a fixed display model, relying primarily on static display panels, manual explanations, or simple touchscreen information interactions to present content. These halls typically have the following characteristics: Fixed exhibit arrangement: Exhibit information is fixed, all visitors receive the same information, and there is a lack of customized recommendations based on individual interests. Although some exhibition halls use touch screens or voice explanation devices, these interactive methods are limited and cannot proactively sense user interests and dynamically adjust. Each information screen or exhibit display terminal is relatively independent, lacking a unified user data management system to integrate user access data and preference information. Although modern exhibition halls have made improvements in digitization and interactivity, the following core issues still exist: the existing exhibition system cannot identify the interests of different users, and all users can only receive the same information without adjusting the exhibit content based on personal preferences. Visitors need to manually search or rely on manual explanations to obtain in-depth exhibit information, and personalized recommendations are impossible. Summary of the Invention

[0003] The present invention proposes an intelligent interactive system for smart exhibition hall sensor screens to solve the above-mentioned problems: existing exhibition systems are unable to identify the interests of different users. All users can only receive the same information, and the exhibit content cannot be adjusted according to personal preferences. Visitors need to manually search or rely on human explanations to obtain in-depth exhibit information, and personalized recommendations cannot be achieved.

[0004] Technical solution: A smart interactive system for smart exhibition hall induction screens, including several induction screens and a central server, wherein the central server is provided with a personalized recommendation module; Each of the sensing screens is equipped with a touch recognition module, a voice recognition module, and a facial recognition module to recognize user behavior and collect user interaction data based on the user's touch, voice input, and facial expressions; The central server is connected to the sensor screen, and receives and analyzes the user's interactive data in real time through the network, and transmits the interactive data to the personalized recommendation module; The personalized recommendation module analyzes the user's interest data based on the user's interaction data and stores the data in the central server. When the user uses the system again, the personalized recommendation module automatically pushes exhibit information related to the user's interests based on the historical interest data and the user's current interaction data; The interaction data includes touch operation data, voice input data, dwell time and emotional feedback; The interest data includes exhibit preferences and sentiment levels.

[0005] Preferably, the personalized recommendation module obtains the user's exhibit preferences through the following calculation method:

[0006] in, Score exhibit preference, For users to The number of touch operations, is the number of voice inputs, is the residence time, Rate the sentiment feedback, , , , is the preset weight coefficient, preference Used to generate personalized exhibit recommendation data.

[0007] Preferably, the personalized recommendation module obtains the user's emotion level by the following calculation method:

[0008] in, Score the user's facial expression recognition score, Score for voice and intonation analysis, Score the sentiment of user interaction behavior. , , is the preset weight coefficient, emotional level Exhibits for users The emotional level ranges from The larger the value, the more positive the emotion.

[0009] Preferably, the facial expression recognition score The calculation includes the following steps: Step 1: Face detection: Use the facial recognition module to identify the user's facial area; Step 2: Feature extraction: Use convolutional neural network to extract key facial feature points, including eyes, eyebrows, and mouth corner change information; Step 3: Emotion classification: Based on the facial expression recognition model, the expressions are classified into positive emotions such as smiles and surprise, neutral emotions such as calmness, and negative emotions such as anger and sadness. Step 4: Calculate sentiment scores: assign corresponding numerical scores based on the classification results , including: negative emotions Neutral sentiment Positive emotions .

[0010] Preferably, the speech and intonation analysis score The calculation includes the following steps: Step 1: Voice collection: The voice recognition module collects the user's voice input data; Step 2: Feature extraction: Perform time-frequency analysis on the speech signal to extract pitch, volume, and speaking speed parameters; Step 3: Emotion classification: Based on the emotional speech recognition model, the speech emotions are classified into positive emotions such as excitement and happiness, neutral emotions such as calmness and stability, and negative emotions such as anxiety and anger. Step 4: Calculate sentiment scores: assign corresponding numerical scores based on the classification results , including: negative emotions Neutral sentiment Positive emotions .

[0011] Preferably, the user interaction behavior emotion score The calculation is as follows: Step 1: Touch and interaction frequency analysis: Count the number of times users touch, drag, slide, and click on points of interest on the exhibits; Step 2: Dwell time analysis: Record the length of time users spend in front of an exhibit. Longer dwell time generally indicates higher interest. Step 3: Operational pattern analysis: Detect whether users frequently return, quickly skip, or stay for a long time to determine their emotional tendencies. Frequent return and quick skip indicate negative emotions, general browsing indicates neutral emotions, and long stays and deep interactions indicate negative emotions. Step 4: Calculate sentiment scores: Assign corresponding numerical scores based on user interaction patterns , including: negative emotions Neutral sentiment Positive emotions .

[0012] Preferably, the personalized recommendation module is based on the user's emotional level Exhibit preference score Perform weighted calculations and weighted preference scores The specific calculation formula is as follows:

[0013] in, is the basic exhibit preference score, For the emotional level, is the emotion weighting coefficient.

[0014] Preferably, the personalized recommendation module automatically pushes exhibit information related to the user's interests, specifically including the following steps: Step 1: Historical interest data extraction: Retrieve the user's historical interest data from the central server, including past weighted preference scores ; Collection of exhibits visited by the user Sort by displaying exhibits with high weighted preference scores first; Step 2: Current interaction data analysis: Record the user's current interaction data in the exhibition hall, including real-time touch operation data , voice input data , residence time , emotional feedback ; Calculate the user's preference for the current exhibit and conduct comprehensive analysis with historical preference data; Step 3: Personalized recommendation calculation: Combine historical interest data and current interaction data to calculate the overall preference :

[0015] in: Represents the user's preference for current interaction; and is a weighting factor used to balance the influence of historical interest and current behavior; Step 4: Generate exhibit recommendations: Based on comprehensive preference Sort the exhibits and select the exhibits with the highest priority to push to the sensor screen.

[0016] Compared with the existing technology, the advantages of the present invention are: (1) Based on the user's interactive data such as touch operation, voice input, facial expression and dwell time, the present invention analyzes user interests in real time and intelligently recommends exhibit information that matches the user's preferences, making the exhibition hall experience more personalized and dynamically adjusting exhibit information so that each visitor can obtain exhibition content that is more in line with personal interests, thereby enhancing immersion and participation.

[0017] (2) The present invention introduces emotion recognition, voice analysis and behavioral data analysis technologies to comprehensively evaluate the user's emotional state and interest tendencies, and makes personalized recommendations based on historical visit data. By accurately recommending exhibits that match the user's interests, it can effectively increase the audience's participation, making them willing to browse the exhibition hall for a longer time, thereby improving the dissemination effect of the exhibits and the operation efficiency of the exhibition hall.

[0018] (3) The present invention collects and analyzes user interaction data through a central server, which can provide exhibition hall managers with accurate audience interest analysis reports, helping to optimize exhibit layout, adjust display strategies, and improve exhibition hall operation efficiency. If a certain type of exhibit has been favored by users with high positive emotional feedback for a long time, managers can consider adding related exhibits or expanding its display content; conversely, for exhibits with poor feedback, their display methods can be adjusted or optimized in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a module block diagram of the present invention; Figure 2 This is the internal module block diagram of the sensor screen. DETAILED DESCRIPTION Example

[0020] See also Figure 1 , an intelligent interactive system for intelligent exhibition hall sensor screens, including several sensor screens and a central server, wherein the central server is provided with a personalized recommendation module; Each sensor screen is equipped with a touch recognition module, a voice recognition module, and a facial recognition module to identify user behavior and collect user interaction data based on the user's touch, voice input, and facial expressions; The central server is connected to the sensor screen. The central server receives and analyzes the user's interactive data in real time through the network and transmits the interactive data to the personalized recommendation module. The personalized recommendation module analyzes the user's interest data based on the user's interaction data and saves the data in the central server. When the user uses the system again, the personalized recommendation module automatically pushes exhibit information related to the user's interests based on the historical interest data and the user's current interaction data; Interaction data includes touch operation data, voice input data, dwell time and emotional feedback; Interest data includes exhibit preferences and sentiment ratings.

[0021] The personalized recommendation module obtains the user's exhibit preferences through the following calculation method:

[0022] in, Score exhibit preference, Exhibits for users The number of touch operations, is the number of voice inputs, is the residence time, Rate the sentiment feedback, , , , is the preset weight coefficient, preference Used to generate personalized exhibit recommendation data.

[0023] The personalized recommendation module obtains the user's emotion level through the following calculation method:

[0024] in, Score the user's facial expression recognition score, Score for voice and intonation analysis, Score the sentiment of user interaction behavior. , , is the preset weight coefficient, emotional level Exhibits for users The emotional level ranges from The larger the value, the more positive the emotion.

[0025] Facial expression recognition score The calculation includes the following steps: Step 1: Face detection: Use the facial recognition module to identify the user's facial area; Step 2: Feature extraction: Use convolutional neural network to extract key facial feature points, including eyes, eyebrows, and mouth corner change information; Step 3: Emotion classification: Based on the facial expression recognition model, the expressions are classified into positive emotions such as smiles and surprise, neutral emotions such as calmness, and negative emotions such as anger and sadness. Step 4: Calculate sentiment scores: assign corresponding numerical scores based on the classification results , including: negative emotions Neutral sentiment Positive emotions .

[0026] Voice and Intonation Analysis Score The calculation includes the following steps: Step 1: Voice collection: The voice recognition module collects the user's voice input data; Step 2: Feature extraction: Perform time-frequency analysis on the speech signal to extract pitch, volume, and speaking speed parameters; Step 3: Emotion classification: Based on the emotional speech recognition model, the speech emotions are classified into positive emotions such as excitement and happiness, neutral emotions such as calmness and stability, and negative emotions such as anxiety and anger. Step 4: Calculate sentiment scores: assign corresponding numerical scores based on the classification results , including: negative emotions Neutral sentiment Positive emotions .

[0027] User interaction behavior sentiment score The calculation is as follows: Step 1: Touch and interaction frequency analysis: Count the number of times users touch, drag, slide, and click on points of interest on the exhibits; Step 2: Dwell time analysis: Record the length of time users spend in front of an exhibit. Longer dwell time generally indicates higher interest. Step 3: Operational pattern analysis: Detect whether users frequently return, quickly skip, or stay for a long time to determine their emotional tendencies. Frequent return and quick skip indicate negative emotions, general browsing indicates neutral emotions, and long stays and deep interactions indicate negative emotions. Step 4: Calculate sentiment scores: Assign corresponding numerical scores based on user interaction patterns , including: negative emotions Neutral sentiment Positive emotions .

[0028] Personalized recommendation module based on user's emotional level Exhibit preference score Perform weighted calculations and weighted preference scores The specific calculation formula is as follows:

[0029] in, is the basic exhibit preference score, For the emotional level, is the emotion weighting coefficient.

[0030] The personalized recommendation module automatically pushes exhibit information related to the user's interests, specifically including the following steps: Step 1: Historical interest data extraction: Retrieve the user's historical interest data from the central server, including past weighted preference scores ; Collection of exhibits visited by the user Sort by displaying exhibits with high weighted preference scores first; Step 2: Current interaction data analysis: Record the user's current interaction data in the exhibition hall, including real-time touch operation data , voice input data , residence time , emotional feedback ; Calculate the user's preference for the current exhibit and conduct comprehensive analysis with historical preference data; Step 3: Personalized recommendation calculation: Combine historical interest data and current interaction data to calculate the overall preference :

[0031] in: Represents the user's preference for current interaction; and is a weighting factor used to balance the influence of historical interest and current behavior; Step 4: Generate exhibit recommendations: Based on comprehensive preference Sort the exhibits and select the exhibits with the highest priority to push to the sensor screen.

[0032] Example 2: An intelligent interactive system for intelligent exhibition hall sensor screens, comprising a plurality of sensor screens and a central server, wherein the central server is provided with a personalized recommendation module; Each sensor screen is equipped with a touch recognition module, a voice recognition module, and a facial recognition module to identify user behavior and collect user interaction data based on the user's touch, voice input, and facial expressions; The central server is connected to the sensor screen. The central server receives and analyzes the user's interactive data in real time through the network and transmits the interactive data to the personalized recommendation module. The personalized recommendation module analyzes the user's interest data based on the user's interaction data and saves the data in the central server. When the user uses the system again, the personalized recommendation module automatically pushes exhibit information related to the user's interests based on the historical interest data and the user's current interaction data; Interaction data includes touch operation data, voice input data, dwell time and emotional feedback; Interest data includes exhibit preferences and sentiment ratings.

[0033] Implementation steps: 1. Upon entering the museum, the system uses facial recognition to identify the user (if necessary) and creates a temporary user profile for the user. If this is a first-time visitor, the system initializes the visit using behavioral data and recommends a general list of exhibits.

[0034] 2. When a user approaches an exhibit and browses its information on the touchscreen, the system records their touch operations (e.g., clicks, swipes, viewing duration, etc.). If the user asks a question by voice, the speech recognition module converts the speech into text and analyzes it to understand the user's areas of interest.

[0035] At the same time, the facial recognition module detects the user's expression and determines their emotional state: if the user is smiling, the system assesses this as positive emotion and increases the recommendation weight of the exhibit. If the user's expression is cold or doubtful, the system may reduce the recommendation weight of the exhibit.

[0036] 3. The system calculates a sentiment level based on facial expressions, voice tone, and dwell time, and then uses a formula to adjust the exhibit preference score. If a user spends a long time in front of an exhibit and expresses positive emotions, the system will recommend more similar exhibits.

[0037] For example, if a user shows extremely high emotional feedback when viewing a modern art exhibit, the system will recognize the user's preference for modern art and prioritize recommending similar works of art when the user browses other exhibits.

[0038] 4. Based on the user's historical interaction data and real-time sentiment feedback, the personalized recommendation module generates a dynamically adjusted list of exhibit recommendations for the user. If the user shows interest in subsequent exhibits, the system will gradually update the recommendations.

[0039] For example, if the user shows a strong interest in the sculpture exhibits in the first part of the exhibition, the system will recommend more sculpture-related content in the future, and may even push information about artists or other exhibits related to sculptures to the user.

[0040] 5. The system generates and pushes personalized exhibit recommendations based on the exhibit preference score and sentiment weighting. If an exhibit's preference score is greater than zero, the system proactively pushes more relevant content to the user, increasing the frequency of that exhibit's display. If the preference score is low, the exhibit's display frequency is reduced, and in some cases, it may even be hidden.

[0041] 6. All user interaction data is uploaded to a central server in real time. Museum managers can use the backend data analysis system to analyze visitor interest trends, emotional feedback, and other data to optimize exhibit layout and display strategies. If certain exhibits consistently perform poorly, managers can adjust the display or provide more explanations and interactive content.

[0042] Among them, facial expression recognition: uses convolutional neural networks (CNN) for image processing and facial feature point extraction to judge the user's emotions (such as smiling, frowning, relaxation, etc.).

[0043] Speech and Intonation Analysis: Applying Emotional Speech Recognition (SER) technology, the user's emotions are judged by analyzing the pitch, speed, and tone of the voice, and the final score is generated based on the emotional feedback.

[0044] Personalized recommendation algorithm: Based on the user's historical interest data, interactive behavior and current emotional feedback, the exhibit preference score is calculated through weighted calculation and intelligent recommendation results are generated.

[0045] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent exhibition hall induction screen intelligent interactive system, including several induction screens and a central server, characterized in that: The central server is provided with a personalized recommendation module; Each of the sensing screens is configured with a touch recognition module, a voice recognition module and a facial recognition module, which recognize the user's behavior and collect the user's interaction data based on the user's touch, voice input and facial expression; The central server is connected to the sensor screen, and receives and analyzes the user's interactive data in real time through the network, and transmits the interactive data to the personalized recommendation module; The personalized recommendation module analyzes the user's interest data based on the user's interaction data and stores the data in the central server. When the user uses the system again, the personalized recommendation module automatically pushes exhibit information related to the user's interest based on the historical interest data and the user's current interaction data; The interaction data includes touch operation data, voice input data, dwell time and emotional feedback; The interest data includes exhibit preferences and sentiment ratings.

2. According to claim 1, the intelligent exhibition hall induction screen intelligent interactive system is characterized in that: The personalized recommendation module obtains the user's exhibit preferences through the following calculation method: in, Score exhibit preference, For users to The number of touch operations, is the number of voice inputs, is the dwell time, Rate the sentiment feedback, , , , is the preset weight coefficient, preference Used to generate personalized exhibit recommendation data.

3. The intelligent interactive system of the intelligent exhibition hall sensing screen according to claim 1 is characterized in that: The personalized recommendation module obtains the user's emotion level through the following calculation method: in, Score the user's facial expression recognition. Score for voice and intonation analysis, Score the sentiment of user interaction behavior. , , is the preset weight coefficient, emotional level For users to The emotional level ranges from The larger the value, the more positive the emotion.

4. The intelligent interactive system of the intelligent exhibition hall sensing screen according to claim 3 is characterized in that: The facial expression recognition score The calculation includes the following steps: Step 1: Face detection: Use the facial recognition module to identify the user's facial area; Step 2: Feature extraction: Use convolutional neural network to extract key facial feature points, including eyes, eyebrows, and mouth corner change information; Step 3: Emotion classification: Based on the facial expression recognition model, the expressions are classified into positive emotions such as smile and surprise, neutral emotions such as calmness, and negative emotions such as anger and sadness; Step 4: Calculate sentiment scores: assign corresponding numerical scores based on the classification results , including: negative emotions ; Neutral sentiment ; Positive emotions .

5. The intelligent interactive system of the intelligent exhibition hall sensing screen according to claim 4 is characterized in that: The speech and intonation analysis score The calculation includes the following steps: Step 1: Voice collection: collect the user's voice input data through the voice recognition module; Step 2: Feature extraction: Perform time-frequency analysis on the speech signal to extract pitch, volume, and speech rate parameters; Step 3: Emotion classification: Based on the emotional speech recognition model, the speech emotions are classified into positive emotions such as excitement and happiness, neutral emotions such as calmness and stability, and negative emotions such as anxiety and anger; Step 4: Calculate sentiment scores: assign corresponding numerical scores based on the classification results , including: negative emotions ; Neutral sentiment ; Positive emotions .

6. The intelligent interactive system of the intelligent exhibition hall sensing screen according to claim 5 is characterized in that: The user interaction behavior sentiment score The calculation is as follows: Step 1: Touch and interaction frequency analysis: Count the number of times users touch the exhibits, drag and slide, and click on points of interest; Step 2: Dwell time analysis: record the length of time users spend in front of an exhibit. Longer dwell time usually indicates higher interest. Step 3: Operation mode analysis: Detect whether users frequently return, quickly skip, or stay for a long time to judge their emotional tendencies. Frequent return and quick skip indicate negative emotions, general browsing indicates neutral emotions, and long stay and deep interaction indicate negative emotions. Step 4: Calculate sentiment scores: Assign corresponding numerical scores based on user interaction patterns , including: negative emotions ; Neutral sentiment ; Positive emotions .

7. The intelligent interactive system of the intelligent exhibition hall induction screen according to claim 6 is characterized in that: The personalized recommendation module is based on the user's emotional level Exhibit preference score Perform weighted calculations, weighted preference scores The specific calculation formula is as follows: in, is the basic exhibit preference score, For the emotional level, is the emotion weighting coefficient.

8. The intelligent interactive system of the intelligent exhibition hall sensing screen according to claim 7 is characterized in that: The personalized recommendation module automatically pushes exhibit information related to the user's interests, specifically including the following steps: Step 1: Historical interest data extraction: Retrieve the user’s historical interest data from the central server, including past weighted preference scores ; A collection of exhibits that the user has visited Sorting is performed to give priority to exhibits with high weighted preference scores; Step 2: Current interaction data analysis: record the user's current interaction data in the exhibition hall, including real-time touch operation data , voice input data , residence time , emotional feedback ; Calculate the user's preference for the current exhibit and conduct a comprehensive analysis with historical preference data; Step 3: Personalized recommendation calculation: Combine historical interest data and current interaction data to calculate the overall preference : in: Represents the user's preference for current interaction; and is a weighting factor used to balance the impact of historical interest and current behavior; Step 4: Generate exhibit recommendations: Based on comprehensive preference Sort the exhibits and select the exhibits with the highest priority to push to the sensor screen.

Images