Interactive LED man-machine interaction display system with 3D effect

Abstract

The invention discloses an interactive LED man-machine interaction display system with a 3D effect, which belongs to the technical field of man-machine interaction display systems and comprises a sensor module, a personalized recommendation module, a positioning tracking module, a state recognition module and an interaction control module. The sensor module, the personalized recommendation module, the positioning tracking module, the state recognition module and the interaction control module are sequentially in communication connection, and the data acquisition module acquires video and image data and handle data of five groups of camera devices and transmits the data to the personalized recommendation module, the positioning tracking module and the state recognition module. The personalized recommendation module analyzes the collected image data to obtain player data, including player gender data; according to the interactive LED man-machine interaction display system with the 3D effect, personalized recommendation can be automatically carried out according to the gender and age of the user by recognizing the gender and age of the user, and game experience can be conveniently and rapidly carried out.

Description

Technical Field

[0001] The invention belongs to the technical field of human-machine interactive display systems, and in particular is an interactive LED human-machine interactive display system with 3D effects. Background Art

[0002] In today's era of rapid digital development, 3D display technology is gradually emerging. 3D technology can create realistic three-dimensional visual effects, making the audience feel as if they are in the scene. By using 3D technology, people can feel a strong sense of reality, so 3D technology is also applied to the game environment;

[0003] However, the existing human-computer interaction system with 3D technology usually only has the function of giving people an immersive feeling, and does not have the function of recommending games according to the age and gender of different users. At the same time, the existing human-computer interaction system with 3D technology does not have the function of adjusting the game frequency according to the user's game operation status. If all users experience the game with the same frequency, the experience may be poor. Therefore, an interactive LED human-computer interaction display system with 3D effect is proposed. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes an interactive LED human-machine interactive display system with 3D effect, which can effectively improve the user experience.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] An interactive LED human-machine interactive display system with 3D effect, comprising a sensor module, a personalized recommendation module, a positioning tracking module, a state recognition module, and an interactive control module, wherein the sensor module, the personalized recommendation module, the positioning tracking module, the state recognition module, and the interactive control module are sequentially connected in communication;

[0007] The data acquisition module collects video and image data of five camera devices and handle data, and transmits the data to the personalized recommendation module, the positioning tracking module and the state recognition module;

[0008] The personalized recommendation module obtains player data, including player gender data and age data, by analyzing the collected image data, and makes personalized recommendations based on the known data to recommend appropriate game items;

[0009] The positioning tracking module receives the video data collected in real time, processes the video data, and performs positioning tracking on the user;

[0010] The state recognition module recognizes the user's facial expression and voice through real-time collected camera data, determines the user's state, and adjusts the frequency progress of the game according to the user's state;

[0011] The interactive control module is used to receive various input signals from handles, gestures, and 3D glasses, convert these signals into recognizable instructions, and finally execute corresponding operations.

[0012] Furthermore, the data acquisition module collects video and image data of five camera devices and handle data, and transmits the data to the personalized recommendation module, the positioning tracking module and the state recognition module. The processing process is as follows:

[0013] Capture video and image data of the camera device in real time, and obtain operation data of the handle in real time, the operation data of the handle includes position, movement direction, and button status;

[0014] The data collected from the five sets of cameras and handles are integrated into a unified data stream, which is then transmitted to the personalized recommendation module, positioning tracking module, and state recognition module for processing.

[0015] Four groups of cameras are used to capture the user's movement status for tracking and positioning, and one group of cameras is used to identify the user's gender and age for personalized recommendations. After identification is completed, status recognition will be performed to adjust the overall frequency of the game.

[0016] Furthermore, the personalized recommendation module obtains player data, including player gender data and age data, by analyzing the collected image data, and makes personalized recommendations based on the known data. The process of recommending suitable game projects is as follows:

[0017] Receiving image data captured by a camera device, using a face detection algorithm to detect and locate facial features of the player in the image, extracting features from the detected face, and applying an age estimation model to analyze the extracted facial features to predict the age range of the player, and finally using a gender recognition algorithm to analyze the facial features to determine the gender of the player;

[0018] The extracted player age and gender data are integrated into a structured data set for personalized recommendations. Based on the known user age and gender, game items that meet the player's characteristics are queried from the game recommendation database, and then matched according to the characteristics of the game and the user's personal data. Based on the output of the recommendation algorithm, the final recommendation list is generated and the recommendation results are displayed to the user through a display for the user to choose.

[0019] Through personalized recommendations, users can quickly get recommended games, improving the player's gaming experience.

[0020] Furthermore, the age estimation model is applied to analyze the extracted facial features to predict the age range of the player, and finally the gender recognition algorithm is used to analyze the facial features to determine the gender of the player. The processing process is as follows:

[0021] The specific formula of the age estimation model is:

[0022] in, is the predicted value of the model, x is the input feature, f(x) is the prediction function, and the specific prediction function f(x) is: Among them, β0 is the intercept, which is a constant term, β1,β2,...,β n are regression coefficients, X1, X2, ..., X n Represents the characteristics of the input;

[0023] The specific formula of the gender recognition algorithm is:

[0024] Among them, p(y=k|x) is the probability that the input feature X belongs to category k, z k is the prediction score for category k, is the exponential function value of the prediction score of category k, It is the sum of the exponential function values ​​of the prediction scores of all categories.

[0025] Furthermore, the positioning tracking module receives the video data collected in real time, processes the video data, and performs positioning tracking processing on the user as follows:

[0026] The user wears marked 3D glasses and holds the operating handle to become the main operator. Four sets of cameras begin to capture the user's behavior and movements, track them using computer vision technology, update the user's position in real time, and then merge the tracked position data of the 3D glasses and the handle. Finally, based on the tracked data, the user's position and posture are determined and converted into three-dimensional space.

[0027] The control host should be an i5 12th generation CPU or above, with a memory ≥ 16G, a 512G SSD or above, and a graphics card of Nvidia GTX1060 or equivalent.

[0028] Furthermore, the state recognition module recognizes the user's facial expression and voice through real-time collected camera data, determines the user's state, and adjusts the frequency progress of the game according to the user's state. The processing process is as follows:

[0029] Process the video data of the user's gaming status, use deep learning models to analyze and identify different facial expressions, use voice emotion analysis technology to analyze voice features, and combine the results of facial expression and voice emotion recognition to determine the user's current status;

[0030] When the user smiles more than 3 times per minute and the tone is high, the speech speed is fast and the volume is loud, it means that the user is in an excited state and the system will automatically increase the game frequency. When the user shows a frustrated expression more than 3 times per minute and does not have a high pitch or high speech speed, it means that the user is in a frustrated state and the system will automatically slow down the game frequency. When the user does not have these states, it means that the user is in a stable state and continues to maintain the current game frequency.

[0031] Automatically adjusting the game frequency according to the user's status can effectively improve the user's gaming experience and meet the user's needs.

[0032] Furthermore, the interactive control module is used to receive various input signals from the handle, gestures, and 3D glasses, and then convert these signals into recognizable instructions, and finally execute the corresponding operation processing process as follows:

[0033] Parse the specific content of the handle input signal, including the handle buttons and joystick directions, and then parse the data of the gesture recognition device into specific gesture actions. At the same time, parse the spatial position and direction data of the 3D glasses, map the parsed data to predefined instructions, pass the generated instructions to the system, and execute the instruction operations.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] By setting up a personalized recommendation module, the user's age and gender can be automatically determined, and games can be recommended based on the user's age and gender, without the need for the user to manually select, saving manual operation time and allowing the user to quickly enter the operation state. At the same time, based on personalized recommendations, the user can also quickly find matching games, improving user satisfaction;

[0036] By setting up a state recognition module, the system can automatically determine the user's game state. By classifying the user's game state into excitement, frustration, and calm, and adjusting the game frequency according to these three states, the game frequency will be increased in the excited state, the current game frequency will be maintained in the calm state, and the game frequency will be slowed down in the frustrated state. This can effectively improve the user's sense of accomplishment and thus improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a block diagram of an interactive LED human-machine interactive display system with 3D effects according to the present invention. DETAILED DESCRIPTION

[0038] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] like Figure 1 As shown, an interactive LED human-computer interactive display system with 3D effect includes a sensor module, a personalized recommendation module, a positioning tracking module, a state recognition module, and an interactive control module, wherein the sensor module, the personalized recommendation module, the positioning tracking module, the state recognition module, and the interactive control module are sequentially connected in communication;

[0040] The data acquisition module collects video and image data of five camera devices and handle data, and transmits the data to the personalized recommendation module, the positioning tracking module and the state recognition module;

[0041] In this embodiment, the data acquisition module collects video and image data and handle data of five sets of camera devices, and transmits the data to the personalized recommendation module, the positioning tracking module and the state recognition module. The processing process is as follows:

[0042] Capture video and image data of the camera device in real time, and obtain operation data of the handle in real time, the operation data of the handle includes position, movement direction, and button status;

[0043] The data collected from the five sets of cameras and handles are integrated into a unified data stream, which is then transmitted to the personalized recommendation module, positioning tracking module, and state recognition module for processing.

[0044] It should be noted that real-time acquisition of controller data can provide more precise control. By continuously acquiring information such as the position, direction, and button status of the controller, the system can accurately understand the participant's intention and convert it into actions in the virtual scene. This allows participants to more finely control the movement, rotation, and operation of virtual characters or objects, improving the playability and challenge of the game.

[0045] The personalized recommendation module obtains player data, including player gender data and age data, by analyzing the collected image data, and makes personalized recommendations based on the known data to recommend appropriate game items;

[0046] In this embodiment, the personalized recommendation module obtains player data, including player gender data and age data, by analyzing the collected image data, and makes personalized recommendations based on the known data. The process of recommending suitable game items is as follows:

[0047] Receiving image data captured by a camera device, using a face detection algorithm to detect and locate facial features of the player in the image, extracting features from the detected face, and applying an age estimation model to analyze the extracted facial features to predict the age range of the player, and finally using a gender recognition algorithm to analyze the facial features to determine the gender of the player;

[0048] The extracted player age and gender data are integrated into a structured data set for personalized recommendations. Based on the known user age and gender, game items that meet the player's characteristics are queried from the game recommendation database, and then matched according to the characteristics of the game and the user's personal data. Based on the output of the recommendation algorithm, the final recommendation list is generated and the recommendation results are displayed to the user through a display for the user to choose.

[0049] It should be noted that when this system is started, the user stands in front of the display, and the camera used to capture images will capture the user's facial image to determine the user's age and gender, thereby facilitating personalized recommendations.

[0050] In this embodiment, the age estimation model is applied to analyze the extracted facial features to predict the age range of the player, and finally the gender recognition algorithm is used to analyze the facial features to determine the gender of the player. The processing process is as follows:

[0051] The specific formula of the age estimation model is:

[0052] in, is the predicted value of the model, x is the input feature, f(x) is the prediction function, and the specific prediction function f(x) is: Among them, β0 is the intercept, which is a constant term, β1,β2,...,β n are regression coefficients, X1, X2, ..., X n Represents the characteristics of the input;

[0053] The specific formula of the gender recognition algorithm is:

[0054] Among them, p(y=k|x) is the probability that the input feature X belongs to category k, z k is the prediction score for category k, is the exponential function value of the prediction score of category k, It is the sum of the exponential function values ​​of the prediction scores of all categories.

[0055] The positioning tracking module receives the video data collected in real time, processes the video data, and performs positioning tracking on the user;

[0056] In this embodiment, the positioning tracking module receives the video data collected in real time, processes the video data, and performs positioning tracking on the user as follows:

[0057] The user wears marked 3D glasses and holds the operating handle to become the main operator. Four sets of cameras begin to capture the user's behavior and movements, track them using computer vision technology, update the user's position in real time, and then merge the tracked position data of the 3D glasses and the handle. Finally, based on the tracked data, the user's position and posture are determined and converted into three-dimensional space.

[0058] It should be noted that the person wearing the marked 3D glasses and holding the operating handle is the main operator, and the entire experience process is controlled by the main operator. The person wearing the unmarked 3D glasses is the participant, and the participant can wear the unmarked 3D glasses to watch the entire experience, and there is no limit on the number of participants.

[0059] The state recognition module recognizes the user's facial expression and voice through real-time collected camera data, determines the user's state, and adjusts the frequency progress of the game according to the user's state;

[0060] In this embodiment, the state recognition module recognizes the user's facial expression and voice through real-time collected camera data, determines the user's state, and adjusts the frequency progress of the game according to the user's state. The processing process is as follows:

[0061] Process the video data of the user's gaming status, use deep learning models to analyze and identify different facial expressions, use voice emotion analysis technology to analyze voice features, and combine the results of facial expression and voice emotion recognition to determine the user's current status;

[0062] When the user smiles more than 3 times per minute and the tone is high, the speech speed is fast and the volume is loud, it means that the user is in an excited state and the system will automatically increase the game frequency. When the user shows a frustrated expression more than 3 times per minute and does not have a high pitch or high speech speed, it means that the user is in a frustrated state and the system will automatically slow down the game frequency. When the user does not have these states, it means that the user is in a stable state and continues to maintain the current game frequency.

[0063] It should be noted that there are five groups of cameras in total, four of which are used to locate and track users, and one group of cameras is used to identify the user's identity and age, as well as facial expressions and sounds made in the game. By processing facial expressions and sounds made in the game, the game frequency can be automatically adjusted to improve the user's gaming experience.

[0064] The interactive control module is used to receive various input signals from the handle, gestures, and 3D glasses, and then convert these signals into recognizable instructions, and finally perform corresponding operations.

[0065] In this embodiment, the interactive control module is used to receive various input signals from the handle, gestures, and 3D glasses, and then convert these signals into recognizable instructions, and finally execute the corresponding operation processing process as follows:

[0066] Parse the specific content of the handle input signal, including the handle buttons and joystick directions, and then parse the data of the gesture recognition device into specific gesture actions. At the same time, parse the spatial position and direction data of the 3D glasses, map the parsed data to predefined instructions, pass the generated instructions to the system, and execute the instruction operations.

[0067] It should be noted that, through these steps, the interactive control module can efficiently process signals from different input devices and perform corresponding operations according to the user's operation requirements, thereby improving the interactivity of the system and the user experience.

[0068] This system always operates in accordance with strict privacy protection and compliance principles. In the process of collecting player image data to obtain player gender, age and other data, explicit authorization has been obtained from the players in advance. This authorization process is clear and transparent. Players voluntarily agree to data collection on the premise of fully knowing the purpose and scope of data use and related security measures. Through this legal and compliant method that respects the player's wishes, the personalized recommendation module can deeply explore the player's characteristics and preferences based on the collected authorized data, so as to carefully screen and recommend the most suitable game projects for the players, aiming to create a personalized, high-quality, safe and reliable game experience environment for players, and ensure that the rights and interests of players are respected and protected in all aspects.

[0069] In the embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation; the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the method of this embodiment.

[0070] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An interactive LED human-computer interactive display system with 3D effect, characterized in that: It includes a sensor module, a personalized recommendation module, a positioning tracking module, a state recognition module, and an interactive control module, wherein the sensor module, the personalized recommendation module, the positioning tracking module, the state recognition module, and the interactive control module are sequentially connected in communication; The data acquisition module collects video and image data of five camera devices and handle data, and transmits the data to the personalized recommendation module, the positioning tracking module and the state recognition module; The personalized recommendation module obtains player data, including player gender data and age data, by analyzing the collected image data, and makes personalized recommendations based on the known data to recommend appropriate game items; The positioning tracking module receives the video data collected in real time, processes the video data, and performs positioning tracking on the user; The state recognition module recognizes the user's facial expression and voice through real-time collected camera data, determines the user's state, and adjusts the frequency progress of the game according to the user's state; The interactive control module is used to receive various input signals from handles, gestures, and 3D glasses, convert these signals into recognizable instructions, and finally execute corresponding operations.

2. The interactive LED human-machine interactive display system with 3D effect according to claim 1, characterized in that: The data acquisition module collects video and image data of five camera devices and handle data, and transmits the data to the personalized recommendation module, the positioning tracking module and the state recognition module. The processing process is as follows: Capture video and image data of the camera device in real time, and obtain operation data of the handle in real time, the operation data of the handle includes position, movement direction, and button status; The data collected from the five sets of cameras and handles are integrated into a unified data stream, which is then transmitted to the personalized recommendation module, positioning tracking module, and state recognition module for processing.

3. The interactive LED human-machine interactive display system with 3D effect according to claim 1, characterized in that: The personalized recommendation module obtains player data, including player gender data and age data, by analyzing the collected image data, and makes personalized recommendations based on the known data. The process of recommending suitable game projects is as follows: Receiving image data captured by a camera device, using a face detection algorithm to detect and locate facial features of the player in the image, extracting features from the detected face, and applying an age estimation model to analyze the extracted facial features to predict the age range of the player, and finally using a gender recognition algorithm to analyze the facial features to determine the gender of the player; The extracted player age and gender data are integrated into a structured data set for personalized recommendations. Based on the known user age and gender, game items that meet the player's characteristics are queried from the game recommendation database, and then matched according to the characteristics of the game and the user's personal data. Based on the output of the recommendation algorithm, the final recommendation list is generated and the recommendation results are displayed to the user through a display for the user to choose.

4. The interactive LED human-machine interactive display system with 3D effect according to claim 3, characterized in that: The age estimation model is applied to analyze the extracted facial features to predict the age range of the player, and finally the gender recognition algorithm is used to analyze the facial features to determine the gender of the player. The processing process is as follows: The specific formula of the age estimation model is: in, is the predicted value of the model, x is the input feature, f(x) is the prediction function, and the specific prediction function f(x) is: Among them, β0 is the intercept, which is a constant term, β1,β2,...,β n are regression coefficients, X1, X2, ..., X n Represents the characteristics of the input; The specific formula of the gender recognition algorithm is: Among them, p(y=k|x) is the probability that the input feature X belongs to category k, z k is the prediction score for category k, is the exponential function value of the prediction score of category k, It is the sum of the exponential function values ​​of the prediction scores of all categories.

5. The interactive LED human-machine interactive display system with 3D effect according to claim 1, characterized in that: The positioning tracking module receives the video data collected in real time, processes the video data, and performs positioning tracking on the user as follows: The user wears marked 3D glasses and holds the operating handle to become the main operator. Four sets of cameras begin to capture the user's behavior and movements, track them using computer vision technology, update the user's position in real time, and then merge the tracked position data of the 3D glasses and the handle. Finally, based on the tracked data, the user's position and posture are determined and converted into three-dimensional space.

6. The interactive LED human-machine interactive display system with 3D effect according to claim 1, characterized in that: The state recognition module recognizes the user's facial expression and voice through real-time collected camera data, determines the user's state, and adjusts the frequency progress of the game according to the user's state. The processing process is as follows: Process the video data of the user's gaming status, use deep learning models to analyze and identify different facial expressions, use voice emotion analysis technology to analyze voice features, and combine the results of facial expression and voice emotion recognition to determine the user's current status; When the user smiles more than 3 times per minute and the tone is high, the speech speed is fast and the volume is loud, it means that the user is in an excited state and the system will automatically increase the game frequency. When the user shows a frustrated expression more than 3 times per minute and does not have a high pitch or high speech speed, it means that the user is in a frustrated state and the system will automatically slow down the game frequency. When the user does not have these states, it means that the user is in a stable state and continues to maintain the current game frequency.

7. The interactive LED human-machine interactive display system with 3D effect according to claim 1, characterized in that: The interactive control module is used to receive various input signals from the handle, gestures, and 3D glasses, and then convert these signals into recognizable instructions, and finally execute the corresponding operation processing process as follows: Parse the specific content of the handle input signal, including the handle buttons and joystick directions, and then parse the data of the gesture recognition device into specific gesture actions. At the same time, parse the spatial position and direction data of the 3D glasses, map the parsed data to predefined instructions, pass the generated instructions to the system, and execute the instruction operations.

Images