Wired gamepad man-machine interaction system based on multi-modal feedback

Abstract

The invention relates to the technical field of gamepad man-machine interaction, in particular to a wired gamepad man-machine interaction system based on multi-modal feedback. The system comprises a reaction finishing unit, a blending unit and a feedback recording unit. Before a game, a user classifies the positions and functions of the keys, then collects the reaction time of clicking each key through the reaction collection module, and then ranks the reaction time through the allocation unit to obtain the reaction time sequence of each key; then, the key adjusting module adjusts the functions of each key according to the response time sequence and the grading data of the key positions and the functions, so that the keys on the gamepad are in accordance with the use of a user, and the contradiction between the fixed key positions on the gamepad and the operation of the user is avoided; the fitness between the gamepad and a user is improved, and the user can play games conveniently.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction of game controllers, and more specifically, to a wired game controller human-computer interaction system based on multimodal feedback. Background Art

[0002] In order to be applicable to current 3D and other various games, game controllers are currently updated with many game buttons added. These include direction control buttons on the front of the game controller, a direction control joystick (left joystick), basic action buttons A, B, X, Y, and an action joystick (right joystick). Some game controllers also add buttons for special games on the front of the controller, such as a joystick or button for changing the viewing angle, as well as LB and RB buttons on the side of the joystick, and multiple buttons such as LT, RT, M1, and M2 on the bottom of the game controller. By setting various buttons, the game controller can be applicable to most current games. After purchasing the game controller, the user will operate the game controller according to the instruction manual of the game controller so that the game controller can be used in the game.

[0003] However, during the production process of the game controller, the instructions corresponding to the game buttons are fixed, that is, the direction keys, jump, crouch, attack, etc. buttons are determined at the time of factory. However, everyone has different reaction times to buttons in different positions. Especially after the user has used it for a period of time, they will form their own control methods. At the same time, there are also certain differences in the functions corresponding to the buttons in different games. If the same set of fixed game key positions is adopted, the functions of the newly added buttons cannot adapt to the user, which will lead to inconvenient game operations. At this time, the buttons in different positions cannot be set according to the specific situation of the user, which will cause conflicts between the user's gaming habits and the game controller, resulting in a reduction in the user's gaming experience and also making the game controller not meet the user's usage. Summary of the Invention

[0004] The purpose of the present invention is to provide a wired game controller human-computer interaction system based on multimodal feedback to solve the problems raised in the above background art.

[0005] To achieve the above purpose, a wired game controller human-computer interaction system based on multimodal feedback is provided, including a reaction sorting unit, a deployment unit, and a feedback recording unit; Before the game: The reaction sorting unit collects the reaction time of each button click by means of game instructions, and at the same time guides the user to divide the priority of the button functions on the game controller; The deployment unit sorts according to the reaction time of each button click collected by the reaction arrangement unit, and recommends button function adjustment according to the arranged order and the priority of the button functions set by the user; In the game: The feedback recording unit counts the number of each button clicked by the user. The deployment unit counts the proficiency according to the data of each button click counted by the feedback recording unit. At the same time, the vibration feedback of the gamepad is used to record the game reaction, and then the best reaction is predicted by means of the number of button clicks and the vibration feedback, and the predicted result is transmitted to the deployment unit; After the game: The deployment unit analyzes the data transmitted by the reaction arrangement unit and the feedback recording unit. After the analysis, it recommends function adjustment for the currently defined button functions. The reaction arrangement unit adjusts the button functions, and the deployment unit records the adjusted buttons.

[0006] As a further improvement of this technical solution, the reaction arrangement unit includes a reaction collection module and a button definition module; The button definition module records the position and corresponding functions of each button of the gamepad, and at the same time collects the data of the function grading performed by the user on different button functions; The reaction collection module collects the reaction time of the user clicking each button, and transmits the collected reaction time data to the deployment unit for storage and analysis by the deployment unit; During the process of the reaction collection module collecting the reaction time of the user for each button, the game instructions will repeatedly display each button, so that the reaction collection module can collect multiple reaction time data of the same button.

[0007] As a further improvement of this technical solution, the deployment unit includes a reaction sorting module and a button adjustment module; The reaction sorting module receives the data of the reaction time of each button collected by the reaction collection module, and sorts them sequentially according to the reaction duration of each button click; The button adjustment module analyzes the adjustment recommendation of the button function by combining the order arranged by the reaction sorting module and the level division of the button function, and the recommended instructions are displayed through the display.

[0008] As a further improvement of this technical solution, after the button adjustment module forms the button adjustment recommendation instruction, the button definition module adjusts the button function. The adjusted button is stored by the button adjustment module, and after the storage, the gamepad executes the instruction, and then the reaction arrangement unit and the deployment unit repeat the execution.

[0009] As a further improvement of this technical solution, the feedback recording unit includes a key click statistics module, a game reaction recording module, and an optimal reaction prediction module; The key click statistics module counts the number of clicks of each key during the user's game process; The game reaction recording module records the vibration condition of the game controller, where the vibration condition includes the vibration time and the number of consecutive vibrations; The optimal reaction prediction module makes a key adjustment strategy for the key function by means of the data statistically obtained in the key click statistics module, the data of the controller vibration collected by the game reaction recording module, and the stub data of the game operation itself, and transmits the made key adjustment strategy to the key adjustment module. After obtaining the key adjustment strategy, the key adjustment module stores its data.

[0010] As a further improvement of this technical solution, the deployment unit further includes a click proficiency statistics module; The click proficiency statistics module receives the data of the number of clicks of each key collected by the key click statistics module, and statistically calculates the proficiency of each key click according to the statistical data, and transmits the statistically calculated proficiency to the key adjustment module; During the game process of the game controller, the reaction acquisition module records the interval duration between each key click and transmits the recorded content to the key adjustment module.

[0011] As a further improvement of this technical solution, the key adjustment module analyzes the data transmitted by the reaction sorting unit and the feedback recording unit. The data analyzed by the key adjustment module transmitted by the reaction sorting unit and the feedback recording unit includes the key adjustment strategy data transmitted by the optimal reaction prediction module, the data of the key click proficiency transmitted by the click proficiency statistics module, and the interval data between each key click collected by the reaction acquisition module; The key adjustment module analyzes by combining the data collected during the game process of the game controller to obtain a new recommended instruction for key adjustment.

[0012] Compared with the prior art, the beneficial effects of the present invention: 1. In the wired game controller human-computer interaction system based on multi-modal feedback, before the game, the user grades the positions and functions of the buttons, then the reaction acquisition module collects the reaction time for clicking each button, and then the allocation unit sorts the reaction times to obtain the reaction time order of each button. Then, the button adjustment module adjusts the functions of each button according to the reaction time sorting and the grading data of the button positions and functions, so that the buttons on the game controller conform to the user's usage, thereby avoiding the contradiction between the fixed button positions on the game controller and the user's operations, improving the compatibility between the game controller and the user, and facilitating the user's gaming.

[0013] 2. In the wired game controller human-computer interaction system based on multi-modal feedback, during the game, the feedback recording unit counts the number of clicks and proficiency of each button, and then predicts the best reaction by optimizing the vibration feedback and the operation stubs of the game itself to obtain the best button position strategy that conforms to the user in this type of game. After the game, the button adjustment module analyzes and adjusts the button functions, so that the button functions on the game controller conform to both the user and this game, thereby improving the user's operation experience in this game. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the overall structure of the present invention.

[0015] The meanings of the various reference numerals in the figure are as follows: 1. Reaction sorting unit; 11. Reaction acquisition module; 12. Button definition module; 2. Allocation unit; 21. Reaction sorting module; 22. Button adjustment module; 23. Click proficiency statistics module; 3. Feedback recording unit; 31. Button click statistics module; 32. Game reaction recording module; 33. Best reaction prediction module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0017] Embodiment 1 To improve the interactivity between the game controller and the user and make the buttons of the game controller more in line with the user's habits, please refer to Figure 1As shown in the figure, the purpose of this embodiment is to provide a wired game controller human-computer interaction system based on multimodal feedback, including a reaction sorting unit 1, a deployment unit 2, and a feedback recording unit 3; Before the game: The reaction sorting unit 1 collects the reaction time of each button click with the help of game instructions, and at the same time guides the user to divide the priority of the button functions on the game controller; The reaction sorting unit 1 includes a reaction collection module 11 and a button definition module 12; The button definition module 12 records the position and corresponding functions of each button on the game controller, and at the same time collects the data of the function grading performed by the user on different button functions. If the user does not perform the grading of button functions, the button definition module 12 performs active grading according to the type of the game; among the buttons of the game controller, there are various types and functions of buttons such as direction control buttons, action instruction buttons (jump, squat), attack instruction buttons, and view buttons. When grading the button functions, they are divided into a first-level button group, a second-level button group, and a third-level button group. Among them, the division of the first-level button group, the second-level button group, and the third-level button group can be actively divided by the user. For example, set the commonly used button functions as the first-level button group, such as direction control buttons, action instruction buttons, etc., the button functions used more frequently as the second-level button group, such as shooting instruction buttons, aiming instruction buttons, etc., and the button functions used less frequently as the third-level button group, such as lying down instruction buttons, view buttons, etc. By setting different levels of button groups, it is convenient to adjust the functions of the buttons on the game controller sequentially later to ensure that the buttons of the adjusted game controller conform to the user's usage.

[0018] Among them, the button definition module 12 serves as the interaction link for the basic information between the game controller and the user. For example, when grading the button functions, the grading of the button functions can be directly defined by the user or defined by the main function buttons recommended by the game; When the user does not define, at this time, according to the game's recommendation, the buttons are directly distributed in the initial state. For example, the direction control buttons are on the left side of the front of the game controller, and the action instruction buttons (jump, squat), attack instruction buttons are set on the right side of the front of the game controller, etc., that is, set according to the normal distribution of game buttons on the market at present. At this time, the game buttons determine the corresponding functions; When the user defines by himself, the button definition module 12 saves the content of the button function grading defined by the user. At this time, the functions corresponding to the game buttons have not been determined, and only the grading of each button function has been determined; When the user determines the level of functions of each button, the user also determines the instruction ranges for the left and right hand operations of the gamepad. For example, on the left side of the gamepad are operations such as directions, aiming, drawing a bow, and charging, and on the right side of the gamepad are operations such as combos, attacks, and flying. By allowing the user to determine the instruction ranges for the left and right hand operations of the gamepad in advance, it is possible to prevent confusion in operations when adjusting the button functions later. For example, if the direction control button and the view adjustment button are together, the user cannot adjust the view adjustment button while clicking the direction control button when using the gamepad, which will increase the difficulty for the user to operate the gamepad. By dividing the left and right sides of the gamepad in advance, it is ensured that the functions of the buttons on the left and right sides of the gamepad are reasonably divided.

[0019] The user clicks the corresponding buttons according to the pre-set game instructions. The reaction acquisition module 11 acquires the reaction time of the user clicking each button, and transmits the data of the acquired reaction time to the deployment unit 2 for storage and analysis by the deployment unit 2.

[0020] During the process of the reaction acquisition module 11 acquiring the reaction time of the user for each button, the game instructions will repeatedly display each button, so that the reaction acquisition module 11 acquires multiple reaction time data for the same button; Among them, the pre-set game instructions are displayed through a display. The game instructions display the graphics and positions of each button correspondingly through the display screen. After the user sees the image of the corresponding button, the hand clicks the corresponding button on the gamepad. The time recorded by the reaction acquisition module 11 is the time period from the display of the image of the corresponding button on the display to the click of the button on the gamepad. This time period is defined as the reaction time of the user; The deployment unit 2 sorts according to the reaction time of each button click acquired by the reaction sorting unit 1, and recommends button function adjustment according to the sorted order and the priority of the button functions set by the user; The deployment unit 2 includes a reaction sorting module 21 and a button adjustment module 22; The reaction sorting module 21 receives the data of the reaction time of each button acquired by the reaction acquisition module 11, and performs sequential sorting according to the reaction duration of each button click; Before sorting the response times of each button, the response sorting module 21 first obtains the response time data corresponding to each button, removes the data of the longest and shortest time periods, and then calculates the average value of the remaining data. After calculating the average value of the response times of all buttons, it sorts according to the average value of the response time of each button. By adopting this method, uncertain factors such as the user's inexperience when initially clicking the button or unclear button position can be excluded, so as to improve the accuracy of the response time of each button obtained, making it easier for the subsequent button functions to be arranged more reasonably in the layout position.

[0021] At the same time, when sorting the response times of each button, three sorting orders are generated in total. One is the sorted list of the response times of all buttons, one is the sorted list of the response times of the button for controlling the left hand of the handle, and one is the sorted list of the response times of the button for controlling the right hand of the handle. The sorting order is from small to large. The button adjustment module 22 combines the sorting order arranged by the response sorting module 21 and the level division of the button functions to analyze the recommended adjustment of the button functions, and the recommended instructions are displayed through the display. When analyzing, the button adjustment module 22 needs to consider the neatness of the same type of buttons, that is, for buttons with the same type of function, when arranging the corresponding button positions, buttons with the same or similar response times are defined as buttons with the same type of function. For example, for the direction control buttons, by defining buttons with the same or similar response times as the same type of buttons, when the user is using, there will be no operation delay due to the time difference in the response of the same type of buttons, ensuring the normal game of the user. For example: the user defines that the number of button functions in the first-level button group is seven, including two buttons for controlling the right hand of the handle, and the other five are buttons for controlling the left hand of the handle. Among the five buttons for controlling the left hand of the handle, three are of the same type, and the other two are of the same type. At this time, look at the corresponding button positions of the first five in the sorted list of the response times of the buttons for controlling the left hand of the handle, and then check whether their response times meet the condition of three being the same or similar and two being the same or similar. If not, continue to extract the sixth one backward, and then judge whether it meets the condition of three being the same or similar and two being the same or similar. If it exists, perform the matching of the button function and the button position. If it does not exist, continue to extract the next one until appropriate data is found. The two buttons for controlling the right hand of the handle also use the above method for the buttons for controlling the left hand of the handle to perform the matching of the button function and the button position.

[0022] After the instruction for button adjustment recommendation is formed by the button adjustment module 22, the user can adjust the button functions with the help of the button definition module 12 by referring to the instruction for button adjustment recommendation. The adjusted buttons are stored by the button adjustment module 22, and after storage, the game controller executes the instruction. Then, the reaction sorting unit 1 and the deployment unit 2 repeat the execution to finely adjust the positions of some of the button functions on the game controller, improving the adaptation effect between the adjusted button functions and the user. At the same time, through the repeated execution of the reaction sorting unit 1 and the deployment unit 2, the user also becomes familiar with the adjusted buttons.

[0023] After the button adjustment is completed, the user can use the game controller to play the game normally.

[0024] During the game: The feedback recording unit 3 counts the number of times each button is clicked by the user. The deployment unit 2 statistically analyzes the proficiency based on the data of each button click counted by the feedback recording unit 3. At the same time, it records the game reaction with the help of the vibration feedback of the game controller, and then predicts the best reaction based on the number of button clicks and the vibration feedback, and transmits the predicted result to the deployment unit 2; During the process of playing the game with the game controller, the reaction acquisition module 11 also records the interval duration of each button click and transmits the recorded content to the button adjustment module 22; The feedback recording unit 3 includes a button click statistics module 31, a game reaction recording module 32, and a best reaction prediction module 33; The button click statistics module 31 counts the number of times each button is clicked during the user's game process; The game reaction recording module 32 records the vibration situation of the game controller, where the vibration situation includes the vibration time and the number of consecutive vibrations; The best reaction prediction module 33 makes a button adjustment strategy for the button functions by means of the data statistically analyzed in the button click statistics module 31, the data of the vibration of the game controller collected by the game reaction recording module 32, and the stub data of the game's own running operations, and transmits the made button adjustment strategy to the button adjustment module 22. After obtaining the button adjustment strategy, the button adjustment module 22 stores its data; Among them, the stub data of the game's own running operations are the data of the coordinates, actions, being hit situation, and attack situation of the character during the game time when the game is running. This data is the stub of the game's operation, and the best reaction prediction module 33 can directly extract it from the game running storage directory.

[0025] When the optimal response prediction module 33 formulates a key adjustment strategy, the game response recording module 32 transmits the collected gamepad vibration situation to the optimal response prediction module 33. The optimal response prediction module 33 searches in the game storage space for data on the actions and hit situations of the character at the corresponding time in the game's own operation stub data. Then, the game response recording module 32 sends the gamepad vibration information to the key click statistics module 31, enabling the key click statistics module 31 to transmit the key click situation at the time of gamepad vibration to the optimal response prediction module 33. After receiving the data from the key click statistics module 31, the game response recording module 32, and the data on the actions and hit situations of the character in the game's own operation stub data found in the game storage space, the optimal response prediction module 33 integrates the data and then formulates a key adjustment strategy based on the integrated data.

[0026] The method for formulating the key adjustment strategy is as follows: ①. Based on the time of gamepad vibration, search in the game's own operation stub for the actions and hit situations of the game character at this time. The hit situation includes the position where the enemy attacks the game character's body. ②. At this time, select from the keys clicked and recorded by the key click statistics module 31 when the gamepad vibrates, and select the key that can be clicked to avoid the attack. ③. Swap the key functions of the key that can avoid the attack and the key function of the key that the user clicks and is attacked, and define this swapped information as the key adjustment strategy.

[0027] For example: When the game character operated by the user is attacked on the head when the user clicks the A key, the gamepad vibrates at this time. When the gamepad vibrates, the user also clicks the B, C, and D keys. It is found through the game's own operation stub data that if the user clicks the B key at that time, the user can avoid the damage of the enemy attacking the head. At this time, the key functions corresponding to the A key and the B key are swapped.

[0028] At the same time, the deployment unit 2 also includes a click proficiency statistics module 23; The click proficiency statistics module 23 receives the data on the number of clicks on each key collected by the key click statistics module 31, and statistically analyzes the click proficiency of each key based on the statistical data, and transmits the statistically analyzed proficiency to the key adjustment module 22; The method for statistically analyzing the proficiency is to record the number of clicks on different keys in a game cycle.

[0029] When the optimal reaction prediction module 33 formulates the key adjustment strategy and the click proficiency statistics module 23 calculates the click proficiency of each key, it is after the end of a game cycle. One game cycle is the end of a game. For example, in the game Honor of Kings, after the game fails or wins, it is a game cycle. For example, in the game PUBG Mobile, after the game character dies or wins, it is a game cycle.

[0030] After the game: The deployment unit 2 analyzes the data transmitted by the reaction sorting unit 1 and the feedback recording unit 3, and after the analysis, makes a recommendation for adjusting the function of the currently defined key functions. The reaction sorting unit 1 adjusts the functions of the keys, and the deployment unit 2 records the adjusted keys.

[0031] Among them, the key adjustment module 22 analyzes the data transmitted by the reaction sorting unit 1 and the feedback recording unit 3. The data analyzed by the key adjustment module 22 from the reaction sorting unit 1 and the feedback recording unit 3 includes the key adjustment strategy data transmitted by the optimal reaction prediction module 33, the data of the click proficiency of the keys transmitted by the click proficiency statistics module 23, and the interval data between the clicks of each key collected by the reaction acquisition module 11. The key adjustment module 22 analyzes the data collected during the game process of the gamepad to obtain a new recommended instruction for key adjustment.

[0032] Among them, the method for the key adjustment module 22 to analyze and obtain a new recommended instruction for key adjustment is as follows: Step 1: The key adjustment module 22 locates and arranges the keys clicked when the game character is hit and the keys that can avoid attacks in the sorted list of the reaction times of all keys, and determines the reaction durations of the clicks of the two keys. Step 2: At the same time, calculate the time difference between the key clicked when the game character is hit and the key that can avoid attacks through the interval data between the clicks of each key collected by the reaction acquisition module 11. Step 3: Then obtain the proficiency data of the key clicks statistically calculated by the click proficiency statistics module 23. Step 4: Combine the data obtained in Step 1, Step 2, and Step 3 to formulate a new recommended instruction for key adjustment. Is the proficiency of the key clicked when the game character is hit lower than the proficiency of the key that can avoid attacks? Is the reaction time of the key clicked when the game character is hit higher than the reaction time of the key that can avoid attacks in the sorted list of the reaction times of all keys? Whether the reaction time obtained by dividing the reaction duration between two button clicks from the button clicked when the game character is hit to the button clicked to avoid the attack by the number of buttons clicked during this reaction time is less than the reaction time of the button clicked when the game character is hit; When two of the above three situations are met, a recommended instruction for new button adjustment can be formulated.

[0033] The recommended instruction for new button adjustment formulated is displayed on the display screen, and then the user determines it through the button definition module 12. After the user agrees or disagrees, the button definition module 12 records the button positions corresponding to the button functions of agreement or disagreement.

[0034] Among them: Whether the proficiency of the button clicked when the game character is hit is lower than that of the button that can avoid the attack, which means that the button that can avoid the attack is clicked frequently; Whether the reaction time of the button clicked when the game character is hit is higher than that of the button that can avoid the attack in the sorted list of reaction times of all buttons, which means that the button that can avoid the attack has a faster click reaction speed; Whether the reaction time obtained by dividing the reaction duration between two button clicks from the button clicked when the game character is hit to the button clicked to avoid the attack by the number of buttons clicked during this reaction time is less than the reaction time of the button clicked when the game character is hit, which means that when the reaction time to click the button that can avoid the attack is less than the reaction time of the button clicked when the game character is hit, that is, clicking the button clicked when the game character is hit is a wrong click, and the user has time to react to click the button that can avoid the attack; By formulating the recommended instruction for new button adjustment through the above method, the adjusted buttons can conform to the reaction situation of the user using the game controller, and the buttons on the game controller can conform to the user's use.

[0035] In summary: Before the game, the user grades the button positions and functions, then the reaction acquisition module 11 acquires the reaction time of clicking each button, and then the allocation unit 2 sorts the reaction times to obtain the reaction time order of each button. Then the button adjustment module 22 adjusts the functions of each button according to the reaction time sorting and the hierarchical data of button positions and functions, so that the buttons on the game controller conform to the user's use, thereby avoiding the contradiction between the fixed button positions on the game controller and the user's operations, improving the compatibility between the game controller and the user, and facilitating the user's game.

[0036] During the game process, the feedback recording unit 3 counts the number of clicks and proficiency of each button, and then obtains the optimal key position strategy that conforms to the user in this type of game by predicting the best response to the vibration feedback and the game's own operation stub; After the game, the button adjustment module 22 is used for analysis and button function adjustment, so that the button functions on the game controller conform to both the user and this game, thereby improving the user's operation experience in this game.

[0037] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions in the specification are only preferred examples of the present invention and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of the present invention claimed is defined by the appended claims and their equivalents.

Claims

1. A wired game controller human-computer interaction system based on multimodal feedback, characterized in that: It includes a reaction sorting unit (1), a deployment unit (2), and a feedback recording unit (3); Before the game: The reaction sorting unit (1) collects the reaction time of each button click by means of game instructions, and at the same time guides the user to divide the priority of the button functions on the game controller; The deployment unit (2) sorts according to the reaction time of each button click collected by the reaction sorting unit (1), and recommends button function adjustment according to the arranged order and the priority of the button functions set by the user; During the game: The feedback recording unit (3) counts the number of each button clicked by the user. The deployment unit (2) counts the proficiency according to the data of each button click counted by the feedback recording unit (3). At the same time, it records the game reaction by means of the vibration feedback of the controller, and then predicts the best reaction by means of the number of button clicks and the vibration feedback, and transmits the predicted result to the deployment unit (2); After the game: The deployment unit (2) analyzes the data transmitted by the reaction sorting unit (1) and the feedback recording unit (3). After the analysis, it recommends function adjustment for the currently defined button functions. The reaction sorting unit (1) adjusts the button functions, and the deployment unit (2) records the adjusted buttons.

2. The human-computer interaction system of a wired game controller based on multimodal feedback according to claim 1, characterized in that: The reaction sorting unit (1) includes a reaction collection module (11) and a button definition module (12); The button definition module (12) records the position and corresponding functions of each button on the game controller, and at the same time collects the data of the function grading of different button functions by the user; The reaction collection module (11) collects the reaction time of the user clicking each button, and transmits the collected reaction time data to the deployment unit (2), which is stored and analyzed by the deployment unit (2); During the process of the reaction collection module (11) collecting the reaction time of the user for each button, the game instructions will repeatedly display each button, so that the reaction collection module (11) can collect multiple reaction time data of the same button.

3. The wired game controller human-computer interaction system based on multi-modal feedback according to claim 2, characterized in that: The deployment unit (2) includes a reaction sorting module (21) and a button adjustment module (22); The reaction sorting module (21) receives the data of the reaction time of each button collected by the reaction collection module (11), and sorts them sequentially according to the reaction duration of each button click; The button adjustment module (22) analyzes the recommended button function adjustment by combining the order arranged by the reaction sorting module (21) and the level division of the button functions, and the recommended instructions are displayed through the display.

4. The human-computer interaction system of a wired game controller based on multimodal feedback according to claim 3, characterized in that: Before sorting the reaction time of each button, the reaction sorting module (21) first obtains the reaction time data corresponding to each button, removes the data of the longest and shortest time periods, and then calculates the average value of the remaining data. After calculating the average value of the reaction time of all buttons, it sorts according to the average value of the reaction time of each button.

5. The wired game controller human-computer interaction system based on multi-modal feedback according to claim 3, wherein: After the instruction for button adjustment recommendation is formed by the button adjustment module (22), the button definition module (12) adjusts the button functions. The adjusted buttons are stored by the button adjustment module (22), and after storage, the game controller executes the instruction, and then the reaction sorting unit (1) and the deployment unit (2) repeat the execution.

6. The wired game controller human-computer interaction system based on multimodal feedback according to claim 1, wherein: The feedback recording unit (3) includes a button click statistics module (31), a game reaction recording module (32), and an optimal reaction prediction module (33); The button click statistics module (31) counts the number of clicks of each button during the user's game process; The game reaction recording module (32) records the vibration conditions of the game controller, where the vibration conditions include the vibration time and the number of consecutive vibrations; The optimal reaction prediction module (33) makes a button adjustment strategy for the button functions by means of the data counted in the button click statistics module (31), the data of the handle vibration collected by the game reaction recording module (32), and the stub data of the game's own operation, and transmits the made button adjustment strategy to the button adjustment module (22). After obtaining the button adjustment strategy, the button adjustment module (22) stores its data.

7. The wired game controller human-computer interaction system based on multimodal feedback according to claim 6, wherein: The method for formulating the button adjustment strategy is as follows: ①. According to the vibration time of the game controller, find the actions and hit conditions of the game character at this time from the stub of the game's own operation. The hit conditions include the position where the enemy attacks the game character's body; ②. At this time, select from the buttons clicked recorded by the button click statistics module (31) when the game controller vibrates, and select the button that can avoid the attack by clicking that button; ③. Swap the button positions of the button function that can avoid the attack and the button function that is attacked after the user clicks the button, and define this swapped information as the button adjustment strategy.

8. The wired game controller human-computer interaction system based on multimodal feedback according to claim 7, wherein: The deployment unit (2) further includes a click proficiency statistics module (23); The click proficiency statistics module (23) receives the data of the number of clicks of each button collected by the button click statistics module (31), and counts the proficiency of each button click according to the counted data, and transmits the counted proficiency to the button adjustment module (22); During the process of the game controller playing the game, the reaction collection module (11) records the interval duration of each button click, and transmits the recorded content to the button adjustment module (22).

9. The wired game controller human-computer interaction system based on multimodal feedback according to claim 8, characterized in that: The button adjustment module (22) analyzes the data transmitted by the reaction sorting unit (1) and the feedback recording unit (3). The data analyzed by the button adjustment module (22) from the reaction sorting unit (1) and the feedback recording unit (3) includes the button adjustment strategy data transmitted by the optimal reaction prediction module (33), the button click proficiency data transmitted by the click proficiency statistics module (23), and the interval data between each button click collected by the reaction collection module (11); The button adjustment module (22) analyzes by combining the data collected during the game controller playing the game to obtain a new recommended instruction for button adjustment.

10. The wired game controller human-computer interaction system based on multimodal feedback according to claim 9, wherein: The method for the key adjustment module (22) to analyze and obtain a recommended instruction for new key adjustment is as follows: Step 1: The key adjustment module (22) locates the keys clicked when the game character is hit and the keys that can avoid attacks in the sorted list of response times of all keys, and determines the response durations of the two keys clicked; Step 2: At the same time, calculate the time difference between the key clicked when the game character is hit and the key that can avoid attacks through the interval data between clicks of each key collected by the response collection module (11); Step 3: Then obtain the proficiency data of key clicks statistically calculated by the click proficiency statistics module (23); Step 4: Combine the data obtained in Step 1, Step 2, and Step 3 to formulate a recommended instruction for new key adjustment.

Images