Game vibration adaptation method and system based on image detection and gamepad key value feedback

Abstract

The invention belongs to the technical field of computer software and human-computer interaction, and particularly relates to a game vibration adaptation method and system based on image detection and gamepad key value feedback. The game vibration adaptation method based on image detection and gamepad key value feedback comprises the steps that S1, the key state of each key on a controller is recorded regularly, and a key input event having a mapping relation with a game event is determined; s2, collecting game pictures at regular time, extracting visual features related to a game scene, and determining a scene key event; and S3, generating a vibration feedback signal according to the key input event and the scene key event, and triggering vibration to be fed back to the controller. Real-time vibration configuration is carried out through two dimensions of key input and game picture analysis, personalized feedback can be generated according to specific game events, and the immersion feeling of a user is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer software and human-computer interaction, and in particular relates to a game vibration adaptation method and system based on image detection and handle key value feedback. Background Art

[0002] At present, vibration feedback in games usually relies on preset event triggers, such as character attacks, injuries, or environmental vibrations. However, existing vibration feedback systems are often relatively simple, with few vibration scenes or weak vibrations, and cannot provide accurate feedback based on the real-time changes and complexity of the battle screen. This results in the vibration effect not matching the actual game experience during the game, affecting the immersion and operation experience of the game.

[0003] Some manufacturers, based on the characteristics of the game, for example, when a character attacks, the game will send vibration data to the controller. By intercepting and counting the characteristics of this type of vibration data and the corresponding character attack behavior, the two are mapped to enhance or optimize the game vibration. The method based on vibration data interception goes deep into the bottom layer of the system and intercepts the game vibration data, which is easy to be mistakenly killed by antivirus software. At the same time, this method can only optimize the existing vibration scenes preset by the game manufacturer, and is powerless for scenes that the game manufacturer has not preset events, but the players feel that vibration is needed.

[0004] With the advancement of image processing technology and game controller technology, how to provide more accurate vibration feedback based on real-time analysis of game screens and the use of controller buttons has become a more valuable research issue.

[0005] Image processing technology:

[0006] (1) Traditional image processing technology mainly relies on mathematical transformation and statistical features. For example, edge detection, image segmentation, morphological processing and other technologies are used to identify specific structures and objects in images. Common traditional image processing methods include: Edge detection (such as Canny and Sobel algorithms): used to detect the edges of objects in images. Color space conversion: such as conversion from RGB to HSV or YUV space, used to enhance specific color features and extract color histograms. Image filtering: such as Gaussian filtering and Laplace filtering, used to smooth and sharpen images, perform noise reduction and other processing operations. Morphological operations: such as corrosion and dilation, used to remove noise or enhance shape features. However, traditional image processing methods have limited performance when processing complex game scenes. For example, when a character releases a big move or a melee in the game, the game screen will be rendered more gorgeously, so the game screen is often full of dynamic changes and detailed textures. These features are difficult to extract efficiently through traditional methods.

[0007] (2) Deep learning, as a branch of machine learning, learns features from large amounts of data by imitating the neural network structure of the human brain. In recent years, with the improvement of computing power, deep learning has achieved remarkable results in fields such as computer vision, speech recognition, and natural language processing. In image processing, convolutional neural networks (CNNs) have become the mainstream model, which can perform excellent results in tasks such as image classification, object detection, and image segmentation. Deep learning algorithms can autonomously learn high-level features in images without relying on manually designed features like traditional algorithms. Therefore, the application of deep learning in games is gradually increasing.

[0008] In addition, modern game controllers are not just simple input devices. Many controllers integrate vibration feedback, accelerometers, gyroscopes and other functions, making them have a richer interactive experience. Vibration feedback technology (HapticFeedback) has become one of the important means to enhance player immersion. Current vibration feedback systems often require preset types. For example, in shooting games, the vibration of pistols, machine guns, sniper rifles or shotguns is preset in advance, and the vibration effect is achieved by triggering the corresponding buttons. However, this vibration system based on pre-set is often too simple and cannot provide more accurate feedback based on real-time dynamic changes in the game screen. Therefore, modern game controller technology requires a more intelligent and real-time vibration feedback solution, so that the vibration of the handle does not only depend on preset events, but also can generate adaptive vibration feedback signals by analyzing the game screen and player operations. Summary of the invention

[0009] The present invention aims to solve the technical problems that the existing game controllers implement vibration feedback based on a pre-set vibration system, which is easily killed by anti-virus software, has insufficient vibration scenes, a relatively single vibration mode, and cannot provide more accurate feedback according to real-time dynamic changes in the game screen. The present invention aims to provide a game vibration adaptation method and system based on image detection and handle key value feedback.

[0010] In order to solve the above technical problems, the first aspect of the present invention provides a game vibration adaptation method based on image detection and handle key value feedback, and the game vibration adaptation method based on image detection and handle key value feedback includes:

[0011] S1, regularly recording the key status of each key on the controller, and determining the key input event having a mapping relationship with the game event;

[0012] S2, regularly collects game images, extracts visual features related to the game scene, and determines key events of the scene;

[0013] S3, generating a vibration feedback signal according to the key input event and the scene key event, and triggering vibration feedback to a controller.

[0014] Optionally, in the game vibration adaptation method based on image detection and handle key value feedback as described above, step S1 includes:

[0015] Using Boolean values ​​to record the current key state of each key on the controller, comparing the current key state with the historical key state, tracking the state changes of the keys in a continuous time period, and determining the current operation type of all keys;

[0016] Determine the current key input event based on the mapping relationship between the preset key operation type and the game event.

[0017] Optionally, in the game vibration adaptation method based on image detection and handle key value feedback as described above, in step S2, the game screen is captured at regular intervals, including:

[0018] Determine whether the game window is pinned to the top. If not, wait for the next game window determination. If it is pinned to the top, obtain the game window coordinates, and then obtain the game screen screenshot in real time at a preset interval.

[0019] Optionally, in the game vibration adaptation method based on image detection and handle key value feedback as described above, in step S2, extracting visual features related to the game scene and determining key scene events include:

[0020] Acquire a plurality of detection areas preset in the game screen, pre-process the plurality of detection areas and then splice them to obtain a detection image of preset specifications;

[0021] Normalizing image pixels of the detection image;

[0022] The normalized detection image is input into a trained convolutional neural network model to extract the visual features, the category probability is obtained through a preset activation function, and the game event corresponding to the category with the highest category probability in each detection area is obtained as the scene key event.

[0023] Optionally, in the game vibration adaptation method based on image detection and handle key value feedback as described above, the detection area includes a game in progress detection area, and the game in progress detection area is detected as the first priority. After the category probability is obtained through a preset activation function, it is detected whether the category with the highest category probability in the game in progress detection area is the game in progress. If it is not the game in progress, wait for the next game screen capture. If it is the game in progress, continue to detect the category probabilities of other detection areas, and obtain the game events corresponding to the categories with the highest category probabilities in each detection area as the scene key events.

[0024] Optionally, in the game vibration adaptation method based on image detection and handle key value feedback as described above, the detection area includes at least one of the following detection areas: a game detection area, a big move detection area, a normal attack detection area, a life bar detection area, and a stamina bar detection area.

[0025] Optionally, in the game vibration adaptation method based on image detection and handle key value feedback as described above, several detection areas are pre-processed and then spliced ​​to obtain a detection image of preset specifications, including:

[0026] Each detection area is first subjected to Gaussian filtering for noise reduction, and then adaptively scaled to a picture of a specified size based on bilinear interpolation;

[0027] The different detection areas are spliced ​​to obtain a detection image of preset specifications.

[0028] Optionally, in the game vibration adaptation method based on image detection and handle key value feedback as described above, the training process of the convolutional neural network model is as follows:

[0029] Collect several game frame area images in normal state and several combat frame area images when various combat actions occur, annotate each frame image with the corresponding game event category, and divide it into training set, verification set and test set;

[0030] A convolutional neural network model is used to train the training set, the cross entropy loss function is used to measure the gap between the model output and the annotation, the gradient descent method is used to optimize the model parameters, and the model weights and biases are updated through back propagation.

[0031] Optionally, in the game vibration adaptation method based on image detection and handle key value feedback as described above, the activation function adopts a softmax function.

[0032] Optionally, in the game vibration adaptation method based on image detection and handle key value feedback as described above, step S3 includes:

[0033] According to the key input event and the scene key event, a vibration sequence is obtained in a preset vibration model as the vibration feedback signal;

[0034] The vibration model is:

[0035] V i =G(E i , D i )

[0036] Among them, V i represents the vibration sequence, Ei Indicates the key input event, D i Indicates the key events of the scene;

[0037] If the vibration sequence is not obtained in the vibration model, it is considered that the combination of the key input event and the scene key event is not configured with a vibration mode, and the data corresponding to the key input event and the scene key event obtained this time are discarded.

[0038] Optionally, in the game vibration adaptation method based on image detection and handle key value feedback as described above, after the key input event is determined in step S1, the key or game event is locked until the vibration feedback signal is obtained or the data is discarded, and then the key or game event is unlocked.

[0039] In order to solve the aforementioned technical problem, the second aspect of the present invention provides a game vibration adaptation system based on image detection and handle key value feedback, comprising:

[0040] A key input detection module is used to regularly record the key status of each key on the controller and determine the key input events that have a mapping relationship with the game events;

[0041] Image acquisition and detection module, used to regularly capture game images, extract visual features related to game scenes, and determine key events in scenes;

[0042] A vibration feedback generation module, used to generate a vibration feedback signal according to the key input event and the scene key event, and trigger vibration feedback to the controller;

[0043] The controller module is used to receive the vibration feedback signal and form a corresponding vibration feeling according to the vibration feedback signal.

[0044] The positive progress of the present invention is that in the field of game controllers, the vibration of the existing technical means is heavily dependent on the preset vibration of the game manufacturer, the vibration scenes are not rich enough, and there are few associations with game events, resulting in a single user experience. The present invention performs real-time vibration configuration through two dimensions of key input and game screen analysis, and can generate personalized feedback according to specific game events to enhance the user's immersion. Based on the dynamic management of key state changes and event trigger detection, it reduces the misjudgment caused by simple image detection, and improves the accuracy of vibration and user interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The disclosure of the present invention will become more apparent with reference to the accompanying drawings. It should be understood that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In the drawings:

[0046] Figure 1 A schematic diagram of a process of the present invention;

[0047] Figure 2 A schematic diagram of splicing detection images of the present invention;

[0048] Figure 3 This is a flow chart of Example 1 of the present invention. DETAILED DESCRIPTION

[0049] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0050] It should be noted that the following embodiments and features therein may be combined with each other if there is no conflict.

[0051] In the description of the present invention, it should be noted that directional words such as the terms "outside", "middle", "inside", "outside", etc., which indicate directions and positional relationships, are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and cannot be understood as limiting the specific protection scope of the present invention.

[0052] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features. Therefore, the definition of "first" and "second" features can explicitly or implicitly include one or more of the features. In the description of the present invention, "several" and "a number" mean two or more, unless otherwise clearly and specifically defined.

[0053] The embodiment of the present invention provides a game vibration adaptation method based on image detection and handle key value feedback, referring to Figure 1 , the method specifically comprises the following steps:

[0054] S1, regularly recording the key status of each key on the controller, and determining a key input event having a mapping relationship with a game event.

[0055] This step is used to capture the user's key input on the handle controller during the game, and trigger the corresponding vibration feedback according to the input information. This step requires real-time and efficient detection of the user's handle controller operation and comprehensive analysis of the input signal.

[0056] In some embodiments, step S1 includes: using Boolean values ​​to record the current key state of each key on the controller, comparing the current key state with the historical key state, tracking the state changes of the keys in a continuous time period, and determining the current operation type of all keys. According to the mapping relationship between the preset key operation type and the game event, the current key input event is determined.

[0057] The buttons in this step include the joystick.

[0058] In specific implementation, each button can be scanned in real time according to a preset scanning interval to detect the current button state of each button on the controller. Each current button state is recorded as a Boolean value, 0 means the button is released, and 1 means the button is pressed. By continuously scanning the button state, the current button state (0 or 1) can be obtained at time t.

[0059] Different operation types of the key, such as pressing, releasing, and long pressing, are distinguished by the changes between the previous state Si(t-1) and the current state Si(t) of the key that have been recorded.

[0060] For example:

[0061] Press event: If the current state Si(t) = 1 and the previous state Si(t-1) = 0, the key is pressed.

[0062] Release event: If the current state Si(t) = 0 and the previous state Si(t-1) = 1, the button is released;

[0063] Long press event: If the state S(t)=1 is maintained from time t1 to time t2, the key is in a long press state.

[0064] According to the above determination rules, the current operation types of all buttons on the controller are determined.

[0065] In the interaction between the controller and the game, button input often triggers specific game actions. The current operation type obtained according to the state change of the button corresponds to different game events, such as "normal attack", "jump", "heavy attack", etc. Therefore, the mapping relationship between the button operation type and the game event can be preset, for example, the A button corresponds to the "jump" game event, the B button corresponds to the "roll" game event, the X button corresponds to the "normal" game event, and the Y button corresponds to the "heavy attack". According to these preset mapping relationships, the current operation type can obtain the corresponding button input event, that is, the button input event is a certain game event, such as "normal attack", "jump", "roll" or "heavy attack", etc.

[0066] S2, regularly collects game images, extracts visual features related to the game scene, and determines key events of the scene.

[0067] The timing in this step refers to collecting screenshots of the game screen at preset intervals, for example, collecting screenshots of the game screen at 10 frames per second.

[0068] The visual features in this step refer to key events such as character attacks, enemy injuries, explosions, etc.

[0069] The scene key events in this step are game events, such as "normal attack", "jump", "roll" or "heavy attack".

[0070] In some embodiments, in step S2, the game screen is captured at regular intervals, including:

[0071] S21, determine whether the game window is pinned to the top. If not, wait for the next game window determination. If it is pinned to the top, obtain the coordinates of the game window, and then obtain a screenshot of the game screen in real time at a preset interval.

[0072] In this embodiment, before collecting the game screen, first determine whether the game is in the top state according to the game currently opened on the terminal desktop. Only when it is in the top state can the subsequent game screen collection and scene key events be determined. If the game is not in the top state, it is necessary to wait for the next determination of whether the game window is top.

[0073] In some embodiments, in step S2, extracting visual features related to the game scene and determining key scene events include:

[0074] S22, obtaining a plurality of detection areas preset in the game screen, pre-processing the plurality of detection areas and then splicing them to obtain a detection image of preset specifications.

[0075] The detection image in this embodiment is an image of a preset size, for example, 640*640, which is formed by splicing the regions of interest at different positions of different game screens. These regions of interest are some key factors used to distinguish which game event the current game screen is in, such as the character's health bar, skill icon, etc.

[0076] The detection area in this embodiment includes at least one of the following detection areas: the game detection area, the ultimate attack detection area, the normal attack detection area, the life bar detection area, the stamina bar detection area, and other areas. Of course, detection areas in different locations can be selected according to different games. After splicing these detection areas, the following is obtained: Figure 2 The detection image shown.

[0077] In step S22, several detection areas are pre-processed and then spliced ​​to obtain a detection image of preset specifications, including: each detection area is first denoised by Gaussian filtering, and then adaptively scaled to a picture of a specified size based on bilinear interpolation. After the image size change is completed, different detection areas are spliced ​​to obtain a detection image of preset specifications.

[0078] Therefore, if Figure 2 As shown, there are detection areas of different sizes in the detection image.

[0079] S23, normalizing the image pixels of the detection image.

[0080] This step normalizes the pixel range of the detection image to [0,1], changing it into a fixed standard form. After normalization, the image features of different dimensions have a certain degree of numerical comparability, which can greatly improve the accuracy of subsequent detection and classification.

[0081] S24, input the normalized detection image into the trained convolutional neural network model to extract visual features, obtain the category probability through a preset activation function, and obtain the game event corresponding to the category with the highest category probability in each detection area as the scene key event.

[0082] In order to more accurately identify complex scenes, this step is based on the convolutional neural network self-training model. The high-level features are extracted based on the trained convolutional neural network model, and the prediction score Z of each category is obtained through the fully connected layer. i , and finally the score is converted into category probability p through the activation function i . According to the probability p output by the activation function i , select the category with the highest probability as the recognition result, that is, the key event of the scene in the game screen. For example, when the probability of a character using a big move is the highest, it is determined that the key event of the scene is that the character is using a big move.

[0083] In this step, the activation function preferably adopts the softmax function.

[0084] In some embodiments, the detection area at least includes a game detection area, and the game detection area is detected as the first priority. After obtaining the category probability p through a preset activation function, i Finally, detect whether the category with the highest probability in the game detection area is the game in progress. If it is not in the game state, wait for the next game screen capture. If it is in the game state, continue to detect the category probabilities of other detection areas, and obtain the game events corresponding to the categories with the highest category probabilities in each detection area as scene key events.

[0085] In this step, the gaming detection area is detected as the first priority, and only when the game is in progress will other detection areas be detected.

[0086] The non-gaming status includes, but is not limited to, the user entering the game setting interface, the user viewing the map, the user entering the store to purchase equipment, etc. Personalized judgments should be made based on the needs of different games.

[0087] In some embodiments, in step S24, the detection image input is a trained convolutional neural network model, and the training process of the convolutional neural network model is as follows:

[0088] Collect several game frame area images in normal state and several combat frame area images when various combat actions occur, annotate each frame image with the corresponding game event category, and divide it into training set, validation set and test set. Use convolutional neural network model to train the training set, use cross entropy loss function to measure the gap between model output and annotation, use gradient descent method to optimize model parameters, and update model weights and biases through back propagation.

[0089] The training of the model requires a large amount of annotated image data. The image data in this embodiment is captured from the game, and different games require different data sets for self-training. The data set includes the game frame area in the normal state, and the battle frame area where the game image elements triggered by combat actions occur, etc. Each frame image needs to be annotated with the corresponding game event category, such as normal attack, big move attack, explosion, etc., and divided into training set, verification set and test set. Assume that the data set D contains n samples, and each sample (x i ,y i ) includes image x i and label y i , where y i ∈(1,2,...,C), C is the number of categories.

[0090] The model structure of this embodiment uses convolutional neural network (CNN) to extract features in the image. The CNN structure is stacked by multiple convolutional layers and pooling layers, and finally outputs the event classification result through a fully connected layer. For classification tasks, a cross entropy loss function is used to measure the gap between the model output and the true label. The gradient descent method (preferably SGD) is used to optimize the model parameters, and the weights and biases are updated by back propagation. Through multiple rounds of training, the model parameters are gradually optimized, so that the classification error is minimized, the accuracy rate is continuously improved, and finally a better self-training model is obtained.

[0091] S3, generating a vibration feedback signal according to the key input event and the scene key event, triggering vibration feedback to the controller.

[0092] The vibration feedback signal of this step is generated by the key input event and the scene key event. That is, the vibration feedback signal should take into account both the user's key input and whether the specific game event on the current game screen corresponds to the key input.

[0093] This step uses the preset vibration model to adapt the personalized vibration to the corresponding game scene.

[0094] In some embodiments, step S3 includes: obtaining a vibration sequence as a vibration feedback signal in a preset vibration model according to the key input event and the scene key event.

[0095] The vibration model is: V i =G(E i , D i )

[0096] Among them, V i represents the vibration sequence, E i Indicates a key input event, D i Indicates the key events of the scene.

[0097] If no vibration sequence is obtained in the vibration model, it is considered that the combination of the key input event and the scene key event is not configured with a vibration mode, and the data corresponding to the key input event and the scene key event obtained this time are discarded.

[0098] Key input events not only trigger actions within the game, but also combine with key scene events to trigger vibration feedback signals.

[0099] This embodiment generates a vibration sequence based on the key input event and the image recognition result of the game scene. The parameters such as the intensity and duration of the vibration sequence can adjust the vibration model according to the event type in the game.

[0100] For example: when pressing the normal attack key, combined with image recognition, the key event detected in the scene is the occurrence of a normal attack behavior. Both the key input event and the key scene event are normal attack game events. The vibration sequence found in the vibration model is a short vibration to reflect the attack of the weapon. Another example: when the character uses a big move to attack, through key and image detection and recognition, the key input event and the key scene event are both big move game events. The vibration sequence found in the vibration model is a strong and long vibration. The vibration sequence is sent to the controller as a vibration feedback signal to form a corresponding vibration.

[0101] The vibration model of this embodiment is pre-set and stored in the system. i Indicates a key input event, that is, which key performs what operation. iIndicates the key event of the scene, which is the result of image detection and recognition of the game screen. In the game, different key values ​​of the controller correspond to different game behaviors. Only when a specific key value state is detected and the corresponding event is detected in the game image, the game event is judged to have occurred. Each game event triggers a vibration sequence sent to the controller. V i The vibration sequence refers to a series of vibration value combinations. For example, {23, 54, 65, 255}, the vibration sequence sends vibration values ​​every 100ms, G() represents the detection judgment logic, and can also be used as a function to map a vibration sequence through key input events and scene key events.

[0102] When no corresponding vibration sequence is found in the vibration model, it may be a misjudgment, such as the game event recognized by the button and the game image does not correspond, or it may be considered that the event is not configured with vibration. In this embodiment, it is uniformly considered that the event is not configured with vibration and the recognition result is discarded.

[0103] In some embodiments, after a key input event is determined in step S1, the key or game event is locked until a vibration feedback signal is obtained or data is discarded, and then the key or game event is unlocked.

[0104] Most existing game mechanisms have a short cooldown period after a move is made. During this period, pressing the button again will not trigger the game character's behavior. The same is true for game events. After a game event occurs, such as a character's ultimate move, normal attack, etc., there will be a short cooldown period. For example, after a charged attack in a certain game, there will be a cooldown period of more than 1 second before a charged heavy attack can be performed again. The cooldown of a normal attack may be 200 to 300 milliseconds. The cooldown time of different skills in the game is different, and the cooldown time of different games is also different.

[0105] This embodiment locks the key or game event similarly to the above-mentioned cooling time. After the current key state is triggered by the game event, the key state of the subsequent preset time is triggered to lock. During this preset time, even if the user presses the key again, step S1 will not be continued. The preset time can be determined according to the game event generated after the key input. For example, the preset time of the heavy attack game event is longer than the preset time of the general attack game event.

[0106] An embodiment of the present invention provides a game vibration adaptation system based on image detection and handle key value feedback, the system comprising:

[0107] The key input detection module is used to regularly record the key status of each key on the controller and determine the key input events that have a mapping relationship with the game events.

[0108] The image acquisition and detection module is used to regularly capture game images, extract visual features related to the game scene, and determine key events in the scene.

[0109] The vibration feedback generation module is used to generate a vibration feedback signal according to key input events and scene key events, and trigger vibration feedback to the controller.

[0110] The controller module is used to receive the vibration feedback signal and form a corresponding vibration feeling according to the vibration feedback signal.

[0111] Embodiment 1:

[0112] Reference Figure 3 , a game vibration adaptation method based on image detection and handle key value feedback, specifically comprising the following steps:

[0113] 1. Initialize the controller interface, detect whether the controller is connected, etc. Scan the status of all buttons on the controller, determine whether the current operation type of each button is a press event, a release event, or a long press event, and determine the current button input event based on the mapping relationship with the game event.

[0114] Determine whether the game window is pinned. If not, wait for the next game window judgment. If it is pinned, obtain the coordinates of the game window and obtain a screenshot of the game screen. Obtain several preset detection areas in the game screen, pre-process several detection areas and then splice them to obtain a detection image of preset specifications. Normalize the image pixels of the detection image. Use the convolutional neural network model to infer the detection image. Obtain the category probability of each detection area through the softmax function. Detect whether the category with the highest category probability in the game detection area is the game in progress. If it is not in the game state, wait for the next game screen to be collected. If it is in the game state, continue to detect the category probabilities of other detection areas, and obtain the game events corresponding to the categories with the highest category probabilities in each detection area as the scene key events.

[0115] 2. Lock keys or game events.

[0116] 3. According to the key input event and scene key event, the vibration sequence is searched in the preset vibration model. If there is no vibration sequence, the key or game event is unlocked and the next step of scanning the key state is waited for. If there is a vibration sequence, the vibration sequence is sent to the controller as a vibration feedback signal and the key or game event is unlocked, and the controller forms the corresponding vibration.

[0117] The present invention is described in detail above in conjunction with the embodiments of the accompanying drawings. A person skilled in the art can make various variations of the present invention according to the above description. Therefore, some details in the embodiments should not be construed as limiting the present invention, and the present invention shall be protected by the scope defined by the attached claims.

Claims

1. A game vibration adaptation method based on image detection and handle key value feedback, characterized in that: The game vibration adaptation method based on image detection and handle key value feedback includes: S1, regularly recording the key status of each key on the controller, and determining the key input event having a mapping relationship with the game event; S2, regularly collects game images, extracts visual features related to the game scene, and determines key events of the scene; S3, generating a vibration feedback signal according to the key input event and the scene key event, and triggering vibration feedback to a controller.

2. The game vibration adaptation method based on image detection and handle key value feedback as claimed in claim 1, characterized in that: Step S1 includes: Using Boolean values ​​to record the current key state of each key on the controller, comparing the current key state with the historical key state, tracking the state changes of the keys in a continuous time period, and determining the current operation type of all keys; Determine the current key input event based on the mapping relationship between the preset key operation type and the game event.

3. The game vibration adaptation method based on image detection and handle key value feedback as claimed in claim 1, characterized in that: In step S2, the game screen is collected at a fixed time, including: Determine whether the game window is pinned to the top. If not, wait for the next game window determination. If it is pinned to the top, obtain the game window coordinates, and then obtain the game screen screenshot in real time at a preset interval.

4. The game vibration adaptation method based on image detection and handle key value feedback as claimed in claim 1, characterized in that: In step S2, visual features related to the game scene are extracted to determine key events of the scene, including: Acquire a plurality of detection areas preset in the game screen, pre-process the plurality of detection areas and then splice them to obtain a detection image of preset specifications; Normalizing image pixels of the detection image; The normalized detection image is input into a trained convolutional neural network model to extract the visual features, the category probability is obtained through a preset activation function, and the game event corresponding to the category with the highest category probability in each detection area is obtained as the scene key event.

5. The game vibration adaptation method based on image detection and handle key value feedback as claimed in claim 4, characterized in that: The detection area includes a game detection area, and the game detection area is detected as the first priority. After obtaining the category probability through a preset activation function, it is detected whether the category with the highest category probability in the game detection area is the game in progress. If it is not in the game state, wait for the next game screen to be collected. If it is in the game state, continue to detect the category probabilities of other detection areas, and obtain the game event corresponding to the category with the highest category probability in each detection area as the scene key event; And / or, the detection area includes at least one of a game detection area, a big move detection area, a normal attack detection area, a life bar detection area, and a stamina bar detection area.

6. The game vibration adaptation method based on image detection and handle key value feedback as claimed in claim 4, characterized in that: Preprocessing and splicing the plurality of detection areas to obtain a detection image of a preset specification includes: Each detection area is first subjected to Gaussian filtering for noise reduction, and then adaptively scaled to a picture of a specified size based on bilinear interpolation; The different detection areas are spliced ​​to obtain a detection image of preset specifications.

7. The game vibration adaptation method based on image detection and handle key value feedback as claimed in claim 4, characterized in that: The training process of the convolutional neural network model is as follows: Collect several game frame area images in normal state and several combat frame area images when various combat actions occur, annotate each frame image with the corresponding game event category, and divide it into training set, verification set and test set; The convolutional neural network model is used to train the training set, the cross entropy loss function is used to measure the gap between the model output and the annotation, the gradient descent method is used to optimize the model parameters, and the model weights and biases are updated through back propagation; And / or, the activation function adopts a softmax function.

8. The game vibration adaptation method based on image detection and handle key value feedback according to any one of claims 1 to 7, characterized in that: Step S3 includes: According to the key input event and the scene key event, a vibration sequence is obtained in a preset vibration model as the vibration feedback signal; The vibration model is: V i =G(E i ,D i ) Among them, V i represents the vibration sequence, E i Indicates the key input event, D i Indicates the key events of the scene; If the vibration sequence is not obtained in the vibration model, it is considered that the combination of the key input event and the scene key event is not configured with a vibration mode, and the data corresponding to the key input event and the scene key event obtained this time are discarded.

9. The game vibration adaptation method based on image detection and handle key value feedback as claimed in claim 8, characterized in that: After the key input event is determined in step S1, the key or game event is locked until the vibration feedback signal is obtained or the data is discarded, and then the key or game event is unlocked.

10. A game vibration adaptation system based on image detection and handle key value feedback, characterized in that: include: A key input detection module is used to regularly record the key status of each key on the controller and determine the key input events that have a mapping relationship with the game events; Image acquisition and detection module, used to regularly capture game images, extract visual features related to game scenes, and determine key events in scenes; A vibration feedback generation module, used to generate a vibration feedback signal according to the key input event and the scene key event, and trigger vibration feedback to the controller; The controller module is used to receive the vibration feedback signal and form a corresponding vibration feeling according to the vibration feedback signal.

Images