A high-precision peripheral feedback adaptation method and system for real-time scene analysis

Abstract

The application discloses a high-precision peripheral feedback adaptation method and system for real-time scene analysis. Through end-to-end feature flow extraction, combined with multi-scale feature fusion based on an attention mechanism, scene determination and regression prediction are directly performed on a full image or a dynamic region of interest. Through lightweight and high-precision real-time feature extraction, millisecond-level dynamic feedback control of peripherals (vibration, adaptive trigger damping) is realized, and the analysis accuracy of specific game elements (such as the number of bullets and the state of a specific vehicle) in a complex texture background is improved. The problems of high delay, low recognition accuracy and inability to adapt in high-dynamic and detail-dense real-time game scenes are solved.

Description

Technical Field

[0001] This invention relates to the field of peripheral control technology, and more specifically, to a high-precision peripheral feedback adaptation method and system for real-time scene analysis. Background Technology

[0002] In gaming, providing sufficiently realistic feedback can enhance immersion and improve the user experience. Vibration feedback, in particular, can overcome the limitations of sight and hearing, providing a tactile experience. Especially in action and shooting games, providing sufficiently real-time and realistic vibration feedback as the player's character performs specific actions or receives corresponding impacts can greatly increase the user's immersion.

[0003] Current vibration adaptation technologies for user input feedback provide accurate feedback to user commands, but cannot achieve synchronization with the screen. On the other hand, there are solutions based on image recognition to determine the vibration feedback method. These solutions mainly use region stitching and traditional classification algorithms. However, using coarse-grained region detection methods often results in high response latency, low recognition accuracy, and inability to adapt to physical triggers when facing highly dynamic and detail-rich real-time game scenes (such as gunfire flashes, bullet count changes, and vehicle physical vibrations). Summary of the Invention

[0004] This invention overcomes the shortcomings of the prior art and provides a high-precision peripheral feedback adaptation method and system for real-time scene analysis, in order to solve the problem of inaccurate feedback for highly dynamic and detail-intensive real-time game scenes in the prior art.

[0005] To solve the above-mentioned technical problems, one embodiment of the present invention adopts the following technical solution:

[0006] A high-precision peripheral feedback adaptation method for real-time scene analysis includes the following steps:

[0007] Scan the current screen to identify points of interest.

[0008] Set the points of attention that meet the criteria as dynamic anchor points, and perform high-frequency sampling on the pixel stream around the dynamic anchor points;

[0009] By using a convolutional feature extractor, the high-frequency sampling results are processed to capture the local feature changes of moving targets in the image in real time.

[0010] By taking local feature changes as input, calling the mapping function, and outputting vibration feedback parameters;

[0011] The output vibration feedback parameters are pushed to the vibration actuator through a low-latency channel.

[0012] A further technical solution is that the points of attention include interactive elements on the screen and visual focal points.

[0013] A further technical solution is that the high-frequency sampling results are processed using a convolutional feature extractor to capture the local feature changes of moving targets in the image in real time, specifically including one or more of the following steps;

[0014] Asynchronous multidimensional feature extraction: extracting the numerical gradient changes of interactive elements in the image;

[0015] Visual focus abrupt change detection: Captures pixel brightness abrupt changes in visual focus areas;

[0016] Global dynamics vector calculation is performed to extract the optical flow displacement vector from the image.

[0017] A further technical solution is that the convolutional feature extractor is a convolutional regression network;

[0018] The convolutional feature extractor is based on an attention mechanism, performs parallel feature extraction on multiple dynamic anchor points, then performs multi-scale feature residual extraction, and fuses multi-scale temporal features.

[0019] The convolutional feature extractor performs regression prediction on the dynamic anchor point image and calls the mapping function based on the regression prediction result.

[0020] A further technical solution is that the mapping function is driven by physical simulation;

[0021] The mapping function adjusts the vibration parameters of the vibration actuator in real time based on the rate of change of the extracted local features.

[0022] A further technical solution is that the mapping function is specifically a shooting vibration mapping function:

[0023] F(total) = (W1 * G_fire) + (W2 * ΔC / Δt) + (W3 * Σ(O_flow)) + (W4* P_state)

[0024] In the formula, F(total) is the composite feedback strength of the final output to the adaptive trigger or motor of the handle;

[0025] G_fire is the instantaneous burst index of gunfire identified by changes in grayscale gradient;

[0026] ΔC / Δt is the rate of change of ammunition count per unit time;

[0027] Σ(O_flow) is the sum of the global optical flow vectors;

[0028] P_state is a predictive correction factor, which is the prediction of the action trend of the next frame based on the data of the previous N frames;

[0029] W1-W4 are the weighting coefficients.

[0030] A further technical solution is that the vibration feedback parameters include physical damping parameters, vibration frequency, vibration amplitude, and vibration direction.

[0031] A further technical solution is that the adaptation method also includes the following step: establishing a state mapping model T = f(S, Δt);

[0032] Where T is the adaptive trigger damping parameter;

[0033] S represents the detected scene element features;

[0034] Δt is the time interval between the occurrence of the event;

[0035] The scene element features S are converted into physical quantities, the event interval Δ_t in consecutive frames is analyzed, the dynamic frequency of the action is calculated, and the vibration parameters are adjusted according to the dynamic frequency.

[0036] The generated parameter T is sent to the hardware execution layer of the controller via a high-speed channel to synchronize the feedback of the physical trigger with the physical simulation in the game.

[0037] A further technical solution is that the state mapping model T = f(S, Δt) includes the following physical logic:

[0038] Resistance lockout logic: When Δt increases abnormally and a bullet zeroing feature is detected, the output parameter T immediately switches to the "complete block" command;

[0039] Linear damping logic: Based on the energy storage action identified by S, the output parameter T increases linearly or exponentially with time.

[0040] Another aspect of the present invention provides a high-precision peripheral feedback adaptation system for real-time scene analysis, comprising: an image acquisition unit, a convolutional feature extractor, and a vibration control unit;

[0041] The image acquisition unit is used to scan the current image and identify points of interest in the current image;

[0042] The convolutional feature extractor is used to process the pixel stream around the dynamic anchor point to identify local feature changes;

[0043] The vibration control unit is used to process local feature changes into vibration feedback parameters and send them to the vibration execution device.

[0044] Compared with existing technologies, this invention has at least the following beneficial effects: This invention, through end-to-end feature flow extraction combined with multi-scale feature fusion based on an attention mechanism, directly performs scene determination and regression prediction on the entire image or dynamic regions of interest. Through lightweight and high-precision real-time feature extraction, it achieves millisecond-level dynamic feedback control of peripherals (vibration, adaptive trigger damping) and improves the accuracy of resolving specific game elements (such as bullet count, specific vehicle status) against complex texture backgrounds. It solves the problems of high latency, low recognition accuracy, and inability to adapt to highly dynamic and detail-intensive real-time game scenes. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating Example 1;

[0046] Figure 2 This is a schematic diagram of the structure and interaction of Embodiment 2; Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] One embodiment of the present invention provides a high-precision peripheral feedback adaptation method for real-time scene analysis.

[0049] A high-precision peripheral feedback adaptation method for real-time scene parsing, see [link to relevant documentation]. Figure 1 This includes the following steps:

[0050] S1: Scan the current screen to identify points of interest.

[0051] Understandably, in game visuals, images that change little over a long period are generally interactive elements, while images that are highlighted or change rapidly are generally visual focal points. Any part that meets these two definitions can be considered a point of attention.

[0052] In this embodiment, the points of attention include interactive elements on the screen and visual focal points.

[0053] For example, screen interaction elements can be various status bars displayed on the screen, such as bullet count and health bar, and visual interaction elements, such as crosshair and vehicle center;

[0054] Visual focus can be a sudden change in pixel brightness, such as gunfire effects, or it can be optical flow displacement.

[0055] S2: Set the points of attention that meet the conditions as dynamic anchor points, and perform high-frequency sampling on the pixel stream around the dynamic anchor points;

[0056] For example, the high-frequency sampling frequency is 120Hz. By sampling the dynamic anchor points at high frequency, the amount of data can be avoided, key information can be obtained from the changes in the current screen, and the latency can be greatly reduced and the accuracy improved.

[0057] S3: Using a convolutional feature extractor, the high-frequency sampling results are processed to capture the local feature changes of moving targets in the image in real time. This extraction process is different from traditional static image classification. For example, its working steps include:

[0058] Asynchronous multidimensional feature extraction and numerical regression of interactive elements: For interactive elements in the scene (such as the HUD bullet counter), extract their numerical gradient changes. By comparing consecutive video frames, calculate the ammunition reduction rate ΔC / Δt in real time, thereby accurately determining the current weapon's rate of fire and remaining ammunition status.

[0059] Transient Detection: For visually important areas (such as around the crosshair), capture pixel brightness transition values ​​G_fire to identify the burst point and duration of gunfire effects in milliseconds.

[0060] Global Kinematic Vectoring: Extracts the optical flow displacement vector Sum(O_flow) from the image, and calculates the vehicle's speed or the character's movement vector by analyzing the relative motion displacement between the background and foreground.

[0061] It is understandable that, in actual processing, one or more of the above methods can be used.

[0062] In a further preferred embodiment, the convolutional feature extractor is not a simple classifier, but rather employs one of the following architectures:

[0063] Convolutional Regression Network: This network directly outputs continuous physical parameter values ​​(such as displacement, brightness, and numerical gradient) instead of discrete class probabilities, thereby enabling more precise feedback control.

[0064] Dynamic Anchor Attention: The system automatically locks multiple dynamic anchors using an attention mechanism, rather than using fixed region truncation. The algorithm extracts features from these anchor regions in parallel, significantly reducing processing latency.

[0065] Multi-scale Residual Extraction: By extracting feature residuals at different spatial scales, the system's ability to capture subtle visual changes (such as distant gunfire or minute numerical fluctuations) is enhanced.

[0066] Multi-scale Temporal Feature Fusion: The convolutional feature extractor in this application does not only identify the current single frame, but also performs deep comparison of consecutive video frames through a temporal network. By fusing feature information in the temporal dimension, it is possible to predict the evolution trend of actions, thereby generating feedback instructions in advance to compensate for hardware latency.

[0067] The convolutional feature extractor performs regression prediction on the dynamic anchor point image and calls the mapping function based on the regression prediction result.

[0068] S4: Take the local feature changes as input, call the mapping function, and output the vibration feedback parameters;

[0069] The mapping function is driven by physical simulation;

[0070] The mapping function adjusts the vibration parameters of the vibration actuator in real time based on the rate of change of the extracted local features.

[0071] The mapping function can be a function that provides feedback based on user actions, or it can be a function that provides feedback based on the current environment.

[0072] In this embodiment, the mapping function is specifically a shooting vibration mapping function:

[0073] F(total) = (W1 * G_fire) + (W2 * ΔC / Δt) + (W3 * Σ(O_flow)) + (W4* P_state)

[0074] In the formula, F(total) is the composite feedback strength of the final output to the adaptive trigger or motor of the handle;

[0075] G_fire is the instantaneous burst index of gunfire identified by changes in grayscale gradient;

[0076] ΔC / Δt is the rate of change of ammunition count per unit time;

[0077] Σ(O_flow) is the sum of the global optical flow vectors;

[0078] P_state is a predictive correction factor, which is the prediction of the action trend of the next frame based on the data of the previous N frames;

[0079] W1-W4 are the weighting coefficients.

[0080] In a further preferred embodiment, a state mapping model T = f(S, Δt) is established;

[0081] Where T is the adaptive trigger damping parameter;

[0082] S represents the detected scene element features;

[0083] Δt is the time interval between the occurrence of the event;

[0084] Detailed operation steps:

[0085] Step 1: Feature Parameter Quantization and Analysis: The system converts the semantic features S output by the convolutional feature extractor into physical quantities. For example, when the weapon feature is detected as "large-caliber sniper rifle", the system assigns it a very high basic drag weight; when the vehicle feature is detected as "off-road vehicle", the system extracts the bump frequency based on the road surface optical flow vector.

[0086] The second step is to calculate the timing frequency. By analyzing the event interval Δ_t in consecutive frames, the system calculates the dynamic frequency of the action and adjusts the vibration parameters according to the dynamic frequency.

[0087] Example (shooting scenario): If the bullet reduction Δt is detected to be extremely small (high rate of fire), the model will reduce the decay time of a single vibration and increase the trigger rebound force, so that the player can feel a high-frequency, continuous physical shoulder-bumping sensation.

[0088] Example (driving scenario): If the visual stretching (optical flow) caused by vehicle acceleration is enhanced, Δt represents the rate of change of speed. The model then increases the pressing resistance of the trigger (accelerator) to simulate the pedal weight at high speed.

[0089] Step 3: Calculation of the dynamic function and issuance of commands. The system calculates the mapping model T = f(S, Δt) in real time. This function is not a simple linear mapping, but contains the following physical logic:

[0090] Resistance lock-up logic: When Δt increases abnormally and the bullet zeroing characteristic is detected, the output parameter T immediately switches to the "complete block" command, driving the trigger motor to perform strong damping, achieving the physical feeling of "cannot be pulled".

[0091] Linear damping logic: Based on the charging action identified by S (such as drawing a bow to its full extent or charging an energy weapon), the output parameter T increases linearly or exponentially with time, realizing real-time coupling between pressing depth and resistance magnitude.

[0092] Step 4: The physical simulation synchronous control system sends the generated parameter T to the hardware execution layer of the controller via a high-speed channel.

[0093] Feedback synchronization: By using predictive factors to offset image processing delays, it ensures that the visual "gunfire" occurs synchronously with the "instantaneous damping" on the trigger.

[0094] Closed-loop adjustment: The hardware layer periodically feeds back the current rotor position of the motor, and the system corrects the torque in real time according to the requirements of f(S,Δt) to ensure that the physical feedback is highly consistent with the complex physics engine simulation in the game (such as recoil model and road feedback model).

[0095] The above steps achieve synchronization between the physical trigger feedback and the in-game physics simulation.

[0096] S5: Push the output vibration feedback parameters to the vibration actuator through a low-latency channel.

[0097] For example, the vibration feedback parameters include physical damping parameters (T), vibration frequency (F), and vibration amplitude (A).

[0098] The hardware execution layer receives a composite vector signal V_cmd, whose control function expression is: V_cmd =Handle(T, F, A)

[0099] (1) Physical damping control function: T = F(S, Δt)

[0100] Operational details: This function controls the servo or DC motor within the adaptive trigger via a pulse width modulation (PWM) signal.

[0101] Execution logic: When a specific scene element S is detected (such as ammunition reaching zero), the function outputs the maximum duty cycle signal, driving the gear set into the physical lock position and generating strong damping; when a charging feature is detected, the function outputs a resistance value that increases linearly with time t.

[0102] (2) Vibration frequency control function: F = f_freq(V_flow, Δt)

[0103] Operational details: This function controls the resonant frequency of the linear motor (LRA).

[0104] Execution logic: The system extracts high-frequency components from the optical flow vector V_flow (such as the visual jitter frequency generated by the vehicle passing over a gravel road surface) and directly maps them to the vibration frequency F of the motor, so that the tactile vibration and the screen jitter frequency are fully coupled.

[0105] (3) Vibration amplitude control function: A = f_amp(G_fire, ΔC)

[0106] Operation details: This function controls the peak-to-peak value of the output voltage.

[0107] Execution logic: The amplitude A is proportional to the visual burst intensity G_fire. When a step decrease in the number of bullets ΔC is detected (such as an explosion or heavy blow), the function instantaneously increases the voltage output, generating a strong, impactful vibration.

[0108] Example 2

[0109] A high-precision peripheral feedback adaptation system for real-time scene analysis, see [link to relevant documentation]. Figure 2 It includes an image acquisition unit, a convolutional feature extractor, a vibration control unit, and a vibration execution device;

[0110] The image acquisition unit is used to scan the current image and identify points of interest in the current image;

[0111] The convolutional feature extractor is used to process the pixel stream around the dynamic anchor point to identify local feature changes;

[0112] The vibration control unit is used to process local feature changes into vibration feedback parameters and send them to the vibration execution device.

[0113] Although the invention has been described herein with reference to illustrative embodiments, it should be understood that many other modifications and implementations can be devised by those skilled in the art, which will fall within the scope and spirit of the principles disclosed herein. More specifically, various variations and modifications can be made to the components and / or layout of the subject matter combination within the scope of this disclosure. Besides variations and modifications to the components and / or layout, other uses will be apparent to those skilled in the art.

Claims

1. A high-precision peripheral feedback adaptation method for real-time scene analysis, characterized in that, Includes the following steps: Scan the current screen to identify points of interest. Set the points of attention that meet the criteria as dynamic anchor points, and perform high-frequency sampling on the pixel stream around the dynamic anchor points; By using a convolutional feature extractor, the high-frequency sampling results are processed to capture the local feature changes of moving targets in the image in real time. By taking local feature changes as input, calling the mapping function, and outputting vibration feedback parameters; The output vibration feedback parameters are pushed to the vibration actuator through a low-latency channel.

2. The high-precision peripheral feedback adaptation method for real-time scene analysis as described in claim 1, characterized in that, The points to note include interactive elements on the screen and visual focal points.

3. The high-precision peripheral feedback adaptation method for real-time scene analysis as described in claim 1, characterized in that, The process of using a convolutional feature extractor to process high-frequency sampling results and capture local feature changes of moving targets in the image in real time includes one or more of the following steps: Asynchronous multidimensional feature extraction: extracting the numerical gradient changes of interactive elements in the image; Visual focus abrupt change detection: Captures pixel brightness abrupt changes in visual focus areas; Global dynamics vector calculation is performed to extract the optical flow displacement vector from the image.

4. The high-precision peripheral feedback adaptation method for real-time scene analysis as described in claim 1, characterized in that, The convolutional feature extractor is a convolutional regression network; The convolutional feature extractor is based on an attention mechanism, performs parallel feature extraction on multiple dynamic anchor points, then performs multi-scale feature residual extraction, and fuses multi-scale temporal features. The convolutional feature extractor performs regression prediction on the dynamic anchor point image and calls the mapping function based on the regression prediction result.

5. The high-precision peripheral feedback adaptation method for real-time scene parsing as described in claim 1, characterized in that, The mapping function is driven by physical simulation; The mapping function adjusts the vibration parameters of the vibration actuator in real time based on the rate of change of the extracted local features.

6. The high-precision peripheral feedback adaptation method for real-time scene parsing as described in claim 5, characterized in that, The mapping function is specifically a shooting vibration mapping function: F(total) = (W1 * G_fire) + (W2 * ΔC / Δt) + (W3 * Σ(O_flow)) + (W4 *P_state) In the formula, F(total) is the composite feedback strength of the final output to the adaptive trigger or motor of the handle; G_fire is the instantaneous burst index of gunfire identified by changes in grayscale gradient; ΔC / Δt is the rate of change of ammunition count per unit time; Σ(O_flow) is the sum of the global optical flow vectors; P_state is a predictive correction factor, which is the prediction of the action trend of the next frame based on the data of the previous N frames; W1-W4 are the weighting coefficients.

7. The high-precision peripheral feedback adaptation method for real-time scene analysis as described in claim 1, characterized in that, The vibration feedback parameters include physical damping parameters, vibration frequency, vibration amplitude, and vibration direction.

8. The high-precision peripheral feedback adaptation method for real-time scene parsing as described in claim 1, characterized in that, The following steps are included to establish a state mapping model T = f(S, Δt); Where T is the adaptive trigger damping parameter; S represents the detected scene element features; Δt is the time interval between the occurrence of the event; The scene element features S are converted into physical quantities, the event interval Δ_t in consecutive frames is analyzed, the dynamic frequency of the action is calculated, and the vibration parameters are adjusted according to the dynamic frequency. The generated parameter T is sent to the hardware execution layer of the controller via a high-speed channel to synchronize the feedback of the physical trigger with the physical simulation in the game.

9. The high-precision peripheral feedback adaptation method for real-time scene parsing as described in claim 8, characterized in that, The state mapping model T = f(S, Δt) contains the following physical logic: Resistance lockout logic: When Δt increases abnormally and a bullet zeroing feature is detected, the output parameter T immediately switches to the "complete block" command; Linear damping logic: Based on the energy storage action identified by S, the output parameter T increases linearly or exponentially with time.

10. A high-precision peripheral feedback adaptation system for real-time scene analysis, characterized in that, This includes an image acquisition unit, a convolutional feature extractor, a vibration control unit, and a vibration execution device; The image acquisition unit is used to scan the current image and identify points of interest in the current image; The convolutional feature extractor is used to process the pixel stream around the dynamic anchor point to identify local feature changes; The vibration control unit is used to process local feature changes into vibration feedback parameters and send them to the vibration execution device.

Images