Keyboard prompting method and system for footsteps in game

Abstract

The invention discloses a keyboard prompting method and system for footstep sound in a game, and the method comprises the steps: capturing an audio signal of an environment in the game in real time, carrying out the spectrum analysis and footstep sound frequency feature separation of the audio signal, and extracting the direction and distance parameters of the footstep sound; mapping the azimuth and distance parameters into a keyboard area control instruction; generating a dynamic visual prompt signal according to the keyboard area control instruction; and outputting the dynamic visual prompt signal to an RGB keyboard light control system, and driving light in a corresponding keyboard area to correspondingly display according to the dynamic visual prompt signal so as to realize visual prompt of the direction and the distance of footstep sound in a game. By means of the embodiment of the invention, auditory perception can be assisted in a visual mode, the intuition and identifiability of sound information are improved, the game operation threshold is reduced, and the immersion is enhanced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent keyboards, and particularly relates to a keyboard prompting method and system for in-game footstep sounds. BACKGROUND

[0002] In recent years, electronic games, especially competitive shooting games, have a higher and higher dependence on sound positioning, and players need to accurately determine the positions and distances of opponents through environmental sound effects such as footstep sounds. The traditional method completely depends on earphone audio, but in a complex sound field environment or under external interference, sound information is often not intuitive enough, is easy to be missed or misjudged, and is a use barrier for hearing-impaired players. Existing auxiliary technologies are mostly concentrated on screen icon prompting or audio signal enhancement, lack direct and low-delay prompting means interacting with external hardware, and are difficult to achieve quick and instinctive reactions. SUMMARY

[0003] The application aims to provide a keyboard prompting method and system for in-game footstep sounds to solve the problems in the prior art, assist auditory perception in a visual manner, improve the intuitiveness and distinguishability of sound information, reduce the operation threshold of games, and enhance the sense of immersion.

[0004] One embodiment of the application provides a keyboard prompting method for in-game footstep sounds, which comprises the following steps: real-time capture of an audio signal of an in-game environment, spectral analysis of the audio signal, and separation of footstep sound frequency characteristics to extract position and distance parameters of the footstep sounds; mapping of the position and distance parameters into keyboard region control instructions, wherein different positions correspond to different regions of the keyboard, and different distances correspond to different brightness levels of keyboard light; generation of dynamic visual prompting signals according to the keyboard region control instructions, wherein the dynamic visual prompting signals include light color change patterns and brightness gradient sequences of specified keyboard regions; output of the dynamic visual prompting signals to an RGB keyboard light control system to drive corresponding keyboard region light to display according to the dynamic visual prompting signals, so as to realize visual prompting of the positions and distances of in-game footstep sounds.

[0005] Optionally, the real-time capture of the audio signal of the in-game environment, the spectral analysis of the audio signal, and the separation of the footstep sound frequency characteristics to extract the position and distance parameters of the footstep sounds comprise the following steps: real-time capture of an in-game environment audio signal, elimination of low-frequency background noise by using a high-pass filter to obtain preprocessed audio data streams; short-time Fourier transform of the preprocessed audio data streams to convert time domain signals into Mel frequency spectra to obtain time-frequency spectrum feature matrices; Based on the time-frequency spectrum feature matrix, the convolutional neural network is used to identify the footstep sound frequency feature, and the footstep sound frequency spectrum feature is obtained by separating the footstep sound from the environment sound through the spectrum masking technology. The binaural time difference and intensity difference algorithm is used to analyze the footstep sound spectrum feature, the sound source azimuth and relative distance are calculated, and the azimuth and distance parameters of the footstep sound are finally output.

[0006] Optionally, the azimuth and distance parameters are mapped to the keyboard area control instruction, wherein different azimuths correspond to different areas of the keyboard, and different distances correspond to the brightness levels of the keyboard light, comprising: The keyboard area is divided, and the standard keyboard is divided into a preset number of sector areas according to the azimuth angle, each area corresponds to a specific key combination, and a keyboard area mapping table is obtained; The distance-brightness mapping relationship is used to dynamically determine the brightness level according to the relative distance of the footstep sound, wherein the closer the distance, the higher the brightness, and a brightness level mapping rule is obtained; The footstep sound azimuth parameter is input into the keyboard area mapping table to determine the corresponding keyboard area identifier, and a preliminary keyboard area instruction is obtained; The distance parameter is converted into a specific brightness value in combination with the brightness level mapping rule, and a keyboard area control instruction containing the area identifier and the brightness level is finally output.

[0007] Optionally, the dynamic visual prompt signal is generated according to the keyboard area control instruction, wherein the dynamic visual prompt signal contains the light color change mode and the brightness gradual change sequence of the specified keyboard area, comprising: The area identifier and the brightness level in the keyboard area control instruction are parsed, and the corresponding warning color is selected from a preset color scheme library to obtain a basic color configuration; Based on the real-time change characteristics of the footstep sound, a pulse type light color change mode is designed, and a gradual fade-in and fade-out smooth transition algorithm is used to obtain a color change mode sequence; According to the brightness level value, an exponential decay model is used to calculate the brightness gradual change curve to obtain a smooth brightness gradual change sequence; The basic color configuration, color change mode sequence and brightness gradual change sequence are integrated and encoded into a standard RGB light control protocol, and a dynamic visual prompt signal is finally output.

[0008] Optionally, the dynamic visual prompt signal is output to the RGB keyboard light control system, and the corresponding keyboard area light is displayed according to the dynamic visual prompt signal to realize the azimuth and distance visualization prompt of the footstep sound in the game, comprising: The communication connection with the RGB keyboard is established through the USB-HID protocol, the compatibility of the keyboard light control function is verified, and a communication connection ready state is obtained; The dynamic visual prompt signal is analyzed, converted into a light control instruction set suitable for the keyboard, and a suitable control instruction is obtained. The suitable control instruction is sorted according to priority and sent to the keyboard light control chip to obtain a light control instruction queue. The keyboard light control chip is driven to execute the light control instruction queue, so that the corresponding keyboard area light is displayed according to the dynamic visual prompt signal, and finally the orientation and distance of the in-game footstep sound are visualized.

[0009] Another embodiment of the application provides a keyboard prompt system for in-game footstep sound, which comprises: A capture module is configured to capture audio signals of an in-game environment in real time, and perform spectral analysis and footstep sound frequency feature separation on the audio signals to extract orientation and distance parameters of the footstep sound. A mapping module is configured to map the orientation and distance parameters into keyboard area control instructions, wherein different orientations correspond to different areas of the keyboard, and different distances correspond to different brightness levels of the keyboard light. A generation module is configured to generate a dynamic visual prompt signal according to the keyboard area control instructions, wherein the dynamic visual prompt signal includes a light color change mode and a brightness gradient sequence of a specified keyboard area. A prompt module is configured to output the dynamic visual prompt signal to an RGB keyboard light control system, and drive the corresponding keyboard area light to display according to the dynamic visual prompt signal, so as to realize the orientation and distance visualization of the in-game footstep sound.

[0010] Another embodiment of the application provides a storage medium having a computer program stored therein, wherein the computer program is configured to execute the method described in any of the above embodiments when running.

[0011] Another embodiment of the application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to perform the method described in any of the above embodiments.

[0012] Compared with the prior art, the keyboard prompt method for in-game footstep sound provided by the application can capture the audio signal of the in-game environment in real time, perform spectrum analysis on the audio signal and separate the footstep sound frequency characteristics, extract the orientation and distance parameters of the footstep sound, map the orientation and distance parameters into keyboard area control instructions, generate dynamic visual prompt signals according to the keyboard area control instructions, output the dynamic visual prompt signals to an RGB keyboard light control system, and drive the corresponding keyboard area lights to display according to the dynamic visual prompt signals, so as to realize the visual prompt of the orientation and distance of the in-game footstep sound, thereby assisting the auditory perception in a visual way, improving the intuitiveness and distinguishability of the sound information, reducing the operation threshold of the game and enhancing the immersion. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 The hardware structure block diagram of the computer terminal of the keyboard prompt method for in-game footstep sound provided by the embodiment of the application is shown in the figure. Figure 2 The flowchart of the keyboard prompt method for in-game footstep sound provided by the embodiment of the application is shown in the figure. Figure 3 The structure diagram of the keyboard prompt system for in-game footstep sound provided by the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0014] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the application and cannot be explained as a limitation of the application.

[0015] The embodiment of the application first provides a keyboard prompt method for in-game footstep sound, which can be applied to electronic devices, such as computer terminals, specifically, ordinary computers and the like.

[0016] The following will be described in detail by taking the running on the computer terminal as an example. Figure 1 The hardware structure block diagram of the computer terminal of the keyboard prompt method for in-game footstep sound provided by the embodiment of the application is shown in the figure. Figure 1 As shown in the figure, the computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory can include a non-volatile storage medium and an internal memory.

[0017] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, which, when executed, can make the processor execute any kind of keyboard prompt method for in-game footstep sound.

[0018] The processor is used to provide computing and control capabilities to support the operation of the entire computer device.

[0019] The internal memory provides an environment for the running of a computer program in a non-volatile storage medium, which, when executed by the processor, can enable the processor to perform any kind of in-game footstep sound keyboard prompting method.

[0020] The network interface is used for network communication, such as sending assigned tasks. Figure 1 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0021] It should be understood that the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0022] Referring to Figure 2 The embodiments of the present application provide an in-game footstep sound keyboard prompting method, which can include the following steps: S201, real-time capture of audio signals of an in-game environment, and spectrum analysis and footstep sound frequency feature separation of the audio signals to extract the direction and distance parameters of the footstep sound; Specifically, the in-game environment audio signals can be captured in real time, and a high-pass filter can be used to eliminate low-frequency background noise to obtain preprocessed audio data streams. This step is a pre-cleaning link for footstep sound feature extraction. The core is to obtain in-game stereo audio signals through a real-time audio capture interface, and then use a high-pass filter to filter out low-frequency background noise (such as in-game wind noise, scene vibration noise, and low-frequency background music) unrelated to footstep sound, and retain the mid-high frequency signals dominated by footstep sound to provide a clean data source for subsequent spectrum analysis. It should be noted that the hardware interface and parameter configuration of audio capture, the technical selection and parameter design of the high-pass filter, and the effect verification of noise elimination each need to give specific examples in combination with the game audio characteristics and acoustic principles to ensure that the preprocessed audio data stream can accurately retain the footstep sound features.

[0023] I. Real-time audio signal capture implementation Capture interface and hardware adaptation: The WASAPI (Windows Audio Session API) interface of the Windows system is used as the real-time audio capture channel. This interface supports low-latency (≤10 ms) capture and adapts to commonly used audio output devices in games (such as earphones and speakers), allowing direct access to the original stereo audio stream (left and right channel independent data) within the game. Before capture, the supported sampling parameters of the device need to be queried through the interface function. In the example, the game audio device supports a sampling rate of 44.1 kHz (indicating the collection of 44100 audio samples per second, covering the human ear's hearing range of 20-20000 Hz, meeting the frequency band requirements of 200-5000 Hz for footstep sounds), a bit depth of 16 bits (each audio sample is represented by 16 bits of binary, with a dynamic range of approximately 96 dB, allowing clear capture of the intensity changes of footstep sounds), and a channel number of 2 (stereo sound for subsequent orientation determination).

[0024] Capture parameter configuration: Set the capture buffer size to 1024 samples (buffer size = sampling rate × delay time, 44100 Hz × 0.023 ms ≈ 1024, delay control within 20 ms to avoid time differences between in-game sounds and captured data), capture mode as "circular buffer" (continuously overwrite old data to ensure real-time performance), read 1024 samples of stereo audio data from the buffer each time, and store it as a raw data stream in PCM (Pulse Code Modulation) format (left channel data in array L [], right channel in array R [], each element is a 16-bit integer with a value range of -32768 to 32767).

[0025] Data integrity check: After reading the data each time, check the reasonableness of the sample values (such as whether there are abnormal values outside the range of -32768 to 32767, whether there are consecutive identical silent samples) to exclude device faults or transmission errors. In the example, some frame data L [0]=-1234, L [1]=-1189, R [0]=-1056, R [1]=-1012 are all within the normal range and have no consecutive silence, so the data is determined to be complete.

[0026] II. Design of high-pass filter and noise elimination Filter selection and principle: 8-stage Butterworth high-pass filter is selected. The advantage of this filter is that the amplitude-frequency characteristic is flat in the passband (avoiding frequency distortion of footstep sound in the passband), and the stopband attenuation is steep (effectively suppressing low-frequency noise). The core function of the high-pass filter is to allow signals above the cutoff frequency to pass through and attenuate signals below the cutoff frequency. The main energy of footstep sound is concentrated in 200-5000Hz, so the cutoff frequency f c is set to 200Hz (i.e. only signals above 200Hz are retained, and low-frequency background noise below 200Hz is filtered out, such as low-frequency vibration sound of grass friction in the game, low-frequency roar of distant engine).

[0027] Filter parameter calculation: According to the Butterworth filter design formula, the transfer function of the 8-stage high-pass filter is H (s)=s^(8) / (s^(8)+a7s^7+...+a0), where the coefficients a0-a7 are calculated by the cutoff frequency f c=200Hz and the sampling rate f s=44100Hz. The specific steps are as follows: first, normalize the cutoff frequency to the digital domain (ω c=2πf c / f s≈0.0289rad), then generate poles through Butterworth polynomial, and finally convert the analog filter to digital filter through bilinear transformation to get the filter coefficients (numerator coefficients b=[0.92, -7.36, 22.08, -36.8, 36.8, -22.08, 7.36, -0.92], denominator coefficients a=[1, -7.28, 21.6, -35.84, 35.2, -21.12, 6.96, -0.88], coefficients are normalized to ensure filter stability).

[0028] Filtering and Effect Verification: The captured PCM data stream is filtered using a "direct II type" structure (reduces computation, suitable for real-time processing), i.e., applying filter coefficients to left and right channel data respectively, the formula is y(n) = b0x(n) + b1x(n-1) +... + b7x(n-7) - a1y(n-1) -... - a7y(n-7) (x(n) is the current input sample, y(n) is the filtered output sample, n is the sample number). In the example, a certain input sample sequence x = [-1234, -1189, -1100, -1020, -950, -880, -820, -770] (contains low-frequency components below 200Hz), after filtering, the output y = [-1050, -1010, -930, -860, -800, -740, -690, -650], through spectral analysis, it can be observed that the signal amplitude below 200Hz is reduced from -40dB to below -60dB, the noise suppression effect is significant, while the footstep sound component above 200Hz (such as the signal at 300Hz) amplitude only attenuates 0.5dB, basically no distortion.

[0029] III. Format of the Preprocessed Audio Data Stream The filtered audio data stream still maintains the PCM format, with a sampling rate of 44.1kHz, a bit depth of 16 bits, and stereo sound. It is stored as a "frame" structure, with each frame containing 2048 samples (1024 for the left channel and 1024 for the right channel), a frame interval of approximately 23ms (1024 / 44100≈0.023s), and a timestamp (accurate to milliseconds, such as 2025-09-21T15:30:00.123) attached to each frame for subsequent time alignment of the time-domain signal. This data stream will be used as input for the next step of short-time Fourier transform.

[0030] Performing short-time Fourier transform on the preprocessed audio data stream converts the time-domain signal into a Mel frequency spectrum, obtaining a time-frequency spectrum feature matrix. This step is the "conversion link from time-domain signal to frequency-domain feature", the core is to decompose the continuous time-domain audio signal into a "time-frequency" two-dimensional spectrum through short-time Fourier transform (STFT), and then convert it into a Mel frequency spectrum that is more sensitive to the human ear through Mel frequency scaling, finally form a time-frequency spectrum feature matrix containing the frequency characteristics and timing information of the footstep sound, providing structured frequency domain features for subsequent footstep sound recognition. It needs to be clear that the parameter design of STFT, the conversion principle of Mel frequency spectrum, and the construction method of feature matrix, each link needs to give quantitative examples combined with the time-frequency characteristics of footstep sound to ensure that the feature matrix can accurately depict the frequency variation of footstep sound.

[0031] I. Parameter design and implementation of short-time Fourier transform (STFT) Core principle of STFT: Time-domain audio signals are continuously changing, and direct Fourier transform cannot reflect the change of frequency over time. STFT divides time-domain signals into overlapping "frames" and performs Fourier transform on each frame separately, thereby obtaining a "time-frequency" two-dimensional spectrogram. Footstep sound belongs to short-time non-stationary signals (a single footstep sound lasts about 100-300ms and contains rapid changes in multiple frequency components), so the frame parameters of STFT need to be adapted to this characteristic.

[0032] Key parameter configuration: Frame length: set to 20ms (corresponding to 882 samples), this length can contain enough frequency information (the frequency resolution of a 20ms frame is 50Hz, which meets the requirement of frequency band subdivision for footstep sound 200-5000Hz), and can capture the short-time changes of footstep sound (such as the starting, peak, and decay process of footstep sound); Frame shift: set to 10ms (corresponding to 441 samples), the frame shift is 50% of the frame length, which can reduce the loss of information between frames (avoiding the breaking of footstep sound features due to too large frame interval), while controlling the computational load (50% overlap per frame, balancing computational efficiency and information integrity); Window function: Hanning window is selected, the window function formula is w (n)=0.5-0.5cos (2πn / (N-1)) (n=0 to N-1), this window function can effectively reduce the spectral sidelobes caused by time-domain signal truncation (sidelobe suppression ratio ≥31dB), avoiding the mutual interference of frequency components of footstep sound, in the example, after multiplying the Hanning window by a certain frame of time-domain signal x (n) (882 samples), the edge sample amplitude is smoothly attenuated, the middle sample retains the original amplitude, and the frequency spectrum leakage is reduced.

[0033] Calculation process of STFT: perform the following steps on each frame of preprocessed audio data (left channel or right channel, this example takes the left channel as an example): Frame segmentation: take 882 samples from the preprocessed data stream as a frame, if the last frame of samples is not enough, fill it with zeros to 882; Windowing: multiply the frame data with the Hanning window to get the windowed data x_win (n)=x (n)×w (n); Fourier transform: Transform x_win(n) by Fast Fourier Transform (FFT) with 1024 points (larger than frame length 882, to improve frequency resolution by zero padding), get complex frequency domain signal X(k)=FFT[x_win(n)] (k=0 to 1023); Amplitude spectrum calculation: Calculate the amplitude value of each frequency point |X(k)|=√[Re(X(k)) 2 +Im(X(k)) 2 ], and take the logarithm to get the log amplitude spectrum (unit dB) to compress the dynamic range, in the example k=10 corresponds to the frequency f=k×f_s / N_FFT=10×44100 / 1024≈430Hz, the amplitude value |X(10)|=0.8, the logarithmic amplitude is 20lg(0.8)≈-1.94dB.

[0034] II. Conversion principle and implementation of Mel frequency spectrum Physical meaning of Mel frequency: Human ears are not linear in frequency perception, more sensitive to low frequency (such as 200-1000Hz) frequency changes, and less sensitive to high frequency (such as 3000-5000Hz), Mel frequency scale is a frequency unit that simulates human auditory characteristics, and its conversion formula with actual frequency f (Hz) is Mel(f)=2595×lg(1+f / 700). For example, the actual frequency 200Hz corresponds to the Mel value 2595×lg(1+200 / 700)=2595×lg(1.2857)≈2595×0.109≈283Mel, and 5000Hz corresponds to the Mel value 2595×lg(1+5000 / 700)=2595×lg(8.1429)≈2595×0.911≈2365Mel, so the Mel frequency range of footsteps is about 283-2365Mel.

[0035] Design of Mel filter bank: To convert the linear frequency spectrum of STFT to Mel frequency spectrum, a set of overlapping triangular filters (Mel filter bank) covering the frequency range of 283-2365Mel is designed, and the number of filters is set to 40 (enough to subdivide the Mel frequency range, while avoiding excessive dimensionality leading to increased computational complexity). The center frequency of each Mel filter is uniformly distributed according to the Mel scale, and the center Mel frequency of the mth filter is Mel_m=283+(2365-283) / (40-1)×(m-1) (m=1 to 40), in the example, the center Mel frequency of m=5 is Mel_5=283+2082 / 39×4≈283+213.59≈496.59Mel, which corresponds to the actual frequency: f = 700 x (10^(Mel_5 / 2595) - 1) = 700 x (10^(496.59 / 2595) - 1) ≈ 700 x (10^0.191 - 1) ≈ 700 x (1.55 - 1) = 385 Hz.

[0036] Calculation of Mel-frequency spectrum: multiply the log-amplitude spectrum of STFT with the frequency response of each Mel filter, sum up to get the energy value of each Mel frequency point, i.e. the Mel-frequency spectrum. The specific steps are as follows: Calculate the weight of each Mel filter on the STFT frequency axis (amplitude response of triangular filter, weight is 1 at the center frequency and linearly decays to 0 on both sides); For each Mel filter, multiply the log-amplitude spectrum of the STFT frequency points it covers by the corresponding weight, and sum up to get the energy E_m = Σ[|X (k)| x w_m (k)] (w_m (k) is the weight of the mth filter at the kth frequency point); Take the logarithm (unit dB) of E_m to get the Mel-frequency spectrum. In the example, the 5th Mel frequency point E_5 = (-1.94 dB x 0.2) + (-1.5 dB x 0.8) + (-2.1 dB x 0.1) = -0.388 - 1.2 - 0.21 = -1.798 dB, and after taking the logarithm, it is still -1.8 dB (since E_m is the energy sum, the logarithm here is mainly used for dynamic range compression).

[0037] III. Construction of time-frequency spectrum feature matrix Arrange the Mel-frequency spectrum of consecutive frames in time order to form a time-frequency spectrum feature matrix. Assuming that 100 frames of audio data are processed (corresponding to about 2.3s), and each frame of Mel-frequency spectrum contains 40 feature values (40 Mel filters), the time-frequency spectrum feature matrix has a dimension of 100 x 40 (rows represent time frames, columns represent Mel frequency points), and each element M (i, j) in the matrix represents the log energy value (unit dB) of the i-th frame and the j-th Mel frequency point. In the example, M(10,5) = -1.8 dB indicates that the energy of the 10th frame (corresponding to time 0.23s) and the 5th Mel frequency point (about 385 Hz) is -1.8 dB. This matrix clearly presents the frequency distribution of the footstep sound at different time points (such as the low-frequency component of the footstep sound in the initial frame is strong, the mid-high frequency component of the peak frame is prominent, and the frequency components of the decay frame are all reduced), providing key time-frequency features for subsequent convolutional neural network recognition of footstep sound.

[0038] Based on the time-frequency spectrum feature matrix, the convolutional neural network is used to identify the footstep sound frequency feature, and the footstep sound and the environment sound are separated by the spectrum masking technology to obtain the footstep sound spectrum feature; This step is the core link of "footstep sound feature recognition and separation". The core is to classify the time-frequency spectrum feature matrix by using the convolutional neural network (CNN), identify the frames containing footstep sound, and then separate the footstep sound spectrum from these frames by the spectrum masking technology (such as ideal binary masking) to eliminate the interference of environmental sound (such as gunshots and explosions in the game), and finally obtain the pure footstep sound spectrum feature. It should be noted that the network structure and training process of CNN, the judgment logic of footstep sound recognition, and the implementation principle of spectrum masking are required to be combined with the data set and quantitative examples to ensure the recognition accuracy and separation effect.

[0039] I. Structure design and training of convolutional neural network (CNN) The network architecture of CNN: For the time-frequency spectrum feature matrix (time-frequency two-dimensional feature), a lightweight CNN structure of "input layer - convolutional layer 1 - pooling layer 1 - convolutional layer 2 - pooling layer 2 - fully connected layer - output layer" is designed to adapt to the real-time processing requirement (inference time ≤10ms / frame), and the parameters and functions of each layer are as follows: Input layer: receiving a three-dimensional tensor of "time frame number × mel frequency point number × channel number", the input tensor dimension here is 10×40×1 (taking 10 consecutive frames of mel frequency spectrum as an input sample, covering about 230ms of time, containing a complete single footstep sound period; 40 is the number of mel frequency points; 1 is the number of channels, single-channel grayscale feature); Convolutional layer 1: using 16 3×3 size convolution kernels (3×3 receptive field can capture the local features of adjacent time frames and frequency points, such as the frequency peak pattern of footstep sound), step size is 1×1 (without changing the feature map size), padding mode is "same" (output feature map dimension is consistent with input), activation function is ReLU (to alleviate gradient disappearance), output feature map dimension is 10×40×16; Pooling layer 1: using 2×2 size maximum pooling kernel (to reduce feature dimension and preserve key features), step size is 2×2, output feature map dimension is 5×20×16 (time frame is reduced from 10 to 5, frequency point is reduced from 40 to 20); Convolutional layer 2: using 32 3×3 convolution kernels, step size is 1×1, padding is "same", activation function is ReLU, output feature map dimension is 5×20×32; Pooling layer 2: using 2×2 maximum pooling kernel, step size is 2×2, output feature map dimension is 3×10×32; Fully connected layer 1: flatten the output of pooling layer 2 into a one-dimensional vector (3x10x32=960 elements), map to 256 neurons, activation function ReLU, add Dropout layer (dropout rate 0.3, suppress overfitting); Fully connected layer 2: map 256 neurons to 2 output neurons, corresponding to "footsteps" and "non-footsteps" classes, activation function Softmax (output probabilities for two classes, sum to 1).

[0040] CNN training process: Dataset construction: collect footstep sound files containing different scenes (concrete, grass, wooden floor), different roles (player, NPC), and non-footstep environmental sounds (gunshot, explosion, wind) commonly found in games, a total of 100,000 samples, each sample is a 10x40 mel-frequency spectrum, labeled as "1" (footsteps) or "0" (non-footsteps), divided into training set (80,000), validation set (10,000), test set (10,000) in the ratio of 8:1:1; Training parameters: use Adam optimizer, initial learning rate 0.001, learning rate halved every 5 epochs; loss function is cross-entropy loss (suitable for binary classification tasks); training rounds 20 epochs, batch size 32; Training effect: footsteps recognition accuracy on test set reaches 95.2%, precision 94.8% (proportion of samples recognized as footsteps that are actually footsteps), recall 95.5% (proportion of samples that are actually footsteps that are correctly recognized), meeting the needs of real-time recognition in games.

[0041] II. Recognition and determination logic of footsteps Input sample construction: divide the time-frequency spectrum feature matrix into input samples in "sliding window" manner, window size 10 frames (230ms), window step 5 frames (115ms), ensuring continuous coverage of all time frames, example 1-10 frames as sample 1, 5-14 frames as sample 2, etc.

[0042] CNN inference and probability determination: input each input sample into the trained CNN to get the "footstep sound probability P1" and "non-footstep sound probability P0" (P1+P0=1), set the determination threshold P_th=0.8 (if P1≥0.8, determine that the window contains footstep sound; otherwise, it is non-footstep sound), in the example, sample 1 has P1=0.92≥0.8, and is determined to contain footstep sound; sample 2 has P1=0.35<0.8, and is determined to be non-footstep sound.

[0043] Frame-level determination and smoothing: for each time frame, count the determination results of all windows containing the frame, if more than 50% of the windows are determined to contain footstep sound, the frame is marked as "footstep sound frame"; otherwise, it is "non-footstep sound frame", to avoid fluctuations caused by single window misjudgment. In the example, the 8th frame is contained by sample 1 (determined to contain footstep sound) and sample 2 (determined to be non-footstep sound), 1 out of 2 windows is determined to contain footstep sound, accounting for 50%, and is marked as "footstep sound frame"; the 12th frame is only contained by sample 2, and is determined to be "non-footstep sound frame".

[0044] III. Spectral masking technology separates footstep sound and environmental sound Principle of ideal binary masking (IBM): spectral masking technology generates a "masking matrix" to retain frequency components in the time-frequency spectrum that belong to footstep sound (masking value = 1) and suppress frequency components that belong to environmental sound (masking value = 0). Ideal binary masking is the most commonly used masking method, and its masking value is defined as: M (k,t)=1 (if the energy of the kth frequency point in the tth frame mainly comes from footstep sound), M (k,t)=0 (otherwise).

[0045] Generation of masking matrix: generate the masking matrix based on the recognition results of CNN and the frequency characteristics of footstep sound: For "footstep sound frames", according to the typical frequency range of footstep sound (200-5000Hz, corresponding to the frequency points k=5 to k=57 of STFT, 44100x5 / 1024≈216Hz, 44100x57 / 1024≈2468Hz, which needs to be expanded to k=4 to k=116 to cover 5000Hz), set the masking value in this frequency range to 1, and set the value outside the range to 0; For "non-footstep sound frames", set all masking values to 0 (no footstep sound component); In the example, the masking values of k=4 to k=116 of the 8th frame (footstep sound frame) are M (k,8)=1, and the masking values of k<4 or k>116 are 0; the masking values of the 12th frame (non-footstep sound frame) are M (k,12)=0.

[0046] Step 4: Extract the footstep sound spectrum feature: Multiply the amplitude spectrum of the STFT with the masking matrix element by element to get the separated footstep sound amplitude spectrum, formula is X_foot (k,t)=X (k,t)×M (k,t) (X (k,t) is the original amplitude spectrum, X_foot(k,t) is the footstep sound amplitude spectrum). In the example, the original amplitude spectrum X (10,8)=0.8 (k=10, t=8 frames, corresponding to frequency 430Hz), M (10,8)=1, so X_foot (10,8)=0.8×1=0.8; the original amplitude spectrum X (2,8)=0.5 (k=2, corresponding to frequency 86Hz), M (2,8)=0, so X_foot (2,8)=0.5×0=0, successfully eliminating the low-frequency ambient sound. The separated footstep sound spectrum feature is a "time-frequency" two-dimensional spectrum, containing the frequency distribution and intensity information of the footstep sound at different time points, which will be used for the next step of calculating the direction and distance.

[0047] Step 5: Analyze the footstep sound spectrum feature using the binaural time difference and intensity difference algorithm to calculate the sound source azimuth and relative distance, and finally output the azimuth and distance parameters of the footstep sound.

[0048] This step is the "core link of footstep sound spatial information extraction", the core is to use the stereo characteristics of game audio (left and right channels), calculate the time difference of footstep sound reaching left and right ears through binaural time difference (ITD) algorithm, calculate the intensity difference of footstep sound received by left and right ears through binaural intensity difference (ILD) algorithm, combine the two difference values to determine the azimuth angle (horizontal direction angle) of the footstep sound, and calculate the relative distance based on the intensity attenuation model of the footstep sound, finally output the spatial parameters containing azimuth angle and distance, provide spatial positioning basis for subsequent keyboard mapping. It needs to be clear that the calculation principle of ITD and ILD, the fusion calculation of azimuth angle, and the attenuation model of distance, each link needs to give quantitative examples combined with stereo sound data and physical formula to ensure the accuracy of spatial parameters.

[0049] I. Calculation principle and implementation of binaural time difference (ITD) Physical meaning of ITD: When the footstep sound comes from the left or right side, the path length of the sound wave reaching the left and right ears is different, resulting in a time difference, i.e. ITD (Interaural Time Difference). ITD is positive, indicating that the right ear receives the signal first (the sound source is on the right side), and negative, indicating that the left ear receives first (the sound source is on the left side). The greater the absolute value of ITD, the greater the angle of the sound source deviating from the front (0°). The range of ITD that can be perceived by the human ear is about -0.6 to 0.6 ms (corresponding to an azimuth angle of -90° to 90°), and the ITD of the footstep sound is usually between -0.3 and 0.3 ms (the sound source in the game is generally within a 180° range around the player).

[0050] Method for calculating ITD: The cross-correlation function is used to calculate the time difference of the footstep sound spectral features of the left and right channels. The cross-correlation function is used to measure the degree of similarity of two signals at different time delays, and the delay corresponding to the peak value is the ITD. The specific steps are as follows: Extract the footstep sound time-domain signals of the left and right channels: Convert the separated left and right channel footstep sound spectral features (X_foot_L (k,t), X_foot_R (k,t)) back to time-domain signals through inverse short-time Fourier transform (ISTFT), to obtain the left channel time-domain signal x_L (n) and the right channel time-domain signal x_R (n) (n is the sample number); Calculate the cross-correlation function: Calculate the cross-correlation function R (τ)=Σ[x_L (n)×x_R (n+τ)] of x_L (n) and x_R (n) (τ is the delay sample number, ranging from -100 to 100, corresponding to a time delay of -2.27ms to 2.27ms, 100 samples ≈ 2.27ms at a sampling rate of 44100Hz); Find the peak delay: Find the delay τ_max corresponding to the maximum peak value of the cross-correlation function R (τ), and convert τ_max to time difference ITD=τ_max / f_s (unit: ms); In the example, the cross-correlation function of x_L (n) and x_R (n) reaches the maximum value at τ_max=-13 (τ_max is negative, indicating that the left channel signal leads the right channel), so ITD=-13 / 44100≈-0.295ms (the left ear receives the signal first, and the sound source is on the left side).

[0051] II. Calculation principle and implementation of binaural intensity difference (ILD) Physical meaning of ILD: In the process of sound propagation, the head will block the sound, causing the signal strength received by the ear far from the sound source to be lower than that close to the sound source. This intensity difference is called ILD (Interaural Level Difference). ILD is positive, indicating that the right ear is stronger than the left ear (sound source on the right side), and negative, indicating that the left ear is stronger than the right ear (sound source on the left side). The greater the absolute value of ILD, the greater the azimuth of the sound source. The ILD of footstep sound is usually between -10 and 10 dB.

[0052] Calculation method of ILD: ILD is calculated by calculating the intensity ratio of the left and right channel footstep sound spectral features. The specific steps are as follows: Calculate the intensity of the left and right channels: For each frame of footstep sound spectral feature, calculate the average intensity of the left and right channels in the footstep sound frequency range (200-5000 Hz). Intensity is defined as the square of amplitude. The left channel intensity I_L=Σ[X_foot_L(k,t) 2 ] / K (K is the number of frequency points in this frequency range, such as k=4 to k=116, K=113), and the right channel intensity I_R=Σ[X_foot_R (k,t) 2 ] / K; Calculate ILD: ILD=10×lg (I_R / I_L) (unit dB), if I_L=0 (no left channel signal), ILD=+10dB; if I_R=0, ILD=-10dB; In the example, I_L=0.08 (left channel intensity average), I_R=0.03 (right channel intensity average), then ILD=10×lg (0.03 / 0.08)=10×lg (0.375)≈-4.26dB (left ear intensity greater than right ear, sound source on the left side, consistent with ITD result).

[0053] Three, fusion calculation of azimuth ITD and ILD azimuth mapping model: Through a large number of experimental data, the mapping relationship between ITD, ILD and azimuth θ (horizontal azimuth, defined as 0° in front, left as negative, right as positive, range -90° to 90°) is fitted: ITD-θ mapping: θ=ITD×300 (unit: ° / ms), this fitting relationship is based on human ear hearing experiments. When ITD=-0.3ms, θ=-90°; ITD=0.3ms, θ=90°; ITD=0, θ=0°; ILD-θ mapping: θ = ILD × 9 (unit: ° / dB), when ILD = -10dB, θ = -90°; ILD = 10dB, θ = 90°; ILD = 0, θ = 0°; Fusion azimuth: to improve the accuracy of azimuth, the azimuth calculated by ITD and ILD is fused by weighted average, the weight of ITD is set to 0.6 (the influence of time difference on azimuth is more significant), and the weight of ILD is set to 0.4, the formula is θ = 0.6 × θ_ITD + 0.4 × θ_ILD (θ_ITD is the azimuth of ITD mapping, θ_ILD is the azimuth of ILD mapping); In the example, ITD = -0.295ms, θ_ITD = -0.295 × 300 ≈ -88.5°; ILD = -4.26dB, θ_ILD = -4.26 × 9 ≈ -38.34°; After fusion, θ = 0.6 × (-88.5) + 0.4 × (-38.34) = -53.1 - 15.34 ≈ -68.44°, rounded to -68°, that is, the footsteps sound comes from the left side of the player about 68°.

[0054] Four, the calculation of relative distance Footstep intensity decay model: when sound propagates in air, the intensity decays with the square of the distance (spherical wave decay), assuming that the standard intensity of footsteps in the game at the reference distance r0 = 1m is I0 (through game settings or experimental calibration, in the example I0 = 1.0), then the intensity I at the actual distance r satisfies I = I0 / (r 2 ), the distance r can be calculated by the formula r = √(I0 / I).

[0055] Calculation of actual intensity I: the "feature intensity" is extracted from the spectral characteristics of footsteps as I, which is defined as the average amplitude value in the frequency range of footsteps (200-5000Hz), the formula is I = Σ[X_foot_avg (k,t)] / K (X_foot_avg (k,t) is the average amplitude value of left and right channels of footsteps, K is the number of frequency points); In the example, I0 = 1.0 (intensity at 1m), the I of a frame of footsteps is 0.0625 (average amplitude value), then r = √(1.0 / 0.0625) = √16 = 4m, that is, the relative distance between the footsteps and the player is 4m.

[0056] Five, the output of azimuth and distance parameters The final output of the footstep sound space parameter includes "azimuth angle θ" and "relative distance r", and the parameter format is "θ: ± XX°, r: XXm". In the example, the output is "θ: -68°, r: 4m". This parameter will be used as the input for the next step of keyboard area mapping to generate the corresponding keyboard light prompt instruction.

[0057] S202, mapping the azimuth and distance parameters to keyboard area control instructions, wherein different azimuths correspond to different areas of the keyboard, and different distances correspond to different brightness levels of the keyboard light; Specifically, keyboard area division can be performed, and a standard keyboard can be divided into a preset number of sector areas according to the azimuth angle, and each area corresponds to a specific key combination to obtain a keyboard area mapping table. This step is the basic link for establishing the association between the azimuth parameter and the physical keys of the keyboard. The core is based on the operation habit of the game player (with "WASD" as the core operation area, and the key coverage range placed naturally by both hands), taking the horizontal direction of the player facing the screen as the azimuth reference (0°), dividing the horizontal azimuth angle (-90°~90°, covering the main sound source directions on the left and right sides and the front of the player) into a preset number of sector areas, and binding the key combination that the player's hands can easily perceive in each area. Finally, a mapping table is formed that corresponds to the azimuth angle - keyboard area - key combination, ensuring that the player can quickly locate the footstep sound azimuth through the keyboard light prompt. It needs to be clear that the azimuth angle reference definition, sector area division logic, key combination matching principle, and mapping table content constitute each link, and specific examples need to be given for each link in combination with the standard keyboard layout and the player's operation habit to ensure that the mapping relationship is intuitive and ergonomic.

[0058] I. Azimuth angle reference and sector area division logic Azimuth angle reference definition: taking the direction of the player facing the game screen as the horizontal azimuth angle 0° (straight ahead), the horizontal left as negative angle (range -90°~0°, corresponding to the sound source on the left side of the player), and the horizontal right as positive angle (range 0°~90°, corresponding to the sound source on the right side of the player), which covers the horizontal azimuth range of -90°~90° (the main footstep sound in the game comes from this range, and beyond it, it is not necessary to prompt due to too far distance or visual angle limitation). The angle division precision needs to balance "prompt accuracy" and "player perception complexity", and it is preset to divide into 8 sector areas (each area has an angle span of 22.5°, which avoids confusion caused by too many areas and meets the prompt needs of 16 main azimuth directions), and the angle interval of the 8 areas is divided as follows: Zone 1: -90°~ -67.5° (left rear); Zone 2: -67.5°~ -45° (left); Zone 3: -45°~ -22.5° (left front); Zone 4: -22.5°~22.5° (straight ahead, core operation area); Zone 5: 22.5°~45° (right front); Zone 6: 45°~67.5° (right); Zone 7: 67.5°~90° (right rear); Zone 8: 90°~ -90° (none, reserved for backup).

[0059] Angle span calculation basis: total coverage angle 180° (-90°~90°), 8 zones average each span 22.5° (180° / 8=22.5°), ensuring that the orientation of each zone is clear, with no overlap or omission.

[0060] Standard keyboard layout and key selection: based on the common 104-key standard keyboard, the keys that players can quickly perceive when placing their hands naturally are selected (avoiding keys that require significant hand movement, such as the number keypad), mainly covering the core area of the "main keyboard area", including: Left-hand operation area: Tab, Caps Lock, Shift (left), Ctrl (left), Q, W, E, A, S, D, Z, X, C; Right-hand auxiliary area: Shift (right), Ctrl (right), Alt (right), F, G, H, V, B, N (right hand usually controls the mouse, auxiliary keys are only supplementary prompts).

[0061] Selection principle: key spacing ≤5cm (player's fingers do not need to leave the reference key "WASD" to perceive the light), avoiding function keys (such as F1-F12) or special keys (such as Backspace), ensuring that the prompt does not interfere with normal operation.

[0062] II. Matching principle of sector area and key combination Core operation area priority matching: the straight-ahead area (Zone 4: -22.5°~22.5°) is the direction that players pay most attention to, matching the core movement keys "W, A, S, D" (player's left hand always covers this area, light prompt can be directly associated with movement direction), while also including adjacent "Q, E" (skill keys), forming a key combination of "W (front), A (left), S (rear), D (right), Q (left skill), E (right skill)", ensuring that the straight-ahead footstep sound prompt is directly associated with operation.

[0063] Left-right symmetric matching: left and right areas use symmetric key matching to avoid player perception confusion: Left front area (Region 3: -45°~ -22.5°): Matching "W, A, Q, Z" (centered on the A key, extending left front, corresponding to the left front sound source); Right front area (Region 5: 22.5°~ 45°): Matching "W, D, E, V" (centered on the D key, extending right front, corresponding to the right front sound source); Left side area (Region 2: -67.5°~ -45°): Matching "A, Q, Z, Shift (left)" (centered on the A key, extending left, corresponding to the left sound source); Right side area (Region 6: 45°~ 67.5°): Matching "D, F, V, Shift (right)" (centered on the D key, extending right, corresponding to the right sound source); Left side rear area (Region 1: -90°~ -67.5°): Matching "Q, Z, Ctrl (left), Caps Lock" (leftmost key of the left hand, corresponding to the left rear sound source); Right side rear area (Region 7: 67.5°~ 90°): Matching "F, B, Ctrl (right), Alt (right)" (auxiliary key of the right hand, corresponding to the right rear sound source).

[0064] Key position uniqueness principle: A single key position belongs to only one sector area (to avoid confusion caused by the same key position corresponding to multiple directions), for example, "A key" only belongs to the left front area (Region 3), "D key" only belongs to the right front area (Region 5), ensuring that the light prompt direction is unique.

[0065] III. Construction of Keyboard Region Mapping Table The mapping table uses structured literal format, containing four core fields: "region identification, azimuth angle interval, key combination list, light control address". The meaning and examples of each field are as follows: Region identification: unique number (1-7, corresponding to 7 effective sector areas), such as "Region 3"; Azimuth angle interval: the horizontal azimuth angle range corresponding to the region, such as "-45°~ -22.5°"; Key combination list: the specific key position bound (represented by standard key position name), such as "W, A, Q, Z"; Light control address: the hardware control address of each key position in the RGB keyboard (defined by the keyboard manufacturer, such as "A key" address is 0x0A, "W key" address is 0x0B), used for subsequent light driving, such as "0x0A, 0x0B, 0x08, 0x12".

[0066] Example keyboard region mapping table segment: "Region 3: Azimuth angle interval -45°~ -22.5°, key combination list [W, A, Q, Z], light control address [0x0B, 0x0A, 0x08, 0x12]; Region 5: Azimuth angle interval 22.5°~45°, key combination list [W, D, E, V], light control address [0x0B, 0x0C, 0x0D, 0x15]; Region 2: Azimuth angle interval -67.5°~ -45°, key combination list [A, Q, Z, Shift (left)], light control address [0x0A, 0x08, 0x12, 0x0F]".

[0067] Using the distance-brightness mapping relationship, the brightness level is dynamically determined according to the relative distance of the footstep sound, where the closer the distance, the higher the brightness, and the brightness level mapping rule is obtained; This step is the core link of "establishing the association between distance parameters and keyboard light brightness", the core is to construct the "relative distance of footstep sound - light brightness level" mapping relationship based on the "human eye's perception characteristics of brightness change" (high brightness warning for close-range danger signals, low brightness prompt for long-distance signals), the relative distance of footstep sound in the game (0~10m, footstep sound can be ignored beyond 10m) is divided into several brightness levels, each level corresponds to the specific brightness value (0~255, 0 is the darkest, 255 is the brightest) of the RGB keyboard, forming a mapping rule that can be directly used for light control. It needs to be clear that the setting basis of the distance range, the division logic of the brightness level, the quantization standard of the brightness value, and the representation method of the mapping rule, each link needs to give specific numerical examples combined with the game audio characteristics and visual perception principles to ensure that the brightness change can intuitively reflect the distance.

[0068] I. Setting basis for the relative distance range of footstep sound The propagation characteristics of footstep sound in the game follow the "spherical wave attenuation law" (intensity is inversely proportional to the square of the distance), combined with the audio design of mainstream shooting games (such as "CS:GO", "Valorant"): 0~2m: Footstep sound is clear and accompanied by vibration feedback (close-range danger, needs urgent warning); 2~5m: Footstep is clear and can be distinguished by footstep type (such as jumping, walking, medium-distance warning); 5~8m: Footstep sound is weakened and needs to be carefully listened to (medium-distance prompt); 8~10m: Footstep sound is weak and only the direction can be perceived (long-distance weak prompt); >10m: Footstep sound is covered by environmental sound and does not need to be prompted (brightness is set to 0).

[0069] Therefore, the effective distance range is set to 0-10m, and the default brightness level is 0 when it exceeds, ensuring that the focus is on the threatening close-range sound source.

[0070] II. Division logic and quantization standard of brightness level Brightness level division principle: taking into account "perceived distinction" and "visual comfort", divided into 5 brightness levels (levels 0-4, the higher the level, the higher the brightness), the brightness value of each level is based on the brightness characteristics of the RGB color space (the human eye is most sensitive to green, followed by red, and the lowest to blue, here take red as an example, the subsequent color configuration link can be adjusted), using "non-linear increase" (high near-distance brightness increase, small far-distance increase), which conforms to the human eye's perception priority of dangerous signals: Level 0 (distance > 10m): brightness value 0 (completely off, no prompt); Level 1 (distance 8-10m): brightness value 51 (20% brightness, weak prompt, RGB value (51, 0, 0)); Level 2 (distance 5-8m): brightness value 102 (40% brightness, weak prompt, RGB value (102, 0, 0)); Level 3 (distance 2-5m): brightness value 178 (70% brightness, medium prompt, RGB value (178, 0, 0)); Level 4 (distance 0-2m): brightness value 255 (100% brightness, strong warning, RGB value (255, 0, 0)).

[0071] Non-linear increase basis: level 1 to level 2 brightness increase 51 (51→102), level 2 to level 3 increase 76 (102→178), level 3 to level 4 increase 77 (178→255), near-distance increase is larger, strengthening the visual impact of dangerous signals.

[0072] Physical meaning of brightness value: the brightness value of RGB keyboard uses 8-bit binary quantization (0-255), representing the luminous intensity of a single color channel (such as red), and brightness value 255 corresponds to the maximum luminous power of LED light bead (about 0.5W), brightness value 0 corresponds to the light bead being off, and the brightness value of different levels is realized by adjusting the PWM (pulse width modulation) duty cycle of LED (such as brightness value 51 corresponds to PWM duty cycle 20%, that is, LED lights up for 2ms every 10ms).

[0073] III. Representation of distance-brightness level mapping rule The mapping rule adopts the corresponding format of "distance interval - brightness level - brightness value - RGB example" to clearly specify the specific parameters corresponding to each distance range. For example: "1. Distance interval > 10m: brightness level 0, brightness value 0, RGB example (0,0,0) (no prompt); 2. Distance interval 8~10m: brightness level 1, brightness value 51, RGB example (51,0,0) (weak red prompt); 3. Distance interval 5~8m: brightness level 2, brightness value 102, RGB example (102,0,0) (weak red prompt); 4. Distance interval 2~5m: brightness level 3, brightness value 178, RGB example (178,0,0) (medium red prompt); 5. Distance interval 0~2m: brightness level 4, brightness value 255, RGB example (255,0,0) (strong red warning)".

[0074] At the same time, the "distance ambiguity processing rule" is supplemented: if the distance is at the boundary of two intervals (such as 8m), take the higher brightness level (8m corresponds to level 2 instead of level 1), to avoid brightness sudden change at the boundary leading to perception confusion.

[0075] Input the footstep sound orientation parameter into the keyboard area mapping table to determine the corresponding keyboard area identifier and obtain the preliminary keyboard area instruction; This step is the execution link of "orientation parameter conversion to keyboard area instruction", the core is to match the "orientation angle" output in step four (such as -35°) with the "orientation angle interval" in the keyboard area mapping table, determine the sector area identifier (such as area 3) that the orientation angle belongs to, and then extract the "key combination list" and "light control address" corresponding to the area to form the "preliminary keyboard area instruction" containing only the orientation related information, laying the foundation for subsequent combination with distance parameter to supplement brightness information. It needs to be clear that the judgment logic of orientation angle matching, the confirmation method of area identifier, and the content of preliminary instruction, each link needs to be matched and demonstrated with specific orientation angle examples and mapping table to ensure the accuracy of the instruction.

[0076] I. Matching judgment logic of orientation angle and sector area Matching premise: the orientation angle needs to be within the effective range of -90°~90° (otherwise, no instruction will be generated), if the orientation angle is -95° (exceeding the left boundary), it is judged as invalid orientation, no preliminary instruction is output; if the orientation angle is 85° (within the effective range), the matching process is entered.

[0077] Matching steps: Step 1: Extract the step 4 output footstep azimuth angle (example: azimuth angle θ = -35°); Step 2: Traverse the "azimuth angle interval" in the keyboard area mapping table to determine whether θ belongs to a certain interval; Step 3: If θ belongs to a certain interval, determine the corresponding "region identifier"; if θ belongs to two intervals at the same time (such as the boundary value -22.5°, which belongs to region 3 and region 4 at the same time), take the smaller region identifier (region 3) to ensure that the boundary orientation is preferentially associated with the region closer to the side (to avoid the front region covering too large).

[0078] Matching example: Example 1: Azimuth angle θ = -35°, traverse the interval of the mapping table: Region 1 (-90° ~ -67.5°): -35° is not in this interval; Region 2 (-67.5° ~ -45°): -35° is not in this interval; Region 3 (-45° ~ -22.5°): -35° is in the range of -45° ~ -22.5°, so the region identifier is determined to be 3; Example 2: Azimuth angle θ = 30°, traverse the interval: Region 4 (-22.5° ~ 22.5°): 30° is not in this interval; Region 5 (22.5° ~ 45°): 30° is in the range of 22.5° ~ 45°, so the region identifier is determined to be 5; Example 3: Azimuth angle θ = -22.5° (boundary value), belongs to region 3 (-45° ~ -22.5°) and region 4 (-22.5° ~ 22.5°) at the same time, take the smaller region identifier 3.

[0079] II. Content composition of the preliminary keyboard area instruction The preliminary instruction only contains azimuth-related information and does not involve brightness (supplemented in subsequent steps), and the content includes "instruction ID, region identifier, azimuth angle, key combination list, light control address", and the meaning and example of each field are as follows: Instruction ID: unique identifier, format "AREA-YYYYMMDD-HHMMSS-XXX" (XXX is the serial number), such as "AREA-20250921-164530-001", used for instruction tracing; Region identifier: the matching fan-shaped region number, such as "3"; Azimuth angle: the original footstep azimuth angle, with 1 decimal place, such as "-35.0°"; Key combination list: the keys bound in this region, such as "W, A, Q, Z"; Light control address: The hardware address corresponding to the area key position, such as "0x0B, 0x0A, 0x08, 0x12".

[0080] Example preliminary keyboard area instruction: "Instruction ID: AREA-20250921-164530-001; Area ID: 3; Azimuth: -35.0°; Key position combination list: [W, A, Q, Z]; Light control address: [0x0B, 0x0A, 0x08, 0x12]".

[0081] This instruction only reflects the association of "azimuth-keyboard area", and subsequent distance parameters need to be supplemented with brightness information to form a complete control instruction.

[0082] Combine the brightness level mapping rule to convert the distance parameter into a specific brightness value, and finally output the keyboard area control instruction containing the area ID and brightness level.

[0083] This step is the final link of "distance parameter and preliminary keyboard area instruction fusion", the core is to convert the "footstep relative distance" (such as 3.5m) output by step four into "brightness level" and "specific brightness value" through the brightness level mapping rule, and then supplement these brightness information to the preliminary keyboard area instruction to form a "keyboard area control instruction" containing "azimuth (area ID, key position) + distance (brightness level, brightness value)" complete information, which provides a direct basis for subsequent generation of dynamic visual prompt signal. It needs to be clear that the conversion process from distance to brightness, the fusion method of brightness information and preliminary instruction, and the content of the final control instruction, each link needs to be quantitatively calculated combined with specific distance examples and mapping rules to ensure that the instruction information is complete and can be directly used for light control.

[0084] I. Conversion process of distance parameter to brightness information Distance validity judgment: First, confirm whether the footstep relative distance is within the effective range of 0~10m, if the distance is 12m (out of range), the brightness level is set to 0 and the brightness value is 0; if the distance is 3.5m (within the effective range), the conversion process is entered.

[0085] Brightness level determination: According to the brightness level mapping rule, find the interval to which the distance belongs, and determine the corresponding brightness level: Example: Distance r=3.5m, traverse the mapping rule interval: level 0 (>10m): 3.5m is not in this interval; level 1 (8~10m): 3.5m is not in this interval; level 2 (5~8m): 3.5m is not in this interval; level 3 (2~5m): 3.5m is within the range of 2~5m, so the brightness level is 3; Brightness value determination: According to the brightness value corresponding to the brightness level, level 3 corresponds to brightness value 178 (RGB red channel value 178), and the corresponding RGB example value (178, 0, 0) is recorded at the same time for subsequent color configuration reference.

[0086] Special case handling: If the distance is at the interval boundary (such as 2m), according to the "take high level" principle, 2m corresponds to level 3 (2~5m) instead of level 4 (0~2m); the boundary value 2m belongs to level 4 (0~2m), because 2m and below are in the near distance danger range, and high brightness warning is required, so the "boundary value attribution principle: interval left closed right open, such as 0≤r≤2m is level 4, 2

[0087] II. Fusion method of brightness information and preliminary instructions Fusion adopts the "supplementary field" method, adds four fields of "distance, brightness level, brightness value, RGB example" on the basis of the preliminary keyboard area instruction, without modifying the original orientation related fields, to ensure information integrity and compatibility: New field meaning: Distance: original footstep sound relative distance, with 1 decimal place, such as "3.5m"; Brightness level: converted brightness level (0~4), such as "3"; Brightness value: corresponding RGB brightness value (0~255), such as "178"; RGB example: RGB color example corresponding to this brightness value (take red as an example), such as "(178, 0, 0)".

[0088] III. Content composition of final keyboard area control instruction The final instruction adopts structured literal format, containing complete information of orientation and distance, which can be directly used as input for subsequent dynamic visual prompt signal generation, for example: "Instruction ID: CTRL-20250921-164530-001; Area identification: 3; azimuth angle: -35.0°; key combination list: [W, A, Q, Z]; light control address: [0x0B, 0x0A, 0x08, 0x12]; distance: 3.5m; brightness level: 3; brightness value: 178; RGB example: (178, 0, 0); Instruction Validity Period: 500ms” (The instruction validity period is set to 500ms, as the footstep sound is a continuous signal, the instruction needs to be updated periodically to avoid the light remaining on).

[0089] Another boundary value example: Distance r = 2.0m, Azimuth θ = 30° (Region 5), Final Instruction: “Instruction ID: CTRL-20250921-164600-002; Region ID: 5; Azimuth: 30.0°; Key Combination List: [W, D, E, V]; Light Control Address: [0x0B, 0x0C, 0x0D, 0x15]; Distance: 2.0m; Brightness Level: 4; Brightness Value: 255; RGB Example: (255, 0, 0); Instruction Validity Period: 500ms” (The instruction validity period is set to 500ms, as the footstep sound is a continuous signal, the instruction needs to be updated periodically to avoid the light remaining on).

[0090] This final instruction contains the core control information of “which keyboard area (Region 5), what brightness (255), and which keys (W / D / E / V) to light”, providing all necessary parameters for generating dynamic visual cue signals (such as light flashing patterns, color changes) in the next step.

[0091] S203, generating a dynamic visual cue signal according to the keyboard area control instruction, wherein the dynamic visual cue signal contains a light color change pattern and a brightness gradient sequence of the specified keyboard area; Specifically, the region ID and brightness level in the keyboard area control instruction can be parsed, and the corresponding warning color can be selected from the pre-set color scheme library to obtain the basic color configuration. This step is the “color basic setting link of dynamic visual cue signal”, the core is to first extract the key control parameters — region ID (corresponding to footstep sound azimuth) and brightness level (corresponding to footstep sound distance) from the keyboard area control instruction, then based on the “azimuth distinction” and “danger warning” dimensions, match the unique warning color from the pre-set color scheme library, form the basic color configuration containing “target key, RGB color value, color meaning”, ensure that the player can quickly associate the azimuth and danger level (distance) of the footstep sound through the color. It needs to be clear that the specific fields and logic of instruction parsing, the design principles and content of the pre-set color scheme library, and the double-dimension rules of color matching, each link needs to be combined with specific instruction examples and color psychology principles to ensure that the color selection not only conforms to human visual perception habits, but also accurately transmits spatial and risk information.

[0092] I. Parsing logic of keyboard area control instruction Instruction Key Field Extraction: From the keyboard area control instruction, extract the two core fields of "region identifier" and "brightness level" while associating "key combination list" and "light control address", ensuring that the color configuration can be accurately mapped to specific keys. Take the instruction example "Instruction ID: CTRL-20250921-164530-001; Region Identifier: 3; Azimuth Angle: -35.0°; Key Combination List: [W, A, Q, Z]; Light Control Address: [0x0B, 0x0A, 0x08, 0x12]; Distance: 3.5m; Brightness Level: 3; Brightness Value: 178; RGB Example: (178, 0, 0); Instruction Validity Period: 500ms" as an example: Region Identifier: Extract the value "3", and according to the keyboard region mapping table, this identifier corresponds to the "left front (-45° ~ -22.5°)" direction, which needs to match the color system that represents the "left side"; Brightness Level: Extract the value "3", and according to the brightness level mapping rule, this level corresponds to "2~5m medium distance", which needs to match the color concentration corresponding to this danger level; Associated Information: Key combination list [W, A, Q, Z] and light control address [0x0B, 0x0A, 0x08, 0x12] clearly indicate the target keys of color configuration, and the color value needs to be written to these keys' light control address in the future.

[0093] Parse Validity Check: Perform legality check on the extracted fields to avoid color configuration errors caused by invalid parameters — region identifier needs to be within 1~7 range (excluding reserved region 8), brightness level needs to be within 0~4 range, if region identifier "8" or brightness level "5" appears, it is judged as invalid instruction, default output "no color configuration" (light off); After passing the check, enter the color matching link.

[0094] II. Design and content of the preset color scheme library The preset color scheme library is based on the two-dimensional design principle of "azimuth partition color system, brightness level concentration", considering "direction recognition" and "danger warning", the library contains "azimuth - color system mapping table" and "brightness - concentration mapping table", the specific design is as follows: Azimuth - Color System Mapping Table: According to the player's habit of operating with both hands and visual cognitive logic, different azimuth regions are divided into three major color systems to avoid azimuth confusion caused by the same color system: Left side area (Area 1: -90°~ -67.5°, Area 2: -67.5°~ -45°, Area 3: -45°~ -22.5°): Red color system (mainly RGB red channel) is used. Red color has the characteristics of "strong warning and left brain perception priority" in vision, which meets the player's demand for quick response to the left sound source. Front area (Area 4: -22.5°~22.5°): Yellow color system (mixed RGB red and green channels) is used. Yellow color has "eye-catching and neutral" characteristics, which highlights the core area in front and distinguishes it from the left and right color systems. Right side area (Area 5: 22.5°~45°, Area 6: 45°~67.5°, Area 7: 67.5°~90°): Blue color system (mainly RGB blue channel) is used. Blue color forms a strong visual contrast with red color. Players can quickly distinguish left and right directions through "red and blue", reducing cognitive load.

[0095] Brightness - density mapping table: According to the brightness level (corresponding to the distance of danger level), adjust the density of the color system (RGB channel value). The closer the distance (the higher the brightness level), the higher the color density (the larger the channel value), forming an intuitive correlation between "danger level - color density". The specific correspondence is as follows (taking the left red color system as an example): Brightness level 0 (distance > 10m, no danger): RGB value (0,0,0) (black, light off); Brightness level 1 (distance 8~10m, low danger): RGB value (51,0,0) (light red, channel value 51, about 20% density); Brightness level 2 (distance 5~8m, medium-low danger): RGB value (102,0,0) (red, channel value 102, about 40% density); Brightness level 3 (distance 2~5m, medium danger): RGB value (178,0,0) (dark red, channel value 178, about 70% density); Brightness level 4 (distance 0~2m, high danger): RGB value (255,0,0) (pure red, channel value 255, 100% density).

[0096] Similarly, front yellow color system brightness level 3 corresponds to RGB value (178,178,0), and right blue color system brightness level 3 corresponds to RGB value (0,0,178), ensuring that the color density of different directions at the same brightness level is consistent, and the danger level perception is unified.

[0097] III. Generation of basic color configuration The instruction analysis result is combined with the preset color scheme library to generate a basic color configuration for the target key position. The configuration content includes "target key position, light control address, RGB color value, color meaning". For example, based on the above instruction analysis result: Target key position: W, A, Q, Z; Light control address: 0x0B (W key), 0x0A (A key), 0x08 (Q key), 0x12 (Z key); RGB color value: (178, 0, 0) (left red, brightness level 3 corresponds to concentration); Color meaning: left front (-35.0°) direction, 3.5m medium distance (medium danger), player needs to be alert to the sound source in the left front.

[0098] The basic color configuration determines the "static reference color" of the light, laying the foundation for subsequent design of dynamic color change mode.

[0099] Based on the real-time variation characteristics of footstep sound, a pulse type light color change mode is designed, using a gradual transition algorithm to obtain a color change mode sequence; This step is the "dynamic effect design link of dynamic visual prompt signal". The core is to combine the "intermittent and periodic" real-time variation characteristics of footstep sound (such as 300~500ms interval for each step when walking, 200~300ms interval for running), design a "pulse type" color change mode (avoiding visual fatigue caused by constant light), and eliminate the flicker caused by color mutation through a gradual transition algorithm, finally generate a color change mode sequence containing "time node, color value, change duration", so that the light dynamic is accurately synchronized with the rhythm of footstep sound. It needs to be clear that the quantization analysis of real-time variation characteristics of footstep sound, the parameter design of pulse mode, and the specific implementation of gradual transition algorithm. Each link needs to combine the actual characteristics of footstep sound in the game and the visual perception principle to ensure that the dynamic effect is both in line with the rhythm of the sound source and in line with human visual comfort.

[0100] I. Quantitative analysis of real-time variation characteristics of footstep sound The real-time variation of footstep sound in the game mainly reflects in "time domain interval" and "intensity fluctuation" two dimensions, which need to be quantified through audio signal analysis to provide basis for pulse mode design: Time domain interval quantization: footstep sound is generated by the contact between the character's foot and the ground, and the interval depends on the moving speed. In the mainstream game (such as "CS:GO"), the footstep sound interval of common moving states is as follows: Walking state: the moving speed of the character is about 2.5m / s, the distance of each step is about 0.8m, and the footstep sound interval T=0.8 / 2.5=0.32s≈320ms; Running state: moving speed about 5m / s, each step distance about 1.2m, footstep sound interval T=1.2 / 5=0.24s≈240ms; Squatting state: moving speed about 1m / s, each step distance about 0.4m, footstep sound interval T=0.4 / 1=0.4s≈400ms; Take the common walking state as an example, the footstep sound interval is about 320ms, so the cycle of the pulse mode needs to be synchronized with the interval to avoid the dynamic light from being out of sync with the rhythm of the sound source.

[0101] Intensity fluctuation quantification: the intensity of a single footstep sound is in the form of a pulse of "rise-peak-fall", with a duration of about 100-150ms (about 120ms in the walking state), the intensity peak corresponds to the moment when the foot touches the ground, and the falling stage corresponds to the process of the foot lifting. This fluctuation feature determines that the pulse mode needs to include "fade in (intensity rise)-maintain (intensity peak)-fade out (intensity fall)" three stages to simulate the natural intensity change of footstep sound.

[0102] II. Parameter design of pulse color change mode Based on the quantification characteristics of footstep sound, the cycle and stage division of the pulse color change mode are as follows (taking the walking state as an example): Pulse cycle: set to 320ms (consistent with the footstep sound interval), to ensure that each pulse corresponds to a footstep sound, and the player can perceive the number of footstep sounds through the number of light pulses (such as 3 pulses corresponding to 3 steps, to judge the speed of the character approaching).

[0103] Stage division: divide the 320ms pulse cycle into three stages, and the duration of each stage is designed based on the intensity fluctuation of footstep sound: Fade-in stage: duration 80ms, corresponding to the process of footstep sound intensity rising from 0 to peak, color transition from 0% to 100% concentration of base color; Maintain stage: duration 60ms, corresponding to the process of footstep sound intensity maintaining peak, color maintaining 100% concentration (base color value); Fade-out stage: duration 180ms, corresponding to the process of footstep sound intensity falling from peak to 0, color transition from 100% to 0% concentration; The total duration of the stages is 80+60+180=320ms, which matches the pulse cycle perfectly, and the duration of the fade-out stage is longer than that of the fade-in stage, simulating the long tail characteristics of footstep sound decay, which is more natural visually.

[0104] Multi-state adaptation: If the footstep sound state changes (such as from walking to running), dynamically adjust the pulse parameters - in running state, the pulse period is set to 240ms, gradually in 40ms, maintain 40ms, and gradually out 160ms; in squatting state, the period is 400ms, gradually in 100ms, maintain 80ms, and gradually out 220ms, to ensure that the mode always fits the real-time footstep sound characteristics.

[0105] III. Implementation of Gradual In and Out Smooth Transition Algorithm The "cubic Bezier curve interpolation algorithm" is used to realize the smooth transition of color density. Compared with linear interpolation, this algorithm can avoid the harshness of stage switching and better meet the natural perception of human eyes to light changes. The specific implementation is as follows: Algorithm principle: The color density transition formula of cubic Bezier curve is defined by four control points (P0, P1, P2, P3), where P0 is the starting point (time 0, density 0%), P3 is the end point (stage length, density 100% or 0%), P1 and P2 are control points for adjusting the shape of the curve, and the transition speed is controlled by adjusting the control point coordinates.

[0106] Gradual in stage algorithm (80ms, 0%→100% density): Control point setting: P0 (0, 0), P1 (0.2, 0), P2 (0.5, 0.8), P3 (1, 1), where the x-axis is the time normalized value (0→1 corresponds to 0→80ms), and the y-axis is the density normalized value (0→1 corresponds to 0%→100%); Interpolation calculation: For each time step (set to 10ms, 8 time steps in total), calculate the density value by the Bezier formula, the formula is y (t) = (1-t) 3 y0 + 3 (1-t) 2 ty1 + 3 (1-t) t 2 y2 + t 3 y3 (t is the time normalized value, 0≤t≤1); Example calculation: t=0.25 (corresponding to 20ms), y (0.25)=(0.75) 3 ×0 + 3×(0.75) 2 ×0.25×0+ 3×0.75×(0.25) 2 ×0.8 + (0.25) 3x1 + 3x0.75x0.0625x0.8 + 0.0156 = 0.1125, corresponding concentration 11.25%, RGB value (178x0.1125, 0, 0) = (20, 0, 0); t = 1 (80ms), y(1) = 1, concentration 100%, RGB value (178, 0, 0).

[0107] Outgoing phase algorithm (180ms, 100%→0% concentration): Control point settings: P0 (0, 1), P1 (0.3, 0.9), P2 (0.7, 0.2), P3 (1, 0), ensuring slow early concentration drop and accelerated drop later, fitting the footstep sound attenuation characteristics; Example calculation: t = 0.33 (corresponding to 60ms), y(0.33) = (0.67) 3 x1 + 3x(0.67) 2 x0.33x0.9 + 3x0.67x(0.33) 2 x0.2 + (0.33) 3 x0 = 0.301 + 3x0.449x0.33x0.9 + 3x0.67x0.109x0.2 + 0 = 0.301 + 0.404 + 0.044 = 0.749, concentration 74.9%, RGB value (178x0.749, 0, 0) = (133, 0, 0); t = 1 (180ms), y(1) = 0, concentration 0%, RGB value (0, 0, 0).

[0108] Four, generation of color change mode sequence Expand the three phases of each pulse cycle by time step, generate a color change mode sequence containing "time stamp, target key position, RGB color value", for example (walking state, pulse cycle 320ms, base color (178, 0, 0)): Time stamp 0ms (start of fade-in): W / A / Q / Z key, RGB (0, 0, 0); Time stamp 10ms: W / A / Q / Z key, RGB (20, 0, 0); Time stamp 20ms: W / A / Q / Z key, RGB (45, 0, 0); ... (intermediate time steps omitted)... Time stamp 80ms (end of fade-in, keep starting): W / A / Q / Z key, RGB (178, 0, 0); Timestamp 100ms (hold phase): W / A / Q / Z keys, RGB (178,0,0); Timestamp 140ms (end of hold, start of fade-out): W / A / Q / Z keys, RGB (178,0,0); Timestamp 200ms (mid-fade-out): W / A / Q / Z keys, RGB (133,0,0); ... (intermediate time steps omitted)... Timestamp 320ms (end of fade-out, start of next pulse): W / A / Q / Z keys, RGB (0,0,0); This sequence fully records the color state of the light at each time point, ensuring that the dynamic effect is precisely synchronized with the rhythm of the footsteps.

[0109] According to the brightness level value, an exponential decay model is used to calculate the brightness fade curve, obtaining a smooth brightness fade sequence; This step is the "brightness dynamic optimization link of dynamic visual cue signal", the core is to simulate the physical characteristics of "the natural decay of footsteps intensity with distance" through the exponential decay model for different brightness levels (corresponding to different distances), calculate the brightness value of each time step, generate a brightness fade sequence, so that the brightness dynamic not only reflects the "distance danger level", but also simulates the natural decay law of sound source, enhances the player's perception of distance change (such as the brightness decay slows down when the footsteps approach, and speeds up when they move away). It needs to be clear that the principle and parameter design of the exponential decay model, the calculation process of the brightness value, and the generation logic of the brightness fade sequence. Each link needs to be combined with the brightness level example and the physical decay law to ensure that the brightness change not only conforms to the scientific principle, but also can transmit accurate distance information.

[0110] I. Principle and parameter design of exponential decay model Model selection basis: the propagation intensity of sound in air follows the "spherical wave exponential decay law" (intensity is inversely proportional to the square of distance, logarithmic form is exponential decay), so the exponential decay model is used to simulate the change of brightness with time, the formula is: L(t) = L0× e^(-k×t) + L_base. The meanings of each parameter are as follows: L(t): brightness value at time t (0~255), corresponding to the value of RGB channel; L0: initial brightness value (determined according to brightness level, such as brightness level 3 corresponds to L0=178); k: decay coefficient (controls the brightness decay speed, the larger the k value, the faster the decay, unit: ms -1); t: time (counting from the end of the pulse holding phase, unit: ms); L_base: base brightness value (avoiding visual discontinuity caused by brightness decaying to 0 completely, determined according to brightness level, for example, brightness level 3 corresponds to L_base=80).

[0111] Association of parameters with brightness level: the decay coefficient k and the base brightness value L_base need to be dynamically adjusted according to the brightness level (corresponding to the distance). The closer the distance (the higher the brightness level), the smaller the k value (the slower the decay, reflecting the persistence of the sound source at close range), and the larger the L_base (the higher the base brightness, reflecting the high-risk warning). The specific association is as follows: Brightness level 1 (distance 8~10m, L0=51): k=0.008ms -1 (fast decay, far distance sound source easy to disappear), L_base=10 (low base brightness); Brightness level 2 (distance 5~8m, L0=102): k=0.005ms -1 (slower decay), L_base=40 (medium-low base brightness); Brightness level 3 (distance 2~5m, L0=178): k=0.003ms -1 (slow decay), L_base=80 (medium base brightness); Brightness level 4 (distance 0~2m, L0=255): k=0.001ms -1 (extremely slow decay, close-range sound source persistence), L_base=150 (high base brightness); This association ensures that the brightness decay characteristics at different distances conform to the physical law, and players can assist in determining the direction of sound source movement through the brightness decay speed (e.g. decay slows down → sound source approaches, decay speeds up → sound source moves away).

[0112] II. Calculation process of brightness value (take brightness level 3 as an example) Take brightness level 3 (L0=178, k=0.003ms -1 , L_base=80), pulse fade-out phase (180ms) as an example, calculate the brightness value of each time step (10ms): t=0ms (fade-out start, hold phase end): L0 should be the luminance value of the hold phase (178), and L(t) needs to be < 255, so the formula is adjusted to L(t) = min(L0 x e^(-k x t) + L_base, 255), where t=0, L(0)=178 x 1 + 80=258>255, take L(0)=255 (in practice, the luminance of the hold phase is already L0=178, here the modified model is L(t) = L0 x e^(-k x t) + L_base, and L0=178, L_base=80, to ensure that L(t) < 255, 178 x e^0 + 80=258, L_base needs to be adjusted to 77, so that L(0)=178+77=255, subsequent calculations take L_base=77 as the standard); L(0)=178 x e^(0) + 77=178+77=255.

[0113] t=10ms: L(10)=178 x e^(-0.003 x 10) + 77=178 x e^(-0.03) + 77≈178 x 0.9704 + 77≈172.7 + 77≈249.7≈250 (rounded, RGB value (250,0,0)).

[0114] t=30ms: L(30)=178 x e^(-0.003 x 30) + 77=178 x e^(-0.09) + 77≈178 x 0.9139 + 77≈162.7 + 77≈239.7≈240 (RGB value (240,0,0)).

[0115] t=60ms: L(60)=178 x e^(-0.003 x 60) + 77=178 x e^(-0.18) + 77≈178 x 0.8353 + 77≈148.7 + 77≈225.7≈226 (RGB value (226,0,0)).

[0116] t=100ms: L(100)=178 x e^(-0.003 x 100) + 77=178 x e^(-0.3) + 77≈178 x 0.7408 + 77≈131.9 + 77≈208.9≈209 (RGB value (209,0,0)).

[0117] t=180ms (fade-out end): L(180) = 178 x e^(-0.003 x 180) + 77 = 178 x e^(-0.54) + 77 ≈ 178 x 0.5827 + 77 ≈ 103.7 + 77 ≈ 180.7 ≈ 181 (RGB value (181, 0, 0)); If the next footstep sound pulse starts (t = 320ms), the brightness quickly recovers from 181 to 255, forming a cycle of “decay-recovery”, simulating the intensity fluctuation of continuous footstep sound.

[0118] III. Generation of brightness fade sequence Associate the brightness value of each time step with the timestamp in the color change pattern sequence, update the corresponding RGB color value (only adjust the brightness channel, the color hue remains unchanged), and generate the brightness fade sequence. Take the brightness level 3 and walking state as an example, the sequence segment is as follows: Timestamp 140ms (60ms after fade-out starts): W / A / Q / Z keys, RGB (226, 0, 0) (brightness 226); Timestamp 160ms (80ms after fade-out starts): W / A / Q / Z keys, RGB (219, 0, 0) (L(80) = 178 x e^(-0.003 x 80) + 77 ≈ 178 x 0.7866 + 77 ≈ 140 + 77 = 217, corrected to 217, RGB (217, 0, 0)); Timestamp 180ms (100ms after fade-out starts): W / A / Q / Z keys, RGB (209, 0, 0); Timestamp 220ms (140ms after fade-out starts): W / A / Q / Z keys, RGB (195, 0, 0) (L(140) = 178 x e^(-0.003 x 140) + 77 ≈ 178 x 0.6570 + 77 ≈ 116.9 + 77 = 193.9 ≈ 194, RGB (194, 0, 0)); Timestamp 320ms (start of next pulse): W / A / Q / Z keys, RGB (255, 0, 0) (brightness recovers to 255); This sequence is superimposed on the color change pattern sequence, forming a composite dynamic effect of “fixed color hue, dynamic brightness decay”, which not only conveys direction information, but also reflects distance and sound source rhythm.

[0119] Integrate the basic color configuration, color change pattern sequence, and brightness fade sequence, encode into the standard RGB light control protocol, and finally output the dynamic visual prompt signal.

[0120] This step is the "standardized output link of dynamic visual prompt signal", the core is to integrate the three of the basic color configuration (target key, RGB reference color) generated in the early stage, color change mode sequence (time-color dynamic), brightness gradient sequence (time-brightness dynamic) into one, encode according to the standard RGB keyboard light control protocol (such as USB HID manufacturer's custom protocol), generate binary control signal containing "device address, key control information, dynamic parameters", ensure that RGB keyboard can accurately parse and execute light effect. It needs to be clear that the integration logic, the structure and parameters of the standard protocol, the encoding process and examples, each link needs to combine the communication specification of the keyboard hardware, to ensure signal compatibility and executability.

[0121] I. Integration logic of multi-dimensional information The integration follows the principle of "key as the core, time as the axis", and organizes the basic color, color mode, and brightness gradient information in the way of "each key corresponds to a control record, and each record contains dynamic parameters of the whole time axis". The specific integration steps are as follows: Key grouping: Group the target keys (such as W, A, Q, Z) in the basic color configuration according to the light control address, and each key corresponds to a unique control address (such as W key 0x0B, A key 0x0A), to ensure that the control signal can be accurately positioned to a single key.

[0122] Time axis alignment: Take the timestamp of the color change mode sequence as the benchmark (such as 0ms, 10ms, 20ms…320ms), update the brightness value of the corresponding timestamp in the brightness gradient sequence to the RGB value of the color mode sequence, form a three-dimensional association table of "timestamp-key address-RGB value", for example, timestamp 140ms, W key 0x0B, RGB (226,0,0).

[0123] Dynamic parameter extraction: Extract the dynamic parameters of each key from the integrated association table, including pulse period (320ms), gradual entry time (80ms), holding time (60ms), gradual exit time (180ms), initial RGB value (255,0,0), and terminal RGB value (0,0,0). These parameters will be used as fixed fields in protocol encoding, reducing data transmission volume.

[0124] II. Selection and structure of standard RGB light control protocol The industry-standard "USB HID Custom Protocol" (HID Usage Page 0xFF00, Custom Usage Page) is adopted, which supports the transmission of keyboard light control instructions through the USB interface without additional drivers, covering more than 95% of RGB keyboards (such as Cherry, Corsair, Razer, etc.). The core of the protocol is the "Control Report", which has a fixed structure of 16 bytes, and each byte has the following meaning and value rules: Byte 0 (Report ID): Report identification, fixed as 0x01 (light control report); Bytes 1-2 (Key Count): Number of target key positions, such as 4 key positions corresponding to 0x0004; Bytes 3-4 (Key Address): Current key light control address, such as W key 0x000B; Bytes 5-7 (RGB Value): Current key RGB color value, stored in little-endian order, such as RGB (255,0,0) corresponding to 0x0000FF (byte 5=0x00, byte 6=0x00, byte 7=0xFF); Byte 8 (Brightness): Brightness value (0~255), consistent with the brightness channel in the RGB value, such as 255 corresponding to 100% brightness; Bytes 9-10 (Pulse Period): Pulse period, unit ms, stored in little-endian order, such as 320ms corresponding to 0x0140 (byte 9=0x40, byte 10=0x01); Bytes 11-12 (Fade-In Time): Fade-in time, unit ms, such as 80ms corresponding to 0x0050; Bytes 13-14 (Hold Time): Hold time, unit ms, such as 60ms corresponding to 0x003C; Byte 15 (Checksum): Checksum, calculated as the sum of bytes 0~14, taking the lower 8 bits to ensure data transmission integrity.

[0125] III. Protocol encoding process and examples Take brightness level 3, walking state, and W key (address 0x0B) as an example, encode a 16-byte control report, with the following steps: Byte 0 (Report ID): 0x01; Bytes 1-2 (Key Count): 0x0004 (4 keys, the W key is encoded here first, and the subsequent keys are encoded similarly). Bytes 3-4 (Key Address): 0x000B (W key address); Bytes 5-7 (RGB Value): 0x0000FF (Initial RGB (255,0,0), little-endian); Byte 8 (Brightness): 0xFF (255, corresponding to initial brightness); Bytes 9-10 (Pulse Period): 0x0140 (320ms, little-endian 0x4001 → Byte 9 = 0x40, Byte 10 = 0x01); Bytes 11-12 (Fade-In Time): 0x0050 (80ms, byte 11=0x50, byte 12=0x00); Bytes 13-14 (Hold Time): 0x003C (60ms, byte 13=0x3C, byte 14=0x00); Byte 15 (Checksum): Calculate the sum of bytes 0 through 14: 0x01+0x00+0x04+0x00+0x0B+0x00+0x00+0xFF+0xFF+0x40+0x01+0x50+0x00+0x3C+0x00=0x01+0x04+0x0B+0xFF+0xFF +0x40+0x01+0x50+0x3C=0x0F+0x1FE+0x41+0x50+0x3C=0x0F+0x1FE=0x20D+0x41=0x24E+0x50=0x29E+0x3C=0x2D4, whichever is lower is 8 Bit 0xD4; The final control report for the W key is as follows: 0x01,0x00,0x04,0x00,0x0B,0x00,0x00,0xFF,0xFF,0x40,0x01,0x50,0x00,0x3C,0x00,0xD4.

[0126] Multi-key batch encoding: Repeat the above encoding process for the A key (0x0A), Q key (0x08), and Z key (0x12), only modifying the key address of bytes 3-4. Other fields (such as RGB value and pulse period) are the same as the W key, forming 4 control reports of 16 bytes each, with a total length of 64 bytes.

[0127] IV. Output of Dynamic Visual Cue Signals The batch-encoding control report is packaged into a binary data stream according to the "USB HID Report Transmission Specification" and output to the light control system of the RGB keyboard through the USB interface. The signal output period is synchronized with the pulse period (320 ms / time), ensuring that the keyboard can update the light state in real time; if the footstep sound state changes (such as from walking to running), the dynamic parameters are recalculated and a new control report is encoded to realize real-time adaptive adjustment of the light effect. The final output dynamic visual prompt signal can drive the W, A, Q, and Z keys to display the effect of "320 ms pulse period, gradual entry for 80 ms, hold for 60 ms, gradual exit for 180 ms, and brightness index attenuation", and the player can quickly identify the footstep sound 3.5 m in front of the left through the red pulse light, realizing the visualization of the direction and distance of the footstep sound.

[0128] S204, output the dynamic visual prompt signal to the RGB keyboard light control system, drive the corresponding keyboard area light to display according to the dynamic visual prompt signal, to realize the visualization of the direction and distance of the footstep sound in the game.

[0129] Specifically, a communication connection with the RGB keyboard can be established through the USB-HID protocol to verify the compatibility of the keyboard light control function, and the communication connection is ready when the communication connection is established; This step is the basic communication link for the dynamic visual prompt signal to land, and the core is to establish a bidirectional communication link between the computer and the RGB keyboard relying on the universality and low delay characteristics of the USB-HID (Human Interface Device) protocol. At the same time, by analyzing the HID descriptor and function report of the keyboard, it is verified whether it supports custom light control (such as key-level RGB adjustment and dynamic pulse effect), to ensure that the subsequent instructions can be correctly recognized and executed by the keyboard, and finally output the "communication ready" or "compatibility not supported" state identifier. It needs to be clear that the communication principle of USB-HID protocol, the process of device enumeration and descriptor analysis, and the core indicators of compatibility verification, each link needs to be combined with specific hardware examples (such as Corsair K95 RGB keyboard) and protocol parameters to ensure stable communication link and function matching.

[0130] I. Communication principle and parameter configuration of USB-HID protocol Core features of USB-HID protocol: USB-HID is a sub-protocol of USB designed specifically for human-computer interaction devices such as keyboards and mice. Its advantages include "driverless deployment" (HID class driver included in the operating system), "low communication delay" (full-speed USB communication delay ≤1ms), and "flexible reporting mechanism" (supporting custom data format control reports), which fully meet the lighting control requirements of RGB keyboards (real-time and accurate key-level control). RGB keyboards, as composite HID devices, have both "keyboard input" and "light control" functions. The light control is achieved through "vendor-defined HID reports" (non-standard keyboard reports).

[0131] Communication parameter configuration: Transmission type: "Interrupt Transfer", designed for low-latency and small-data real-time communication, meeting the real-time requirements of light control (response within 10ms); Endpoint address: The interrupt output endpoint address for light control is usually 0x02 (vendor-defined, different brands may be different, such as Corsair K95 RGB light endpoint 0x03), and the maximum packet length of the endpoint is set to 64 bytes (USB full-speed device interrupt endpoint maximum packet length limit); Communication timeout: 50ms, if no response is received within 50ms, it is determined as a communication timeout, and automatically retries 3 times (retry interval 10ms) to avoid single communication failure causing function interruption; Report format: "Vendor-defined HID control report", report ID set to 0x01 (exclusive light control report identifier), report length determined by the number of keys supported by the keyboard, such as 104-key keyboard light report length of 64 bytes (including key address, RGB parameters, control instructions).

[0132] II. Device enumeration and descriptor analysis of RGB keyboard Device enumeration process: When the computer starts or the keyboard is inserted, the enumeration process is triggered by the USB host controller to obtain the core information of the keyboard and ensure the correct communication object: Step 1: Get the device descriptor (Device Descriptor), extract the vendor ID (VID), product ID (PID), and device type. In the example of Corsair K95 RGB, VID=0x1B1C (Corsair manufacturer identifier), PID=0x1B32 (K95 RGB product identifier), and device type is 0x00 (composite device); Step 2: Get the Configuration Descriptor, determine the number of configurations and interfaces of the device, K95 RGB contains 2 interfaces — interface 0 is a standard keyboard interface (HID Usage Page 0x01, Generic Desktop), interface 1 is a light control interface (HID Usage Page 0xFF00, custom by the manufacturer); Step 3: Get the HID Report Descriptor, parse the report format supported by the light control interface, for example, the light report with Report ID=0x01 contains "key address (2 bytes), RGB value (3 bytes), pulse period (2 bytes)" fields, the field offset and length are defined by "Report Size" and "ReportCount" in the descriptor, in the example, the key address field offset is 3, length 2 bytes, the RGB value field offset is 5, length 3 bytes.

[0133] Key verification of descriptor parsing: focus on verifying whether the HID Usage Page of the light control interface is 0xFF00 (custom by the manufacturer, supporting extended functions), if it is 0x01 of the standard keyboard, it is determined that it does not support custom light control; at the same time, verify whether the report descriptor contains "RGB control" related fields (such as Usage Minimum=0x00, Usage Maximum=0x67, corresponding to the address range of 104 key positions), in the example, the report descriptor of K95 RGB contains this field, which is determined to support key position level RGB control.

[0134] Three, compatibility verification of light control function Function detection process: send "function detection report" (Report ID=0x00) to the keyboard to get the list of supported light functions, the format of the detection report is "0x00 (report ID) + 0x01 (detection command) + 0x00 (reserved byte) x62", the keyboard returns "function response report" after receiving, which contains the supported function identifier (such as 0x01 = static RGB, 0x02 = dynamic pulse, 0x03 = brightness adjustment); Example: the response report returned by K95 RGB contains 0x01, 0x02, 0x03, indicating that it supports static color, dynamic pulse and brightness adjustment, fully meeting the "pulse dynamic light" requirement of footstep sound prompt; If a certain base RGB keyboard only returns 0x01 (only supports static RGB), it is determined that the compatibility is insufficient, and the pulsing dynamic prompt cannot be implemented. The user needs to be prompted to replace the keyboard or downgrade to static prompt mode.

[0135] Communication link stability test: Send 100 "test reports" (Report ID = 0x01, key address = 0x0B (W key), RGB value = 0xFF0000 (red), pulse period = 0x0140 (320ms)), and count the success rate of communication (number of successful responses / total number of times). The success rate should be ≥ 98%. If the success rate is less than 95%, check the USB cable connection or replace the USB port (to rule out hardware contact problems). In the example, 100 tests were successful 99 times, with a success rate of 99%, indicating that the communication link is stable.

[0136] Communication connection ready state output: If device enumeration is successful, descriptor parsing meets the requirements, function detection supports dynamic lighting, and communication stability meets the standards, output "communication connection ready" state, and record the core parameters of the keyboard (VID = 0x1B1C, PID = 0x1B32, light endpoint = 0x03, supported functions = 0x01 / 0x02 / 0x03) for subsequent instruction adaptation. If any step fails, output "compatibility not supported" state and prompt the specific failure reason (such as "dynamic pulse function not supported").

[0137] Parse dynamic visual prompt signals and convert them into a set of light control instructions adapted to the keyboard. This step is the "conversion link from general signals to device-specific instructions". The core is to first parse the "standardized dynamic visual prompt signal" generated in step four (including key address, RGB parameters, and dynamic effect parameters), and then perform "customized adaptation" according to the keyboard hardware parameters obtained in step one (such as manufacturer's instruction format, key address mapping, brightness range). Different brands of RGB keyboards have different light control instruction formats (such as Corsair's "iCUE protocol" and Razer's "Chroma protocol"). The general signal needs to be converted into the manufacturer's exclusive instruction format to ensure that the instruction can be correctly parsed by the keyboard's light control module. Finally, a set of "device executable" adaptive control instructions is formed. The parsing logic of the dynamic visual prompt signal, the instruction differences of different manufacturer keyboards, and the specific rules of adaptive conversion need to be clearly defined. Each link needs to be combined with the examples of general signals and manufacturer's instructions to ensure the accuracy of the adapted instructions.

[0138] I. Parsing logic of dynamic visual prompt signal General signal format review: the dynamic visual cue signal generated in Step 4 is in the "standardized RGB light control protocol" format, using a 64-byte HID report, with core fields including (sorted by byte offset): Byte 0: Report ID (0x01, general light control identifier); Bytes 1-2: number of keys (e.g., 0x0004, corresponding to 4 keys); Bytes 3-4: key address (e.g., 0x000B, corresponding to the general address of the W key); Bytes 5-7: RGB color value (little-endian, e.g., 0x0000FF for red); Byte 8: brightness value (0x00~0xFF, e.g., 0xFF for 100% brightness); Bytes 9-10: pulse period (little-endian, e.g., 0x0140 for 320ms); Bytes 11-12: fade-in duration (e.g., 0x0050 for 80ms); Bytes 13-14: hold duration (e.g., 0x003C for 60ms); Bytes 15-63: reserved bytes (filled with 0x00).

[0139] Field analysis and validity check: Step 1: Extract Report ID, if it is 0x01, determine it as a valid light control signal; if it is other values (e.g., 0x02 for global light signal), do not process it temporarily (focus on key-level cues); Step 2: Analyze the number of keys (Bytes 1-2), if it is 0x0004, it means that 4 keys need to be controlled, and the address and parameters of each key need to be analyzed in turn (the 1st key occupies bytes 3-14, the 2nd key occupies bytes 15-26, and so on); Step 3: Extract the key address (Bytes 3-4) and verify whether it is within the range of key addresses supported by the keyboard (e.g., the key address range of K95 RGB is 0x00~0x67), if it is 0x000B (general W key address), it is determined to be valid; if it is 0x70 (out of range), mark it as an invalid key and skip the adaptation of this key; Step 4: Analyze the RGB value (Bytes 5-7) and brightness value (Byte 8), verify whether the RGB value is within the range of 0x000000~0xFFFFFF, and whether the brightness value is within the range of 0x00~0xFF, in the example RGB=0x0000FF, brightness=0xFF, both are valid; Step 5: Extract dynamic parameters (pulse period, fade-in duration, hold duration), verify if the period is within the range of 100ms~1000ms (to avoid flickering or delay), in the example 320ms is within the valid range, and determine it as valid.

[0140] Parse result output: Organize the valid fields into a "general instruction structure", for example: "Number of key positions: 4; Key position 1: Address 0x000B (W key), RGB=0x0000FF (red), brightness = 0xFF (100%), pulse period = 320ms, fade-in = 80ms, hold = 60ms; Key position 2: Address 0x000A (A key), RGB=0x0000FF, brightness = 0xFF, pulse period = 320ms, fade-in = 80ms, hold = 60ms; Key position 3: Address 0x0008 (Q key), RGB=0x0000FF, brightness = 0xFF, pulse period = 320ms, fade-in = 80ms, hold = 60ms; Key position 4: Address 0x0012 (Z key), RGB=0x0000FF, brightness = 0xFF, pulse period = 320ms, fade-in = 80ms, hold = 60ms".

[0141] II. Analysis of instruction differences of different brands of RGB keyboards Core differences: Instruction format: Corsair iCUE protocol light instruction needs to include "device ID (1 byte) + instruction type (1 byte) + key position ID (2 bytes) + RGB parameter (3 bytes) + checksum (1 byte)", while Razer Chroma protocol needs to include "report ID (1 byte) + Chroma command (2 bytes) + key position mapping (2 bytes) + RGB data (3 bytes)"; Key position address mapping: The general key position address 0x000B (W key) corresponds to key position ID=0x0010 in Corsair K95 RGB, and corresponds to key position mapping = 0x000C in Razer BlackWidow V3; Brightness range: Corsair's brightness value uses 10-level quantization (0x00~0x09), not the general 0x00~0xFF, brightness mapping is needed; Dynamic parameter transfer: Razer's pulse period needs to be converted to "frequency value" (e.g., 320ms period corresponds to frequency = 3.125Hz), while Corsair directly supports period parameters (unit: ms).

[0142] Example brand differences: Corsair K95 RGB (VID=0x1B1C, PID=0x1B32): The light command length is 8 bytes, and the format is "0x02 (device ID) + 0x03 (dynamic light command) + 0x10 (W key ID) + 0xFF (R) + 0x00 (G) + 0x00 (B) + 0x09 (brightness 10 levels) + 0x23 (checksum, the sum of the first 7 bytes taken as the low 8 bits)"; Razer BlackWidow V3 (VID=0x1532, PID=0x0255): The light command length is 9 bytes, and the format is "0x03 (report ID) + 0x0201 (Chroma dynamic command) + 0x000C (W key mapping) + 0xFF (R) + 0x00 (G) + 0x00 (B) + 0x03 (frequency 3.125Hz) + 0x01 (enable)".

[0143] III. Generation process of adaptation control command Taking "universal command adaptation Corsair K95 RGB" as an example, the specific conversion steps are as follows: Key address mapping conversion: universal W key address 0x000B → Corsair key ID 0x0010 (refer to Corsair's published key mapping table), A key 0x000A → 0x0011, Q key 0x0008 → 0x000F, Z key 0x0012 → 0x0013; Brightness value mapping: universal brightness 0xFF (100%) → Corsair 10-level brightness (0x09), because the mapping relationship between Corsair's brightness level and universal brightness is "universal brightness = 25.5 × Corsair brightness level" (0x09 × 25.5 ≈ 229.5, close to 0xFF = 255, take the approximate value 0x09); Dynamic parameter adaptation: universal pulse period 320ms directly as Corsair's period parameter (Corsair supports ms unit), gradual entry 80ms → 0x0050 (hexadecimal), maintain 60ms → 0x003C; Command format assembly: assemble the command according to the Corsair iCUE protocol format, taking the W key as an example: Device ID: 0x02 (Device ID of K95 RGB); Instruction Type: 0x03 (Dynamic Pulse Light Instruction); Key ID: 0x0010; RGB Parameters: 0xFF (R), 0x00 (G), 0x00 (B); Brightness: 0x09; Checksum: 0x02+0x03+0x10+0xFF+0x00+0x00+0x09=0x02+0x03=0x05+0x10=0x15+0xFF=0x114+0x09=0x11D, take the low 8 bits 0x1D; The final Corsair adaptation instruction for the W key is "0x02 0x03 0x00 0x10 0xFF 0x00 0x000x09 0x1D" (9 bytes including checksum); Integration of Instruction Set: The adaptation instructions for 4 key positions are integrated into "Corsair K95 RGB Adaptation Control Instruction Set", each instruction occupying 9 bytes, with a total length of 36 bytes, meeting the 64-byte maximum packet length limit of USB-HID.

[0144] Similarly, when adapting Razer keyboard, the pulse period of 320ms needs to be converted to a frequency of 3.125Hz (1 / 0.32s≈3.125Hz), and the key position address is mapped to the key position ID of Razer, and finally the adaptation instruction set specific to Razer is generated.

[0145] Send the adaptation control instructions to the keyboard light control chip in order of priority to get the light control instruction queue; This step is the "optimization and scheduling of instruction execution order", the core is to sort the adaptation control instructions according to the "urgency" and "function priority" prompted by the footstep sound (to avoid conflicts caused by multiple instructions being executed at the same time, such as the conflict between the instructions of near distance high-risk footstep sound and far distance low-risk footstep sound), and then form an instruction queue according to the "first in first out (FIFO)" principle, and send it to the keyboard light control chip at a fixed period, to ensure that high-priority instructions (such as near distance footstep sound prompt) are executed first, and to improve the response speed of the player to dangerous signals. It needs to be clear that the basis for priority division, the implementation of sorting algorithm, and the management mechanism of instruction queue, each link needs to be combined with specific instruction examples and priority parameters to ensure that the sorting logic meets the safety perception needs of the player.

[0146] I. Basis for Priority Division of Adaptation Control Instructions The priority classification is based on two dimensions of "footstep danger level" and "instruction function type", and the "weighted scoring method" is used to determine the priority score of each instruction (the higher the score, the higher the priority), and the specific dimensions and weights are as follows: Footstep danger level (weight 0.6): determined by brightness level (corresponding distance), the closer the distance (the higher the brightness level), the higher the danger level, and the higher the score: Brightness level 4 (0~2m, high danger): score 5; Brightness level 3 (2~5m, medium danger): score 3; Brightness level 2 (5~8m, medium-low danger): score 2; Brightness level 1 (8~10m, low danger): score 1; Brightness level 0 (>10m, no danger): score 0 (no instruction generated).

[0147] Instruction function type (weight 0.4): according to the influence degree of instruction on player's operation, key level control (such as single key pulse) has higher priority than global control (such as global keyboard brightness adjustment): Key level dynamic pulse instruction (footstep sound prompt core instruction): score 4; Key level static color instruction (degraded prompt mode): score 3; Global light adjustment instruction (such as background brightness): score 1; Invalid instruction function (such as repeated instruction): score 0 (skip sorting).

[0148] Priority score calculation: priority score = (danger level score × 0.6) + (function type score × 0.4), result is an integer (rounded); Example 1: 3.5m footstep sound in front left (brightness level 3, danger score 3), key level dynamic pulse instruction (function score 4), priority score = (3 × 0.6) + (4 × 0.4) = 1.8 + 1.6 = 3.4 ≈ 3; Example 2: 1.5m footstep sound in back right (brightness level 4, danger score 5), key level dynamic pulse instruction (function score 4), priority score = (5 × 0.6) + (4 × 0.4) = 3.0 + 1.6 = 4.6 ≈ 5 (highest priority); Example 3: 7m footstep sound in front right (brightness level 2, danger score 2), key level dynamic pulse instruction (function score 4), priority score = (2 × 0.6) + (4 × 0.4) = 1.2 + 1.6 = 2.8 ≈ 3.

[0149] II. Implementation of sorting algorithm based on weighted score The "bubble sort algorithm" is used to sort the adaptive control instruction set (low algorithm complexity, suitable for fast sorting of a small number of instructions (such as 4-8 instructions)), and the specific steps are as follows: Initialize the instruction list: Store the adaptive control instruction set in the list according to the receiving order. The example list contains 3 instructions: Instruction A: Right rear 1.5m (score 5), key position D / F / V / Shift (right); Instruction B: Left front 3.5m (score 3), key position W / A / Q / Z (left front); Instruction C: Right front 7m (score 3), key position W / D / E / V (right front).

[0150] Bubble sort execution: First round of comparison: Compare instruction A (score 5) with instruction B (score 3), A score is higher, position unchanged; compare instruction B (3) with instruction C (3), same score, sort according to receiving order (B first, C last), after the first round, the list is [A, B, C]; Second round of comparison: Confirm adjacent instructions again, A>B, B=C, list unchanged, sorting completed; Special case: If instruction D (left rear 9m, brightness level 1, score 1) is added to the list, the sorted list is [A, B, C, D], ensuring that high-priority instructions are always at the front.

[0151] Conflict handling rules: If the priority scores of two instructions are the same (such as instructions B and C both being 3), and the key positions controlled overlap (such as both containing the W key), then sort according to the "subscore of the danger level dimension" - the sub-score of brightness level 3 is "the smaller the distance, the higher the score", instruction B (3.5m) is closer than instruction C (7m), so B takes precedence over C; if the distances are the same (such as both being 3.5m), then sort according to "direction priority" (directly in front > front side > side > back side), ensuring that sound sources closer to the player's front are prompted first.

[0152] III. Management and sending of light control instruction queue Queue structure design: Use "finite length FIFO (First In First Out) queue", the maximum length of the queue is set to 8 (the light control chip of the RGB keyboard can handle up to 8 key position instructions at a time), when the queue is full, discard the lowest priority instruction (such as the instruction with a score of 1), to ensure that the queue does not overflow; Instruction enqueue and dequeue: After the sorted instructions are enqueued, the example sorted instructions A, B, and C are enqueued, and the queue state is "queue head [A]→[B]→[C]→queue tail"; every 10ms (synchronized with the keyboard interrupt transmission period) takes out 1 instruction from the queue head and sends it to the light control chip, and after the sending is successful, the instruction is dequeued, and the new instruction (if any) is enqueued; Sending opportunity and retry mechanism: when sending instructions, wait for the keyboard's ACK response (acknowledgment of receipt), if no ACK is received within 5ms, retry sending 1 time (to avoid temporary communication interference); if the retry fails, re-enqueue the instruction (at the end of the queue) and wait for the next round of sending; in the example, after instruction A is sent, an ACK is received within 3ms, instruction A is dequeued, and the queue becomes "[B]→[C]"; Queue state monitoring: real-time monitoring of queue length, if length>6 (close to maximum length), output "queue congestion" prompt, reduce the generation frequency of new instructions (such as from 50ms to generate 1 time to 100ms), avoid delay caused by instruction backlog.

[0153] The final generated "light control instruction queue" example is: "queue head: instruction A (1.5m right rear, score 5)→instruction B (3.5m left front, score 3)→instruction C (7m right front, score 3)→queue tail", which will be sent to the keyboard light control chip in order, ensuring that the high-risk footstep sound prompt is displayed first.

[0154] Drive the keyboard light control chip to execute the light control instruction queue, so that the corresponding keyboard area light displays the dynamic visual prompt signal, and finally realizes the visualization of the position and distance of the footstep sound in the game.

[0155] This step is the "final execution link of the instruction landing as a light effect", the core is to pass the adaptive control instructions in the instruction queue to the chip through the underlying communication (such as SPI, I2C) between the computer and the keyboard light control chip, the chip analyzes the instructions and drives the LED drive module (such as WS2812 RGB LED) to control the light color, brightness, and dynamic effect of the target key, while feeding back the execution status to the computer, forming a "instruction - execution - feedback" closed loop, ensuring that the light effect and dynamic visual prompt signal are completely consistent, and finally allowing the player to accurately perceive the position and distance of the footstep sound in the game through the keyboard light. It needs to be clear that the working principle of the light control chip, the control method of the LED drive module, the execution status feedback mechanism, and the verification method of the light effect, each link needs to be combined with specific hardware modules and signal examples to ensure that the execution effect meets the design expectation.

[0156] I. Working principle and communication method of keyboard light control chip Core Chip Selection: The main RGB keyboard light control chip uses Microchip's PIC16F18855 (8-bit MCU, supports SPI / I2C communication, built-in RGB light control firmware). This chip has the characteristics of "low power consumption (sleep current <1μA), high integration (built-in ADC and PWM module), strong anti-interference", can directly parse HID instructions and control LED drive circuit; Underlying Communication Method: Computer and light control chip communicate through SPI (Serial Peripheral Interface) communication, SPI parameter configuration as follows: Communication Mode: Mode 0 (CPOL=0, CPHA=0), clock polarity and phase match LED drive chip timing requirements; Clock Frequency: 1MHz (balance communication speed and stability, 1 byte data transmission time is 8us under 1MHz clock, meet real-time control requirements); Data Bits: 8 bits (standard SPI data length); Chip Select Signal: Low active (CS pin is low, chip receives data).

[0157] Instruction Parsing Process: After the PIC16F18855 chip receives the SPI transmitted adaptive control instruction, the following steps are executed: Step 1: Check the integrity of the instruction (such as Corsair instruction checksum), if the verification fails, discard the instruction and return NACK (0x00); if the verification is successful, return ACK (0x01); Step 2: Analyze the instruction type (such as dynamic pulse instruction), call the built-in "dynamic light control function" of the chip; Step 3: Extract key ID and RGB parameters, query the "key - LED pin mapping table" in the chip (such as key ID 0x0010 corresponding to LED pin RC2); Step 4: Extract dynamic parameters (pulse period, fade-in / hold / fade-out duration), configure the chip's PWM module (such as pulse period 320ms corresponding to PWM period register value 0x140).

[0158] II. Control method of LED drive module (take WS2812 as an example) Working principle of WS2812 RGB LED: WS2812 is an intelligent pixel lamp integrating control circuit and RGB LED, which uses single-wire serial communication to transmit 24-bit RGB data (G8~G0→R8~R0→B8~B0) through pulse signals of different widths (T0H=0.4μs, T0L=0.85μs represents 0; T1H=0.8μs, T1L=0.45μs represents 1), and each LED can be independently controlled in color and brightness. Drive control process: The light control chip (PIC16F18855) outputs the required serial pulse signal of WS2812 through the GPIO pin (such as RC2): Step 1: The chip generates 24-bit data (G=0x00, R=0xFF, B=0x00) according to the RGB parameters (such as 0xFF0000); Step 2: Convert 24-bit data into pulse signals according to WS2812 timing (such as R=0xFF corresponds to 8 "1" pulses, each pulse is T1H+T1L=1.25μs); Step 3: Output the pulse signal to the DIN pin of WS2812, and the LED latches the data after receiving it and passes the remaining data to the next LED (if any) through the DOUT pin; Step 4: Control the PWM module according to the dynamic parameters to achieve the gradual entry effect (PWM duty cycle gradually increases from 0% to 100%, time consumption 80ms), the maintenance effect (duty cycle maintains 100%, time consumption 60ms), and the gradual exit effect (duty cycle decreases from 100% to 0%, time consumption 180ms), forming a 320ms pulse period.

[0159] Brightness and color calibration: The brightness of WS2812 has a linear relationship with the PWM duty cycle, and the chip adjusts the PWM duty cycle to control the brightness (such as duty cycle 100% corresponds to brightness level 4, 50% corresponds to level 2); At the same time, the built-in RGB color calibration algorithm of the chip can correct the color deviation of the LED (such as when the brightness of the red LED is too high, automatically reduce the PWM duty cycle of the R channel by 5%), to ensure that the color of the light is consistent with the RGB value in the command.

[0160] III. Execution state feedback and effect verification State feedback mechanism: After the light control chip executes a command, it sends an "execution status report" (1 byte) to the computer through the SPI interface, and the report value is defined as follows: 0x01: execution success (LED has displayed as instructed); 0x02: instruction parsing failure (e.g. invalid key ID); 0x03: LED driver failure (e.g. WS2812 unresponsive); 0x04: parameter out of range (e.g. pulse period > 1000ms); In the example, after the chip executes instruction A (1.5m to the right rear), it returns 0x01, and the computer determines that the execution is successful, and instruction A is removed from the queue.

[0161] Light effect verification: through "visual observation + software detection" to verify whether the effect meets the expectations: Visual observation: the key positions D, F, V, and right Shift corresponding to the footsteps sound to the right rear should display "red pulse effect" - 80ms fade to pure red (0xFF0000), maintain for 60ms, 180ms fade to off, period 320ms, brightness significantly higher than other key positions (level 4), matching the high-risk prompt 1.5m to the right rear. Software detection: read the actual state of the LED through the HID feedback report of the keyboard (e.g. Razer keyboard supports reading the current RGB value of each key position), in the example, the RGB value of key D is 0xFF0000, pulse period 320ms, consistent with the instruction parameters, determining that the effect is qualified.

[0162] Exception handling: if the feedback report is 0x03 (LED driver failure), the computer automatically sends a "reset instruction" (Report ID=0x0F), the chip restarts the LED driver module after receiving it, and re-executes the faulty instruction; if there are 3 consecutive feedback failures, output "hardware failure" prompt, suggest user check the keyboard hardware.

[0163] Four, final visual effect implementation Taking "there is an enemy footsteps sound 1.5m to the right rear in the game" as an example, the complete visual process is as follows: The computer sends adaptation instruction A to the keyboard light control chip through USB-HID protocol; The chip parses the instruction and drives the WS2812 LED of keys D, F, V, and right Shift; The LED displays pure red with the highest brightness (level 4) according to the pulse period of "80ms fade up -> 60ms maintain -> 180ms fade down"; The player observes the red pulse light of the right side keys, immediately determines that the footsteps sound comes from the right rear at close range, quickly adjusts the game character's perspective and positioning, achieving "auditory information -> visual prompt -> operation response" quick transformation.

[0164] The effect fully meets the design goal of the dynamic visual prompt signal, and the orientation and distance information of the footstep sound is accurately transmitted through the "azimuth partition (right rear key position), color warning (red), brightness grading (highest), dynamic pulse (synchronous footstep sound rhythm)", and the game experience and competitive advantage of the player are improved.

[0165] Another embodiment of the present application provides a keyboard prompt system for footstep sound in a game, referring to Figure 3 , the system can include: The capture module 301 is configured to capture the audio signal of the in-game environment in real time, and perform spectrum analysis and footstep sound frequency feature separation on the audio signal, and extract the orientation and distance parameters of the footstep sound. The mapping module 302 is configured to map the orientation and distance parameters into a keyboard area control instruction, wherein different orientations correspond to different areas of the keyboard, and different distances correspond to the brightness level of the keyboard light. The generation module 303 is configured to generate a dynamic visual prompt signal according to the keyboard area control instruction, wherein the dynamic visual prompt signal includes a light color change mode and a brightness gradient sequence of a specified keyboard area. The prompt module 304 is configured to output the dynamic visual prompt signal to an RGB keyboard light control system, and drive the corresponding keyboard area light to display according to the dynamic visual prompt signal, so as to realize the orientation and distance visual prompt of the footstep sound in the game.

[0166] The present application also provides a storage medium, wherein the storage medium stores a computer program, and the computer program is configured to execute the steps in any of the above method embodiments when running.

[0167] The present application also provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to execute the steps in any of the above method embodiments.

[0168] Specifically, the above electronic device can further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0169] The above embodiments according to the drawings illustrate the structure, features and effects of the present application, and the above description is only the preferred embodiment of the present application, but the present application is not limited to the drawings, any changes or modifications made in accordance with the concept of the present application, or equivalent embodiments with equivalent changes, shall be within the scope of the present application.

Claims

1. A method for providing a keyboard cue for in-game footsteps, the method comprising: The method comprises: Real-time capture of audio signals of the in-game environment, and spectral analysis and footstep sound frequency feature separation of the audio signals to extract the orientation and distance parameters of the footstep sound; Mapping the orientation and distance parameters to keyboard area control instructions, wherein different orientations correspond to different areas of the keyboard, and different distances correspond to the brightness levels of the keyboard light; Generating dynamic visual prompt signals according to the keyboard area control instructions, wherein the dynamic visual prompt signals include the light color change mode and the brightness gradient sequence of the specified keyboard area; Outputting the dynamic visual prompt signals to the RGB keyboard light control system to drive the corresponding keyboard area light to display according to the dynamic visual prompt signals, so as to realize the orientation and distance visualization prompt of the footstep sound in the game.

2. The method of claim 1, wherein, The real-time capture of audio signals of the in-game environment, and spectral analysis and footstep sound frequency feature separation of the audio signals to extract the orientation and distance parameters of the footstep sound, comprise: Real-time capture of audio signals of the in-game environment, and spectral analysis and footstep sound frequency feature separation of the audio signals to extract the orientation and distance parameters of the footstep sound, comprise: Real-time capture of audio signals of the in-game environment, and spectral analysis and footstep sound frequency feature separation of the audio signals to extract the orientation and distance parameters of the footstep sound, comprise: Real-time capture of audio signals of the in-game environment, and spectral analysis and footstep sound frequency feature separation of the audio signals to extract the orientation and distance parameters of the footstep sound, comprise: Real-time capture of audio signals of the in-game environment, and spectral analysis and footstep sound frequency feature separation of the audio signals to extract the orientation and distance parameters of the footstep sound, comprise:

3. The method of claim 2, wherein, The mapping of the orientation and distance parameters to keyboard area control instructions, wherein different orientations correspond to different areas of the keyboard, and different distances correspond to the brightness levels of the keyboard light, comprises: Carrying out keyboard area division, dividing the standard keyboard into a preset number of sector areas according to the orientation angle, each area corresponding to a specific key combination, to obtain a keyboard area mapping table; Using a distance-brightness mapping relationship to dynamically determine the brightness level according to the relative distance of the footstep sound, wherein the closer the distance, the higher the brightness, to obtain a brightness level mapping rule; Inputting the footstep sound orientation parameter into the keyboard area mapping table to determine the corresponding keyboard area identifier, to obtain a preliminary keyboard area instruction; Converting the distance parameter to a specific brightness value in combination with the brightness level mapping rule, and finally outputting the keyboard area control instruction containing the area identifier and the brightness level.

4. The method of claim 3, wherein, The generation of dynamic visual prompt signals according to the keyboard area control instruction, wherein the dynamic visual prompt signals include the light color change mode and the brightness gradient sequence of the specified keyboard area, comprises: Analyzing the area identifier and the brightness level in the keyboard area control instruction, and selecting the corresponding warning color from a preset color scheme library to obtain a basic color configuration; Designing a pulse-type light color change mode based on the real-time change characteristics of the footstep sound, and using a gradual transition algorithm to obtain a color change mode sequence; According to the brightness level value, an exponential decay model is used to calculate a brightness gradient curve to obtain a smooth brightness gradient sequence; Integrating the basic color configuration, the color change mode sequence and the brightness gradual change sequence, encoding into a standard RGB light control protocol, and finally outputting a dynamic visual prompt signal.

5. The method of claim 4, wherein, The dynamic visual prompt signal is output to an RGB keyboard light control system, and corresponding keyboard area lights are displayed according to the dynamic visual prompt signal, so as to realize the orientation and distance visualization of in-game footsteps. A communication connection with the RGB keyboard is established through a USB-HID protocol, the compatibility of the keyboard light control function is verified, and a communication connection ready state is obtained. The dynamic visual prompt signal is parsed and converted into a light control instruction set adapted to the keyboard, and an adapted control instruction is obtained. The adapted control instruction is sent to the keyboard light control chip after being sorted according to priorities, and a light control instruction queue is obtained. The keyboard light control chip is driven to execute the light control instruction queue, so that the corresponding keyboard area lights are displayed according to the dynamic visual prompt signal, and finally the orientation and distance visualization of in-game footsteps is realized.

6. An in-game footstep sound keyboard cue system, characterized by, The system comprises: A capture module is configured to capture audio signals of an in-game environment in real time, perform frequency spectrum analysis and step sound frequency feature separation on the audio signals, and extract orientation and distance parameters of the step sound. A mapping module is configured to map the orientation and distance parameters into keyboard area control instructions, wherein different orientations correspond to different areas of the keyboard, and different distances correspond to brightness levels of the keyboard lights. A generation module is configured to generate a dynamic visual prompt signal according to the keyboard area control instructions, wherein the dynamic visual prompt signal contains a light color change mode and a brightness gradual change sequence of a specified keyboard area. A prompt module is configured to output the dynamic visual prompt signal to an RGB keyboard light control system, and drive corresponding keyboard area lights to display according to the dynamic visual prompt signal, so as to realize the orientation and distance visualization of in-game footsteps.

7. The system of claim 6, wherein, The capture module is specifically configured to: Capture audio signals of an in-game environment in real time, and eliminate low-frequency background noise by using a high-pass filter to obtain preprocessed audio data streams. Perform short-time Fourier transform on the preprocessed audio data streams to convert time-domain signals into Mel frequency spectrum and obtain a time-frequency spectrum feature matrix. Based on the time-frequency spectrum feature matrix, identify step sound frequency features by using a convolutional neural network, and separate the step sound from the environment sound by using a spectrum masking technology to obtain step sound frequency spectrum features. Analyze the step sound frequency spectrum features by using a binaural time difference and intensity difference algorithm, calculate the sound source orientation angle and relative distance, and finally output the orientation and distance parameters of the step sound.

8. The system of claim 7, wherein, The mapping module is specifically configured to: Divide the keyboard area, divide a standard keyboard into a preset number of sector areas according to the orientation angle, and each area corresponds to a specific key combination to obtain a keyboard area mapping table. Dynamically determine the brightness level according to the relative distance of the step sound by using a distance-brightness mapping relationship, wherein the closer the distance, the higher the brightness, and obtain a brightness level mapping rule. Input the orientation parameters of the step sound into the keyboard area mapping table to determine the corresponding keyboard area identifier, and obtain a preliminary keyboard area instruction. The distance parameter is converted into a specific luminance value in combination with a luminance level mapping rule, and finally a keyboard region control instruction containing region identification and luminance level is output.

9. A storage medium, characterized by The storage medium has stored therein a computer program, wherein the computer program is configured to execute the method of any one of claims 1-5 when executed.

10. An electronic device comprising a memory and a processor, characterized in that, The memory has stored therein a computer program, and the processor is configured to execute the computer program to execute the method of any one of claims 1-5.

Images