Handle control methods, electronic devices and computer-readable storage media

Abstract

This invention discloses a control method for a gamepad, an electronic device, and a computer-readable storage medium. The method includes: acquiring a trigger process currently corresponding to a control action received by the gamepad; determining a corresponding audio segment from the audio to be played corresponding to the control action based on the trigger process; and controlling the vibration module of the gamepad to vibrate based on the spectral characteristics of the audio segment. This invention aims to improve the realism of the user experience.

Description

Technical Field

[0001] This invention relates to the field of electronic devices, and more particularly to a method for controlling a handle, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Virtual reality (VR) devices typically come with controllers. After pairing, users can interact with the VR device using the controllers. However, the interaction provided by controllers is generally a one-way transmission from the user to the virtual scene. For example, using a controller to control game items, cursors, and other objects in the virtual scene. Controllers generally do not provide tactile feedback to the user regarding the virtual scene's controls. For instance, in real shooting scenarios, different guns have different grip feels. In scenarios like simulated shooting, a controller can only vibrate in a single way, unable to provide different tactile sensations, resulting in a poor user experience and lack of realism.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a control method for a handle, an electronic device, and a computer-readable storage medium, with the aim of improving the realism of operation.

[0005] To achieve the above objectives, the present invention provides a method for controlling a handle, the method comprising:

[0006] Obtain the trigger process corresponding to the control action received by the controller on the controller;

[0007] The corresponding audio segment is determined from the audio to be played corresponding to the control action according to the triggering process;

[0008] The vibration module of the handle is controlled to vibrate based on the spectral characteristics of the audio segment.

[0009] Optionally, the vibration module includes multiple vibration elements installed at different positions on the handle, and the step of controlling the vibration module of the handle to vibrate according to the spectral characteristics of the audio segment includes:

[0010] Determining the spectral characteristics determines the audio channel of the audio segment;

[0011] The vibration of the target vibration element corresponding to the vocal tract is controlled based on the spectral characteristics.

[0012] Optionally, the step of determining the spectral characteristics to determine the vocal channel of the audio segment includes:

[0013] Obtain the device identifier of the handle;

[0014] The audio channel is determined based on the device identifier and the spectral characteristics.

[0015] Optionally, the step of obtaining the triggering process corresponding to the control action received by the controller on the controller currently includes:

[0016] Obtain the pressure data of the trigger button on the handle;

[0017] The triggering process is determined based on the pressure data.

[0018] Optionally, the step of determining the triggering process based on the pressure data includes:

[0019] The pressure change value of the trigger key and the compression direction of the trigger key within a preset time period are determined based on the pressure data.

[0020] The triggering process is determined based on the pressure change value and the compression direction.

[0021] Optionally, before the step of controlling the vibration module of the handle to vibrate according to the spectral characteristics of the audio segment, the steps include: when the pressure value is less than a preset value, determining the pressure change rate according to the pressure change value; determining the playback speed of the audio segment according to the pressure change rate; and adjusting the spectral characteristics of the audio segment according to the playback speed.

[0022] Optionally, after the step of determining the corresponding audio segment from the audio to be played corresponding to the control action based on the triggering process, the method further includes:

[0023] While controlling the vibration module to vibrate, the associated device of the handle and / or the speaker of the handle are simultaneously controlled to play the audio clip.

[0024] Optionally, in an associated device applied to the handle, the step of controlling the vibration module of the handle to vibrate based on the spectral characteristics of the audio segment includes:

[0025] The audio clip is sent to the handle; when the handle receives the audio clip, it controls the vibration module to vibrate according to the spectral characteristics.

[0026] In addition, to achieve the above objectives, the present invention also provides an electronic device, the electronic device including a memory, a processor, and a control program for a handle stored in the memory and executable on the processor, wherein when the control program for the handle is executed by the processor, it implements the steps of the handle control method described above.

[0027] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a control program for a handle, wherein the control program for the handle, when executed by a processor, implements the steps of the handle control method described above.

[0028] This invention discloses a control method for a controller, an electronic device, and a computer-readable storage medium. The method first acquires the triggering process corresponding to a control action received by the controller. Then, based on the triggering process, it determines a corresponding audio segment from the audio to be played corresponding to the control action. Finally, it controls the vibration module of the controller to vibrate based on the spectral characteristics of the audio segment. In this way, after the controller receives a control action, it selects a corresponding audio segment from the audio to be played based on the triggering process of the control action, and then controls the vibration module of the controller to vibrate based on the spectral characteristics of the audio segment. This allows the control action to be broken down into different stages, providing the user with tactile feedback at different triggering times, thereby improving the realism of the operation experience. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of the present invention;

[0030] Figure 2 This is a flowchart illustrating an embodiment of the control method for the handle of the present invention;

[0031] Figure 3 This is a flowchart illustrating another embodiment of the control method for the handle of the present invention;

[0032] Figure 4 This is a schematic diagram of the handle structure according to an embodiment of the present invention;

[0033] Figure 5 This is a schematic diagram illustrating the preset correspondence between audio segments, sound channels, and vibration elements in an embodiment of the present invention.

[0034] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0035] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0036] In related technologies, gamepads generally provide an interactive experience that is transmitted unidirectionally from the user to the virtual scene. Gamepads generally do not provide the user with real tactile feedback from the virtual scene. Gamepads can only vibrate in a single way and cannot provide different vibration sensations, resulting in poor realism of the operation provided by gamepads.

[0037] To improve the realism of operation, embodiments of the present invention provide a control method for a handle, an electronic device, and a computer-readable storage medium, wherein the main steps of the method include:

[0038] Obtain the trigger process corresponding to the control action received by the controller on the controller;

[0039] The corresponding audio segment is determined from the audio to be played corresponding to the control action according to the triggering process;

[0040] The vibration module of the handle is controlled to vibrate based on the spectral characteristics of the audio segment.

[0041] This allows the control action to be broken down into different triggering processes, and each triggering process to correspond to the audio to be played for the control action. When different triggers occur, the vibration module of the controller is controlled to vibrate according to different audio segments, providing the user with tactile feedback at different triggering times, thereby improving the realism of the operation experience.

[0042] The claims of this invention will be described in detail below with reference to the accompanying drawings.

[0043] like Figure 1 As shown, Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of the present invention.

[0044] In this embodiment of the invention, the terminal can be an electronic device.

[0045] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU, a memory 1003, and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The memory 1003 may be a high-speed RAM or a stable, non-volatile memory, such as a disk drive. Optionally, the memory 1003 may also be a storage device independent of the aforementioned processor 1001.

[0046] Those skilled in the art will understand that Figure 1 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0047] like Figure 1 As shown, the memory 1003, which serves as a computer storage medium, may include an operating system and a controller program for the gamepad.

[0048] exist Figure 1In the terminal shown, the processor 1001 can be used to call the control program of the handle stored in the memory 1003 and perform the following operations:

[0049] Obtain the trigger process corresponding to the control action received by the controller on the controller;

[0050] The corresponding audio segment is determined from the audio to be played corresponding to the control action according to the triggering process;

[0051] The vibration module of the handle is controlled to vibrate based on the spectral characteristics of the audio segment.

[0052] Furthermore, the processor 1001 can call the handle control program stored in the memory 1003 and also perform the following operations:

[0053] Determining the spectral characteristics determines the audio channel of the audio segment;

[0054] The vibration of the target vibration element corresponding to the vocal tract is controlled based on the spectral characteristics.

[0055] Furthermore, the processor 1001 can call the handle control program stored in the memory 1003 and also perform the following operations:

[0056] Obtain the device identifier of the handle;

[0057] The audio channel is determined based on the device identifier and the spectral characteristics.

[0058] Furthermore, the processor 1001 can call the handle control program stored in the memory 1003 and also perform the following operations:

[0059] Obtain the pressure data of the trigger button on the handle;

[0060] The triggering process is determined based on the pressure data.

[0061] Furthermore, the processor 1001 can call the handle control program stored in the memory 1003 and also perform the following operations:

[0062] The pressure change value of the trigger key and the compression direction of the trigger key within a preset time period are determined based on the pressure data.

[0063] The triggering process is determined based on the pressure change value and the compression direction.

[0064] Furthermore, the processor 1001 can call the handle control program stored in the memory 1003 and also perform the following operations:

[0065] When the pressure value is less than the preset value, the pressure change rate is determined based on the pressure change value;

[0066] The playback speed of the audio segment is determined based on the rate of pressure change;

[0067] The spectral characteristics of the audio segment are adjusted according to the playback speed.

[0068] Furthermore, the processor 1001 can call the handle control program stored in the memory 1003 and also perform the following operations:

[0069] While controlling the vibration module to vibrate, the associated device of the handle and / or the speaker of the handle are simultaneously controlled to play the audio clip.

[0070] Furthermore, the processor 1001 can call the handle control program stored in the memory 1003 and also perform the following operations:

[0071] The audio clip is sent to the handle; when the handle receives the audio clip, it controls the vibration module to vibrate according to the spectral characteristics.

[0072] The following explanation, through specific exemplary solutions, clarifies the scope of protection claimed in the claims of this invention, so that those skilled in the art can better understand the scope of protection of the claims. It is understood that the following exemplary solutions do not limit the scope of protection of this invention, but are only used to explain this invention.

[0073] For example, referring to Figure 2 In one embodiment of the control method for the handle of the present invention, the control method for the handle includes the following steps:

[0074] Step S10: Obtain the triggering process corresponding to the control action received by the controller on the controller;

[0075] In this embodiment, the control method of the controller can be applied to an electronic device, which can be a controller or a controller-related device. The related device can be a virtual reality device, such as VR glasses, or other electronic devices, such as a game console. The related device is mainly used to provide users with operating scenarios, such as game scenarios, and there is a need to recreate the realism of the operating scenario to immerse the user. The user controls the related device through the controller and interacts with the operating scenario of the related device. The increased realism of the operation by the controller affects the user's sense of immersion in the operating scenario.

[0076] When interacting with the operating environment, users trigger control actions through buttons, controller movements, etc. These control actions are pre-defined actions within the operating environment that require active user intervention. For example, in a shooting game, the pre-defined action of pulling the trigger to fire is the firing action; the user triggers this action by pulling the trigger. A certain amount of time is required from when the controller receives the control action to when it ends. Different time periods or varying degrees of completion of different control actions correspond to different triggering processes. The triggering process can be divided based on factors such as time, controller movement distance, and the degree of button press.

[0077] Example 1: In the long jump game, when the user is charging up for the long jump, they will press and hold the trigger button. Different pressing times correspond to different pressing degrees, and the tactile feedback that the controller needs to provide to the user will also be different. The triggering process of the control action can be determined based on the time after the received control action.

[0078] Example 2: In the fruit-cutting game, the user cuts fruit by swinging the controller. The distance the controller moves from the start of the swing to the end corresponds to different degrees of completion of the cutting operation. Therefore, the triggering process of the control action can be determined based on the controller's movement data.

[0079] Example 3: In a shooting game, each time a user pulls the trigger, it is considered one shot. From the moment the user starts pulling the trigger until the trigger is fully released, the pressure data received by the trigger changes continuously. The triggering process of the control action can be determined based on the pressure data of the trigger on the controller.

[0080] Optionally, pressure data of the trigger button on the handle is acquired; the triggering process is determined based on the pressure data.

[0081] Most control actions are triggered by the controller's trigger buttons. For example, in shooting scenarios, trigger button 1 needs to be used as a trigger. Figure 4 It can monitor the pressure data of the trigger button on the controller in real time and determine the triggering process based on the pressure data. As the user performs a control action by pressing the trigger button, the pressure value gradually increases, and then decreases briefly when the user releases the trigger. Therefore, the triggering process of the control action can be determined based on the pressure data.

[0082] When the controller receives a control action, it marks the beginning of the action's triggering process. For example, when the user pulls the trigger, the controller receives the firing control action and begins the shooting process. When the user fully releases the trigger, the controller ends the control action. Each node corresponds to a different triggering process. The entire triggering process of the control action can be segmented according to certain characteristics, with each segment corresponding to a triggering process. This reduces computational power consumption while still providing the user with a realistic operating experience.

[0083] Taking the Gatling gun shooting scenario as an example: Triggering process 1, when the player slowly pulls the Gatling gun trigger, the Gatling disc initially spins faster, and no bullets are fired. At this time, the player can only feel the vibration caused by the rotation of the disc, and cannot experience the recoil vibration after the bullets are fired. The corresponding vibration and sound effects are relatively smooth. Triggering process 2, until a certain amount of pressure is applied, the disc will spin at a constant speed, and the player can feel the vibration caused by the rotation of the disc, and bullets are fired. At this time, the player can not only feel the vibration caused by the rotation of the disc, but also experience the recoil vibration after the bullets are fired. Triggering process 3, when the trigger is slowly released, the disc will decelerate, and no bullets are fired. At this time, the player can only feel the vibration caused by the rotation of the disc, and cannot experience the recoil vibration after the bullets are fired.

[0084] Specifically, the entire process of controlling the action can be divided into multiple triggering processes. The pressure range corresponding to each process is determined, and the pressure range of the current trigger key's pressure value is determined based on the pressure data. The corresponding triggering process is then determined based on this pressure range. If the pressure range has not changed compared to before, it is determined that the previous triggering process has not ended, and the next step is not continued.

[0085] Optionally, the pressure change value of the trigger key and the compression direction of the trigger key within a preset time period are determined based on the pressure data; the triggering process is determined based on the pressure change value and the compression direction.

[0086] Most trigger buttons are equipped with an automatic rebound feature. Throughout the entire control action, the trigger button's compression direction includes both compression and release. The trigger button may change direction at any time during this process. Even if the pressure change value is the same within a preset time period, the compression direction of the trigger button within that period should be distinguished. The compression direction and pressure change value are crucial for determining the triggering process. For example, within a preset time period, the pressure change value during the trigger button's rebound will be greater than the pressure change value during the pressing process, and the compression direction will be opposite. The compression direction can be determined by the difference between the initial pressure value at the start of the predicted time period and the final pressure value at the end of the preset time period. If the difference is positive, the compression direction is compression; if the difference is negative, the direction is release.

[0087] Furthermore, when the pressure value is less than a preset value, the pressure change rate is determined based on the pressure change value; the playback speed of the audio segment is determined based on the pressure change rate; and the spectral characteristics of the audio segment are adjusted based on the playback speed.

[0088] The current pressure value of the trigger button is determined based on the pressure data of the trigger button on the controller. When the pressure value is greater than the preset value, the user has essentially bottomed out the trigger button, but the pressure value can continue to increase. At this level, the playback speed of the audio clip cannot be adjusted according to the pressure value, and the corresponding audio clip has a sufficiently high frequency, so there is no need to adjust it according to the pressure value. When the pressure value is less than the preset value, such as in trigger processes 1 and 3 of a shooting scene, generally only the dial rotates. In trigger process 1, the dial will rotate faster as the pressure value increases, and the corresponding audio playback speed should also increase. In trigger process 2, the dial will rotate slower as the pressure value decreases, and the corresponding audio playback speed should also decrease. For the audio to be played, the action can be controlled to set corresponding sound effects for different trigger processes. For example, the sound effect for the entire dial is the sound of the dial rotating faster and faster, and then slower and slower. For some audio playback effects, the sound effect is uniform, so its playback speed needs to be adjusted. This can be achieved by determining the corresponding pressure change rate based on the pressure change value within a preset time period, and adjusting the playback speed of the audio segment accordingly. The preset time period can be very short, creating a real-time control effect for the audio segment's playback speed. Then, the spectral characteristics of the audio segment are adjusted based on the sound effect playback speed, i.e., the audio segment's spectrum is compressed in the time domain. The vibration module is controlled to vibrate based on the adjusted spectral characteristics of the audio segment, or, while controlling the vibration module's vibration, the associated device of the control handle and / or the handle's speaker are simultaneously controlled to play the audio segment according to the adjusted spectral characteristics.

[0089] Step S20: Determine the corresponding audio segment from the audio to be played corresponding to the control action according to the triggering process;

[0090] In this embodiment, to provide a better user experience, audio, such as game sound effects, accompanies the completion of control actions. The audio is determined based on the control action at the start of the action. However, audio playback takes time; therefore, it is the audio to be played. In most scenarios, the audio corresponds to the control action. To simulate different scenarios or provide a better experience, the audio content may change, resulting in different audio frequencies. Taking a Gatling gun shooting scenario as an example, in a real scenario, the sound of the Gatling gun firing changes from a low-frequency audio at the start of the dial rotation to a high-frequency audio when the bullet is released, and the corresponding trigger press intensity also varies. Therefore, the audio segment to be played differs in different triggering processes. The corresponding audio segment can be determined from the audio to be played based on the current triggering process.

[0091] Optionally, the audio to be played is divided into multiple consecutive audio segments according to the spectral characteristics of the audio to be played; the audio segments are matched with the triggering process of the control action according to the order of the audio segments to obtain a preset correspondence between the triggering process and the audio segments.

[0092] The audio to be played can be the sound effect corresponding to the entire control action triggering process. Throughout this process, the waveform of the audio segment may change significantly in both the time and frequency domains. However, the spectral characteristics within a given time period will not change much, indicating that the control action has not progressed significantly within that time period. To save power, the audio to be played can be divided into multiple consecutive audio segments based on its spectral characteristics. The duration of each audio segment may differ, but within the same audio segment, the difference between the maximum and minimum audio frequencies will not exceed a preset value, and the audio segments are continuous in time. The order of the audio segments is determined based on their time intervals within the audio to be played. A preset correspondence between the triggering process and the audio segments is determined based on the order of the audio segments within the audio to be played and the order of the triggering process within the entire control action triggering process. Step S20 includes determining the audio segment corresponding to the current triggering process based on the preset correspondence between the triggering process and the audio segments.

[0093] Step S30: Control the vibration module of the handle to vibrate according to the spectral characteristics of the audio segment.

[0094] In this embodiment, a vibration module is provided on the handle to provide vibration feedback to the user, including simulating the vibration of the object corresponding to the handle in the operating scenario. The audio to be played is a sound effect set to match the control action. The audio to be played is used to provide narration for the control action, indicating the intensity of the control action in the scenario, restoring the realistic scenario, and improving the user experience. When the audio to be played is relatively low-pitched, it indicates that the object corresponding to the handle in the scenario, such as a gun, does not emit a loud sound or vibrate violently, and the vibration feedback from the handle to the user should not be too large. Because the audio to be played generally uses the sound produced by the vibration of objects in the scenario, the vibration frequency of the scenario is the same as the audio frequency of the audio to be played. The audio segment is the audio excerpted from the audio to be played during the triggering process. The audio frequency of the audio segment at each hour can be determined based on the spectral characteristics of the audio segment, thereby determining the vibration frequency of the handle's vibration module at each hour in the triggering process, and then controlling the vibration of the vibration module according to the vibration frequency in the triggering process.

[0095] Optionally, while controlling the vibration module to vibrate, the associated device of the handle and / or the speaker of the handle are simultaneously controlled to play the audio clip.

[0096] After determining the current triggering process of a control action, the corresponding audio segment is identified, and the vibration frequency of the controller's vibration module during that triggering process is determined based on the audio segment. The controller's vibration can be controlled according to this frequency during the triggering process, providing the user with a more realistic tactile experience. Users can also distinguish different control actions based on the tactile feedback. Furthermore, to enhance immersion and integrate the user's tactile and auditory senses, in addition to controlling the vibration module to vibrate at the determined frequency during the triggering process, an audio segment is simultaneously played through the controller's associated devices and / or the controller's speakers, further enhancing the realism of the operation.

[0097] It should be noted that if the controller does not have a speaker installed, the audio clips need to be played through the speaker of the controller's associated device, which is more compatible with the device structure. If the controller has a speaker installed, the audio clips can be played through the controller's speaker. When the audio is delivered to the user, the user can also identify the directionality of the audio clips, which is more conducive to recreating the scene.

[0098] In this embodiment, the control of the handle can be applied to the handle itself or its associated device. If applied to the handle, after determining an audio segment, the handle can directly determine the vibration parameters of the vibration module based on the spectral characteristics of the audio segment, and then directly control the vibration module to vibrate based on the vibration parameters. If applied to the associated device, step S10 includes: receiving pressure data from the trigger key on the handle sent by the handle, and determining the trigger process based on the pressure data. When executing step S30, the control operation of the handle involves sending the determined audio segment or the vibration parameters determined based on the audio segment to the handle. When the handle receives the audio segment, it needs to determine the vibration parameters of the vibration module based on the audio segment and control the vibration module to vibrate based on the vibration parameters; or, upon receiving the vibration parameters, it controls the vibration module to vibrate based on the vibration parameters. This achieves control of the handle by the associated device.

[0099] In the technical solution disclosed in this embodiment, the triggering process corresponding to the control action received by the controller is first obtained; the corresponding audio segment is determined from the audio to be played corresponding to the control action based on the triggering process; and the vibration module of the controller is controlled to vibrate based on the spectral characteristics of the audio segment. Thus, after the controller receives a control action, the corresponding audio segment is selected from the audio to be played corresponding to the control action based on the triggering process of the control action on the controller, and then the vibration module of the controller is controlled to vibrate based on the spectral characteristics of the audio segment. This allows the control action to be broken down into different stages, providing the user with tactile feedback at different triggering times, thereby improving the realism of the operation provided by the controller and its associated devices, and immersing the user in the operating scenario.

[0100] Optionally, refer to Figure 3Based on any of the above embodiments, in another embodiment of the control method for the handle of the present invention, the control method for the handle further includes:

[0101] Step S31: Determine the spectral characteristics to determine the audio channel of the audio segment;

[0102] Step S32: Control the vibration of the target vibration element corresponding to the sound channel according to the spectral characteristics.

[0103] In this embodiment, the control handle includes multiple vibrating elements installed at different positions on the handle. When these different vibrating elements vibrate at different positions, they can provide the user with vibration feedback from different locations. In many scenarios, the vibration of an object held by the user varies at different locations. Therefore, the vibrating elements at different positions can vibrate according to different frequencies, meaning that the vibration of the vibrating elements at different positions can be controlled to vibrate only according to audio segments within their corresponding audio frequency range. In this way, when the vibration of the corresponding vibrating element is controlled according to the audio frequency of the audio segment, the vibration frequency will not exceed a preset range, achieving different vibration experiences at different positions and further restoring the realism of the operation experience.

[0104] Optionally, refer to Figure 4 The handle includes a trigger button 1 and a grip. The trigger button 1 is connected to a pressure sensor, which collects pressure data from the trigger button. A narrow-frequency vibration element 2 is located between the grip and the trigger button 1, and a wide-frequency vibration element 3 is located on the grip. The narrow-frequency vibration element 2 vibrates according to audio segments with frequencies lower than a preset value, while the wide-frequency vibration element 3 vibrates according to audio segments with frequencies greater than or equal to the preset value. In a shooting scenario, the triggering process corresponding to audio segments with frequencies lower than the preset value involves vibration caused by the rotation of a dial. Since these audio segments are low-frequency, controlling the narrow-frequency vibration element 2 based on the low-frequency audio segments allows the user to feel a slight vibration at the front, simulating the charging of the gun before firing. The triggering process corresponding to audio segments with frequencies greater than or equal to the preset value involves the firing of a bullet, resulting in recoil vibration. Since these audio segments are high-frequency, controlling the wide-frequency vibration element 3 based on the high-frequency audio segments allows the user to feel a strong vibration at the rear, simulating the recoil of firing.

[0105] Specifically, to distinguish different vibrating elements, the vibration frequency range of the vibrating element is determined based on its location, and the corresponding audio frequency range is determined based on its vibration frequency range. The audio range of the audio segment is determined based on the spectral characteristics of the audio segment. It is then determined whether the audio range is within the vibration range of the vibrating element. If it is, the vibration of the vibrating element is controlled according to the spectral characteristics of the audio segment. In actual implementation, to achieve audio segment splitting in advance and reduce the processor load, especially when the controller method is applied to the controller's associated devices, it is necessary to reduce the controller's computing power consumption. This can be achieved by changing the audio segment's channel, determining the target vibrating element based on the audio segment's channel, and controlling the target vibrating element's vibration based on the spectral characteristics of the audio segment.

[0106] Prior to this, the corresponding audio frequency range is determined based on the location of each vibrating element. A pre-defined correspondence is then established between each vibrating element, each channel, and the audio frequency range. Based on the spectral characteristics of the audio segment and the pre-defined correspondence, the audio frequency range and corresponding channel of the audio segment are determined, and then the corresponding target vibrating element is identified.

[0107] Optionally, the device identifier of the handle is obtained; the audio channel is determined based on the device identifier and the spectral characteristics.

[0108] The associated device for a handle can be associated with not only that handle but also other handles. Typically, there are two handles: a left handle and a right handle. If the handle control method is applied to the associated device, different handles and their different vibrating elements can be distinguished by audio channels. The device identifier of the handle that records the transmitted pressure data or triggers the process is used. Based on the device identifier and the spectral characteristics of the audio segment, the corresponding audio channel is determined. The audio segment is then transmitted to the operating device based on this channel. The operating device determines whether the vibrating element of the audio segment includes the target vibrating element corresponding to that channel. If so, the target vibrating element is controlled to vibrate based on the spectral characteristics of the audio segment.

[0109] It should be noted that the number of audio clips to be played corresponding to the control action can be one or more. During the entire control action triggering process, there may be multiple overlapping audio clips. For example, in a shooting scenario, there may only be the sound of the dial rotating, or there may be the sound of the dial rotating and the sound of bullet casings falling simultaneously. The audio segments corresponding to the control action triggering process can also be one or more. If there are multiple audio clips to be played corresponding to the control action, the corresponding audio segments are determined from the multiple audio clips to be played according to the triggering process, and the vibration of different vibration modules of the vibration module is controlled according to the spectral characteristics of the audio frequency band.

[0110] To better understand, here is a specific application scenario:

[0111] The virtual reality device is equipped with two controllers, left and right. Each controller needs to have two vibration elements (a narrow-frequency motor and a wide-frequency motor). The left controller has a narrow-frequency motor and a wide-frequency motor, and the right controller has a narrow-frequency motor and a wide-frequency motor. (See reference...) Figure 4 Taking the right-hand controller as an example, the Trigger button (trigger button 1) has a pressure sensor that can provide real-time feedback on pressure values. The narrow-frequency motor (narrow-frequency vibration element 2) of the right-hand controller is located at the front of the controller, while the wide-frequency motor (wide-frequency vibration element 3) is located in the grip area, which is the area where the user's hand grips. The left-hand controller has the same function as the right-hand controller's trigger button, both having pressure sensors that can provide real-time feedback on pressure values, and the relative positions of the narrow-frequency and wide-frequency motors are the same. Different audio segments are sent to different motors according to their corresponding channels. The controller analyzes the audio and outputs a motor vibration waveform based on the waveform (spectral characteristics) of the audio segment. The greater the waveform vibration of the audio segment, the greater the output motor vibration waveform. Therefore, the controller motor can vibrate according to the waveform of the audio segment; the greater the waveform vibration, the greater the controller vibration; the smaller the waveform vibration, the smaller the controller vibration.

[0112] Reference Figure 5 Audio clip 1 is sent to the narrowband motor of the left controller; audio clip 2 is sent to the narrowband motor of the right controller; audio clip 3 is sent to the wideband motor of the left controller; and audio clip 4 is sent to the wideband motor of the right controller. When shooting in a virtual scene, the user can send the audio clip of the desired waveform to the motor they want to make vibrate.

[0113] Taking a Gatling gun firing scenario as an example: To simulate a realistic Gatling gun firing scenario, the trigger button pressure value on the controller was monitored in real time, and the trigger button's range was divided into 0-100%. Different segments provide different vibration sensations. Figure 4 Following the standard gun-holding technique, place your right hand on the trigger (fire button 1) to fire. The specific firing action is as follows:

[0114]

[0115] When triggering the Gatling gun's firing control action, the triggering process is divided into the initial stage, the middle stage, and the final stage. The audio segments of each triggering stage are determined, including sound effects such as gunshots and the sound of shell casings hitting the ground. These sound effects are very short audio clips, generally around tens of milliseconds in length.

[0116] By obtaining the Trigger key pressure value, the Trigger key progress is obtained. When the Trigger key progress is below 10%, no audio operation or motor vibration operation is performed.

[0117] When the trigger progress is in the early stage, that is, when the trigger button progress is between 10% and 80%, the dial spins faster, dynamically controlling the audio playback frequency of the Gatling gun. The larger the progress, the faster the music playback frequency, and the stronger the vibration of the narrow frequency motor on the right handle, giving people a feeling of accelerated rotation.

[0118] During the mid-stage trigger process, when the Trigger button progress is greater than 80%, the dial rotates at a constant speed. At the same time, there is a bullet firing action. The audio playback rate of the Gatling gun in the early stage is dynamically accelerated to keep the playback frequency of the Gatling gun's mid-stage audio and the bullet firing sound constant. The corresponding narrow-frequency motor and wide-frequency motor of the controller will vibrate in the same way. At the same time, the sound of the cartridge case hitting the ground should also be played at the same playback rate, but it should lag behind the bullet firing sound. The lag time can be adjusted according to the actual effect.

[0119] In the later stage of the trigger process, when the Trigger button progress decreases from 80% to 0, the dial slows down and the bullets stop firing. This dynamically controls the audio playback rate in the early stage of the Gatling gun. The smaller the progress, the slower the music playback frequency, and the weaker the vibration of the narrow frequency motor on the right handle, giving a feeling of slowing down the rotation.

[0120] In the technical solution disclosed in this embodiment, the audio channel of the audio segment is determined by determining the spectral characteristics; the vibration of the target vibration element corresponding to the audio channel is controlled according to the spectral characteristics. This achieves audio segment splitting, saves computational resources, allows control of vibration elements at different locations, and further improves the realism of the user experience.

[0121] Furthermore, this invention also proposes an electronic device, which includes a memory, a processor, and a control program for a handle stored in the memory and executable on the processor. When the control program for the handle is executed by the processor, it implements the steps of the handle control method described in the above embodiments.

[0122] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a control program for a handle, wherein when the control program for the handle is executed by a processor, it implements the steps of the handle control method described in the above embodiments.

[0123] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0124] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause an electronic device to execute the methods described in the various embodiments of the present invention.

[0126] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for controlling a handle, characterized in that, The method comprises: The process of obtaining the triggering process corresponding to the control action received by the controller on the controller includes obtaining the pressure data of the trigger button on the controller and determining the triggering process based on the pressure data. The corresponding audio segment is determined from the audio to be played corresponding to the control action according to the triggering process, including dividing the audio to be played into multiple consecutive audio segments according to the spectral characteristics of the audio to be played; When the pressure value is less than the preset value, the pressure change rate is determined based on the pressure change value. The playback speed of the audio segment is determined based on the rate of pressure change; Adjust the spectral characteristics of the audio segment according to the playback speed; The vibration parameters of the vibration module are determined based on the spectral characteristics of the audio segment, and the vibration of the vibration module of the handle is controlled based on the vibration parameters.

2. The control method for the handle as described in claim 1, characterized in that, The vibration module includes multiple vibration elements installed at different positions on the handle, and the step of controlling the vibration module of the handle to vibrate according to the spectral characteristics of the audio segment includes: Determining the spectral characteristics determines the audio channel of the audio segment; The vibration of the target vibration element corresponding to the vocal tract is controlled based on the spectral characteristics.

3. The control method for the handle as described in claim 2, characterized in that, The step of determining the audio channel by determining the spectral characteristics includes: Obtain the device identifier of the handle; The audio channel is determined based on the device identifier and the spectral characteristics.

4. The control method for the handle as described in claim 1, characterized in that, The step of determining the triggering process based on the pressure data includes: The pressure change value of the trigger key and the compression direction of the trigger key within a preset time period are determined based on the pressure data. The triggering process is determined based on the pressure change value and the compression direction.

5. The control method for the handle as described in claim 1, characterized in that, After the step of determining the corresponding audio segment from the audio to be played corresponding to the control action based on the triggering process, the method further includes: While controlling the vibration module to vibrate, the associated device of the handle and / or the speaker of the handle are simultaneously controlled to play the audio clip.

6. The control method for the handle as described in claim 1, characterized in that, The associated device applied to the handle, wherein the step of controlling the vibration module of the handle to vibrate based on the spectral characteristics of the audio segment includes: The audio clip is sent to the handle; when the handle receives the audio clip, it controls the vibration module to vibrate according to the spectral characteristics.

7. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a control program for a handle stored in the memory and executable on the processor, wherein the control program for the handle, when executed by the processor, implements the steps of the control method for the handle as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a control program for the handle, which, when executed by a processor, implements the steps of the handle control method as described in any one of claims 1 to 6.

Images