Signal generating device, signal generating method, and program

Abstract

Provided is a feature relating to more intuitive setting of presented tactile force. This signal generation device comprises an input unit that accepts setting information including information indicating the direction of tactile force presented from an object, and a generation unit that generates a waveform signal for presenting tactile force based on the setting information accepted by the input unit.

Description

[Technical field]

[0001] The present invention relates to a signal generating device, a signal generating method, and a program. [Background technology]

[0002] Haptics technology that presents sensations such as haptics by generating vibrations is known. Patent Document 1 discloses modifying parameters such as start time, duration, loop count, clip length, signal type, strength type, signal gap, signal width, gap first, link gap width, and signal shape to edit haptic effects. Patent Document 2 discloses changing parameters of vibration frequency and amplitude to customize haptic output. Patent Document 3 discloses customizing haptic parameters such as magnitude, duration, frequency, and waveform. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2018-528534 [Patent Document 2] Special Publication No. 2010-528394 [Patent Document 3] JP 2014-044722 A Summary of the Invention [Problem to be solved by the invention]

[0004] There is a demand for technology that allows the haptic sensations to be presented more intuitively.

[0005] The present invention has been made in consideration of the above circumstances, and has an object to provide a technique for more intuitive setting of the haptic sense to be presented. [Means for solving the problem]

[0006] A signal generating device according to one embodiment of the present invention includes an input unit that accepts setting information including information indicating the direction of a haptic sensation to be presented from an object, and a generation unit that generates a waveform signal for presenting a haptic sensation based on the setting information accepted by the input unit.

[0007] A signal generating method according to one embodiment of the present invention includes receiving setting information including information indicating a direction of a haptic sensation to be presented from an object, and generating a waveform signal for presenting a haptic sensation based on the received setting information.

[0008] A program according to one embodiment of the present invention causes a computer to function as an input unit that accepts setting information including information indicating the direction of the tactile sensation to be presented from an object, and a generation unit that generates a waveform signal for presenting a tactile sensation based on the setting information accepted by the input unit. Effect of the Invention

[0009] According to the present invention, it is possible to provide a technique for more intuitively setting the haptic sensation to be presented. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a game development system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of a signal generating device according to an embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a user interface according to an embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a user interface according to an embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a user interface according to an embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a user interface according to an embodiment. [Figure 7]FIG. 7 is a diagram illustrating an example of a user interface according to an embodiment. [Figure 8] FIG. 8 is a diagram for explaining an example of a waveform signal generation process according to an embodiment. [Figure 9] FIG. 9 is a diagram for explaining an example of a waveform signal generation process according to an embodiment. [Figure 10] FIG. 10 is a block diagram showing an example of a configuration for realizing a signal generating device. [Figure 11] FIG. 11 is a block diagram showing another example of the configuration for realizing a signal generating device. [Figure 12] FIG. 12 is a block diagram showing another example of the configuration for realizing a signal generating device. [Figure 13] FIG. 13 is a block diagram showing another example of the configuration for realizing a signal generating device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the same elements are given the same reference numerals, and duplicated explanations will be omitted as much as possible.

[0012] A game development system according to this embodiment will be described. FIG. 1 is a diagram showing a schematic configuration of the game development system according to this embodiment. As shown in FIG. 1, the game development system 3 mainly includes a computer 11, a speaker 19, a display monitor 20, and a controller 21 (object). Although not shown, the game development system 3 may include an operation unit (e.g., a mouse and a keyboard for user operation) connected to the computer 11 in addition to the controller 21. Although FIG. 1 shows that the game development system 3 includes one computer 11, it may include multiple computers 11 in order to realize the functions of the computer 11 described below.

[0013] The computer 11 is a computer for creating a game program and executing the created game program (for testing). The computer 11 can, for example, display a user interface for creating a game program on the display monitor 20 and create a game program in response to a user input received via an operation unit (not shown) such as a keyboard or a mouse. Furthermore, the computer 11 executes the created game program, displays an image representing a virtual reality in which the game program unfolds on the display monitor 20, and further outputs sound from the speaker 19. For example, a user recognizes the situation of a character in the virtual reality displayed on the display monitor 20, and operates the controller 21 so as to give a movement to the character in response to the situation. The computer 11 executes the game program in response to the operation content performed on the controller 21.

[0014] Furthermore, the computer 11 transmits a control signal including a waveform signal to the controller 21, thereby presenting at least one of the "force sense," "pressure sense," and "touch sense" (hereinafter, sometimes referred to as haptic presentation) from the controller 21 to the user through haptics. Here, the "force sense" is, for example, the feeling of being pulled or pushed, and the feeling of impact when pressing hard against an object or when an object pops. The "pressure sense" is, for example, the sense of contact when touching an object or when feeling the hardness or softness of an object. The "touch sense" is, for example, the feel of the surface of an object, or the feel of touch and roughness, such as the degree of unevenness of the surface of an object.

[0015] The software and hardware hierarchy in the computer 11 is made up of a game program in the application layer, a Software Development Kit (SDK) in the middle layer, a system and game engine, and Hardware (HW) in the physical layer.

[0016] The SDK includes, for example, a plug-in or an authoring tool, and middleware. The middleware includes a program (hereinafter, sometimes referred to as the target program) that vibrates the controller 21 so as to give the user at least one of a "force sense," a "pressure sense," and a "touch sense." For example, when a specific event occurs to a character, the game program calls the target program according to an API (Application Programming Interface). At this time, the game program passes event information indicating, for example, the type of event and the start time of the event to the target program. The type of event is identified, for example, by an ID.

[0017] Examples of specific events include an external force being applied to a character pulling or pushing within the virtual reality, a character firing a gun, a character being hit, an explosion occurring near a character, and a character dancing to music.

[0018] The target program generates a waveform signal for presenting a haptic sensation according to the type of event indicated by the event information, based on the event information. The target program transmits the generated waveform signal to the controller 21 via the game engine, the operating system, and the hardware.

[0019] The controller 21 vibrates based on the waveform signal to provide a haptic sensation. For example, by holding the vibrating controller 21 in his / her hand, a user playing a game can recognize the status of a character in virtual reality not only through sight and hearing, but also through at least one of the "force sense," "pressure sense," and "touch sense."

[0020] Next, the hardware configuration of the game development system according to this embodiment will be described with reference to Fig. 1. The computer 11 includes a CPU (Central Processing Unit) 12, a memory 13, a disk 14, an audio interface (I / F) 15, a GPU (Graphics Processing Unit) 16, a communication interface (I / F) 17, and a bus 18. The controller 21 includes an MCU (Micro Controller Unit) 22, a communication interface (I / F) 23, a haptic output driver 24, a haptic element 25, a sensor input driver 26, and a sensor element 27.

[0021] In the computer 11, the CPU 12, memory 13, disk 14, audio interface 15, GPU 16 and communication interface 17 are connected via a bus 18 so as to be able to transmit and receive data to and from each other.

[0022] In this embodiment, the disk 14 is a readable / writable non-volatile storage device such as a hard disk drive (HDD), and stores programs (codes) such as a game program and an SDK. The disk 14 is not limited to an HDD, and may be another storage medium such as a solid state drive (SSD), a memory card, or a read-only compact disc read-only memory (CD-ROM) or digital versatile disc-read-only memory (DVD-ROM). Programs such as the target program can be installed from outside. Programs such as the target program are distributed in a state stored in a storage medium such as the disk 14 that can be read by the computer 11. Programs such as the target program may be distributed on the Internet connected via a communication interface.

[0023] The memory 13 is a volatile storage device such as a dynamic random access memory (DRAM). The communication interface 17 transmits and receives various data to and from a communication interface 23 in the controller 21. This communication may be performed either wired or wirelessly, and any communication protocol may be used as long as the two can communicate with each other. The communication interface 17 transmits various data to the controller 21 in accordance with instructions from the CPU 12. The communication interface 17 also receives various data transmitted from the controller 21 and outputs the received data to the CPU 12.

[0024] When executing a program, CPU 12 transfers the program and data necessary for executing the program stored on disk 14 to memory 13. CPU 12 reads processing instructions and data necessary for executing the program from memory 13 and executes arithmetic processing according to the contents of the processing instructions. At this time, CPU 12 may newly generate data necessary for executing the program and store it in memory 13. Note that CPU 12 is not limited to a configuration that obtains the program and data from disk 14, and may be configured to obtain the program and data from a server via the Internet.

[0025] Specifically, the CPU 12 creates a game program in response to user operations, for example, and stores the created game program in a storage unit such as the memory 13 or the disk 14. Furthermore, for example, when executing the created game program, the CPU 12 receives the operation contents of the user's controller 21, executes a processing command corresponding to the operation contents, and gives movement to a character in the virtual reality. At this time, the CPU 12 performs processing for presenting haptics, displaying video, and outputting audio according to the status of the character in the virtual reality.

[0026] More specifically, for example, when an external pulling or pushing force is applied to a character in virtual reality, the CPU 12 generates a waveform signal for haptically presenting the force sensation felt when the external force is applied.

[0027] In addition, the CPU 12 generates a waveform signal for haptically presenting the recoil sensation felt when a character fires a gun in virtual reality, for example.

[0028] Furthermore, the CPU 12 generates a waveform signal for haptically presenting the sensation of impact felt when a character receives a blow in virtual reality, for example.

[0029] Furthermore, the CPU 12 generates a waveform signal for haptic presentation of the sense of dynamism felt in response to musical beats and rhythms, for example, when a character is dancing to music in virtual reality.

[0030] The CPU 12 generates haptic information by digitally encoding the generated waveform signal, and transmits the generated haptic information to the controller 21 via the communication interface 17 .

[0031] Furthermore, the CPU 12 generates screen information necessary for displaying images of characters and backgrounds moving within the virtual reality, and outputs the generated screen information to the GPU 16. The GPU 16, for example, receives the screen information from the CPU 12, performs rendering based on the screen information, and generates a digital video signal including images such as 3D graphics. The GPU 16 transmits the generated digital video signal to the display monitor 20, thereby causing the display monitor 20 to display the 3D graphics.

[0032] Furthermore, the CPU 12 generates audio information indicating audio corresponding to the environment, movement, and situation of the character in the virtual reality, and outputs the generated audio information to the audio interface 15. The audio interface 15 receives audio information from the CPU 12, performs rendering based on the received audio information, and generates an audio signal. The audio interface 15 transmits the generated audio signal to the speaker 19, thereby causing the speaker 19 to output audio.

[0033] The haptic element 25 in the controller 21 is a vibration actuator that converts an electric signal into a mechanical vibration, for example, a voice coil actuator that has a wide frequency band of vibration relaxation. The haptic element 25 may be an eccentric motor, a linear resonant actuator, an electromagnetic actuator, a piezoelectric actuator, an ultrasonic actuator, an electrostatic actuator, or a polymer actuator. The number of haptic elements 25 in the controller 21 may be one or more.

[0034] The MCU 22 controls the haptic output driver 24 and the sensor input driver 26. Specifically, for example, when power is supplied to the MCU 22, the MCU 22 reads a program stored in a ROM (not shown) and executes arithmetic processing according to the contents of the program.

[0035] In this embodiment, when the MCU 22 receives haptic information from the computer 11 via the communication interface 23, for example, the MCU 22 controls the haptic output driver 24 based on the received haptic information and presents haptics using the haptic element 25.

[0036] Specifically, the MCU 22 outputs the haptic information to the haptic output driver 24. The haptic output driver 24 receives the haptic information from the MCU 22, generates an analog electrical signal that is an electrical signal corresponding to a waveform signal and can drive the haptic element 25 based on the received haptic information, and outputs the analog electrical signal to the haptic element 25. This causes the haptic element 25 to vibrate based on the electrical signal, and a haptic presentation is performed.

[0037] The sensor element 27 detects the movement of an operation unit that receives user operations, such as a joystick and buttons provided on the controller 21, and outputs an analog electrical signal indicating the detection result to the sensor input driver .

[0038] The sensor input driver 26 operates, for example, under the control of the MCU 22, supplies the sensor element 27 with power required for driving, and receives an electric signal from the sensor element 27 and converts the received electric signal into a digital signal. The sensor input driver 26 outputs the converted digital signal to the MCU 22. The MCU 22 generates operation information indicating the content of the user's operation on the controller 21 based on the digital signal received from the sensor input driver 26, and transmits the operation information to the computer 11 via the communication interface 23.

[0039] [Configuration of signal generator] 2 is a block diagram showing the configuration of a signal generating device according to this embodiment. The signal generating device 1 is realized, for example, by causing a CPU 12 in a computer 11 to execute a signal generating program, which is an example of a target program.

[0040] The signal generating device 1 includes, as functional blocks, a direction input unit 202 that receives setting information including information indicating the direction of the haptic sense to be presented from an object, and a signal generating unit 206 that generates a waveform signal for presenting the haptic sense based on the setting information received by the direction input unit 202. The setting information indicating the direction of the haptic sense to be presented from an object makes it easier for a user to intuitively grasp the sensation to be experienced by the haptic sense, compared with setting information such as the frequency and amplitude of vibration for generating the haptic sense. Therefore, according to the signal generating device 1 of this embodiment, it is possible to perform more intuitive setting of the haptic sense to be presented. In addition, by being able to intuitively set the direction in which the haptic sense is presented, it becomes easier to present a more realistic haptic sense to the user.

[0041] The signal generating device 1 also includes an intensity input unit 203 that receives setting information including information indicating the intensity of the haptic sensation presented from the object. The signal generating device 1 also includes a time axis setting unit 201, a parameter generating unit 204, and a basic waveform registration unit 205. The time axis setting unit 201, the direction input unit 202, and the intensity input unit 203 may also include a function for controlling the display of a user interface (graphical user interface) on the display monitor 20 by the signal generating device 1. Each functional block will be described in more detail below.

[0042] The time axis setting unit 201 sets the length of a signal generated for presenting a haptic sense, that is, the time axis of the haptic sense to be presented. The time axis of the haptic sense may include information on the timing of starting or ending the presentation of the haptic sense. The time axis setting unit 201 may set the time axis of the haptic sense in response to a user input via an operation unit, for example, or may set a predetermined time axis. The time axis setting unit 201 can also set the resolution (frequency) of the haptic sense to be presented in response to a user input.

[0043] The resolution of the haptic sensation to be presented is the granularity (unit) of the haptic sensation that can be set or edited by the method described below. For example, if the resolution is set to 1 / 30 [seconds], it is possible to set the haptic sensation to be presented (for example, the direction or intensity) in units of 1 / 30 seconds.

[0044] The time axis of the haptic sense to be presented may be set based on a reference waveform. The reference waveform is, for example, a waveform obtained by analyzing data that can be converted into a waveform, such as audio data or video data. The time axis setting unit 201 sets the reference waveform, for example, by analyzing audio data or video data acquired in response to a user input. The time axis setting unit 201 may set the time axis of the haptic sense to be presented based on the length (time length) of the set reference waveform. Note that the audio data or video data is acquired, for example, via media or in a streaming format, but the data acquisition method or format may be any.

[0045] The time axis setting unit 201 performs processing for transmitting information on the time axis, resolution, and reference waveform set as described above from the signal generating device 1 to the display monitor 20. The display monitor 20 controls the display based on the received information on the time axis, resolution, and reference waveform.

[0046] FIG. 3 shows an example of a user interface image (hereinafter, simply referred to as "user interface") displayed on the display monitor 20 based on the information of the time axis, resolution, and reference waveform received by the display monitor 20. In FIG. 3, the waveforms of "reference waveform A" and "reference waveform B" are shown along the time axis direction as reference waveforms. Furthermore, in FIG. 3, the time axis of the "directional intensity waveform" is shown as the time axis of the haptic sense. In the example shown in FIG. 3, the time axis of the haptic sense is displayed side by side with the time axes of "reference waveform A" and "reference waveform B" corresponding to the start and end. In other words, the time axis of the haptic sense is set to the same length as the time axes of "reference waveform A" and "reference waveform B".

[0047] The user can generate or edit the haptic sensation to be presented on a set time axis while referring to an image such as that shown in Fig. 3 displayed on the display monitor 20. For example, the user can specify the direction and strength of the haptic sensation to be presented on the time axis while referring to "reference waveform A" and "reference waveform B".

[0048] It is also possible to generate a haptic waveform based on a reference waveform. For example, it is also possible to generate a haptic waveform based on data obtained by filtering "reference waveform A" or "reference waveform B", or on a portion of the data of the reference waveform. It is also possible to edit the direction or intensity of the haptic waveform generated based on the reference waveform by a method described later. The generated or edited haptic waveform is displayed along the time axis direction as a "direction intensity waveform" in an image such as that shown in FIG. 3.

[0049] As another example, the time axis setting unit 201 can set the time axis or resolution of the haptic sense based on a user operation, without displaying or using a reference waveform.

[0050] The direction input unit 202 sets the direction in which the haptic sense is presented. The direction in which the haptic sense is presented is a direction based on the position of an object (e.g., the controller 21) to which the haptic sense is presented. For example, "presenting the haptic sense in the forward direction" means continuously presenting the haptic sense (i.e., the sensation that the object is being pulled forward, or a physical quantity such as a torque that pulls the object forward) in the forward direction based on the position of the object for a set period. The period is set along the time axis for presenting the haptic sense, as described later. The direction input unit 202 can set the direction in which the haptic sense is continuously presented to any direction, such as up / down, left / right, front / back, or diagonal.

[0051] The direction input unit 202 sets the direction of the haptic sense to be presented at a specified timing based on, for example, a user input. That is, the set haptic sense direction (setting information) is associated with time on a timeline (time axis) of the haptic sense to be presented from the target object. The direction input unit 202 performs processing for transmitting necessary information from the signal generating device 1 to the display monitor 20 in order to display on the display monitor 20 a user interface for setting the haptic sense direction based on the user input.

[0052] FIG. 4 shows an example of an image displayed on the display monitor 20 as a user interface for setting the direction of a haptic sensation based on a user input. For example, when a user selects a desired position on the image shown in FIG. 4 via an operation unit, a haptic sensation in a direction corresponding to the selected position is set. For example, when an area displayed as "forward (+)" is selected, a haptic sensation in a forward direction is set to be presented, and when an area displayed as "downward (-)" is selected, a haptic sensation in a downward direction is set to be presented. Also, a haptic sensation in a set direction is set to be presented at a timing according to a position on a time axis (i.e., a position on a horizontal axis) corresponding to the selected position. That is, a user can set a direction of a haptic sensation and a timing for presenting a haptic sensation in that direction via a user interface as shown in FIG. 4. As a result, according to this embodiment, a haptic sensation to be presented can be set more intuitively.

[0053] The intensity input unit 203 sets the intensity of the haptic sense to be presented. The intensity input unit 203 sets the intensity of the haptic sense to be presented at a specified timing, for example, based on a user input. That is, the set intensity of the haptic sense (setting information) is associated with the time on the timeline of the haptic sense to be presented from the target object. The intensity input unit 203 performs processing for transmitting necessary information from the signal generating device 1 to the display monitor 20 in order to display on the display monitor 20 a user interface for setting the intensity of the haptic sense based on the user input.

[0054] FIG. 5 shows an example of an image displayed on the display monitor 20 as a user interface for setting the intensity of the haptic sense based on a user input. The waveform shown in the image of FIG. 5 indicates the intensity of the haptic sense presented at each timing on the time axis. The horizontal axis of the image of FIG. 5 is the time axis of the presented haptic sense, and the vertical axis is the intensity of the presented haptic sense. For example, a user can change the shape of the waveform shown in FIG. 5 to a desired shape via the operation unit, thereby setting a haptic sense of intensity according to the shape of the changed waveform. That is, a user can set the intensity of the haptic sense and the timing of presenting the haptic sense of that intensity via a user interface as shown in FIG. 5. As a result, according to this embodiment, the haptic sense to be presented can be set more intuitively.

[0055] The parameter generating unit 204 generates parameters of the haptic sense to be presented based on the information set by the direction input unit 202 and the information set by the intensity input unit 203. More specifically, the parameter generating unit 204 generates parameters of the direction and intensity of the haptic sense associated with the timing of presenting the haptic sense. For example, the direction of presenting the haptic sense is indicated by a sign (e.g., "+" (plus) or "-" (minus)), the intensity of the haptic sense to be presented is indicated by a numerical value, and parameters indicating the direction and intensity of the haptic sense can be generated by combining the sign and the numerical value. Furthermore, the parameter generating unit 204 can generate parameters of the direction and intensity of the haptic sense associated with the timing of presenting the haptic sense by associating the parameters indicating the direction and intensity of the haptic sense with parameters indicating the timing.

[0056] According to this embodiment, as described with reference to Figures 4 and 5, information (setting information) on the direction and intensity of the haptic sense set along the time axis may include information based on user input via a user interface, and the parameter generation unit 204 may generate parameters of the haptic sense to be presented based on the setting information.

[0057] In the present embodiment, the method of setting the direction of the haptic sense by the direction input unit 202 and the intensity of the haptic sense by the intensity input unit 203 has been described, but the present invention is not limited to this. For example, the parameter generating unit 204 may simultaneously set the direction and intensity of the haptic sense to be presented at a predetermined timing and generate parameters based on the settings. In this case, the parameter generating unit 204 performs a process of transmitting necessary information from the signal generating device 1 to the display monitor 20 in order to display on the display monitor 20 a user interface for setting the direction and intensity of the haptic sense based on a user input.

[0058] FIG. 6 shows an example of an image displayed on the display monitor 20 as a user interface for setting the direction and intensity of the haptic sensation based on a user input. The horizontal axis of the image in FIG. 6 is the time axis of the presented haptic sensation, and the vertical axis is the intensity of the presented haptic sensation. The waveform shown in the image in FIG. 6 shows the direction and intensity of the haptic sensation presented at each timing on the time axis. In the image in FIG. 6, the upper (plus side) area which is the "forward intensity" area indicates a forward haptic sensation, and the higher the position, the stronger the intensity of the haptic sensation. The lower (minus side) area which is the "backward intensity" area indicates a backward haptic sensation, and the lower the position, the stronger the intensity of the haptic sensation. That is, in the example of FIG. 6, during the period when the waveform is in the "forward intensity" area, it is set so that the forward haptic sensation is continuously presented with an intensity according to the vertical axis position, and during the period when the waveform is in the "backward intensity" area, it is set so that the backward haptic sensation is continuously presented with an intensity according to the vertical axis position.

[0059] For example, the user can deform the shape of a waveform image as shown in Fig. 6 into a desired shape via the operation unit, thereby setting a haptic sense of a direction and intensity according to the shape of the deformed waveform image. The parameter generating unit 204 generates parameters of the haptic sense to be presented according to the shape of the deformed waveform image. That is, the user can set information (setting information) including the direction and intensity of the haptic sense, as well as the timing for presenting the haptic sense of that direction and intensity, based on image operations via a user interface as shown in Fig. 6. As a result, according to this embodiment, the haptic sense to be presented can be set more intuitively.

[0060] As another example of a method for setting the direction and intensity of the haptic sense via a user interface, the user may specify curve points (curve points) of a waveform representing the direction, intensity, and timing of the haptic sense to be presented via an operation unit, and the parameter generating unit 204 may generate a waveform that connects (complements) the specified curve points. In addition, the steepness or steepness of the curve (curve radius) of the curve points may be set according to user input. The parameter generating unit 204 generates parameters of the haptic sense to be presented based on the generated waveform. By making it possible to generate a waveform by specifying curve points in this way, it is possible to minimize the operation input by the user and generate haptic parameters in a short time.

[0061] In the example described with reference to Fig. 6, two directions, the forward direction and the backward direction, can be set, but this is not limiting. The parameter generating unit 204 may set the direction in which the haptic sense is presented to any direction in three dimensions, such as up / down, left / right, front / back, or diagonal, via a user interface, and generate parameters based on the setting. In this case, the upper side (plus side) and the lower side (minus side) of the user interface shown in Fig. 6 can be associated with the direction in which the haptic sense is presented, depending on user operations, etc. Also, the number of settable directions may be one direction, or three or more directions, instead of two directions.

[0062] With reference to FIG. 7, an example of a method for associating each of the upper side (plus side) and the lower side (minus side) of the user interface shown in FIG. 6 with a direction for presenting a haptic sense in response to a user operation will be described. FIG. 7 shows an example of a user interface displayed on the display monitor 20. FIG. 7 shows an image of a controller presenting a haptic sense and an image of an arrow. In the example shown in FIG. 7, one of the two arrows shows that the presentation of a haptic sense in the forward direction is associated with the plus side, and the presentation of a haptic sense in the backward direction is associated with the minus side. A user can change the association between each of the upper side (plus side) and the lower side (minus side) of the user interface shown in FIG. 6 with a direction for presenting a haptic sense by changing the "forward direction" to the "upward direction" or the "backward direction" to the "downward direction" or changing the position of the arrow image, etc., via the operation unit.

[0063] In the above, a description has been given of generating a waveform that indicates the direction and intensity of a haptic sensation on a time axis by manipulating an image or specifying curve points via a user interface, but the method of generating the waveform is not limited to this.

[0064] For example, the parameter generating unit 204 may automatically (based on a predetermined algorithm) generate a waveform indicating the direction and intensity of the haptic sensation to be presented on the time axis. As an example of a method for automatically generating a waveform, the parameter generating unit 204 may generate information (setting information) for setting the direction and intensity of the haptic sensation to be presented along the time axis based on audio data or video data (or a reference waveform) as described above.

[0065] In this case, the parameter generating unit 204 may generate a waveform indicating the direction and intensity of the haptic sense on a time axis (i.e., setting information of the direction and intensity of the haptic sense along the time axis) based on the data after filtering of the audio data or video data. For example, data of a predetermined frequency component may be extracted by filtering of the audio data or video data, and a waveform may be generated based on the extracted data.

[0066] The parameter generating unit 204 may also extract features of the audio data or video data and generate a waveform indicating the direction and intensity of the haptic sense on a time axis (i.e., setting information of the direction and intensity of the haptic sense along the time axis) based on the features. A pattern recognition technique may be employed to extract the features.

[0067] In addition, artificial intelligence techniques such as machine learning or deep learning may be adopted to generate setting information of the haptic direction and intensity along the time axis. For example, haptic information (including setting information of the haptic direction and intensity along the time axis) may be specified (generated) from audio data or video data using a model trained with feature quantities extracted from audio data or video data as explanatory variables and haptic information as a target variable. The trained model may be stored in a storage unit (not shown) of the signal generating device 1, or may be acquired from outside the signal generating device 1. The specification (generation) of haptic information using the trained model may be performed by the parameter generating unit 204.

[0068] In addition, in order to automatically set the direction and intensity of the haptic sense along the time axis, setting information or a template stored in a database may be selected by a user input, and the selected setting information or template may be joined together. Also, the direction and intensity of the haptic sense along the time axis may be automatically set based on cooperation with an input device connected to the signal generating device 1 (for example, input according to the movement of a motion capture device or a mouse). Also, the direction and intensity of the haptic sense along the time axis may be automatically set by cooperation with a remote device using a remote network system.

[0069] It is also possible to manually set the direction and intensity of the haptic sense along the time axis. For example, the parameter generating unit 204 may obtain information on the direction and intensity of the haptic sense associated with time information based on a user input (e.g., sequence data of parameters indicating time, direction, and intensity), and generate parameters based on the information.

[0070] Returning to the explanation of FIG. 2, basic waveform registration unit 205 stores and registers a basic waveform signal of vibration of a haptic element (e.g., haptic element 25) for presenting a haptic sensation in a predetermined direction from an object. Since each haptic element has different characteristics, a different haptic sensation may be presented for each haptic element even if the haptic elements are vibrated with the same basic waveform signal. Therefore, in order to present a haptic sensation in the same direction, basic waveform registration unit 205 may register a different basic waveform for each haptic element, or may register one basic waveform so that it can be changed depending on the haptic element.

[0071] The basic waveform signal of the vibration of the haptic element 25 for presenting a haptic sensation in a predetermined direction from an object includes a waveform signal of a vibration for continuously presenting a haptic sensation in any direction in three dimensions, such as up-down, left-right, front-back, and diagonal directions. The basic waveform signal of the vibration of the haptic element 25 for continuously presenting a haptic sensation in a predetermined direction is generated by any method. For example, the basic waveform signal may be generated based on the method described in Japanese Patent No. 4111278.

[0072] The signal generating unit 206 generates a waveform signal for presenting a haptic sensation based on the setting information (or generated parameters) received by an input unit (the direction input unit 202, the intensity input unit 203, or the parameter generating unit 204). For example, the signal generating unit 206 generates a waveform signal for presenting a haptic sensation based on the setting information (or generated parameters) received by the input unit, based on a combination of basic waveform signals for presenting a haptic sensation in a predetermined direction from an object. More specifically, the signal generating unit 206 refers to the basic waveform registration unit 205, specifies one or more basic waveform signals necessary for presenting a haptic sensation based on the setting information, etc. received by the input unit, and generates a waveform signal for presenting a haptic sensation based on the setting information, etc. received by the input unit by combining the specified basic waveform signals.

[0073] A method for generating a waveform signal for presenting a haptic sense with a predetermined intensity in the forward direction (+ direction) or backward direction (- direction) by the signal generating unit 206 will be described in more detail with reference to Fig. 8 and Fig. 9. As shown in Fig. 8, the signal generating unit 206 acquires setting information for the haptic sense (information on directional intensity at each time) received by the input unit. Fig. 9 (3) conceptually shows an example of a waveform showing the direction and intensity of the haptic sense on the time axis based on the setting information.

[0074] Next, the signal generating unit 206 refers to the basic waveform registering unit 205 and acquires one or more signals of a single forward waveform and a single backward waveform as basic waveform signals necessary for presenting a haptic sensation based on the above setting information. Fig. 9 (1) conceptually shows an example of a single forward waveform, and Fig. 9 (2) conceptually shows an example of a single backward waveform.

[0075] Next, the signal generating unit 206 connects the corresponding forward direction single waveforms (basic waveform signals) acquired above according to the set time axis in order to generate a waveform signal for a period in which the setting information indicates the forward direction (+ direction). Also, the signal generating unit 206 connects the corresponding backward direction single waveforms (basic waveform signals) acquired above according to the set time axis in order to generate a waveform signal for a period in which the setting information indicates the backward direction (- direction). (4) of FIG. 9 conceptually shows an example of the connected basic waveform signal. In the period in which the forward direction single waveforms are connected, a waveform signal for continuously presenting a forward haptic sensation is generated, and in the period in which the backward direction single waveforms are connected, a waveform signal for continuously presenting a backward haptic sensation is generated.

[0076] Next, signal generating unit 206 multiplies the fundamental waveform signals that have been joined together by the intensity associated with the time axis indicated by the setting information (i.e., sets the intensity). (5) in Fig. 9 conceptually shows an example of a waveform signal obtained by setting the intensity to the fundamental waveform signals that have been joined together.

[0077] Through the above processing, a waveform signal can be generated to present a haptic sensation in the manner intended by the user (intended direction, strength, and time axis) of shaking, based on intuitive settings such as forward and backward directions.

[0078] <Example> Specific examples of the present embodiment are described below. The present embodiment can be implemented in various ways, and the examples are not limited to the following examples.

[0079] Example 1 A first embodiment will be described with reference to FIG. 10. In the first embodiment, the game development system 3 includes at least one computer 11a and a controller 21. The computer 11a is a specific example of the computer 11. The game development system 3 may include other configurations described above. In this embodiment and other embodiments, the game development system 3 is applicable not only as a system for game development but also as a system for game execution. The computer 11a communicates with the controller 21. The communication between the computer 11a and the controller 21 is performed by any method, and may be either wired communication or wireless communication, or may be communication via a network such as the Internet. The same applies to the communication between the computer 11 and the controller 21 and the communication between multiple computers 11 in other embodiments.

[0080] In the first embodiment, the computer 11a includes a parameter input unit 111, a signal generation unit 112, and a device output unit 113 as functional components.

[0081] The parameter input unit 111 inputs parameters of the haptic sense to be presented to the signal generation unit 112. The processing by the parameter input unit 111 may correspond to the processing by at least a part of the time axis setting unit 201, the direction input unit 202, the intensity input unit 203, the parameter generation unit 204, and the basic waveform registration unit 205 described with reference to FIG.

[0082] The parameters input from the parameter input unit 111 to the signal generation unit 112 may be haptic parameters related to direction or strength based on features extracted from audio data or video data. Furthermore, the user's movement, which is specified by an operation on the controller 21 or by motion capture, may be detected with respect to a time axis, and haptic parameters related to direction or strength generated to correspond to the time axis based on the detection result may be input from the parameter input unit 111 to the signal generation unit 112. The parameters input to the signal generation unit 112 may be generated randomly in association with the time axis. The movement of a character animation appearing in a game may be detected, and haptic parameters related to direction or strength generated to correspond to the time axis based on the detection result may be input from the parameter input unit 111 to the signal generation unit 112. The parameters input to the signal generation unit 112 may be haptic parameters registered in advance in association with the time axis.

[0083] The signal generating unit 112 generates a waveform signal for presenting a haptic sensation based on the parameters input from the parameter input unit 111, and outputs the waveform signal to the device output unit 113. The processing by the signal generating unit 112 may correspond to the processing by the signal generating unit 206 described above.

[0084] The device output unit 113 is a communication interface that transmits the waveform signal output from the signal generating unit 112 to the controller 21. The device output unit 113 is realized by, for example, the communication interface 17 described above.

[0085] Example 2 A second embodiment will be described with reference to FIG. 11. In the second embodiment, the functions of the computer 11 described in the first embodiment are realized by two computers 11. In the second embodiment, the game development system 3 includes a computer 11b, a computer 11c, and a controller 21. The computers 11b and 11c are specific examples of the controller 11. The computers 11b and 11c each include a hardware configuration similar to at least a part of the hardware configuration of the controller 11 described above. The computer 11b communicates between the computer 11c and the controller 21. The communication in this embodiment is performed by any method, and may be either wired communication or wireless communication, or may be communication via a network such as the Internet.

[0086] As shown in FIG. 11, the computer 11b includes a parameter input unit 111 and a device output unit 113 as functional components. The computer 11c includes a signal generating unit 112 as functional components. That is, in this embodiment, parameters are input from the computer 11b to the computer 11c, and signal generation is performed by the computer 11c. The generated signal is transmitted from the computer 11c to the computer 11b. Communication between the computer 11b and the computer 11c is realized via a receiving interface or a transmitting interface provided in the computer 11b and the computer 11c. The processing by the parameter input unit 111, the signal generating unit 112, and the device output unit 113 shown in FIG. 11 is the same as that in the first embodiment, and therefore will not be described here.

[0087] Example 3 A third embodiment will be described with reference to FIG. 12. In the third embodiment, the functions of the computer 11 described in the first embodiment are realized by three computers 11. In the third embodiment, the game development system 3 includes a computer 11d, a computer 11e, a computer 11f, and a controller 21. The computers 11d, 11e, and 11f are specific examples of the controller 11. The computers 11d, 11e, and 11f each include a hardware configuration similar to at least a part of the hardware configuration of the controller 11 described above. The computer 11d communicates with the computer 11e. The computer 11e communicates with the computer 11f. The computer 11f communicates with the controller 21. The communication in this embodiment is performed by any method, and may be either wired communication or wireless communication, or may be communication via a network such as the Internet.

[0088] As shown in FIG. 12, the computer 11d includes a parameter input unit 111 as a functional configuration. The computer 11e includes a signal generating unit 112 as a functional configuration. The computer 11f includes a device output unit 113 as a functional configuration. That is, in this embodiment, parameters are input from the computer 11d to the computer 11e, and signal generation is performed by the computer 11e. The generated signal is transmitted from the computer 11e to the computer 11f. Communication between the multiple computers 11 is realized via a receiving interface or a transmitting interface that each computer 11 includes. The processing by the parameter input unit 111, the signal generating unit 112, and the device output unit 113 shown in FIG. 12 is the same as that of the first embodiment, and therefore will not be described here.

[0089] Example 4 A fourth embodiment will be described with reference to FIG. 13. As shown in FIG. 13, the game development system 3 includes a computer 11g and a storage medium 31. In the fourth embodiment, a waveform signal for presenting a haptic sense generated by any of the methods of the first to third embodiments is stored in the storage medium 31 and output from the storage medium 31 to the computer 11g. The storage medium 31 may be configured in any manner, and may be, for example, a non-volatile storage device capable of reading and writing, such as an HDD or an SSD, or another storage medium, such as a portable storage medium, such as a memory card or a read-only CD-ROM (or DVD-ROM). Communication for outputting a waveform signal from the storage medium 31 to the computer 11g is performed by any manner. The computer 11g includes a device output unit 113 as a functional configuration. The process by the device output unit 113 is the same as that of the first embodiment, and therefore will not be described here.

[0090] <Modification> In this embodiment, a process for generating a waveform signal for presenting a tactile sensation based on information (setting information) set in accordance with image operations or automatic processing via a user interface has been described, but the process method for generating a waveform signal is not limited to this.

[0091] For example, the setting information (the direction and intensity of the haptic sense along the time axis) may be generated based on character input via a user interface. In this case, a user programming a game program can character input a function or method for generating the setting information via an operation unit (by typing, mouse operation, etc.), and further character input setting information for the direction and intensity of the haptic sense along the time axis as a parameter of the function or method. After that, the signal generating unit 206 may generate a waveform signal for presenting the haptic sense based on the character input.

[0092] The above-described embodiments are intended to facilitate understanding of the present invention, and are not intended to limit the present invention. The present invention may be modified / improved without departing from the spirit thereof, and equivalents are also included in the present invention. That is, those in which a person skilled in the art appropriately changes the design of each embodiment are also included in the scope of the present invention as long as they have the characteristics of the present invention. For example, the elements and their arrangements, materials, conditions, shapes, sizes, etc. of each embodiment are not limited to those exemplified, and can be appropriately changed. In addition, each embodiment is an example, and it goes without saying that partial replacement or combination of the configurations shown in different embodiments is possible, and these are also included in the scope of the present invention as long as they include the characteristics of the present invention. [Explanation of symbols]

[0093] 1...Signal generation device 3. Game Development System 11. Computer 19…Speaker 20...Display monitor 21…Controller 201...Time axis setting section 202...Direction input section 203...Intensity input section 204...parameter generation unit 205...Basic waveform registration section 206...signal generating unit

Claims

1. an input unit that receives setting information including first information indicating a first direction and a second direction of a haptic sense to be presented from an object; a generation unit that generates a waveform signal for presenting a haptic sensation based on the setting information received by the input unit; Equipped with the haptic sense is a sensation that the object is pulled in the first direction or the second direction, or a physical quantity that pulls the object in the first direction or the second direction; The setting information further includes second information indicating an intensity of a haptic sensation to be presented from the object, the first direction, the second direction, and the intensity are associated with time on a timeline of a haptic sensation presented from the object; an image generating unit that generates an image including one waveform indicating the first direction and the second direction, which are respectively a plus side and a minus side, and which indicates the first direction, the second direction, and the intensity of the haptic sense presented at each timing on the timeline; The signal generating device further comprises an operation unit for editing the one waveform.

2. An input unit that accepts setting information including first information indicating a first direction and a second direction of a haptic sensation presented from an object; a generation unit that generates a waveform signal for presenting a haptic sensation based on the setting information received by the input unit; Equipped with the haptic sense is a sensation that the object is pulled in the first direction or the second direction, or a physical quantity that pulls the object in the first direction or the second direction; the first direction and the second direction are associated with times on a timeline of a haptic sensation presented from the object; an image generating unit that generates an image in which the first direction and the second direction are a plus side and a minus side, respectively, and the first direction and the second direction are associated with time on the timeline; the signal generating device further comprising an operation unit for editing a correspondence relationship between the first direction and the second direction and time on the timeline.

3. An input unit that accepts setting information including first information indicating a first direction and a second direction of a haptic sensation presented from an object; a generation unit that generates a waveform signal for presenting a haptic sensation based on the setting information received by the input unit; Equipped with the haptic sense is a sensation that the object is pulled in the first direction or the second direction, or a physical quantity that pulls the object in the first direction or the second direction; A signal generating device, wherein the first information is generated based on character input representing a function or method via a user interface.

4. The operation unit accepts an operation to change the first direction to a third direction, The signal generating device of claim 1, wherein the image generating unit generates the image including the one waveform indicating the third direction and the second direction as the positive side and the negative side, respectively, and the third direction and the second direction and the intensity of the tactile sensation presented at each timing on the timeline.

5. The operation unit accepts an operation to change the first direction to a third direction, the image generating unit generates the image in which the third direction and the second direction are the plus side and the minus side, respectively, and the third direction and the second direction are associated with time on the timeline, The signal generating device according to claim 2 , wherein the operation unit accepts an operation for editing a correspondence relationship between the third direction and the second direction and time on the timeline.

6. The signal generating device comprises: The signal generating device according to claim 1 , further comprising a time axis setting unit that sets the timeline based on a first waveform that changes over time.

7. The signal generating device according to claim 6 , wherein the first waveform is generated based on audio data or video data.

8. 5. The signal generating device according to claim 1, wherein the one waveform is generated based on audio data or video data.

9. 5. The signal generating device according to claim 1, wherein the one waveform is generated based on filtered audio data or filtered video data.

10. 5. The signal generating device according to claim 1, wherein the one waveform is generated based on a feature extracted from audio data or video data.

11. 5. The signal generating device according to claim 1, wherein the one waveform is generated using a model trained with features extracted from audio data or video data as explanatory variables and a haptic sense as a target variable.

12. The signal generating device of claim 1 or 4, wherein the generation unit generates a waveform signal for presenting a tactile sensation based on the first information and the second information received by the input unit based on a combination of basic waveform signals for presenting a tactile sensation in a predetermined direction from the object according to the first direction, the second direction, and the intensity.

13. The signal generating device according to claim 1 , wherein each of the first direction and the second direction is an arbitrary direction based on a position of the object.

14. receiving setting information including first information indicating a first direction and a second direction of a haptic sense to be presented from an object; generating a waveform signal for presenting a haptic sensation based on the received setting information; Including, the haptic sense is a sensation that the object is pulled in the first direction or the second direction, or a physical quantity that pulls the object in the first direction or the second direction; The setting information further includes second information indicating an intensity of a haptic sensation to be presented from the object, the first direction, the second direction, and the intensity are associated with time on a timeline of a haptic sensation presented from the object; generating an image including one waveform indicating the first direction and the second direction being a positive side and a negative side, respectively, and indicating the first direction, the second direction, and the intensity of the haptic sense presented at each timing on the timeline; and subjecting said one waveform to an operation for editing said one waveform.

15. Computer, an input unit that receives setting information including first information indicating a first direction and a second direction of a haptic sense to be presented from an object; a generation unit that generates a waveform signal for presenting a haptic sensation based on the setting information received by the input unit; It is a program to function as the haptic sense is a sensation that the object is pulled in the first direction or the second direction, or a physical quantity that pulls the object in the first direction or the second direction; The setting information further includes second information indicating an intensity of a haptic sensation to be presented from the object, the first direction, the second direction, and the intensity are associated with time on a timeline of a haptic sensation presented from the object; The computer, an image generating unit that generates an image including one waveform indicating the first direction and the second direction, which are respectively a plus side and a minus side, and which indicates the first direction, the second direction, and the intensity of the haptic sense presented at each timing on the timeline; an operation unit for editing the one waveform; A program that functions further as a

Images