INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING SYSTEM, INFORMATION PROCESSING METHOD AND PROGRAM

Abstract

An information processing device is provided, comprising: a detection unit that detects an angular velocity of a user's head; a prediction unit that predicts a final angle of movement of the user's head at the end of the head's rotation using an integral value of angular velocities detected from the beginning of the head's rotation until a first angular velocity detection time at which the angular velocity becomes a first velocity; and a correction unit that corrects the position of a virtual sound source presented to the user based on the final angle of movement.

Description

Technical field

[0001] The present disclosure relates to an information processing device, an information processing system, an information processing method and a program. State of the art

[0002] For example, a head-tracking technique has been proposed in which the movement of a user's head is tracked by an inertial measurement device built into headphones, and the direction of a sound emitted by the headphones is changed according to the direction of the head. Using such a technique can provide the user with a clearer and richer sense of presence. List of oppositions patent literature

[0003] PTL 1: JP 2010-147529 A Brief description of the technical problem

[0004] Prior art tracking techniques had a limitation in providing a richer sense of presence for a user, as factors affecting the sense of presence were not addressed.

[0005] Therefore, the present disclosure proposes an information processing device, an information processing system, an information processing method and a program that can provide a user with an audio experience that offers a higher sense of presence. Solution to the problem

[0006] According to the present disclosure, an information processing device is provided which includes: a detection section that detects an angular velocity of a user's head, a prediction section that predicts a final angle of movement of the head at a rotation end time of the head using an integral value of the angular velocity detected from a rotation start time of the head to a first angular velocity detection time at which the angular velocity reaches a first velocity, and a correction section that corrects a position of a virtual sound source presented to the user based on the final angle of movement.

[0007] According to the present disclosure, an information processing system is further provided which includes: a detection section designed to detect an angular velocity of a user's head, a prediction section designed to predict a final angle of movement of the head at a rotation end time of the head using an integral value of the angular velocity detected from a rotation start time of the head to a first angular velocity detection time at which the angular velocity reaches a first velocity, and a correction section designed to correct a position of a virtual sound source presented to the user based on the final angle of movement.

[0008] According to the present disclosure, an information processing method is further provided which causes an information processing device to perform the following: detecting an angular velocity of a user's head, predicting a final angle of movement of the head at a rotation end time of the head using an integral value of the angular velocity detected from a rotation start time of the head to a first angular velocity detection time at which the angular velocity reaches a first velocity, and correcting a position of a virtual sound source presented to the user based on the final angle of movement.

[0009] According to the present disclosure, a program is further provided which causes a computer to perform the following: a function for detecting an angular velocity of a user's head, a function for predicting a final angle of movement of the head at a rotation end time of the head using an integral value of the angular velocity detected from a rotation start time of the head to a first angular velocity detection time at which the angular velocity reaches a first velocity, and a function for correcting a position of a virtual sound source presented to the user based on the final angle of movement. Brief description of the drawings [ Fig. 1] Fig. Figure 1 is an explanatory diagram (Part 1) to explain some background that leads to a first embodiment of the present disclosure. [ Fig. 2] Fig. 2 is an explanatory diagram (Part 2) to explain the background that leads to the first embodiment of the present disclosure. [ Fig. 3] Fig. Figure 3 is an explanatory diagram illustrating an overview of an information processing system according to the first embodiment of the present disclosure. [ Fig. 4] Fig. Figure 4 is an explanatory diagram illustrating an appearance of a sound processing device according to the first embodiment of the present disclosure. [ Fig. 5] Fig. Figure 5 is a block diagram illustrating functional blocks of the sound processing device according to the first embodiment of the present disclosure. [ Fig. 6] Fig. Figure 6 is a block diagram illustrating functional blocks of an information processing device according to the first embodiment of the present disclosure. [ Fig. 7] Fig. Figure 7 is a flowchart showing the sequence of an information processing procedure according to the first embodiment of the present disclosure. [ Fig. 8] Fig. Figure 8 is an explanatory graph (part 1) showing the information processing procedure according to the first embodiment of the present disclosure. [ Fig. 9] Fig. Figure 9 is an explanatory graph (part 2) showing the information processing procedure according to the first embodiment of the present disclosure. [ Fig. 10] Fig. Figure 10 is an explanatory graph (part 3) showing the information processing procedure according to the first embodiment of the present disclosure. [ Fig. 11] Fig. Figure 11 is an explanatory graph (part 4) showing the information processing procedure according to the first embodiment of the present disclosure. [ Fig. 12] Fig. Figure 12 is an explanatory graph (part 5) showing the information processing procedure according to the first embodiment of the present disclosure. [ Fig. 13] Fig. Figure 13 is an explanatory graph (part 6) showing the information processing procedure according to the first embodiment of the present disclosure. [ Fig. 14] Fig. Figure 14 is an explanatory graph (part 7) showing the information processing procedure according to the first embodiment of the present disclosure. [ Fig. 15] Fig. Figure 15 is an explanatory graph showing an information processing method according to a second embodiment of the present disclosure. [ Fig. 16] Fig. Figure 16 is an explanatory diagram showing an information processing method according to a third embodiment of the present disclosure. [ Fig. 17] Fig. Figure 17 is an explanatory diagram illustrating an example of a hardware configuration of an information processing device 900 according to an embodiment of the present disclosure. Description of embodiments

[0010] Preferred embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be noted that in this document and the drawings, components with substantially the same functional configuration are designated by the same reference numeral, and any redundant description thereof is omitted. Furthermore, in this document and the drawings, a plurality of components with substantially the same or similar functional configurations can be distinguished from one another by adding different letters after the same reference numeral. However, if it is not necessary to distinguish the plurality of components with substantially the same or similar functional configurations, only the same reference numeral is assigned.

[0011] Furthermore, the drawings described below are intended to illustrate embodiments of the present disclosure and to facilitate understanding thereof, and the modes illustrated in the drawings may differ from the actual modes to facilitate understanding. Moreover, designs of devices illustrated in the drawings may be modified as appropriate, taking into account the following description and known techniques.

[0012] It should be noted that the description follows in the order below. 1. Background 2. First embodiment 2.1 Overview of the information processing system 2.2 Detailed configuration of the sound processing device 2.3 Detailed configuration of the information processing device 2.4 Information processing methods 4. Third embodiment 5. Summary 6. Hardware configuration 7th Supplement 1. Background

[0013] First, the background that led to the creation of the first embodiment of the present disclosure by the present inventors is described with reference to Figures 1 and 2. Fig. 1 and Fig. Figure 2 are explanatory diagrams to explain the background that leads to the first embodiment of the present disclosure.

[0014] With reference to Fig. 1 A description of the output of a tone is given to cause a user 80 listening using headphones 10a or the like to feel as if the position of a virtual sound source 82 is spatially fixed regardless of the direction of the user 80's head.

[0015] As on the left side of Fig. As illustrated in Figure 1, user 80, who is wearing headphones 10a and listening to the sound, feels as if the sound emitted by headphones 10a is being emitted from the front in front of user 80's face. That is, the virtual sound source 82 of the sound emitted by headphones 10a is positioned in front of user 80's face. It should be noted that in this document, a spatial position of the virtual sound source 82 that corresponds to the sound output from headphones 10a is referred to as a virtual sound source position.

[0016] If no special processing is performed on the audio to be output (in particular, no head tracking is performed) and the user's head rotates 80 degrees, the virtual sound source position also rotates with the user's head rotation, as in the middle of Fig. 1 illustrates.

[0017] Therefore, a head-tracking technique is applied to the headphones 10a. In this technique, the rotation angle of the user's head (80) is detected, and signal processing is performed on the audio data. This makes the user (80) perceive that the virtual sound source position is rotated by the detected angle in a direction opposite to the rotation of the user's head. The headphones (10a) then output a tone to the user (80) based on the processed audio data. In this way, the user (80) hears the tone as if the virtual sound source position were spatially fixed, as on the right side of Fig. 1 illustrates.

[0018] For example, such a head-tracking technique is applied to a sound recorded using a sound source, such as a surround sound source, which can cause listeners to perceive a sense of spatial extension. As a result, even if the user moves their head or body while listening to the sound through headphones, they will feel as if the sound source, such as a loudspeaker or musical instrument, is spatially fixed. That is, according to the head-tracking technique, it is possible to provide the user with an audio experience that offers a sense of presence.

[0019] Next, a system that enables head tracking will be described, with reference to Fig. 2 described. For example, it is assumed that the user 80 is wearing headphones 10a, which are equipped with an (not illustrated) inertial measuring device (sensor) compatible with a communication system, such as Bluetooth (registered trademark), and capable of detecting the posture and movement of the user 80's body. It is also assumed that the user 80 is watching a video on an information processing device (information processing terminal device), such as a smartphone 40a. Audio from the video is recorded using a surround sound source, and audio data from a (not illustrated) server or the like is received by the smartphone 40a. The received audio data is then transmitted to the headphones 10a, and the user 80 can enjoy stereophonic sound with a sense of presence using the headphones 10a.

[0020] In such a situation, when user 80 turns their head, the inertial measuring device (sensor) built into the headphones 10a detects the rotational movement of the head and determines the direction (angle) of user 80's head through arithmetic processing. Furthermore, the headphones 10a transmit the detected direction (angle) of user 80's head to the smartphone 40a using a communication system such as Bluetooth.

[0021] The smartphone 40a then processes the audio data recorded using the surround sound source, rotating the virtual sound source position of the audio data by an angle corresponding to the rotation of the user's head 80 in a direction opposite to the received direction of the user's head 80. Furthermore, the smartphone 40a mixes the audio data processed in this way with, for example, a 2-channel stereo signal and transmits the mixed audio data to the headphones 10a using a communication system such as Bluetooth.

[0022] Furthermore, the headphones 10a receive the audio data and output a tone to the user 80. As a result, the user 80 can determine the direction of the virtual sound source position of the tone by listening to it.

[0023] As can be seen from the processing flow described above, which is in Fig. As illustrated in Figure 2, many processes are carried out from the moment user 80's head begins to turn until user 80 hears the tone that reflects the head turn. Therefore, there is a delay between the turn start time, when user 80's head begins to turn, and the tone arrival time, when the tone reflecting the head turn reaches user 80's ears. The present inventors have conducted studies and found that this delay is approximately several hundred milliseconds. This delay causes a discrepancy between user 80's head turn and the virtual sound source position (the direction of the tone) perceived by user 80, and thus there is concern that this could also affect user 80's audio experience and impair their sense of presence.

[0024] Therefore, the present inventors have conducted intensive studies in light of such a situation and have created embodiments of the present disclosure that can reduce the delay time, i.e., reduce the deviation between the rotation of the user's head 80 and the virtual sound source position (the direction of the sound) perceived by the user 80. According to the embodiments of the present disclosure, it is possible to reduce the delay time, that is, to reduce the deviation described above and to provide the user 80 with an audio experience that offers a greater sense of presence. Details of the embodiments of the present disclosure created by the present inventors are described below in sequence. 2. First Implementation 2.1 Overview of the Information Processing System

[0025] First, an overview of an information processing system 1 according to a first embodiment of the present disclosure is given with reference to Fig. 3 described. Fig. Figure 1 is an explanatory diagram illustrating a schematic configuration of the information processing system 1 according to an embodiment of the present disclosure.

[0026] As in Fig. As illustrated in Figure 3, the information processing system 1 according to the present embodiment includes a sound processing device 10 and an information processing device 40, which are communicatively connected to each other via a communication network 70. It is noted that any system can be used as the communication system in the communication network 70, whether wired or wireless (for example, Bluetooth, WiFi (registered trademark), or the like), but it is desirable to use a wireless communication system, such as Bluetooth, which can maintain stable operation and has fewer limitations. Furthermore, the number of sound processing devices 10 and the number of information processing devices 40 included in the information processing system 1 are not each limited to one, as shown in Figure 3. Fig. Figure 3 illustrates the devices, and each can be a plurality. The following is an overview of each of the devices included in the information processing system 1 according to the present embodiment. (Sound processing device 10)

[0027] In the present embodiment, the sound processing device 10 can, for example, be the headphones 10a, which are worn on the head of the user 80 and can output acoustic signals to the left and right ear of the user 80. Furthermore, in the present embodiment, the sound processing device 10 is not limited to the headphones 1a and can, for example, be earphones, a hearing aid, a sound collector, a head-mounted display (HMD), or the like. It should be noted that details of the functional configuration of the sound processing device 10 will be described later. (Information processing device 40)

[0028] The information processing device 40 is an information processing terminal configured to communicate with the sound processing device 10. In the present embodiment, the information processing device 40 can be, for example, a device such as a smartphone, a tablet personal computer (PC), a mobile phone, or a notebook PC. Furthermore, in the present embodiment, the sound processing device 10 and the information processing device 40 can be configured as an integrated device. Additionally, in the present embodiment, the information processing device 40 can be a server or the like in a cloud and, in this case, can be composed of one or a plurality of devices. It should be noted that the functional configuration of the information processing device 40 will be described in detail later.

[0029] It is noted that in one embodiment of the present disclosure, the functional configuration of the information processing system 1 does not depend on the one described in Fig. The mode illustrated in section 3 is limited. For example, the information processing system 1 may include another communication device, such as a relay device, which is used when information is transferred between the sound processing device 10 and the information processing device 40. 2.2 Detailed configuration of the sound processing device

[0030] Next, the sound processing device 10 according to the present embodiment will be described with reference to Fig. 4 and Fig. 5 described in detail. Fig. Figure 4 is an explanatory diagram illustrating an appearance of the sound processing device 10 according to the present embodiment, and Fig. Figure 5 is a block diagram illustrating functional blocks of the sound processing device 10 according to the present embodiment.

[0031] The appearance of the sound processing device 10 according to the present embodiment is described with reference to Fig. 4 described. Here, it is assumed that the sound processing device 10 according to the present embodiment is a headphone.

[0032] As in Fig. As illustrated in Figure 4, the sound processing device 10 includes, for example, sliders 3, suspension elements 4L, 4R, housings 5L, 5R, ear pads 6L, 6R, and a headband 7. For example, a driver unit (not shown) and the like are provided within the housings 5L, 5R. When the sound processing device 10 is used, the user 80 wears the headband 7 on the top of their head and brings the ear pads 6L, 6R into contact with their ears. Thus, the user 80 can hear tones emitted by the housings 5L, 5R.

[0033] The sliders 3 are provided at both ends of the headband 7. The suspension element (suspension element 4L or 4R) is then attached to a connecting section at one end of the slider 3. The sliders 3 are attached to the two ends of the headband 7 and are each designed to be slidable along a guide element (not shown), the axis of which coincides with the central axis of the headband 7. By sliding the sliders 3 along the guide elements, the suspension elements 4L and 4R can be moved away from or closer to the headband 7. When the sound processing device 10 is worn, the positions of the sliders 3 are adjusted according to the size of the user's head 80, the distance between each ear and the top of the head, and similar factors. In this way, the user 80 can comfortably wear the sound processing device 10 according to their physical characteristics and preferences.On the other hand, when the sound processing device 10 is not in use, the slides 3 are brought into a retracted state, thereby saving storage space for the sound processing device 10.

[0034] The suspension elements 4L, 4R are provided at the tips of the slides 3 via the (not shown) connecting sections and can rotatably support the housings 5L, 5R.

[0035] Enclosures 5L and 5R have an internal storage space and house a driver unit (not shown) that converts an electrical signal into a sound wave and outputs the sound wave, and the like. Enclosures 5L and 5R are formed, for example, by using synthetic resin, such as plastic. Enclosures 5L and 5R prevent air from flowing in and out between the front and rear surfaces of a loudspeaker diaphragm. Enclosures 5L and 5R can also each serve to efficiently transmit acoustic radiation from the front surface of the diaphragm to the eardrum of the user 80 by sealing or partially sealing a space from the front surface of the loudspeaker diaphragm to the eardrum of the user 80, with the enclosure having an acoustic resistance or incorporating a channel or the like.Furthermore, the loudspeaker and similar components are housed in the driver unit.

[0036] The ear pads 6L, 6R are each provided on the surfaces of the housings 5L, 5R on the side facing the user's head. The ear pads 6L, 6R are positioned between the housing 5L and the user's head, and between the housing 5R and the user's head, respectively, and thus act as a buffer element between the housing 5L, 5R and the user's head. That is, when the user wears the sound processing device 10, the ear pads 6L, 6R prevent the housings 5L, 5R, which are made of a hard, non-deformable material, from coming into direct contact with the user's ears and the sides of the user's head, thereby preventing discomfort or pain. The ear pads 6L, 6R are covered with a synthetic leather or a cloth made of sponge or the like.

[0037] As in Fig. As illustrated in Figure 4, the headband 7 is formed in a curved shape along the head of the user 80 and can support the entire sound processing device 10 by contacting the top of the user's head when worn. The headband 7 is made of a synthetic resin, such as a plastic, a metal, or the like, and exhibits flexibility due to a predetermined stiffness and elasticity. It is noted that in the present embodiment, rubber or the like can be provided as a buffer material on a section of the inner surface of the headband 7 that comes into contact with the top of the user's head 80. Furthermore, a control unit (not illustrated) that houses a communication unit 30 and the like, as described later, can be provided within the housings 5L, 5R, or the like.Furthermore, in the present embodiment, the sound processing device 10 can be provided with a hinge (not shown) so that the sound processing device 10 can be folded in the middle when it is transported.

[0038] Next, the functional blocks of the sound processing device 10 according to the present embodiment will be described with reference to Fig. 5 described. As in Fig. As illustrated in Figure 5, the sound processing device 10 mainly comprises a gyroscope 12 and an accelerometer 14 (inertial measuring device), an output section 16, a sound data acquisition section 20, a sound output section 22, and the communication unit 30. Details of each of the functional blocks included in the sound processing device 10 according to the present embodiment are described below in sequence. (Gyrosensor 12)

[0039] The gyro sensor (angular velocity sensor) 12 is an inertial sensor that acquires angular velocity as acquisition data. In particular, the gyro sensor 12 consists, for example, of three sensors for corresponding three axes and can measure angular velocities of rotation about a yaw axis (rotation about a Z-axis perpendicular to a horizontal plane), rotation about a roll axis (rotation about an X-axis on the horizontal plane, when a line of sight direction of the user 80, who is carrying the sound processing device 10, is defined as the X-axis), and rotation about a pitch axis (rotation about a Y-axis on the horizontal plane, when an axis that perpendicularly intersects the X-axis on the horizontal plane is defined as the Y-axis).It should be noted that in the present embodiment, the gyro sensor 12 may include at least one sensor that measures the angular velocity of rotation about the yaw axis, and the sensor may measure the angular velocity about the upright axis of the user's head 80. The acquisition data acquired by the gyro sensor 12 is output to the output section 16 described later. For example, the gyro sensor 12 may be provided in the control unit (not illustrated), which is housed within the enclosure 5L, 5R, or the like. (Accelerometer 14)

[0040] The accelerometer 14 is an inertial sensor that detects acceleration as acquisition data. The acquisition data detected by the accelerometer 14 is output to the output section 16 described later. For example, the accelerometer 14 may be provided in the control unit (not illustrated) or the like, which is housed within the enclosure 5L, 5R, or the like. Furthermore, in the present embodiment, the accelerometer 14 does not necessarily need to be provided. Additionally, in the present embodiment, a sensor of a type different from the gyroscope 12 and the accelerometer 14 may be used. For example, in the present embodiment, a biological sensor (not illustrated) or an imaging sensor (not illustrated), such as a camera, may optionally be used. (Distribution section 16)

[0041] The output section 16 can perform signal conversion processing, such as digital signal conversion, on the acquisition data from the gyro sensor 12 or the accelerometer 14 and transmit the acquisition data via the communication unit 30 to the information processing device 40. (Sound data acquisition section 20)

[0042] The audio data acquisition section 20 performs predetermined signal processing on the audio data (acoustic signals) acquired by the information processing device 40 via the communication unit 30 described later, and outputs the processed audio data to the audio output section 22 described later. The audio data acquisition section 20 can, for example, consist of a memory and a processor, including hardware such as a digital signal processor (DSP). (Audio output section 22)

[0043] The audio output section 22 performs signal conversion processing, such as analog signal conversion, on the audio data from the audio data acquisition section 20 and outputs the analog signal to the (not illustrated) driver unit within the housing 5L, 5R. It is noted that, since the audio data includes audio data for the right ear and audio data for the left ear, the audio output section 22 outputs the corresponding audio data to the loudspeaker of the driver unit within the corresponding housing 5L, 5R. (Communication Unit 30)

[0044] The communication unit 30 can communicate with the information processing device 40 via the communication network 70 according to a predetermined communication standard. It is assumed that the predetermined communication standard is, for example, Bluetooth, WiFi, or the like. Furthermore, the communication unit 30 can, for example, consist of a communication module.

[0045] Although not illustrated, the sound processing device 10 may also include a control unit (not illustrated) that controls each of its functional blocks. The control unit may be located within the enclosure 5L, 5R, or the like. The control unit may, for example, consist of a memory and a processor with hardware such as a central processing unit (CPU) or a digital signal processor (DSP). The sound processing device 10 may also include a battery that supplies electrical power to each of its functional blocks. The battery may, for example, be a rechargeable secondary battery, such as a lithium-ion battery. Furthermore, the battery may be rechargeable by electrical power supplied by a charger via a connecting section (not illustrated).

[0046] It is noted that in the present embodiment, the appearance and functional block configuration of the sound processing device 10 do not correspond to those described in Fig. 4 and Fig. The 5 illustrated modes are limited. In the present embodiment, the sound processing device 10 can, for example, be wireless earphones or the like. 2.3 Detailed configuration of the information processing device

[0047] Next, the information processing device 40 according to the present embodiment will be described with reference to Fig. 6 described in detail. Fig. 6 is a block diagram illustrating functional blocks of the information processing device according to the present embodiment. As shown in Fig. As illustrated in Figure 6, the information processing device 40 mainly comprises a head posture prediction section (capture section, prediction section) 42, an audio data acquisition section 50, an audio data correction section (correction section) 52, an audio data output section (output section) 54, a communication unit 60, and a storage section 62. Details of each of the functional blocks included in the information processing device 40 according to the present embodiment are described below in sequence. (Head posture prediction section 42)

[0048] The head posture prediction section 42 receives the acquisition data from the gyroscope 12 from the sound processing device 10 via the communication unit 60 (described later) and detects the rotation angle of the user's head 80 from the angular velocity of the user's head 80, which is obtained from the acquisition data. For example, the rotation angle of the user's head 80 at the current time can be obtained by integrating the angular velocity recorded from the beginning of the head rotation until the current time. The head posture prediction section 42 then outputs the detected angular velocity and the rotation angle of the user's head 80 to the sound data correction section 52 and the storage section 62, which are described later.

[0049] Furthermore, in the present embodiment, the head posture prediction section 42 predicts a rotation angle (final movement angle) until the user's head 80 finally stops rotating, based on the acquisition data from the gyroscope 12. Specifically, the head posture prediction section 42 detects the angular velocity of the user's head rotation, and when it detects that the angular velocity has reached a predetermined speed (first speed), based on the acquired angular velocity (at a first angular velocity detection time), the head posture prediction section 42 predicts the final movement angle of the head to be maintained when the head rotation ends (at a rotation end time), using an integral value of the angular velocity acquired from the rotation start time to a first angular velocity detection time.More precisely, the head posture prediction section 42 detects the angular velocity of the user's head rotation 80, and when it is detected that the angular velocity is becoming fastest, based on the detected angular velocity (at a fastest angular velocity detection time), the head posture prediction section 42 predicts the final angle of head movement to be obtained when the head rotation ends (at the rotation end time), using the integral value of the angular velocity detected from the rotation start time to the fastest angular velocity detection time.Furthermore, following the fastest angular velocity detection time, the head posture prediction section 42 predicts the final head movement angle to be obtained when the head rotation ceases, using the integral of the angular velocity captured from the rotation start time to the current time. The head posture prediction section 42 then outputs the predicted final head movement angle of the user 80 to the tone data correction section 52, which is described later. It is noted that the head posture prediction section 42 repeatedly performs the prediction of the final movement angle at a predetermined time interval until the head rotation ceases. Details of the prediction procedure in the head posture prediction section 42 are described later. (Audio data correction section 52)

[0050] The audio data correction section 52 corrects the audio data (specifically each of a plurality of frames that constitute an audio data element) so that the virtual sound source positions corresponding to the audio data output to the left and right ears of the user 80—that is, the virtual sound source positions presented to the user—rotate in the opposite direction to the direction of rotation of the user's head 80. The audio data correction section 52 then outputs the corrected audio data sequentially to the audio data output section 54, which is described later.The sound data correction section 52 corrects the sound data sequentially according to the angle of movement of the user's head 80 at the current time, and the corrected sound data are output sequentially to the user 80 through the sound processing device 10, so that the user can hear the sound from the spatially fixed virtual sound source even while the user's head is turning.

[0051] In particular, in the present embodiment, the audio data correction section 52 corrects the audio data up to the fastest angular velocity detection time such that the virtual sound source position rotates by the angle of movement of the user's head 80 in the direction opposite to that of the head rotation in conjunction with the rotation of the user's head 80. Furthermore, the audio data correction section 52 corrects the audio data up to and after the fastest angular velocity detection time such that the virtual sound source position rotates in the direction opposite to that of the head rotation, using the predicted value of the final head movement angle to be obtained when the head rotation ends (at the rotation end time), wherein the predicted value of the final head movement angle was predicted by the head posture prediction section 42.For example, the audio data correction section 52 corrects the audio data so that the virtual sound source position rotates in the opposite direction to accurately follow the head of user 80, using the head's current angle of movement, the head's current angular velocity, and the predicted value of the head's final angle of movement. It is noted that the audio data correction section 52 performs the correction repeatedly at a predetermined time interval until the head rotation ceases. (Audio output section 54)

[0052] The sound data output section 54 transmits the sound data corrected by the sound data correction section 52 to the sound processing device 10 via the communication unit 60 described later.

[0053] It is noted that the head posture prediction section 42, the tone data correction section 52 and the tone data output section 54 described above can be formed using hardware such as a CPU, a ROM and a RAM provided in the information processing device 40. (Communication Unit 60)

[0054] The communication unit 60 can communicate with the sound processing device 10 via the communication network 70 according to a predetermined communication standard. The communication unit 60 can, for example, consist of a communication module. (Storage section 62)

[0055] Storage section 62 stores various types of information, such as audio data like streamed music. Storage section 62 can consist of, for example, random access memory (RAM), read-only memory (ROM), a memory card, or the like. Furthermore, storage section 62 can store programs to be executed by the information processing device 40 and various types of data to be used in information processing within the information processing device 40.

[0056] Although not illustrated, the information processing device 40 is further equipped with a control unit that controls each of the functional blocks of the information processing device 40. The control unit can, for example, consist of a memory and a processor with hardware such as a CPU or a DSP.

[0057] It is noted that in the present embodiment the functional configuration of the information processing device 40 does not correspond to the one described in Fig. The mode illustrated in section 6 is limited. Furthermore, details of the hardware configuration of the information processing device 40 are described later.

[0058] Furthermore, in the present embodiment, as described above, the sound processing device 10 and the information processing device 40 can be designed as an integrated device. Furthermore, in the present embodiment, some of the functional blocks of the information processing device 40 can be provided in the sound processing device 10, or some of the functional blocks of the sound processing device 10 can be provided in the information processing device 40. Finally, in the present embodiment, some or all of the functional blocks of the information processing device 40 and the sound processing device 10 can be performed on a server (not illustrated). 2.4 Information processing methods

[0059] Next, an information processing method according to the present embodiment will be described with reference to Fig. 7 to 14 described. Fig. Figure 7 is a flowchart showing the sequence of the information processing procedure according to the present embodiment, and Fig. Figures 8 to 14 are explanatory graphs that show the information processing procedure according to the present embodiment.

[0060] As in Fig. As shown in Figure 7, the information processing method according to the present embodiment can mainly comprise steps S101 to S105. Furthermore, in the information processing method described below, steps S103 and S104 are executed repeatedly until the rotation of the user's head 80 stops. Details of each of the steps according to the present embodiment are described below.

[0061] First, as in Fig. As shown in Figure 7, the sound processing device 10 included in the information processing system 1 according to the present embodiment begins by measuring the rotation of the user's head 80 (step S101). Subsequently, the information processing device 40, which is included in the information processing system 1 according to the present embodiment, detects the angular velocity of the user's head rotation 80 and detects that the angular velocity has reached its maximum angular velocity based on the detected angular velocity (step S102).

[0062] The information processing device 40 then predicts the final head movement angle to be obtained when the head rotation ends, using the integral value of the angular velocity acquired up to that point. Subsequently, the information processing device 40 corrects the audio data such that the virtual sound source position rotates in the opposite direction to that of the head rotation, using the predicted value of the final movement angle (step S103). Furthermore, the information processing device 40 transmits the corrected audio data to the sound processing device 10 (step S104).

[0063] When the user's head rotation is complete, the sound processing device 10 stops measuring the user's head rotation 80 (step S105) and ends the processing.

[0064] Furthermore, each of the above-described steps in the processing of the present embodiment is not necessarily limited to being carried out by the above-described device and can be carried out by a functional section of another device.

[0065] Furthermore, details of the prediction procedure are given in step S103 described above with reference to Fig. As described in sections 8 to 12, the present inventors have assumed that if an endpoint angle of the head's rotation can be predicted while the user's head is rotating 80°, the delay time described above can be reduced; that is, the deviation between the rotation of the user's head 80° and the virtual sound source position (the direction of the sound) perceived by the user 80° can be reduced by rotating the virtual sound source position using the predicted value. The present inventors have conducted intensive studies based on this idea and independently acquired the following knowledge and designed the present embodiment.

[0066] Fig. Figure 8 shows angular velocities measured when the three-axis gyro sensor 12 is attached to the user's head 80 and the head is shaken from side to side. According to the present inventors, as shown in Fig. As shown in section 8, it has been observed that changes over time (in Fig. 8, specified by Z) of the angular velocity of the rotational movement of the user's head 80 on the horizontal plane, essentially form bilaterally symmetrical peak shapes. Therefore, the present inventors have designed the prediction of the endpoint angle of the rotational movement of the head by exploiting the fact that the changes over time of the angular velocity of the rotational movement of the user's head 80 on the horizontal plane exhibit bilaterally symmetrical peak shapes. In other words, in the present embodiment generated by the present inventors, the endpoint angle of the rotational movement of the user's head 80 is predicted by assuming that the changes in the angular velocity of the head from the fastest angular velocity detection time to the end time of rotation are the same as the changes in the angular velocity from the start time of rotation to the fastest angular velocity detection time.

[0067] The angle of motion due to the rotational motion can be obtained by integrating (area) the changes in angular velocity over time. As in Fig. As shown in Figure 9, if the changes in angular velocity exhibit bilaterally symmetrical mountain shapes, the angle of motion in the first half of the rotation and the angle of motion in the second half of the rotation are equal. Therefore, in the present embodiment, assuming that the angle of motion of the second half can be approximated by the angle of motion of the first half, the final angle of motion of the head, which is to be obtained when the rotation of the head ends (at the end-of-rotation time), is predicted before the rotation ends.

[0068] Next, the prediction procedure from the beginning of the rotation to the end of the rotation will be described with reference to Fig. Sections 10 to 12 describe this in detail. In the present embodiment, it is assumed that the prediction is performed for rotation about the yaw axis (rotation about the Z-axis perpendicular to the horizontal plane), which is rotation of an azimuth angle at a relatively high rotational speed (angular velocity). It is noted that in the present embodiment, the rotation to be predicted (and corrected) is not limited to rotation about the yaw axis and can be rotation about the roll axis (rotation about the X-axis on the horizontal plane) or rotation about the pitch axis (rotation about the Y-axis on the horizontal plane).

[0069] First, an interval is defined from the beginning of the rotation until the point at which the angular velocity reaches its highest angular velocity, with reference to Fig. 10 described. During this interval, prediction processing according to the present embodiment is not performed, and only an angular velocity recording is carried out to use the recording in prediction processing to be performed in a subsequent interval. That is, until the fastest angular velocity detection time, the audio data correction section 52 corrects the audio data so that the virtual sound source position rotates in the opposite direction to that of the head rotation in conjunction with the head turning.

[0070] Next, a time at which the angular velocity reaches its highest angular velocity is determined with reference to Fig. As described in section 11, at the fastest angular velocity detection time, the head posture prediction section 42 predicts the final angle of movement using an integral value of the angular velocity detected from the rotation start time up to the fastest angular velocity detection time t, i.e., an integral value (area) of the angular velocity in the interval of the first half. Specifically, the head posture prediction section 42 adds the integral value (area) of the angular velocity in the interval of the first half to the head's angle of movement at the current time. That is, the head posture prediction section 42 predicts the final angle of movement of the head at the end of the rotation by doubling the integral value of the angular velocity detected from the rotation start time up to the fastest angular velocity detection time.Then, at the fastest angular velocity detection time, the audio data correction section 52 corrects the audio data using such a predicted value, so that the virtual sound source position rotates in the direction opposite to that of the head's rotation. It is noted that in a basic mode of the present embodiment, the final angle of head movement at the end of the rotation is predicted by doubling the integral value of the angular velocity detected from the start of the rotation to the fastest angular velocity detection time t. However, in the present embodiment, it is not necessary to use the fastest angular velocity detection time t and the value twice the integral value of the angular velocity.In the present embodiment, for example, the integral value of the angular velocity, acquired over the period from the start of rotation to the fastest angular velocity detection time t, or over a period from the fastest angular velocity detection time t to a rotation end time T (T ≠ t), can be multiplied by N (predetermined value) (N ≠ 2). Alternatively, in the present embodiment, for example, the integral value of the angular velocity, acquired over a period from the start of rotation to a time (first angular velocity detection time) at which the angular velocity has reached a predetermined speed (first speed), or over a period from the first angular velocity detection time to the rotation end time, can be multiplied by N (N ≠ 2).Here, N can be set to a real value (for example, 1.9, 2.01, or similar) other than an integer. Furthermore, the t, T, and N described above can be conveniently set by the user or dynamically set and changed by the system.

[0071] Furthermore, with reference to Fig. Section 12 describes an interval from the time at which the angular velocity reaches its highest value until the end of the rotation. Within this interval, the head posture prediction section 42 identifies a time (third angular velocity detection time) t' at which the angular velocity (a third angular velocity) is detected. The third angular velocity detection time t' precedes the highest angular velocity detection time, and the third angular velocity has the same value as the angular velocity (a second angular velocity) detected at a current time (second angular velocity detection time) t after the highest angular velocity detection time.Subsequently, the head posture prediction section 42 predicts the final angle of movement using an integral value of the angular velocity recorded from the start of the rotation to time t'. Specifically, the head posture prediction section 42 predicts the final angle of movement of the head at the end of the rotation by adding the integral value (area) of the angular velocity recorded up to time t' to the angle of movement at the current time t. Then, after the fastest angular velocity detection time, the audio data correction section 52 corrects the audio data using such a predicted value so that the virtual sound source position rotates in the opposite direction to that of the head rotation.Furthermore, in the present embodiment, such a prediction and correction are repeatedly performed at a predetermined time interval in the interval from the time at which the angular velocity reaches its highest angular velocity until the end of rotation. It should be noted that in the present embodiment, the time interval at which the prediction and correction are performed is preferably set as appropriate, taking into account the processing time, the processing load, battery consumption, and the like.

[0072] It is noted that if the audio data is corrected using a very large final movement angle, the virtual sound source position rotates rapidly, and the user may experience discomfort. Therefore, to avoid such discomfort, the present embodiment allows for the setting of an upper limit for the final movement angle, which can be predicted using the method described above. It is noted that the upper limit can be conveniently set by the user or dynamically adjusted or changed by the system according to processing time, processing load, battery consumption, the type of content being applied, and the like.

[0073] Next, the effects of the present embodiment will be discussed with reference to Fig. 13 and Fig. 14 described. In Fig. 13 and Fig. In figure 14, the horizontal axis represents the elapsed time, and the vertical axis represents the angle of head movement. Furthermore, in Fig. 13 and Fig. 14 A solid line indicates the angle of movement of the actual rotation of the user's head 80, and a dashed line indicates the angle of movement of the virtual sound source position of the tone perceived by the user 80.

[0074] In a comparative example to which the prediction according to the present embodiment is not applied, as in Fig. As shown in Figure 13, the actual rotation angle of the head (solid line) and the angle of movement of the virtual sound source position of the tone (dashed line), which reflects the actual rotation of the user's head and is perceived by user 80, differ from each other. Therefore, the difference in the vertical direction between the solid line and the dashed line at a given time is perceived by user 80 as the deviation in the direction of the tone.

[0075] On the other hand, in an example to which the prediction according to the present embodiment is applied, the predicted value is indicated by an alternating long and short dashed line, as in Fig. 14 is shown, and the audio data is corrected so that the virtual sound source position is rotated and reflects the predicted value. Then, in the example, as shown in Fig. Figure 14 shows that the deviation between the actual rotation angle of the head (solid line) and the angle of movement of the virtual sound source position of the tone (dashed line), which reflected the predicted value (alternating long and short dashed lines) and is perceived by user 80, is small near the endpoint of the rotation. Therefore, the difference in the vertical direction between the solid line and the dashed line is smaller near the endpoint, and thus it is less likely that user 80 will perceive the deviation in the direction of the tone. In other words, using the prediction according to the present embodiment, the virtual sound source position in the example reaches the vicinity of the endpoint of the rotation at an earlier timing than in the comparison example. That is, according to the present embodiment, the deviation in the direction of the tone is reduced near the endpoint.

[0076] As described above, in the present embodiment, the endpoint angle of the head's rotation is predicted by utilizing the fact that the changes in the angular velocity of the user's head's rotation over time on the horizontal plane form bilaterally symmetrical mountain shapes, and also under the assumption that the changes in the angular velocity of the user's head from the fastest angular velocity detection time to the end of rotation are the same as the changes in angular velocity from the start of rotation to the fastest angular velocity detection time. Then, in the present embodiment, the audio data is corrected using the predicted value described above. Therefore, according to the present embodiment, it is possible to reduce the delay time, i.e.,, to reduce the deviation between the rotation of the user's head 80 and the virtual sound source position (the direction of the sound) perceived by the user 80. As a result, in the present embodiment, it is possible to emphasize the effect of causing the user to hear the sound as if the virtual sound source position were spatially fixed, and to provide the user with an audio experience that offers a greater sense of presence.

[0077] As described above, in the present embodiment, the final angle of movement is predicted under the assumption that the changes in angular velocity over time due to the rotational movement form the bilaterally symmetrical mountain shapes. However, the accuracy of the prediction can be improved by performing further processing. For example, initially, when user 80 begins using the information processing system 1 according to the present embodiment, the prediction is performed only by the method described above. Then, using the information processing system 1, user 80 repeatedly extracts characteristics of the changes in angular velocity over time of the head rotation, which are unique to user 80, by means of a machine learning model (not illustrated) contained in the information processing system 1.Examples of the characteristics include a case where the changes in angular velocity over time due to the rotational motion form mountain shapes that are not bilaterally symmetrical, such as a case where the time interval from when the angular velocity reaches its highest point until the rotation stops is shorter for clockwise rotation than for counterclockwise rotation. In the present embodiment, the accuracy of the prediction can be improved by forecasting the final angle of motion based on the individual characteristics of the user 80, extracted by such a machine learning model.Furthermore, in the present embodiment, the accuracy of the prediction can be increased by using not only the extraction of characteristics by the machine learning model, but also information (attribute information) such as characteristics of the skeletal structure and muscles of user 80 and the dominant hand of user 80. 3. Second embodiment

[0078] In the first embodiment described above, the virtual sound source position is rotated in the opposite direction to the rotation of the user's head (80), so that the user (80) hears the sound as if the virtual sound source position were spatially fixed. In a second embodiment of the present disclosure, the angle of movement of a virtual moving sound source is changed according to the relative angle of the virtual sound source position with respect to the face (head) of the user (80). In this way, the present embodiment makes it possible to improve the directionality of the sound in the sensory experience of the user (80), to emphasize the effect of causing the user to hear the sound as if the virtual sound source position were spatially fixed, and to provide the user (80) with an audio experience that offers a greater sense of presence.

[0079] The present embodiment is described below with reference to Fig. 15 described. Fig. Figure 15 is an explanatory graph showing an information processing method according to the present embodiment, and in particular a graph of the relationship between the rotation angle of the user's head 80 and the angle of movement (the angle of movement of the virtual moving sound source) applied to the tone.

[0080] Humans perceive the movement of a sound differently depending on its direction. For example, a person is generally able to sensitively perceive the movement of a sound coming from the front of them, but it is more difficult for them to sensitively perceive the movement of a sound coming from behind them. Therefore, in the present embodiment, the angle of movement of the virtual moving sound source is changed according to the relative angle of the virtual sound source's position with respect to the user's face (head), taking into account human hearing characteristics. In this way, the directionality of the sound can be improved in the sensory experience of the user according to human hearing characteristics.

[0081] In particular, as in Fig. As shown in Figure 15, when the virtual sound source position is located at the front of the user's face (80°), the audio data correction section (52) corrects the audio data so that the virtual sound source position has a smaller angle of movement than the angle of movement of the head. More precisely, if, for example, the angle of movement of the head ranges from -90° to 90° relative to the front of the user's face (80°), the audio data correction section (52) multiplies the angle of movement of the head by a coefficient (first coefficient) R of 0 or more and less than 1 to obtain the angle of movement of the virtual sound source position.If, on the other hand, the virtual sound source position is on the opposite side from the front of the user's face 80, the audio data correction section 52 corrects the audio data such that the virtual sound source position has a larger angle of movement than the angle of movement of the head. More precisely, if, for example, the angle of movement of the head is from -180 degrees to -90 degrees or from 90 degrees to 180 degrees relative to the front of the user's face 80, the audio data correction section 52 multiplies the angle of movement of the head by a coefficient (second coefficient) R greater than 1 to obtain the angle of movement of the virtual sound source position.

[0082] It is noted that, although specific angle values ​​in Fig. Figure 15 shows only examples, and the present embodiment is not limited to such angles and coefficients R. For example, in the present embodiment, the angles and coefficients R can be selected by the user as appropriate or can be dynamically changed by the system.

[0083] Furthermore, in the present embodiment, the user 80 can be asked in advance to listen to tones corresponding to virtual sound source positions with different directions and to answer questions regarding the degree of perception of the tone, their preference for the tone, and the like, and the coefficients R described above can be adjusted according to their answers. In this way, the directionality of the tone in the sensory experience of the user 80 can be improved according to the individual hearing characteristics of the user 80.

[0084] As described above, in the present embodiment it is also possible to improve the directionality of the sound in the sensory experience of the user 80, to emphasize the effect of causing the user to hear the sound as if the virtual sound source position is spatially fixed, and to provide the user with the audio experience that offers a higher sense of presence. 4. Third embodiment

[0085] In the first embodiment described above, the virtual sound source position is rotated in the opposite direction to the rotation of the user's head (80), so that the user (80) hears the tone as if the virtual sound source position were spatially fixed. In a third embodiment of the present disclosure, the sound pressure of an emitted tone is modified according to the relative angle of the virtual sound source position with respect to the face (head) of the user (80). In this way, the present embodiment makes it possible to improve the directionality and localization of the tone in the sensory experience of the user (80), to emphasize the effect of causing the user to hear the tone as if the virtual sound source position were spatially fixed, and to provide the user with an audio experience that offers a greater sense of presence.

[0086] The present embodiment is described below with reference to Fig. 16 described. Fig. Figure 16 is an explanatory diagram showing an information processing method according to the present embodiment, and is in particular a diagram of the relationship between the virtual sound source position (sound source direction) in relation to the user 80 and the weighting of the sound pressure of the tone.

[0087] It is generally known that, as a characteristic of human hearing, the perceived sound pressure varies depending on the direction of a sound. For example, ITU-R BS.1770 states that even sounds that physically have the same sound pressure exhibit different acoustic sound pressures depending on the direction of the sound.

[0088] To achieve more natural head tracking, the present embodiment employs sound pressure control that takes into account the difference in sound pressure level perceived by the user depending on the direction of the sound, which is due to the nature of human hearing. In this way, the present embodiment makes it possible to improve the directionality and localization of the sound in the user's sensory experience, to emphasize the effect of causing the user to hear the sound as if the virtual sound source position were spatially fixed, and to provide the user with an audio experience that offers a greater sense of presence.

[0089] In particular, in the present embodiment, the sound pressure of the tone to be emitted is changed according to the relative angle of the virtual sound source position in relation to the face (head) of the user 80.

[0090] More precisely, as in Fig. As shown in Figure 16, if the virtual sound source position in the present embodiment is located, for example, on the front of the user's face 80, the audio data correction section 52 corrects the audio data without weighting the sound pressure (0 dB). If, on the other hand, the virtual sound source position is located on the opposite side from the front of the user's face 80, the audio data correction section 52 corrects the audio data so that the sound pressure is weighted (1.5 dB). For example, if the angle of the virtual sound source position relative to the front of the user's face 80 is less than -110 degrees and equal to or greater than +110 degrees, the audio data correction section 52 performs a correction so that the sound is perceived as 1.5 dB louder than if the virtual sound source position were located on the front of the user's face 80.

[0091] It is noted that, although specific angle values ​​and specific sound pressure weighting values ​​in Fig. Figure 16 shows that these are merely examples, and the present embodiment is not limited to such values ​​and the like. For example, in the present embodiment, the angles and weighting of the sound pressures can be selected by the user as appropriate or dynamically changed by the system.

[0092] Furthermore, in the present embodiment, the user 80 can be asked in advance to listen to tones corresponding to virtual sound source positions with different directions and to answer questions regarding the perceived degree of the tone, their preference for the tone, and the like. The angles and weighting can then be adjusted according to their answers. In this way, the directionality of the tone can be enhanced in the sensory experience of the user 80 according to the individual hearing characteristics of the user 80.

[0093] As described above, in the present embodiment it is also possible to improve the directionality of the sound in the sensory experience of the user 80, to emphasize the effect of causing the user to hear the sound as if the virtual sound source position is spatially fixed, and to provide the user with the audio experience that offers a higher sense of presence. 5. SUMMARY

[0094] As described above, according to the embodiments of the present disclosure, it is possible to resolve the factors that impair the feeling of presence, and it is thus possible to provide the user with the audio experience that offers a higher feeling of presence.

[0095] It is noted that in the embodiments of the present disclosure described above, the prediction and correction were described with respect to rotation about the yaw axis (rotation about the Z-axis perpendicular to the horizontal plane), where rotation about the yaw axis is the rotation of the azimuth angle at a relatively high rotational speed (angular velocity). However, in each of the embodiments of the present disclosure, the rotation to be predicted and corrected is not limited to rotation about the yaw axis, and the prediction and correction can be applied to rotation about the roll axis (rotation about the X-axis on the horizontal plane) or rotation about the pitch axis (rotation about the Y-axis on the horizontal plane). 6. Hardware configuration

[0096] Fig. Figure 17 is an explanatory diagram illustrating an example of a hardware configuration of an information processing device 900 according to the present embodiment. Fig. Figure 17 illustrates the information processing device 900 as an example of the hardware configuration of the information processing device 40 according to the embodiment described above in the present disclosure. The information processing device 900 can be a smartphone or the like.

[0097] As in Fig.As shown in Figure 17, the information processing device 900 includes a central processing unit (CPU) 901, a read-only memory (ROM) 902, and a random access memory (R_AM) 903. The information processing device 900 further includes a storage device 904, a communication module 905, and a sensor module 907. The information processing device 900 also includes an imaging device 908, a display device 910, a loudspeaker 911, a microphone 912, an input device 913, and a bus 914. The information processing device 900 may also include a processing circuit, such as a digital signal processor (DSP), instead of or in addition to the CPU 901.

[0098] The CPU 901 functions as an arithmetic processing device and a control device, controlling all or part of the operation in the information processing device 900 according to various programs recorded in the ROM 902, the RAM 903, the storage device 904, or the like. For example, the CPU 901 can be the head posture prediction section 42, the sound data acquisition section 50, the sound data correction section 52, the sound data output section 54, or the like, according to the embodiment of the present disclosure described above. The ROM 902 stores programs, arithmetic parameters, and the like to be used by the CPU 901. The RAM 903 temporarily stores programs to be used by the CPU 901 during execution, parameters that are expediently modified during execution, and the like.The CPU 901, the ROM 902, and the RAM 903 are interconnected by the bus 914. The storage device 904 is a data storage device configured as an example of a storage section of the information processing device 900. The storage device 904 is formed, for example, by a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or the like. The storage device 904 stores programs and various types of data executed by the CPU 901, various types of data acquired from outside, and the like. For example, the storage device 904 can be the storage section 62 or the like according to the embodiment described above in the present disclosure.

[0099] The Communication Module 905, for example, is a communication interface formed by a communication device for connecting to the Communication Network 906. The Communication Module 905 can be, for example, a communication card for a wired or wireless local area network (LAN), Bluetooth, or a Wireless Universal Serial Bus (WUSB). Furthermore, the Communication Module 905 can be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), a modem for various types of communication, or the like. The Communication Module 905 transmits and receives signals to and from, for example, the internet or other communication devices using a predefined protocol, such as TCP (Transmission Control Protocol) / IP (Internet Protocol).The communication network 906, which is connected to the communication module 905, is a network that is connected by wire or wirelessly and is, for example, the Internet, a home LAN, infrared communication, satellite communication, or the like. For example, the communication module 905 can be the communication unit 60 or the like according to the embodiment described above in the present disclosure.

[0100] The sensor module 907 includes any number of different sensors, such as a motion sensor (for example, an accelerometer, a gyroscope, a geomagnetic sensor, or the like), a sensor for biological information (for example, a pulse sensor, a blood pressure sensor, a fingerprint sensor, or the like), and a position sensor (for example, a GNSS (Global Navigation Satellite System) receiver, or the like).

[0101] The imaging device 908 is provided on the front surface of the information processing device 900 and can image an object or the like located on the rear or front of the information processing device 900. Furthermore, the imaging device 908 can include an optical system mechanism (not illustrated) including an imaging lens, a zoom lens, a focusing lens, or the like, and a drive system mechanism (not illustrated) that controls the operation of the optical system mechanism.

[0102] The display device 910 is provided above the front surface of the information processing device 900 and can be a display device such as a liquid crystal display (LCD) or an organic electroluminescent (EL) display. The display device 910 can display an operating screen, a captured image acquired by the imaging device 908 described above, and the like.

[0103] For example, the speaker 911 can output a ringtone, a tone accompanying video content displayed by the display device 910, and the like to the user.

[0104] For example, the microphone 912 can collect a user call tone, a tone including a command to activate a function of the information processing device 900, and a sound from the surrounding environment of the information processing device 900.

[0105] The input device 913 is a device operated by the user, such as a button, a keyboard, a touch panel, or a mouse. The input device 913 includes an input circuit that generates an input signal based on information entered by the user and outputs the input signal to the CPU 901. By operating the input device 913, the user can input various types of data into the information processing device 900 or instruct the information processing device 900 to perform processing operations.

[0106] The above describes a configuration example of the Information Processing Device 900. Each of the components described above can consist of a generic part or of hardware specialized for the function of each component. Such a configuration can be modified as appropriate according to the level of technology available at the time of implementation.

[0107] Furthermore, the information processing device according to the present embodiment can be applied to a system that includes one or a plurality of devices which are assumed to be connected to a network (or communication between devices), such as cloud computing. That is to say, the information processing device described above according to the present embodiment can, for example, be designed as an information processing system that performs processing related to the information processing method according to the present embodiment using a plurality of devices. 7th Supplement

[0108] It should be noted that the embodiment of the present disclosure described above may, for example, include a program for causing a computer to function as the information processing device according to the present embodiment, and a non-volatile material medium on which the program is recorded. The program may be distributed via a communication line (including wireless communication), such as the internet.

[0109] Furthermore, each of the processing steps described above need not necessarily be performed in the order described above, according to the present embodiment. For example, the order of the processing steps can be expedited. Additionally, each processing step can be performed partially in parallel or individually, instead of sequentially. Moreover, the processing in each step need not necessarily be carried out according to the method described above and can, for example, be performed by a different method in a different functional section.

[0110] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is obvious that a person skilled in the art in the field of the present disclosure could devise various changes or modifications within the scope of protection of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0111] Furthermore, the effects described in this document are merely illustrative or exemplary and are not limited. This means that the technology according to this disclosure may exhibit other effects which are apparent to those skilled in the art from the description in this document, either alongside or instead of the effects described above.

[0112] It should be noted that the present technology can also have the following configuration. (1) An information processing device includes a detection section designed to detect the angular velocity of a user's head, a prediction section designed to predict a final angle of movement of the head at a rotation end time of the head using an integral value of the angular velocity detected from a rotation start time of the head to a first angular velocity detection time at which the angular velocity reaches a first velocity, and a correction section designed to correct a position of a virtual sound source presented to the user based on the final angle of movement. (2) In the information processing device according to (1), the correction section performs a correction such that the position of the virtual sound source rotates in a direction opposite to a direction of rotation of the head. (3) In the information processing device according to (2), the prediction section predicts the final angle of motion by multiplying the integral value of the angular velocity, which is recorded from the start of rotation time to the first angular velocity detection time, by a predetermined value. (4) In the information processing device according to (3) the first velocity is an angular velocity at a time when the angular velocity becomes fastest, and the first angular velocity detection time is a fastest angular velocity detection time. (5) In the information processing device according to (4), the prediction section, following the fastest angular velocity detection time, identifies a third angular velocity detection time at which a third angular velocity with the same value as a second angular velocity is detected, wherein the third angular velocity detection time precedes the fastest angular velocity detection time, and wherein the second angular velocity is detected at a second angular velocity detection time after the fastest angular velocity detection time, and predicts the final angle of motion using an integral value of the angular velocity detected from the rotation start time to the third angular velocity detection time. (6) In the information processing device according to (5), the prediction section predicts the final angle of movement by adding the integral value of the angular velocity detected from the start of rotation time to the third angular velocity detection time to an angle of movement of the head at the second angular velocity detection time. (7) In the information processing device according to (5) or (6), the prediction section repeatedly performs a prediction of the final angle of motion with a predetermined time interval after the fastest angular velocity detection time and up to the end of the rotation time. (8) In the information processing device according to any of (4) to (7), the correction section performs a correction up to the fastest angular velocity detection time such that the position of the virtual sound source rotates by an angle of movement of the head in the direction opposite to the direction of rotation of the head in conjunction with the rotation of the head, and performs a correction from the fastest angular velocity detection time such that the position of the virtual sound source rotates in the direction opposite to the direction of rotation of the head, using a predicted value of the final angle of movement. (9) In the information processing device according to (1) to (8), the correction section corrects a movement angle of the position of the virtual sound source according to a relative angle of the position of the virtual sound source in relation to the user. (10) In the information processing device according to (9), when the position of the virtual sound source is on the front of the user, the correction section performs a correction so that the position of the virtual sound source has an angle of movement that is smaller than the angle of movement of the head, and when the position of the virtual sound source is on the opposite side to the front of the user, it performs a correction so that the position of the virtual sound source has an angle of movement that is larger than the angle of movement of the head. (11) In the information processing device according to (10), when the position of the virtual sound source is on the front of the user, the correction section multiplies the angle of movement of the head by a first coefficient, and when the position of the virtual sound source is on the opposite side from the front of the user, it multiplies the angle of movement of the head by a second coefficient which differs from the first coefficient. (12) In the information processing device according to (11) the first coefficient is a value less than 1 and the second coefficient is a value greater than 1. (13) In the information processing device according to paragraphs (1) to (12), the correction section corrects an acoustic signal presented to the user so that the sound pressure of the acoustic signal changes according to a relative angle of the position of the virtual sound source in relation to the user. (14) In the information processing device according to (13), the correction section causes the sound pressure of the acoustic signal to be greater when the position of the virtual sound source is on a side opposite the front of the user than when the position of the virtual sound source is on the front of the user. (15) In the information processing device according to any of (1) to (14), the detection section detects the angular velocity of the user's head from an inertial measuring device provided in a sound processing device attached to the user's head. (16) The information processing device according to (15) also includes the sound processing device. (17) In the information processing device according to (16), the sound processing device shall be at least one selected from the group consisting of headphones, earphones, a hearing aid, a sound collector and a head-mounted display. (18) In the information processing device according to (15), the information processing device is an information processing terminal device designed to be able to communicate with a sound processing device. (19) An information processing system includes a detection section designed to detect the angular velocity of a user's head, a prediction section designed to predict the final angle of movement of the head at the end of the head's rotation using an integral value of the angular velocity detected from the start of the head's rotation to a first angular velocity detection time at which the angular velocity reaches a first velocity, and a correction section designed to correct the position of a virtual sound source presented to the user based on the final angle of movement. (20) An information processing procedure causes an information processing device to perform the following: detecting an angular velocity of a user's head, predicting a final angle of movement of the head at a rotation end time of the head using an integral value of the angular velocity detected from a rotation start time of the head to a first angular velocity detection time at which the angular velocity reaches a first velocity, and correcting a position of a virtual sound source presented to the user based on the final angle of movement. (21) A program causes a computer to implement: a function for detecting an angular velocity of a user's head, a function for predicting a final angle of movement of the head at a rotation end time of the head using an integral value of the angular velocity detected from a rotation start time of the head to a first angular velocity detection time at which the angular velocity reaches a first velocity, and a function for correcting a position of a virtual sound source presented to the user based on the final angle of movement. Reference symbol list 1 Information processing system 3 sliders 4L, 4R suspension element 5L, 5R housing 6L, 6R ear pads 7 headbands 10 Sound processing device 10a Headphones 12 Gyro sensor 14 Accelerometer 16 Output section 20 Sound data acquisition section 22 Audio output section 30, 60 communication unit 40,900 Information processing device 40a Smartphone 42 Head posture prediction section 50 Sound data acquisition section 52 Audio Data Correction Section 54 Audio output section 62 Storage section 70, 906 Communication network 80 users 82 Sound source 901 CPU 902 ROM 903 RAM 904 Storage device 905 Communication module 907 Sensor module 908 Imaging device 910 Display device 911 loudspeakers 912 Microphone 913 Input device 914 Bus QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2010-147529 A

[0003]

Claims

[1] Information processing device comprising a detection section designed to detect an angular velocity of a user's head, a prediction section designed to predict a final angle of movement of the head at a rotation end time of the head using an integral value of the angular velocity detected from a rotation start time of the head to a first angular velocity detection time at which the angular velocity reaches a first velocity, and a correction section designed to correct a position of a virtual sound source presented to the user based on the final angle of movement. [2] Information processing device according to claim 1, wherein the correction section performs a correction such that the position of the virtual sound source rotates in a direction opposite to a direction of rotation of the head. [3] Information processing device according to claim 2, wherein the prediction section predicts the final angle of motion by multiplying the integral value of the angular velocity, which is detected from the start of rotation time to the first angular velocity detection time, by a predetermined value. [4] Information processing device according to claim 3, wherein the first velocity is an angular velocity at a time when the angular velocity is at its fastest, and the first angular velocity detection time is the fastest angular velocity detection time. [5] Information processing device according to claim 4, wherein Following the fastest angular velocity detection time of the prediction section a third angular velocity detection time is identified at which a third angular velocity with the same value as a second angular velocity is detected, wherein the third angular velocity detection time precedes the fastest angular velocity detection time, and wherein the second angular velocity is detected at a second angular velocity detection time after the fastest angular velocity detection time, and predicts the final angle of motion using an integral value of the angular velocity, which is recorded from the start of rotation to the third angular velocity detection time. [6] Information processing device according to claim 5, wherein the prediction section predicts the final angle of movement by adding the integral value of the angular velocity detected from the start of rotation time to the third angular velocity detection time to an angle of movement of the head at the second angular velocity detection time. [7] Information processing device according to claim 5, wherein following the fastest angular velocity detection time and up to the end of the rotation time The prediction section repeatedly performs a prediction of the final angle of movement with a predetermined time interval. [8] Information processing device according to claim 4, wherein the correction section a correction is performed up to the fastest angular velocity detection time, so that the position of the virtual sound source rotates by an angle of movement of the head in the direction opposite to the direction of rotation of the head in conjunction with the rotation of the head, and a correction is performed from the fastest angular velocity detection time using a predicted value of the final movement angle, so that the position of the virtual sound source rotates in the direction opposite to the direction of rotation of the head. [9] Information processing device according to claim 1, wherein the correction section corrects a movement angle of the position of the virtual sound source according to a relative angle of the position of the virtual sound source in relation to the user. [10] Information processing device according to claim 9, wherein the correction section, If the position of the virtual sound source is on the front of the user, a correction is made so that the position of the virtual sound source has a movement angle that is smaller than a movement angle of the head, and If the position of the virtual sound source is on a side opposite the front of the user, a correction is made so that the position of the virtual sound source has a movement angle that is greater than the movement angle of the head. [11] Information processing device according to claim 10, wherein the correction section, If the position of the virtual sound source is at the front of the user, the head movement angle is multiplied by a first coefficient, and If the position of the virtual sound source is on the opposite side from the front of the user, the head movement angle is multiplied by a second coefficient that differs from the first coefficient. [12] Information processing device according to claim 11, wherein the first coefficient is a value less than 1 and the second coefficient is a value greater than 1. [13] Information processing device according to claim 1, wherein the correction section corrects an acoustic signal presented to the user such that a sound pressure of the acoustic signal changes according to a relative angle of the position of the virtual sound source in relation to the user. [14] Information processing device according to claim 13, wherein the correction section causes the sound pressure of the acoustic signal to be greater when the position of the virtual sound source is on a side opposite the front of the user than when the position of the virtual sound source is on the front of the user. [15] Information processing device according to claim 1, further comprising a sound processing device. [16] Information processing device according to claim 15, wherein the sound processing device is at least one selected from the group consisting of headphones, earphones, a hearing aid, a sound collector and a head-mounted display. [17] Information processing device according to claim 1, wherein the information processing device is an information processing terminal device designed to be communicative with a sound processing device. [18] Information processing system comprising a detection section designed to detect an angular velocity of a user's head, a prediction section designed to predict a final angle of movement of the head at a rotation end time of the head using an integral value of the angular velocity detected from a rotation start time of the head to a first angular velocity detection time at which the angular velocity reaches a first velocity, and a correction section designed to correct a position of a virtual sound source presented to the user based on the final angle of movement. [19] Information processing method which causes an information processing device to perform a detection of an angular velocity of a user's head, to predict a final angle of movement of the head at a rotation end time of the head using an integral value of the angular velocity detected from a rotation start time of the head to a first angular velocity detection time at which the angular velocity reaches a first velocity, and to correct a position of a virtual sound source presented to the user based on the final angle of movement. [20] Program that causes a computer to implement a function for detecting an angular velocity of a user's head, a function for predicting a final angle of movement of the head at a rotation end time of the head using an integral value of the angular velocity detected from a rotation start time of the head to a first angular velocity detection time at which the angular velocity reaches a first velocity, and a function for correcting a position of a virtual sound source presented to the user based on the final angle of movement.

Images