Information processing device, information processing system, information processing method, and program

The information processing system addresses the delay issue in head tracking by predicting and correcting the virtual sound source's position based on the user's head rotation, enhancing the realism of audio experiences by aligning it with the head's movement, thus providing a more immersive audio experience.

WO2024247561A1PCT designated stage expired Publication Date: 2024-12-05SONY GROUP CORP

Patent Information

Application Number
PCT/JP2024/015959
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-29
Filing Date
2024-04-24
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Existing head tracking technologies for audio devices suffer from delays in updating the virtual sound source position based on head rotation, leading to a discrepancy that impairs the sense of realism in audio experiences.

Method used

An information processing system that acquires the angular velocity of the user's head, predicts the final movement angle using integral values, and corrects the position of the virtual sound source to reduce delay and improve realism by aligning it with the head's rotation.

Benefits of technology

The system effectively reduces the delay between head rotation and sound source alignment, providing a more realistic and immersive audio experience by maintaining the virtual sound source's position as if it were fixed in space.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024015959_05122024_PF_FP_ABST
    Figure JP2024015959_05122024_PF_FP_ABST
Patent Text Reader

Abstract

Provided is an information processing device comprising: an acquisition unit that acquires an angular velocity of the head of a user; a prediction unit that predicts a final movement angle of the head of the user at the end of rotation of the head by using an integral value of angular velocities acquired from the start of the rotation of the head to a first angular velocity detection time point at which the angular velocity becomes a first speed; and a correction unit that corrects, on the basis of the final movement angle, the position of a virtual sound source presented to the user.
Need to check novelty before this filing date? Find Prior Art

Description

Information processing device, information processing system, information processing method and program

[0001] The present disclosure relates to an information processing device, an information processing system, an information processing method, and a program.

[0002] For example, head tracking technology has been proposed, which uses an inertial measurement unit built into headphones to track the user's head movements and change the direction of the sound output from the headphones depending on the direction of the head. By using such technology, it is possible to provide users with a clearer and richer sense of presence.

[0003] JP 2010-147529 A

[0004] The tracking technologies proposed so far have been limited in their ability to provide a richer sense of realism because they have not been able to eliminate the factors that impair the sense of realism.

[0005] Therefore, the present disclosure proposes an information processing device, an information processing system, an information processing method, and a program that can provide a user with a more realistic audio experience.

[0006] According to the present disclosure, there is provided an information processing device comprising: an acquisition unit that acquires an angular velocity of a user's head; a prediction unit that predicts a final movement angle of the head at the end of the head rotation using an integral value of the angular velocity acquired from the start of the head rotation to a first angular velocity detection point at which the angular velocity becomes a first velocity; and a correction unit that corrects the position of a virtual sound source presented to the user based on the final movement angle.

[0007] Furthermore, according to the present disclosure, an information processing system is provided that includes an acquisition unit that acquires the angular velocity of the user's head, a prediction unit that predicts the final movement angle of the head at the end of the head rotation using the integral value of the angular velocity acquired from the start of the head rotation to a first angular velocity detection point at which the angular velocity becomes a first velocity, and a correction unit that corrects the position of a virtual sound source presented to the user based on the final movement angle.

[0008] Furthermore, according to the present disclosure, an information processing method is provided, which includes an information processing device acquiring an angular velocity of a user's head, predicting a final movement angle of the head at the end of the head rotation using an integral value of the angular velocity acquired from the start of the head rotation to a first angular velocity detection point at which the angular velocity becomes a first velocity, and correcting the position of a virtual sound source presented to the user based on the final movement angle.

[0009] Furthermore, according to the present disclosure, a program is provided that enables a computer to implement the following functions: a function of acquiring the angular velocity of the user's head; a function of predicting the final movement angle of the head at the end of the head rotation using the integral value of the angular velocity acquired from the start of the head rotation to a first angular velocity detection point at which the angular velocity becomes a first velocity; and a function of correcting the position of a virtual sound source presented to the user based on the final movement angle.

[0010] FIG. 1 is an explanatory diagram (part 1) for explaining the background leading to the first embodiment of the present disclosure. FIG. 2 is an explanatory diagram (part 2) for explaining the background leading to the first embodiment of the present disclosure. FIG. 3 is an explanatory diagram explaining an overview of an information processing system according to the first embodiment of the present disclosure. FIG. 4 is an explanatory diagram explaining the appearance of a voice processing device according to the first embodiment of the present disclosure. FIG. 5 is an explanatory diagram explaining the functional blocks of a voice processing device according to the first embodiment of the present disclosure. FIG. 6 is an explanatory diagram explaining the functional blocks of an information processing device according to the first embodiment of the present disclosure. FIG. 7 is an explanatory diagram explaining the flow of an information processing method according to the first embodiment of the present disclosure. 10 is an explanatory diagram illustrating an information processing method according to a third embodiment of the present disclosure. FIG. 11 is an explanatory diagram illustrating an example of a hardware configuration of an information processing device 900 according to an embodiment of the present disclosure.

[0011] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted. Furthermore, in this specification and the drawings, multiple components having substantially the same or similar functional configurations may be distinguished by adding different letters after the same reference numeral. However, when there is no particular need to distinguish between multiple components having substantially the same or similar functional configurations, only the same reference numerals will be used.

[0012] The drawings referred to in the following description are for explaining and facilitating understanding of one embodiment of the present disclosure, and the configurations shown in the drawings may differ from the actual configurations for clarity. Furthermore, the design of the devices shown in the drawings may be modified as appropriate in consideration of the following description and known techniques.

[0013] The description will be given in the following order: 1. Background 2. First embodiment 2.1 Overview of information processing system 2.2 Detailed configuration of audio processing device 2.3 Detailed configuration of information processing device 2.4 Information processing method 3. Second embodiment 4. Third embodiment 5. Summary 6. Regarding hardware configuration 7. Supplementary information

[0014] <<1. Background>> First, the background that led the inventors to create the first embodiment of the present disclosure will be described with reference to Figures 1 and 2. Figures 1 and 2 are explanatory diagrams for explaining the background that led to the first embodiment of the present disclosure.

[0015] Referring to FIG. 1, a description will be given of outputting sound so that a user 80 feels as if the position of a virtual sound source 82 is fixed in space, regardless of the orientation of the user 80's head, when listening using headphones 10a or the like.

[0016] 1, a user 80 wearing headphones 10a and listening to audio feels as if the audio output from the headphones 10a is coming from in front of the face of the user 80. In other words, a virtual sound source 82 of the audio output from the headphones 10a is located directly in front of the face of the user 80. In this specification, the spatial position of the virtual sound source 82 corresponding to the audio output from the headphones 10a is referred to as the virtual sound source position.

[0017] Furthermore, if no special processing is performed on the output audio (more specifically, if head tracking is not performed), when the head of the user 80 rotates, the virtual sound source position also rotates in accordance with the rotation of the user 80's head, as shown in the center of Figure 1.

[0018] Therefore, head tracking technology is applied to the headphones 10a. In head tracking technology, the rotation angle of the head of the user 80 is detected, and signal processing is performed on the audio data so that the user 80 feels as if the virtual sound source position has rotated by the detected rotation angle in the opposite direction to the rotation of the head of the user 80. The headphones 10a then output audio based on the processed audio data to the user 80. In this way, the user 80 hears the virtual sound source position as if it were fixed in space, as shown on the right side of Fig. 1 .

[0019] For example, this type of head tracking technology is applied to audio recorded using a sound source that can create a sense of spatial expansion, such as a surround sound source. As a result, even when listening through headphones 10a, the user 80 feels as if the sound source 82, such as a speaker or musical instrument, is fixed in space, even if the user 80 moves their head or body. In other words, head tracking technology can provide a realistic audio experience.

[0020] Next, a system for realizing head tracking will be described with reference to FIG. 2 . For example, a user 80 is assumed to be wearing headphones 10a on his / her head, which are compatible with a communication method such as Bluetooth (registered trademark) and are equipped with an inertial measurement unit (sensor) (not shown) that can detect the posture and movement of the user 80's body. The user 80 is also assumed to be watching a video on an information processing device (information processing terminal) such as a smartphone 40a. Audio corresponding to the video is recorded using a surround sound source, and the smartphone 40a receives audio data of the audio from a server (not shown) or the like. The received audio data is then transmitted to the headphones 10a, allowing the user 80 to enjoy realistic, three-dimensional sound through the headphones 10a.

[0021] In this situation, when the user 80 turns their head, an inertial measurement unit (sensor) mounted on the headphones 10a detects the rotational movement of the head and performs calculations to detect the orientation (angle) of the head of the user 80. Furthermore, the headphones 10a transmit the detected orientation (angle) of the head of the user 80 to the smartphone 40a using a communication method such as Bluetooth.

[0022] The smartphone 40a then processes the audio data recorded by the surround sound source so that the virtual sound source position of the audio data faces in the opposite direction to the head of the user 80 who received the data and rotates by the angle corresponding to the rotational movement of the head of the user 80. Furthermore, the smartphone 40a mixes the audio data processed in this manner into, for example, a 2-channel stereo signal and transmits it to the headphones 10a using a communication method such as Bluetooth.

[0023] Furthermore, the headphones 10a receive the audio data and output the audio toward the user 80. As a result, the user 80 can recognize the direction of the virtual sound source position of the audio by listening to the audio.

[0024] As can be seen from the processing flow shown in FIG. 2 described above, many processes are performed between the time the user 80 rotates their head and the time the user 80 hears the sound that reflects that rotation. Therefore, there is a delay time between the time the user 80 starts to rotate their head and the time the sound that reflects that rotation reaches the user 80's ears. In fact, the inventors have conducted research and found that this delay time is approximately several hundred milliseconds. This delay time causes a discrepancy between the rotation of the user 80's head and the virtual sound source position (sound direction) perceived by the user 80, which may affect the audio experience of the user 80 and cause a loss of realism.

[0025] Therefore, in consideration of such circumstances, the present inventors have conducted extensive research and have come up with an embodiment of the present disclosure that can reduce the delay time, i.e., reduce the discrepancy between the rotation of the user 80's head and the virtual sound source position (sound direction) perceived by the user 80. According to the embodiment of the present disclosure, the delay time, i.e., the discrepancy, can be reduced, and a more realistic audio experience can be provided to the user 80. Details of the embodiment of the present disclosure created by the present inventors will be described below one by one.

[0026] <<2. First Embodiment>> <<2.1 Overview of Information Processing System>> First, an overview of an information processing system 1 according to a first embodiment of the present disclosure will be described with reference to Fig. 3. Fig. 1 is an explanatory diagram illustrating a schematic configuration of an information processing system 1 according to an embodiment of the present disclosure.

[0027] As shown in FIG. 3 , the information processing system 1 according to this embodiment includes a voice processing device 10 and an information processing device 40, which are communicatively connected to each other via a communication network 70. The communication method used in the communication network 70 can be any method, whether wired or wireless (e.g., Bluetooth, Wi-Fi (registered trademark), etc.). However, it is preferable to use a wireless communication method such as Bluetooth, which can maintain stable operation and has few restrictions. Furthermore, the voice processing device 10 and the information processing device 40 included in the information processing system 1 are not limited to being one each as shown in FIG. 3 , and multiple voice processing devices 10 and information processing devices 40 may be included. Below, an overview of each device included in the information processing system 1 according to this embodiment will be described.

[0028] (Sound processing device 10) In this embodiment, the sound processing device 10 may be, for example, headphones 10a that are worn on the head of the user 80 and that can output acoustic signals corresponding to the left and right ears of the user 80. Furthermore, in this embodiment, the sound processing device 10 is not limited to being headphones 1a, and may be, for example, earphones, hearing aids, sound collectors, head-mounted displays (HMDs), etc. Details of the functional configuration of the sound processing device 10 will be described later.

[0029] (Information Processing Device 40) The information processing device 40 is an information processing terminal configured to be able to communicate with the voice processing device 10. In this embodiment, for example, the information processing device 40 may be a device such as a smartphone, a tablet PC (Personal Computer), a mobile phone, or a notebook PC. In addition, in this embodiment, the voice processing device 10 and the information processing device 40 may be configured as an integrated device. Furthermore, in this embodiment, the information processing device 40 may be a server on a cloud, etc., and in this case, may be configured by one or more devices. Note that details of the functional configuration of the information processing device 40 will be described later.

[0030] In the embodiment of the present disclosure, the functional configuration of the information processing system 1 is not limited to the form shown in Fig. 3. For example, the information processing system 1 may include other communication devices such as a relay device used when transmitting information between the voice processing device 10 and the information processing device 40.

[0031] <<2.2 Detailed Configuration of Audio Processing Device>> Next, details of the audio processing device 10 according to this embodiment will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is an explanatory diagram illustrating the appearance of the audio processing device 10 according to this embodiment, and Fig. 5 is a block diagram illustrating functional blocks of the audio processing device 10 according to this embodiment.

[0032] The external appearance of the audio processing device 10 according to this embodiment will be described with reference to Fig. 4. Here, it is assumed that the audio processing device 10 according to this embodiment is a headphone.

[0033] 4 , the sound processing device 10 includes, for example, a slider 3, hangers 4L and 4R, housings 5L and 5R, ear pads 6L and 6R, and a headband 7. For example, a driver unit (not shown) and the like are provided inside the housings 5L and 5R. When using the sound processing device 10, a user 80 wears the headband 7 on the top of the user's head and places the ear pads 6L and 6R against the user's outer ears, thereby being able to listen to sounds output from the housings 5L and 5R.

[0034] The sliders 3 are provided on both ends of the headband 7. A hanger (hanger 4L or 4R) is attached to the joint at one end of the slider 3. The sliders 3 are fixed to both ends of the headband 7 and are configured to be slidable along guide members (not shown) having an axis aligned with the central axis of the headband 7. By sliding the slider 3 along the guide members, the hangers 4L and 4R can be moved closer or farther from the headband 7. When wearing the audio processing device 10, the position of the slider 3 is adjusted according to the size of the user's head, the distance between the ears and the top of the head, and other factors. This allows the user 80 of the audio processing device 10 to achieve a wearing comfort that suits their physical characteristics and preferences. On the other hand, when the audio processing device 10 is not in use, the slider 3 can be retracted to save storage space for the audio processing device 10.

[0035] The hangers 4L and 4R are attached to the tip of the slider 3 via a joint (not shown), and can rotatably support the housings 5L and 5R.

[0036] The housings 5L and 5R have storage space therein and house a driver unit (not shown) that converts electrical signals into sound waves and outputs them. The housings 5L and 5R are formed using, for example, synthetic resin such as plastic. The housings 5L and 5R have the effect of blocking air from passing through the front and back sides of the speaker diaphragm. The housings 5L and 5R also serve to efficiently transmit acoustic radiation from the front of the diaphragm to the eardrum of the user 80 by sealing or semi-sealing the space from the front of the speaker diaphragm to the eardrum of the user 80 with a housing having acoustic resistance, ducts, etc. The driver units also house speakers, etc.

[0037] The ear pads 6L, 6R are provided on the surfaces of the housings 5L, 5R that face the temporal region of the user 80. By being interposed between the housings 5L, 5R and the temporal region of the user 80, the ear pads 6L, 6R can function as buffer members between the housings 5L, 5R and the temporal region of the user 80. In other words, the ear pads 6L, 6R can prevent the housings 5L, 5R, which are made of a hard material that is difficult to deform, from coming into direct contact with the ears and temporal region of the user 80, causing discomfort or pain to the user 80, when the sound processing device 10 is worn. The ear pads 6L, 6R are configured to be covered with synthetic leather or cloth made of sponge or the like.

[0038] 4, the headband 7 is curved to fit the head of the user 80 and can support the entire sound processing device 10 by contacting the top of the user's head when worn. The headband 7 is made of synthetic resin such as plastic, metal, or the like, and has a predetermined rigidity and elasticity, making it flexible. In this embodiment, rubber or the like may be provided as a cushioning material on the inner surface of the headband 7 that contacts the top of the user's head. A control unit (not shown) that stores a communication unit 30, etc., as described below, may be provided inside the housings 5L, 5R, etc. Furthermore, in this embodiment, the sound processing device 10 may have a hinge (not shown) so that it can be folded in the middle when carried.

[0039] Next, functional blocks of the audio processing device 10 according to this embodiment will be described with reference to Fig. 5. As shown in Fig. 5, the audio processing device 10 mainly includes a gyro sensor 12, an acceleration sensor 14 (inertial measurement unit), an output unit 16, an audio data acquisition unit 20, an audio output unit 22, and a communication unit 30. Details of each functional block included in the audio processing device 10 according to this embodiment will be described below.

[0040] (Gyro Sensor 12) The gyro sensor (angular velocity sensor) 12 is an inertial sensor that acquires angular velocity as sensing data. Specifically, the gyro sensor 12 is composed of, for example, three sensors corresponding to three axes, and can measure the angular velocity of rotation around the yaw axis (rotation around the Z axis perpendicular to the horizontal plane), rotation around the roll axis (rotation around the X axis on a horizontal plane when the X axis is the line of sight of the user 80 wearing the audio processing device 10), and rotation around the pitch axis (rotation around the Y axis on a horizontal plane when the Y axis is an axis perpendicular to the X axis on the horizontal plane). In this embodiment, the gyro sensor 12 is sufficient as long as it includes a sensor that measures at least the angular velocity of rotation around the yaw axis, and this sensor can measure the angular velocity around the upright axis of the user 80's head. The sensing data acquired by the gyro sensor 12 is output to the output unit 16, which will be described later. For example, the gyro sensor 12 may be provided in a control unit (not shown) inside the housings 5L, 5R, etc.

[0041] (Acceleration Sensor 14) The acceleration sensor 14 is an inertial sensor that acquires acceleration as sensing data. The sensing data acquired by the acceleration sensor 14 is output to the output unit 16, which will be described later. For example, the acceleration sensor 14 may be provided in a control unit (not shown) inside the housings 5L, 5R, etc. Furthermore, in this embodiment, the acceleration sensor 14 may not be provided. Furthermore, in this embodiment, a sensor of a type different from the gyro sensor 12 and the acceleration sensor 14 may be used. For example, in this embodiment, a biosensor (not shown) or an imaging sensor (not shown) such as a camera may be used as appropriate.

[0042] (Output Unit 16 ) The output unit 16 can perform signal conversion processing such as digital signal conversion on the sensing data from the gyro sensor 12 and the acceleration sensor 14 and transmit the data to the information processing device 40 via the communication unit 30 .

[0043] (Audio Data Acquisition Unit 20) The audio data acquisition unit 20 performs predetermined signal processing on audio data (acoustic signals) acquired from the information processing device 40 via the communication unit 30 (described later), and outputs the processed audio data to the audio output unit 22 (described later). The audio data acquisition unit 20 can be configured by, for example, a memory and a processor having hardware such as a DSP (Digital Signal Processor).

[0044] (Audio Output Unit 22) The audio output unit 22 performs signal conversion processing such as analog signal conversion on the audio data from the audio data acquisition unit 20, and outputs the audio data to the driver units (not shown) inside the housings 5L, 5R. Note that, since there is audio data for the right ear and audio data for the left ear, the audio output unit 22 outputs the audio data to the corresponding speakers of the driver units inside the housings 5L, 5R.

[0045] (Communication Unit 30) The communication unit 30 can communicate with the information processing device 40 via the communication network 70 in accordance with a predetermined communication standard. Here, examples of the predetermined communication standard include Bluetooth and Wi-Fi. The communication unit 30 can be configured using, for example, a communication module.

[0046] Furthermore, although not shown, in the audio processing device 10, a control unit that controls each functional block of the audio processing device 10 may be provided in a control unit (not shown) inside the housing 5L, 5R, etc. The control unit may be configured using, for example, a memory and a processor having hardware such as a CPU (Central Processing Unit) or a DSP. Furthermore, although not shown, the audio processing device 10 may have a battery that supplies power to each functional block of the audio processing device 10. The battery may be, for example, a rechargeable secondary battery such as a lithium-ion battery. Furthermore, the battery may be configured to be rechargeable with power supplied from a charger via a connection unit (not shown).

[0047] In this embodiment, the appearance and functional block configuration of the audio processing device 10 are not limited to those shown in Figures 4 and 5. In this embodiment, the audio processing device 10 may be, for example, a wireless earphone.

[0048] <<2.3 Detailed Configuration of Information Processing Device>> Next, details of the information processing device 40 according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram illustrating functional blocks of the information processing device according to this embodiment. As shown in Fig. 6, the information processing device 40 mainly includes a head posture prediction unit (acquisition unit, prediction unit) 42, an audio data acquisition unit 50, an audio data correction unit (correction unit) 52, an audio data output unit (output unit) 54, a communication unit 60, and a storage unit 62. Details of each functional block included in the information processing device 40 according to this embodiment will be described below in order.

[0049] (Head posture prediction unit 42) The head posture prediction unit 42 acquires sensing data from the gyro sensor 12 of the audio processing device 10 via the communication unit 60, which will be described later, and detects the rotation angle of the head of the user 80 from the angular velocity of the head of the user 80 obtained from the sensing data. For example, the current rotation angle of the head of the user 80 can be obtained by time integrating the angular velocities acquired from the start of head rotation to the present time. The head posture prediction unit 42 then outputs the detected angular velocity and rotation angle of the head of the user 80 to the audio data correction unit 52 and the memory unit 62, which will be described later.

[0050] Furthermore, in this embodiment, the head posture prediction unit 42 predicts the rotation angle (final movement angle) until the head of the user 80 finally stops rotating, based on the sensing data from the gyro sensor 12. In particular, the head posture prediction unit 42 acquires the angular velocity of the rotation of the head of the user 80, and when it detects that the angular velocity has reached a predetermined velocity (first velocity) based on the acquired angular velocity (first angular velocity detection time point), it predicts the final movement angle of the head when the head rotation ends (rotation end time) using the integral value of the angular velocity acquired from the start of rotation to the first angular velocity detection time point. More specifically, the head posture prediction unit 42 acquires the angular velocity of the rotation of the head of the user 80, and when it detects that the angular velocity has reached its fastest based on the acquired angular velocity (fastest angular velocity detection time point), it predicts the final movement angle of the head when the head rotation ends (rotation end time) using the integral value of the angular velocity acquired from the start of rotation to the fastest angular velocity detection time point. Furthermore, after the fastest angular velocity is detected, head posture prediction unit 42 predicts the final head movement angle when the head rotation ends, using the integral value of the angular velocities acquired from the start of rotation to the present time. Then, head posture prediction unit 42 outputs the predicted final head movement angle of user 80 to audio data correction unit 52, which will be described later. Note that head posture prediction unit 42 repeatedly predicts the final head movement angle at predetermined time intervals until the head rotation ends. Note that the prediction method used by head posture prediction unit 42 will be described in detail later.

[0051] (Audio Data Correction Unit 52) ​​The audio data correction unit 52 corrects the audio data (more specifically, each of the multiple frames constituting one piece of audio data) so that the virtual sound source positions corresponding to the audio data output to the left and right ears of the user 80, i.e., the virtual sound source positions presented to the user, rotate in the direction opposite to the direction of rotation of the head of the user 80. The audio data correction unit 52 then sequentially outputs the corrected audio data to the audio data output unit 54, which will be described later. The audio data correction unit 52 sequentially corrects the audio data according to the current movement angle of the head of the user 80, and the corrected audio data is sequentially output by the audio processing device 10 to the user 80, allowing the user to hear audio from a virtual sound source fixed in space even while their head is rotating.

[0052] In detail, in this embodiment, the audio data correction unit 52 corrects the audio data until the fastest angular velocity is detected so that the virtual sound source position follows the head rotation of the user 80 and rotates in the opposite direction to the head rotation by the movement angle of the head rotation of the user 80. Furthermore, from the time of detecting the fastest angular velocity, the audio data correction unit 52 corrects the audio data so that the virtual sound source position rotates in the opposite direction to the head rotation, using a predicted value of the final head movement angle when the head rotation ends (at the end of rotation) by the head posture prediction unit 42. For example, the audio data correction unit 52 corrects the audio data so that the virtual sound source position rotates in the opposite direction to accurately follow the head of the user 80, using the current head movement angle of the user 80, the current head angular velocity, and the predicted value of the final head movement angle. Note that the audio data correction unit 52 repeatedly performs the correction at predetermined time intervals until the head rotation ends.

[0053] (Audio Data Output Unit 54) The audio data output unit 54 transmits the audio data corrected by the audio data correction unit 52 to the audio processing device 10 via a communication unit 60, which will be described later.

[0054] The head posture prediction unit 42, the audio data correction unit 52, and the audio data output unit 54 described above can be realized by hardware such as a CPU, a ROM, and a RAM provided in the information processing device 40.

[0055] (Communication Unit 60) The communication unit 60 can communicate with the voice processing device 10 in accordance with a predetermined communication standard via the communication network 70. The communication unit 60 can be configured using, for example, a communication module or the like.

[0056] (Storage Unit 62) The storage unit 62 stores various information such as audio data such as streamed music. The storage unit 62 can be configured using, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a memory card, etc. The storage unit 62 can also store programs executed by the information processing device 40 and various data used in information processing by the information processing device 40.

[0057] Although not shown, the information processing device 40 is provided with a control unit and the like that controls each functional block of the information processing device 40. The control unit can be configured using, for example, a memory and a processor having hardware such as a CPU or a DSP.

[0058] In this embodiment, the functional block configuration of the information processing device 40 is not limited to the form shown in Fig. 6. The details of the hardware configuration of the information processing device 40 will be described later.

[0059] In addition, in this embodiment, as described above, the voice processing device 10 and the information processing device 40 may be configured as an integrated device. Furthermore, in this embodiment, some of the functional blocks of the information processing device 40 may be provided in the voice processing device 10, or some of the functional blocks of the voice processing device 10 may be provided in the information processing device 40. Furthermore, in this embodiment, some or all of the functional blocks of the information processing device 40 and the voice processing device 10 may be performed on a server (not shown).

[0060] <<2.4 Information Processing Method>> Next, the information processing method according to this embodiment will be described with reference to Fig. 7 to Fig. 14. Fig. 7 is a flowchart illustrating the flow of the information processing method according to this embodiment, and Fig. 8 to Fig. 14 are explanatory diagrams illustrating the information processing method according to this embodiment.

[0061] 7, the information processing method according to this embodiment may mainly include steps S101 to S105. In the information processing method described below, steps S103 and S104 are repeatedly executed until the rotation of the head of the user 80 stops. Each of these steps according to this embodiment will be described in detail below.

[0062] 7 , the voice processing device 10 included in the information processing system 1 according to this embodiment starts measuring the rotation of the head of the user 80 (step S101). Next, the information processing device 40 included in the information processing system 1 according to this embodiment acquires the angular velocity of the rotation of the head of the user 80, and detects that the angular velocity has reached its fastest state based on the acquired angular velocity (step S102).

[0063] The information processing device 40 then uses the integral value of the angular velocity acquired up to that point to predict the final head movement angle when the head rotation is completed. Next, the information processing device 40 corrects the audio data using the predicted final movement angle so that the virtual sound source position rotates in the direction opposite to the head rotation (step S103). Furthermore, the information processing device 40 transmits the corrected audio data to the audio processing device 10 (step S104).

[0064] When the user's head rotation has finished, the voice processing device 10 ends the measurement of the head rotation of the user 80 (step S105) and ends the process.

[0065] Furthermore, each step in the processing of the above-described embodiment is not necessarily limited to being executed by the described device, but may be performed by a functional unit of another device.

[0066] 8 to 12, the details of the prediction method in step S103 will be described. The inventors of the present invention believed that if the end angle of the rotational movement of the user 80's head could be predicted while the head of the user 80 is rotating, the predicted value could be used to rotate the virtual sound source position, thereby reducing the delay time described above, i.e., reducing the discrepancy between the rotation of the user 80's head and the virtual sound source position (sound direction) perceived by the user 80. The inventors of the present invention then conducted extensive research based on this idea and independently discovered the following, which led to the concept of this embodiment.

[0067] FIG. 8 illustrates angular velocities measured when a user 80 wears a triaxial gyro sensor 12 on their head and swings their head from side to side. The inventors have discovered that, as shown in FIG. 8 , the change over time in angular velocity during a rotational movement of the user 80's head in a horizontal plane (indicated by Z in FIG. 8 ) has a generally symmetrical mountain-like shape. Therefore, the inventors came up with the idea of ​​predicting the end angle of the head rotational movement by utilizing the fact that the change over time in angular velocity during a rotational movement of the user 80's head in a horizontal plane has a symmetrical mountain-like shape. In other words, in this embodiment created by the inventors, the change in angular velocity of the user 80's head from the time when the fastest angular velocity is detected to the time when the rotation ends is assumed to be the same as the change in angular velocity from the time when the rotation starts to the time when the fastest angular velocity is detected, and the end angle of the head rotational movement is predicted.

[0068] The angle of movement due to a rotational movement can be obtained by integrating (taking the area) the change in angular velocity over time. As shown in Figure 9, when the change in angular velocity forms a symmetrical mountain shape, the angle of movement in the first half of the rotation is equal to the angle of movement in the second half of the rotation. Therefore, in this embodiment, by assuming that the angle of movement in the second half can be approximated by the angle of movement in the first half, the final angle of movement of the head when the rotation of the head is completed (at the end of rotation) is predicted before the rotational movement ends.

[0069] Next, a method of predicting from the start to the end of rotation will be specifically described with reference to Figures 10 to 12. In this embodiment, prediction is performed for rotation around the yaw axis (rotation around the Z axis perpendicular to the horizontal plane), which is an azimuth rotation with a relatively large rotational speed (angular velocity). Note that in this embodiment, the rotation to be predicted (and corrected) is not limited to rotation around the yaw axis, but may also be performed for rotation around the roll axis (rotation around the X axis on a horizontal plane) and rotation around the pitch axis (rotation around the Y axis on a horizontal plane).

[0070] First, the section from the start of rotation to when the fastest angular velocity is reached will be described with reference to Fig. 10. During this section, the prediction process according to this embodiment is not performed, and only the angular velocity is recorded for use in the prediction process to be performed in a later section. That is, the audio data correction unit 52 corrects the audio data so that it follows the head rotation until the fastest angular velocity is detected, and the virtual sound source position rotates in the opposite direction to the head rotation.

[0071] Next, the time when the fastest angular velocity is reached will be described with reference to FIG. 11 . At the time of detecting the fastest angular velocity, the head posture prediction unit 42 predicts the final movement angle using the integral value of the angular velocity acquired from the start of rotation to the time t of the fastest angular velocity detection, i.e., the integral value (area) of the angular velocity for the first half section. In particular, the head posture prediction unit 42 adds the integral value (area) of the angular velocity for the first half section to the current head movement angle. That is, the head posture prediction unit 42 predicts the final head movement angle at the end of rotation by doubling the integral value of the angular velocity acquired from the start of rotation to the time t of the fastest angular velocity detection. Then, at the time of detecting the fastest angular velocity, the audio data correction unit 52 corrects the audio data using this predicted value so that the virtual sound source position rotates in the direction opposite to the head rotation. Note that, in the basic form of this embodiment, the final head movement angle at the end of rotation is predicted by doubling the integral value of the angular velocity acquired from the start of rotation to the time t of the fastest angular velocity detection. However, in this embodiment, the values ​​of the time t of the fastest angular velocity detection and the two times the integral of the angular velocity do not necessarily have to be used. In this embodiment, for example, the integral of the angular velocity acquired during the time from the start of rotation to the time t of the fastest angular velocity detection, or during the time from the time t of the fastest angular velocity detection to the time T of the rotation (T≠t), may be multiplied by N (a predetermined value) (N≠2). Alternatively, in this embodiment, for example, the integral of the angular velocity acquired during the time from the start of rotation to the time (first angular velocity detection) when the angular velocity reaches a predetermined velocity (first velocity), or during the time from the first angular velocity detection to the time T of the rotation, may be multiplied by N (N≠2). Here, N may be set to a real value other than an integer (e.g., 1.9 or 2.01). Furthermore, the above-described t, T, and N may be appropriately set by the user, or may be dynamically set or changed by the system.

[0072] 12 , the section from when the fastest angular velocity is reached to when the rotation ends will be described. In this section, the head posture prediction unit 42 identifies a time (third angular velocity detection time) t' before the fastest angular velocity detection time at which an angular velocity (third angular velocity) having the same value as the angular velocity (second angular velocity) acquired at the current time (second angular velocity detection time) t, which is after the fastest angular velocity detection time, was acquired. Next, the head posture prediction unit 42 predicts the final movement angle using the integral of the angular velocities acquired from the start of rotation to time t'. Specifically, the head posture prediction unit 42 predicts the final movement angle of the head at the end of the rotation by adding the integral (area) of the angular velocities acquired up to time t' to the movement angle at the current time t. Then, after the fastest angular velocity detection time, the audio data correction unit 52 uses this predicted value to correct the audio data so that the virtual sound source position rotates in the direction opposite to the head rotation. In this embodiment, such prediction and correction are repeatedly performed at predetermined time intervals during the period from when the fastest angular velocity is reached to the end of rotation. Note that in this embodiment, it is preferable to appropriately set the time intervals for performing prediction and correction, taking into consideration the processing time, processing load, battery consumption, etc.

[0073] Note that if an extremely large final movement angle is used to correct the audio data, the virtual sound source position will rotate abruptly, which may cause discomfort to the user 80. Therefore, in this embodiment, in order to prevent such discomfort, an upper limit may be set for the final movement angle predicted by the above-described method. Note that the upper limit may be set appropriately by the user, or may be dynamically set or changed by the system depending on the processing time, processing load, battery consumption, type of applied content, etc.

[0074] Next, the effects of this embodiment will be described with reference to Figures 13 and 14. In Figures 13 and 14, the horizontal axis represents elapsed time, and the vertical axis represents the angle of head movement. Furthermore, in Figures 13 and 14, the solid line represents the actual angle of head rotation movement of the user 80, and the dashed line represents the angle of movement of the virtual sound source position of the sound perceived by the user 80.

[0075] 13, in a comparative example in which the prediction according to this embodiment is not applied, there is a discrepancy between the actual head rotation angle (solid line) and the movement angle (dotted line) of the virtual sound source position of the sound that reflects the actual head rotation of the user as perceived by the user 80. Therefore, the vertical difference between the solid line and the dotted line at a certain point in time is perceived by the user 80 as a discrepancy in the direction of the sound.

[0076] On the other hand, in an example in which prediction according to this embodiment is applied, the predicted value is represented by a dashed-dotted line as shown in FIG. 14 , and the audio data is corrected so that the virtual sound source position rotates to reflect the predicted value. In the example, as shown in FIG. 14 , the deviation between the actual head rotation angle (solid line) and the movement angle (dotted line) of the virtual sound source position of the audio, which reflects the predicted value (dotted-dotted line) as perceived by the user 80, becomes smaller near the end point of the rotation. Therefore, near the end point, the vertical difference between the solid line and the dotted line becomes smaller, making it difficult for the user 80 to perceive a deviation in the sound direction. In other words, by using prediction according to this embodiment, the virtual sound source position in the example reaches the vicinity of the end point of the rotation at an earlier timing than in the comparative example. That is, according to this embodiment, the deviation in the sound direction is reduced near the end point.

[0077] As described above, in this embodiment, the change over time in the angular velocity of the rotational movement of the user's 80's head in the horizontal plane has a symmetrical mountain-like shape. Taking advantage of this, the change in the angular velocity of the user's 80's head from the time of the fastest angular velocity detection to the time of the end of the rotation is assumed to be the same as the change in the angular velocity from the time of the start of the rotation to the time of the fastest angular velocity detection, and the end angle of the rotational movement of the head is predicted. Then, in this embodiment, the predicted value is used to correct the audio data. Therefore, this embodiment can reduce the delay time, that is, the discrepancy between the rotation of the user's 80's head and the virtual sound source position (sound direction) perceived by the user 80. As a result, this embodiment can clarify the effect of the virtual sound source position sounding fixed in space, providing a more realistic audio experience.

[0078] As described above, in this embodiment, the final movement angle is predicted on the assumption that the change in angular velocity over time due to the rotational movement forms a symmetrical mountain-like shape. However, further processing may be performed to improve the accuracy of the prediction. For example, when the user 80 first begins using the information processing system 1 according to this embodiment, the prediction is performed using only the above-described method. Then, as the user 80 uses the information processing system 1 several times, a machine learning model (not shown) included in the information processing system 1 extracts characteristics of the change in angular velocity over time unique to the user 80. Examples of such characteristics include the fact that the change in angular velocity over time due to the rotational movement may form a non-symmetrical mountain-like shape, such as the time from reaching the maximum angular velocity to stopping in a clockwise rotation being shorter than in a counterclockwise rotation. In this embodiment, the accuracy of the prediction can be improved by predicting the final movement angle based on the characteristics of each individual user 80 extracted by such a machine learning model. Furthermore, in this embodiment, the accuracy of the prediction may be improved not only by extracting characteristics using the machine learning model, but also by using information (attribute information) such as the user 80's skeletal and muscular characteristics and dominant hand.

[0079] <<3. Second Embodiment>> In the first embodiment described so far, the virtual sound source position is rotated in the opposite direction to the rotation of the head of the user 80, thereby making it seem as if the virtual sound source position is fixed in space to the user 80. Furthermore, in the second embodiment of the present disclosure, the movement angle of the virtual moving sound source is changed according to the relative angle of the virtual sound source position with respect to the face (head) of the user 80. In this way, in this embodiment, the directionality of sound experienced by the user 80 is improved, the effect of hearing the virtual sound source position as if it were fixed in space is made clearer, and a more realistic audio experience can be provided.

[0080] This embodiment will be described below with reference to Fig. 15. Fig. 15 is an explanatory diagram for explaining an information processing method according to this embodiment, and in detail is a diagram showing the relationship between the rotation angle of the head of the user 80 and the movement angle applied to the voice (the movement angle of a virtual moving sound source).

[0081] Humans perceive the movement of sound differently depending on the direction of the sound. For example, generally, humans can sensitively sense the movement of the sound position when the sound comes from the front side of the person, but it is difficult for humans to sensitively sense the movement of the sound position when the sound comes from the back side of the person. Therefore, in this embodiment, the movement angle of the virtual moving sound source is changed depending on the relative angle of the virtual sound source position with respect to the face (head) of the user 80, in accordance with the human hearing characteristics. In this way, the directionality of the sound experienced by the user 80 can be improved depending on the human hearing characteristics.

[0082] 15 , when the virtual sound source position is on the front side of the face of the user 80, the audio data correction unit 52 corrects the audio data so that the virtual sound source position has a movement angle smaller than the movement angle of the head. More specifically, when the angle is between −90 degrees and 90 degrees with respect to the front of the face of the user 80, for example, the audio data correction unit 52 multiplies the movement angle of the head by a coefficient (first coefficient) R that is greater than or equal to 0 and less than 1 to obtain the movement angle of the virtual sound source position. On the other hand, when the virtual sound source position is on the opposite side from the front of the face of the user 80, the audio data correction unit 52 corrects the audio data so that the virtual sound source position has a movement angle larger than the movement angle of the head. More specifically, when the angle of the head is, for example, -180 degrees to -90 degrees and 90 degrees to 180 degrees with respect to the front of the face of the user 80 as the reference, the audio data correction unit 52 multiplies the head movement angle by a coefficient (second coefficient) R greater than 1 to obtain the movement angle of the virtual sound source position.

[0083] 15 shows specific angle numbers, but these are merely examples, and the present embodiment is not limited to these angles and coefficient R. For example, in the present embodiment, the angles and coefficient R may be selected appropriately by the user, or may be dynamically changed by the system.

[0084] Furthermore, in this embodiment, the user 80 may be asked to listen in advance to sounds corresponding to virtual sound source positions in various directions, and to respond to questions about the degree of sound perception and preferences for sounds, and the coefficient R may be adjusted according to the response results. In this way, the directionality of sound experienced by the user 80 can be improved according to the hearing characteristics of the individual user 80.

[0085] As described above, in this embodiment as well, the directionality of sound as perceived by the user 80 is improved, the effect of the virtual sound source position sounding as if it were fixed in space is made clearer, and a more realistic audio experience can be provided.

[0086] <<4. Third Embodiment>> In the first embodiment described so far, the virtual sound source position is rotated in the opposite direction to the rotation of the head of the user 80, thereby making it seem as if the virtual sound source position is fixed in space to the user 80. Furthermore, in a third embodiment of the present disclosure, the sound pressure of the output sound is changed according to the relative angle of the virtual sound source position with respect to the face (head) of the user 80. In this way, in this embodiment, the directionality and localization of the sound experienced by the user 80 are improved, the effect of the virtual sound source position being heard as if fixed in space is made clearer, and a more realistic audio experience can be provided.

[0087] This embodiment will be described below with reference to Fig. 16. Fig. 16 is an explanatory diagram for explaining an information processing method according to this embodiment, and in detail, is a diagram showing the relationship between the virtual sound source position (sound source direction) with respect to the user 80 and the weighting of the sound pressure of the voice.

[0088] It is generally known that the actual sound pressure perceived by humans varies depending on the direction of the sound. For example, ITU-R BS.1770 indicates that even sounds with the same physical sound pressure will have different audible sound pressure depending on the direction of the sound.

[0089] Therefore, in this embodiment, in order to achieve more natural head tracking, sound pressure control is performed taking into consideration differences in sound pressure levels perceived depending on the direction of sound due to human hearing characteristics. In this way, this embodiment can improve the directionality and localization of sound experienced by the user 80, clarify the effect of hearing the virtual sound source position as if it were fixed in space, and provide a more realistic audio experience.

[0090] More specifically, in this embodiment, the sound pressure of the output sound is changed according to the relative angle of the virtual sound source position with respect to the face (head) of the user 80 .

[0091] 16 , in this embodiment, for example, when the virtual sound source position is in front of the face of the user 80, the audio data correction unit 52 corrects the audio data without weighting the sound pressure (0 dB). On the other hand, when the virtual sound source position is on the opposite side of the face of the user 80, the audio data correction unit 52 corrects the audio data so as to weight the sound pressure (1.5 dB). For example, when the virtual sound source position is at an angle smaller than −110 degrees and equal to or greater than +110 degrees with respect to the front of the face of the user 80, the audio data correction unit 52 performs correction so that the sound sounds 1.5 dB louder than when the virtual sound source position is in front of the face of the user 80.

[0092] 16 shows specific angle numbers and sound pressure weighting numbers, but these are merely examples, and the present embodiment is not limited to these numbers, etc. For example, in the present embodiment, the angle and sound pressure weighting may be selected appropriately by the user, or may be dynamically changed by the system.

[0093] Furthermore, in this embodiment, the user 80 may be asked to listen in advance to sounds corresponding to virtual sound source positions in various directions, and may be asked to respond regarding the degree of sound perception, preferences for sound, etc., and the angle and weighting may be adjusted according to the response results. In this way, the sound directionality experienced by the user 80 can be improved according to the hearing characteristics of the individual user 80.

[0094] As described above, in this embodiment as well, the directionality of sound as perceived by the user 80 is improved, the effect of the virtual sound source position sounding as if it were fixed in space is made clearer, and a more realistic audio experience can be provided.

[0095] <<5. Summary>> As described above, according to the embodiment of the present disclosure, it is possible to eliminate factors that impair the sense of realism, thereby providing a more realistic audio experience.

[0096] In the above-described embodiments of the present disclosure, prediction and correction have been described for rotation around the yaw axis (rotation around the Z axis perpendicular to the horizontal plane), which is an azimuth rotation with a relatively large rotation speed (angular velocity). However, in each embodiment of the present disclosure, the rotation to be predicted and corrected is not limited to rotation around the yaw axis, and may also be applied to rotation around the roll axis (rotation around the X axis on a horizontal plane) and rotation around the pitch axis (rotation around the Y axis on a horizontal plane).

[0097] <<6. Hardware Configuration>> Fig. 17 is an explanatory diagram showing an example of the hardware configuration of an information processing device 900 according to this embodiment. In Fig. 17, the information processing device 900 shows an example of the hardware configuration of the information processing device 40 according to the embodiment of the present disclosure described above. The information processing device 900 can be a smartphone or the like.

[0098] 17 , the information processing device 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903. The information processing device 900 also includes a storage device 904, a communication module 905, and a sensor module 907. The information processing device 900 also includes an imaging device 908, a display device 910, a speaker 911, a microphone 912, an input device 913, and a bus 914. The information processing device 900 may also include a processing circuit such as a DSP (Digital Signal Processor) instead of or in addition to the CPU 901.

[0099] The CPU 901 functions as an arithmetic processing device and control device, and controls all or part of the operations within the information processing device 900 in accordance with various programs recorded in the ROM 902, RAM 903, storage device 904, or the like. For example, the CPU 901 can be the head pose prediction unit 42, the audio data acquisition unit 50, the audio data correction unit 52, and the audio data output unit 54, etc., according to the above-described embodiments of the present disclosure. The ROM 902 stores programs and calculation parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901, parameters that change as appropriate during the execution, etc. The CPU 901, ROM 902, and RAM 903 are interconnected by a bus 914. The storage device 904 is a data storage device configured as an example of a storage unit of the information processing device 900. The storage device 904 is configured by, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, etc. This storage device 904 stores programs and various data executed by the CPU 901, as well as various data acquired from the outside. For example, the storage device 904 can be the storage unit 62 according to the embodiment of the present disclosure described above.

[0100] The communication module 905 is a communication interface configured with, for example, a communication device for connecting to the communication network 906. The communication module 905 may be, for example, a communication card for a wired or wireless local area network (LAN), Bluetooth, or wireless universal serial bus (WUSB). The communication module 905 may also be a router for optical communication, a router for asymmetric digital subscriber line (ADSL), or a modem for various communications. The communication module 905 transmits and receives signals between the Internet and other communication devices using a predetermined protocol such as TCP (Transmission Control Protocol) / IP (Internet Protocol). Furthermore, the communication network 906 connected to the communication module 905 is a wired or wireless network, such as the Internet, a home LAN, infrared communication, satellite communication, etc. For example, the communication module 905 can be the communication unit 60 according to the embodiment of the present disclosure described above.

[0101] The sensor module 907 includes various sensors such as a motion sensor (e.g., an acceleration sensor, a gyro sensor, a geomagnetic sensor, etc.), a biometric information sensor (e.g., a pulse sensor, a blood pressure sensor, a fingerprint sensor, etc.), or a position sensor (e.g., a GNSS (Global Navigation Satellite System) receiver, etc.).

[0102] The imaging device 908 is provided on the surface of the information processing device 900, and can capture an image of an object or the like located on the back or front side of the information processing device 900. Furthermore, the imaging device 908 can further include an optical system mechanism (not shown) configured with an imaging lens, a zoom lens, a focus lens, etc., and a drive system mechanism (not shown) that controls the operation of the optical system mechanism.

[0103] The display device 910 is provided on the surface of the information processing device 900, and can be, for example, a display device such as an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display, etc. The display device 910 can display an operation screen, an image captured by the above-described imaging device 908, etc.

[0104] The speaker 911 can output, for example, telephone call audio and audio accompanying the video content displayed by the display device 910 described above to the user.

[0105] The microphone 912 can collect, for example, the user's call voice, voice including a command to activate a function of the information processing device 900, and voice from the surrounding environment of the information processing device 900.

[0106] The input device 913 is a device operated by a user, such as a button, keyboard, touch panel, mouse, etc. The input device 913 includes an input control circuit that generates an input signal based on information input by the user and outputs the signal to the CPU 901. By operating the input device 913, the user can input various data to the information processing device 900 and instruct processing operations.

[0107] The above describes an example of the configuration of the information processing device 900. Each of the above components may be configured using general-purpose components, or may be configured using hardware specialized for the function of each component. Such a configuration may be changed as appropriate depending on the technical level at the time of implementation.

[0108] Furthermore, the information processing device according to the present embodiment may be a system consisting of one or more devices that is premised on connection to a network (or communication between devices), such as cloud computing, etc. In other words, the information processing device according to the present embodiment described above can also be realized as an information processing system in which a plurality of devices perform processing related to the information processing method according to the present embodiment.

[0109] <<7. Supplementary Information>> The above-described embodiment of the present disclosure may include, for example, a program for causing a computer to function as an information processing device according to the embodiment, and a non-transitory tangible medium on which the program is recorded. Furthermore, the program may be distributed via a communication line (including wireless communication) such as the Internet.

[0110] Furthermore, the steps in the processing of the present embodiment described above do not necessarily have to be processed in the order described. For example, the steps may be processed in a different order as appropriate. Furthermore, instead of being processed in chronological order, the steps may be partially processed in parallel or individually. Furthermore, the processing method of each step does not necessarily have to be processed in the manner described; for example, the steps may be processed in a different manner by another functional unit.

[0111] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0112] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0113] The present technology may also be configured as follows. (1) An information processing device comprising: an acquisition unit that acquires an angular velocity of a user's head; a prediction unit that predicts a final movement angle of the head at the end of the head rotation using an integral value of the angular velocity acquired from the start of the head rotation to a first angular velocity detection time point at which the angular velocity becomes a first velocity; and a correction unit that corrects a position of a virtual sound source presented to the user based on the final movement angle. (2) The information processing device described in (1), wherein the correction unit corrects the position of the virtual sound source so that it rotates in a direction opposite to the head rotation direction. (3) The information processing device described in (2), wherein the prediction unit predicts the final movement angle by multiplying the integral value of the angular velocity acquired from the start of the rotation to the first angular velocity detection time point by a predetermined value. (4) The information processing device described in (3), wherein the first velocity is an angular velocity at a time point at which the angular velocity becomes fastest, and the first angular velocity detection time point is a time point at which the fastest angular velocity is detected. (5) The information processing device according to (4), wherein the prediction unit, after the fastest angular velocity detection point, identifies a third angular velocity detection point before the fastest angular velocity detection point at which a third angular velocity having the same value as the second angular velocity acquired at a second angular velocity detection point after the fastest angular velocity detection point, and predicts the final movement angle using an integral value of the angular velocity acquired from the start of rotation to the third angular velocity detection point. (6) The information processing device according to (5), wherein the prediction unit predicts the final movement angle by adding an integral value of the angular velocity acquired from the start of rotation to the third angular velocity detection point to the movement angle of the head at the second angular velocity detection point. (7) The information processing device according to (5) or (6), wherein the prediction unit repeatedly predicts the final movement angle at predetermined time intervals from after the fastest angular velocity detection point until the end of the rotation.(8) The information processing device according to any one of (4) to (7), wherein the correction unit corrects the position of the virtual sound source to rotate by an angle of movement of the head in a direction opposite to the rotation of the head, following the rotation of the head, until the time point of detecting the fastest angular velocity, and corrects the position of the virtual sound source to rotate by an angle of movement of the head in a direction opposite to the rotation of the head, using a predicted value of the final movement angle, from the time point of detecting the fastest angular velocity. (9) The information processing device according to any one of (1) to (8), wherein the correction unit corrects the movement angle of the position of the virtual sound source according to an angle of the position of the virtual sound source relative to the user. (10) The information processing device according to (9), wherein the correction unit corrects the position of the virtual sound source to have a smaller movement angle than the movement angle of the head when the position of the virtual sound source is on the front side of the user, and corrects the position of the virtual sound source to have a larger movement angle than the movement angle of the head when the position of the virtual sound source is on the opposite side from the front of the user. (11) The information processing device according to (10), wherein the correction unit multiplies the head movement angle by a first coefficient when the virtual sound source is located on the front side of the user, and multiplies the head movement angle by a second coefficient different from the first coefficient when the virtual sound source is located on the opposite side from the front of the user. (12) The information processing device according to (11), wherein the first coefficient is a value less than 1, and the second coefficient is a value greater than 1. (13) The information processing device according to any one of (1) to (12), wherein the correction unit corrects an acoustic signal to be presented to the user so that sound pressure changes depending on the angle of the virtual sound source relative to the user. (14) The information processing device according to (13), wherein the correction unit increases the sound pressure of the acoustic signal when the virtual sound source is located on the opposite side from the front of the user compared to when the virtual sound source is located on the front side of the user. (15) The information processing device according to any one of (1) to (14), wherein the acquisition unit acquires an angular velocity of the user's head from an inertial measurement unit provided in a sound processing device worn on the user's head.(16) The information processing device according to (15), further comprising the audio processing device. (17) The information processing device according to (16), wherein the audio processing device is at least one of headphones, earphones, hearing aids, sound collectors, and head-mounted displays. (18) The information processing device according to (15), comprising an information processing terminal configured to be able to communicate with the audio processing device. (19) An information processing system comprising: an acquisition unit that acquires an angular velocity of a user's head; a prediction unit that predicts a final movement angle of the head at the end of the head rotation using an integral value of the angular velocities acquired from a start of the head rotation to a first angular velocity detection point at which the angular velocity becomes a first velocity; and a correction unit that corrects a position of a virtual sound source presented to the user based on the final movement angle. (20) An information processing method, comprising: an information processing device acquiring an angular velocity of a user's head; predicting a final movement angle of the head at the end of the head rotation using an integral value of the angular velocity acquired from the start of the head rotation to a first angular velocity detection time point at which the angular velocity becomes a first velocity; and correcting a position of a virtual sound source presented to the user based on the final movement angle. (21) A program that causes a computer to realize the following functions: acquiring the angular velocity of the user's head; predicting a final movement angle of the head at the end of the head rotation using an integral value of the angular velocity acquired from the start of the head rotation to a first angular velocity detection time point at which the angular velocity becomes a first velocity; and correcting the position of a virtual sound source presented to the user based on the final movement angle.

[0114] REFERENCE SIGNS LIST 1 Information processing system 3 Slider 4L, 4R Hanger 5L, 5R Housing 6L, 6R Ear pad 7 Headband 10 Audio processing device 10a Headphones 12 Gyro sensor 14 Acceleration sensor 16 Output unit 20 Audio data acquisition unit 22 Audio output unit 30, 60 Communication unit 40, 900 Information processing device 40a Smartphone 42 Head posture prediction unit 50 Audio data acquisition unit 52 Audio data correction unit 54 Audio data output unit 62 Storage unit 70, 906 Communication network 80 User 82 Sound source 901 CPU 902 ROM 903 RAM 904 Storage device 905 Communication module 907 Sensor module 908 Imaging device 910 Display device 911 Speaker 912 Microphone 913 Input device 914 Bus

Claims

1. An information processing device comprising: an acquisition unit that acquires an angular velocity of a user's head; a prediction unit that predicts a final movement angle of the head at the end of the head rotation using an integral value of the angular velocity acquired from the start of the head rotation to a first angular velocity detection point at which the angular velocity becomes a first velocity; and a correction unit that corrects a position of a virtual sound source presented to the user based on the final movement angle.

2. The information processing device according to claim 1, wherein the correction unit corrects the position of the virtual sound source so as to rotate in a direction opposite to a direction of rotation of the head.

3. The information processing device according to claim 2, wherein the prediction unit predicts the final movement angle by multiplying an integral value of the angular velocity acquired from the start of the rotation to the first angular velocity detection point by a predetermined value.

4. The information processing device according to claim 3, wherein the first velocity is an angular velocity at a point in time when the angular velocity is fastest, and the first angular velocity detection point in time is a point in time when the fastest angular velocity is detected.

5. The information processing device of claim 4, wherein the prediction unit, after the fastest angular velocity detection point, identifies a third angular velocity detection point before the fastest angular velocity detection point at which a third angular velocity having the same value as a second angular velocity acquired at a second angular velocity detection point after the fastest angular velocity detection point was acquired, and predicts the final movement angle using an integral value of the angular velocity acquired from the start of rotation to the third angular velocity detection point.

6. The information processing device of claim 5, wherein the prediction unit predicts the final movement angle by adding the integral value of the angular velocity acquired from the start of the rotation to the third angular velocity detection point to the head movement angle at the second angular velocity detection point.

7. The information processing device according to claim 5, wherein the prediction unit repeatedly predicts the final movement angle at a predetermined time interval from the time point after the fastest angular velocity is detected until the end of the rotation.

8. The information processing device of claim 4, wherein the correction unit corrects the position of the virtual sound source to rotate an amount equal to the movement angle of the head in a direction opposite to the rotation of the head, following the rotation of the head, until the time point of detecting the fastest angular velocity, and corrects the position of the virtual sound source to rotate an amount equal to the movement angle of the head in a direction opposite to the rotation of the head, using a predicted value of the final movement angle from the time point of detecting the fastest angular velocity.

9. The information processing device according to claim 1, wherein the correction unit corrects a movement angle of the position of the virtual sound source in accordance with a relative angle of the position of the virtual sound source with respect to the user.

10. The information processing device according to claim 9, wherein the correction unit corrects the position of the virtual sound source to have a smaller movement angle than the movement angle of the head when the position of the virtual sound source is in front of the user, and corrects the position of the virtual sound source to have a larger movement angle than the movement angle of the head when the position of the virtual sound source is on the opposite side from the front of the user.

11. The information processing device according to claim 10, wherein the correction unit multiplies the head movement angle by a first coefficient when the virtual sound source is located in front of the user, and multiplies the head movement angle by a second coefficient different from the first coefficient when the virtual sound source is located on the opposite side from the front of the user.

12. The information processing device according to claim 11, wherein the first coefficient is a value less than 1, and the second coefficient is a value greater than 1.

13. The information processing device according to claim 1, wherein the correction unit corrects the acoustic signal presented to the user so that sound pressure changes depending on a relative angle of a position of the virtual sound source with respect to the user.

14. The information processing device according to claim 13, wherein the correction unit increases the sound pressure of the acoustic signal when the virtual sound source is located on the opposite side from the front of the user compared to when the virtual sound source is located in front of the user.

15. The information processing device of claim 1, further comprising an audio processing device.

16. The information processing device according to claim 15, wherein the audio processing device is at least one of a headphone, an earphone, a hearing aid, a sound amplifier, and a head-mounted display.

17. The information processing device according to claim 1, comprising an information processing terminal configured to be capable of communicating with the voice processing device.

18. An information processing system comprising: an acquisition unit that acquires an angular velocity of a user's head; a prediction unit that predicts a final movement angle of the head at the end of the head rotation using an integral value of the angular velocity acquired from the start of the head rotation to a first angular velocity detection point at which the angular velocity becomes a first velocity; and a correction unit that corrects a position of a virtual sound source presented to the user based on the final movement angle.

19. An information processing method comprising: an information processing device acquiring an angular velocity of a user's head; predicting a final movement angle of the head at the end of the head rotation using an integral value of the angular velocity acquired from the start of the head rotation to a first angular velocity detection point at which the angular velocity becomes a first velocity; and correcting a position of a virtual sound source presented to the user based on the final movement angle.

20. A program that enables a computer to realize the following functions: a function for acquiring the angular velocity of a user's head; a function for predicting a final movement angle of the head at the end of the head rotation using an integral value of the angular velocity acquired from the start of the head rotation to a first angular velocity detection point at which the angular velocity becomes a first velocity; and a function for correcting the position of a virtual sound source presented to the user based on the final movement angle.

Citation Information

Patent Citations

  • Sound processing apparatus, sound source position control method and sound source position control program

    JP2015233252A

  • System and method for improving headphone spatial impression

    US20120020502A1

  • Angle detector and audio playback apparatus using the detector

    WO1995013690A1

  • Sensor data prediction

    WO2022197987A1

Cited By

  • Audio playing method and device, electronic equipment and computer readable storage medium

    CN120832118A

  • Audio playing method and device, electronic equipment and computer readable storage medium

    CN120832118B