Information processing device, information processing method, and program

Abstract

The present technology relates to an information processing device, an information processing method, and a program which enable a user to adjust an individualized transfer function. This information processing device comprises an adjustment unit that adjusts, in response to a user operation, an individualized transfer characteristic, which is a sound transfer characteristic in a measurement environment and which is individualized for a user. This information processing device further comprises a presentation unit that presents to the user, together with the individualized transfer characteristic, the content of reference information referenced when the individualized transfer characteristic is adjusted. The present technology can be applied, for example, to a system that carries out audio mixing for content such as movies.

Description

Technical Field

[0001] The present technology relates to an information processing apparatus, an information processing method, and a program, and particularly relates to an information processing apparatus, an information processing method, and a program that enable a user to adjust a personalized transfer function.

Background Art

[0002] A personalized head-related transfer function (HRTF) is obtained, for example, by convolving the inverse characteristics of the HRTF from headphones to the ears with the HRTF from speakers to the ears.

[0003] By performing an operation using the personalized HRTF, it becomes possible to accurately localize the sound image at a predetermined position and stereophonically reproduce the sound heard from headphones. The sound heard from headphones is a reproduction of the sound from the sound source in the HRTF measurement environment.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] A personalized HRTF is unique to a user and is usually used as a fixed value in calculations. Therefore, the user cannot adjust the sound field and sound quality by adjusting the personalized HRTF itself.

[0006] The present technology has been made in view of such a situation, and enables a user to adjust a personalized transfer function.

Means for Solving the Problems

[0007] One aspect of this technology is an information processing device that includes an adjustment unit for adjusting the personalized transmission characteristics, which are the sound transmission characteristics in the measurement environment and which are the transmission characteristics that have been personalized for the user, in accordance with the user's operation.

[0008] In one aspect of this technology, the sound transmission characteristics in the measurement environment, which are the personalized transmission characteristics that are individualized for the user, are adjusted according to the user's operation. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example configuration of an audio production system according to one embodiment of this technology. [Figure 2] This diagram shows the measurement process in the measurement environment. [Figure 3] This diagram shows the adjustment process in the measurement environment. [Figure 4] This figure shows an example of adjustments made in the measurement environment. [Figure 5] This diagram shows the playback process in a playback environment. [Figure 6] This is a block diagram showing an example of the functional configuration of an information processing device. [Figure 7] This block diagram shows an example of the file generation section's configuration. [Figure 8] This figure shows an example of reference information for sound field adjustment. [Figure 9] This is a block diagram showing an example configuration of the adjustment value recording unit. [Figure 10] This figure shows an example of information recorded in a personalized HRTF file. [Figure 11] This flowchart explains the process of generating personalized HRTF files performed by an information processing device in a measurement environment. [Figure 12] This flowchart explains the personalized HRTF adjustment process performed by the information processing device in the measurement environment. [Figure 13] This is a flowchart explaining the file information display process. [Figure 14] It is a diagram showing an example of display of attribute information. [Figure 15] It is a flowchart for explaining the sound quality adjustment process. [Figure 16] It is a diagram showing an example of display of the sound quality adjustment screen. [Figure 17] It is a diagram showing the content of each piece of information on the sound quality adjustment screen. [Figure 18] It is a flowchart for explaining the sound field adjustment process. [Figure 19] It is a diagram showing an example of display of the sound field adjustment screen. [Figure 20] It is a diagram showing the content of each piece of information on the sound field adjustment screen. [Figure 21] It is a block diagram showing an example of the functional configuration of a playback device. [Figure 22] It is a flowchart for explaining the playback process performed by a playback device in a playback environment. [Figure 23] It is a block diagram showing an example of the configuration of a computer's hardware.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments for carrying out the present technology will be described. The description will be made in the following order. 1. Configuration of an audio production system 2. Overall flow of operations in an audio production system 3. Configuration and operation of an information processing device 4. Configuration and operation of a playback device 5. Variation

[0011] <<1. Configuration of an audio production system>> FIG. 1 is a diagram showing an example of the configuration of an audio production system according to an embodiment of the present technology.

[0012] The audio production system in FIG. 1 is composed of devices on the measurement environment side and devices on the playback environment side. The audio production system in FIG. 1 is, for example, a system used for producing the audio of a movie.

[0013] Film audio includes not only the voices of actors and narrators, but also various other sounds such as sound effects, ambient sounds, and background music. In the following, where it is not necessary to distinguish between different types of sounds, they will be collectively referred to as "audio," but in reality, film audio also includes other types of sounds besides voices.

[0014] As shown on the left side of Figure 1, the measurement environment is a movie theater, often called a dubbing stage, used for sound production. The movie theater is equipped with a screen and multiple speakers. The movie theater is also equipped with an information processing device 1 that acquires measurement results of HRTF (Head-Related Transfer Function), which represents the sound transmission characteristics of the measurement environment, and generates information such as an HRTF file. The information processing device 1 is configured, for example, as a PC.

[0015] In the measurement environment of the sound production system shown in Figure 1, a personalized HRTF, which is a personalized HRTF tailored to the film's sound producer, is measured. Furthermore, the personalized HRTF is adjusted to reproduce the same sound quality as the measurement environment, as well as the same sound field as the measurement environment. The adjustment of the personalized HRTF is performed, for example, by the producer themselves who is editing in the playback environment, while listening to the playback sound using the personalized HRTF.

[0016] By recording the adjustment values ​​for the personalized HRTF along with the personalized HRTF data, the personalized HRTF file is generated by the information processing device 1.

[0017] As shown by the arrow in Figure 1, the personalized HRTF file, which contains personalized HRTF data representing the measurement results of the personalized HRTF and adjustment values, is provided to the playback device 31 installed in the playback environment. The provision of the personalized HRTF file to the playback device 31 may be done via a network such as the Internet, or using a recording medium such as flash memory.

[0018] The playback environment is a different location from a movie theater, such as a studio or the producer's home. The playback environment may be set up in the same location as the measurement environment.

[0019] The playback environment includes a playback device 31, which is used for editing the audio of the film. The playback device 31 is also configured, for example, by a PC. The producer edits the audio of the film using headphones 32 in the playback environment, such as at home. The headphones 32 are an output device provided in the playback environment.

[0020] In the playback device 31, audio signals are reproduced using a personalized HRTF. By performing playback using a personalized HRTF, the sound output from the movie theater speakers used to measure the personalized HRTF is reproduced.

[0021] When playing back an audio signal, the playback device 31 adjusts the personalized HRTF used for playing back the audio signal based on the adjustment value. Because the audio signal is played back using the adjusted personalized HRTF, the sound quality of the audio heard from the headphones 32 becomes the same as the sound quality in the measurement environment. In addition, the sound field of the sound heard from the headphones 32 becomes the same as the sound field in the measurement environment.

[0022] Thus, in the sound production system shown in Figure 1, it is possible to adjust the personalized HRTF itself. Normally, audio played back using a personalized HRTF is a more faithful reproduction of the audio in the measurement environment compared to audio played back using a non-personalized HRTF (an HRTF commonly used by many people). However, depending on the acoustic characteristics of the measurement environment and the device characteristics of the headphones used during playback, a difference may be perceived between the audio played back using a personalized HRTF and the audio in the measurement environment.

[0023] By adjusting the personalized HRTF itself, the audio from a measurement environment with adjusted sound quality and sound field is reproduced. Creators can then edit while listening to the audio adjusted based on their own perception.

[0024] This allows creators to edit using headphones 32 under the same sound environment as a movie theater. In other words, the same sound environment as a movie theater is virtually reproduced in the playback environment.

[0025] Typically, in film audio production, the sound output from the theater's speakers is used as a reference. However, with this technology's audio production system, there is no need to go to a movie theater, allowing producers to edit from home or other locations.

[0026] <<2. Overall workflow in sound production systems>> This section describes the workflow performed in both the measurement environment and the playback environment. In the measurement environment, measurement and adjustment tasks are carried out separately.

[0027] <Measurement procedure in the measurement environment> Figure 2 shows the measurement flow in the measurement environment.

[0028] Measurements in the measurement environment primarily include HRTF measurement and recording of reference information.

[0029] • HRTF measurement As shown on the left side of Figure 2, HRTF measurement is performed, for example, with the listener sitting in a designated seat in a movie theater and a microphone 21 attached to their ear canal.

[0030] In this process, the film's audio producer acts as the listener. By having the producer act as the listener, their personalized HRTF (Head-Related Frequency Response) is measured. Since HRTF differs depending on factors such as ear shape, using personalized HRTF allows for precise sound localization.

[0031] In this state, sound is output from speaker 23 of the movie theater, and the personalized HRTF from speaker 23 to the ear (e.g., ear canal position, eardrum position) is measured.

[0032] After the measurement of the personalized HRTF from speaker 23 to the ear is performed, the listener puts on headphones 22, which are placed over the ears to which microphones 21 are attached. Headphones 22 are output devices provided in the measurement environment.

[0033] In this state, playback sound is output from headphones 22, and the personalized HRTF from headphones 22 to the ear is measured. For example, the same sound as the playback sound output from speaker 23 is used as the playback sound from headphones 22.

[0034] The personalized HRTF from speaker 23 to ear and the personalized HRTF from headphones 22 to ear, measured in this manner, are acquired by the information processing device 1. The information processing device 1 generates personalized HRTF data that includes the personalized HRTF from speaker 23 to ear and inverse correction data.

[0035] The inverse correction data represents the inverse characteristics of the personalized HRTF from the headphones 22 to the ear. The inverse correction data is used to correct the personalized HRTF during playback in the playback environment. This correction is performed by superimposing the inverse correction data onto the personalized HRTF from the speaker 23 to the ear, that is, by canceling out the personalized HRTF from the headphones 22 to the ear.

[0036] By correcting the personalized HRTF, it becomes possible to obtain a highly accurate HRTF from the speaker 23 to the ear, which is personalized by the creator and takes into account the individual differences of the headphones 22.

[0037] • Recording of reference information As shown on the right side of Figure 2, reference information, which is information referred to by the creator when adjusting the personalized HRTF, is acquired. Reference information is acquired based on, for example, the characteristics of the measurement environment and the device characteristics of the headphones 22. Details of the reference information will be described later.

[0038] By recording reference information along with the personalized HRTF data generated by the HRTF measurement, a personalized HRTF file is generated, as shown at the tip of arrow #1.

[0039] <Adjustment process in the measurement environment> Figure 3 shows the adjustment process in the measurement environment.

[0040] In the information processing device 1 prepared in the measurement environment, as shown on the left side of Figure 3, the inverse correction data of the personalized HRTF from speaker 23 to ear and the personalized HRTF from headphones 22 to ear are read out as personalized HRTF data from the personalized HRTF file.

[0041] As shown on the right side of Figure 3, headphones 22 are connected to the information processing device 1. Audio is output from the headphones 22, which is played back using the personalized HRTF data read from the personalized HRTF file.

[0042] The creator operates the information processing device 1 while looking at the display on display 1A, and adjusts the sound field and sound quality of the playback sound output from headphones 22 to reproduce the sound field and sound quality of the playback sound output from speaker 23, for example, as shown in Figure 4. The adjustment screen displayed on display 1A shows information based on, for example, personal HRTF data and reference information.

[0043] For example, the producer adjusts the reverberation component in the playback sound output from the headphones 22 by referring to reference information used for adjusting the sound field. In the information processing device 1, adjustment values ​​used for adjusting the personalized HRTF from the speaker 23 to the ear are generated according to the producer's operation.

[0044] Furthermore, the creator adjusts the sound quality, taking into account the characteristics of the headphones 22, by referring to reference information used for sound quality adjustment. In the information processing device 1, adjustment values ​​used to adjust the inverse correction data of the personalized HRTF from the headphones 22 to the ear are generated according to the creator's operation.

[0045] The adjusted values ​​for the personalized HRTF data generated in this way are recorded in the personalized HRTF file along with the personalized HRTF data and reference information, as shown at the end of arrow #2 in Figure 3.

[0046] <The playback process in a playback environment> Figure 5 shows the regeneration process in a regeneration environment.

[0047] Headphones 32 are connected to the playback device 31 provided in the playback environment. Headphones 32 are, for example, headphones of the same model number (headphones with the same specifications) manufactured by the same company as headphones 22 used in the measurement environment. Alternatively, headphones 22 that the creator took home may be used as headphones 32.

[0048] In the playback device 31, the personalized HRTF data recorded in the personalized HRTF file is adjusted based on adjustment values. The adjusted personalized HRTF data is used to play back the audio signals of the film to be edited, such as object audio and channel audio. The audio data that makes up the film's audio includes object audio and channel audio data.

[0049] The producer can edit the film's audio while listening to the output sound, which is generated in a way that replicates the environment of a movie theater, as the production environment for film audio.

[0050] In this way, by adjusting personalized HRTF data, it becomes possible to reproduce the sound field and sound quality required in the production of film audio and music.

[0051] For example, by adjusting the personalized HRTF from speaker 23 to the ear, the producer can edit the audio with a sound field adjusted so that the sound is not blurred. Also, by adjusting the inverse correction data of the personalized HRTF from headphones 22 to the ear, the producer can improve the accuracy of reproduction of the low-frequency sound output from the subwoofer installed in the measurement environment.

[0052] <<3. Configuration and Operation of Information Processing Devices>> <Configuration of the information processing device> Figure 6 is a block diagram showing an example of the functional configuration of the information processing device 1.

[0053] In the information processing device 1, the information processing unit 101 is realized when a predetermined program is executed by the CPU of the PC that constitutes the information processing device 1.

[0054] The information processing unit 101 consists of a file generation unit 111 and an adjustment value recording unit 112. At least some of the components of the information processing unit 101 may be implemented by other equipment such as an amplifier provided in the measurement environment.

[0055] The file generation unit 111 measures the personalized HRTF and generates a personalized HRTF file. The personalized HRTF file generated by the file generation unit 111 is supplied to the adjustment value recording unit 112.

[0056] The adjustment value recording unit 112 adjusts the personalized HRTF data according to the creator's operations and records the adjustment values ​​in the personalized HRTF file.

[0057] • Configuration of the file generation section Figure 7 is a block diagram showing an example configuration of the file generation unit 111.

[0058] The file generation unit 111 consists of a playback processing unit 121, an output control unit 122, an HRTF acquisition unit 123, an HRTF data generation unit 124, a reference information acquisition unit 125, and an HRTF file generation unit 126.

[0059] The playback processing unit 121 controls the playback of sound output from the headphones 22 and speakers 23. The audio signal obtained by playing back audio data such as specified signal data is supplied to the output control unit 122.

[0060] The output control unit 122 outputs playback sound corresponding to the audio signal supplied from the playback processing unit 121 through the headphones 22 and speaker 23.

[0061] The HRTF acquisition unit 123 acquires personalized HRTF data from the headphones 22 to the ears and personalized HRTF data from the speaker 23 to the ears, based on the sound collection results from the microphone 21. The information representing the personalized HRTF acquired by the HRTF acquisition unit 123 is supplied to the HRTF data generation unit 124.

[0062] The HRTF data generation unit 124 generates personalized HRTF data, which includes inverse correction data and personalized HRTF data from speaker 23 to ear. The personalized HRTF data generated by the HRTF data generation unit 124 is supplied to the HRTF file generation unit 126.

[0063] The reference information acquisition unit 125 acquires reference information based on the characteristics of the measurement environment and the device characteristics of the headphones 22.

[0064] Specifically, the reference information acquisition unit 125 acquires reference information for sound quality adjustment based on the characteristics of the headphones 22. The reference information for sound quality adjustment is acquired based on device characteristics such as individual variability, linearity, and THD (Total Harmonic Distortion).

[0065] For example, reference information for sound quality adjustment is obtained based on the sound pressure level (SPL) and THD when using signals of each voltage.

[0066] Furthermore, the reference information acquisition unit 125 acquires reference information for sound field adjustment based on the reverberation characteristics of the measurement environment. For example, reference information for sound field adjustment is acquired based on the personalized HRTF from speaker 23 to the ear.

[0067] Figure 8 shows an example of reference information for sound field adjustment.

[0068] Figure 8A shows the decay characteristics of reverberation components in three spaces: "Room A," "Room B," and "Room C." Figure 8B shows the information used to convert the reverberation components in "Room A" to the reverberation components in "Room B" and "Room C." "Room A" represents a large room, such as a movie theater, which is the measurement environment. "Room B" represents a room of medium size, and "Room C" represents a small room.

[0069] In this way, the reference information acquisition unit 125 acquires, as reference information, information on the reverberation components in the measurement environment, as well as information for converting the reverberation components in the measurement environment into reverberation components in a specified space.

[0070] The information describing the characteristics of the measurement environment and the device characteristics of the headphones 22, as described above, is input to the reference information acquisition unit 125 in Figure 7, for example, before measuring the personalized HRTF. The reference information acquired by the reference information acquisition unit 125 is supplied to the HRTF file generation unit 126.

[0071] The HRTF file generation unit 126 generates a personalized HRTF file by adding a header section containing reference information supplied by the reference information acquisition unit 125 to the personalized HRTF data supplied by the HRTF data generation unit 124. The header section includes attribute information such as the measurement location, user name, and headphone model name 22, along with the reference information.

[0072] • Configuration of the adjustment value recording unit Figure 9 is a block diagram showing an example configuration of the adjustment value recording unit 112.

[0073] The adjustment value recording unit 112 consists of a personalized HRTF file acquisition unit 141, a playback processing unit 142, an output control unit 143, an adjustment unit 144, and a recording unit 145.

[0074] The personalized HRTF file acquisition unit 141 acquires the personalized HRTF file supplied from the file generation unit 111. The personalized HRTF file acquired by the personalized HRTF file acquisition unit 141 is supplied to the playback processing unit 142, the adjustment unit 144, and the recording unit 145.

[0075] The playback processing unit 142 acquires an audio signal to be used in adjusting the personalized HRTF. For example, the same audio signal used in the measurement of the personalized HRTF is acquired.

[0076] The playback processing unit 142 reads personalized HRTF data from the personalized HRTF file supplied by the personalized HRTF file acquisition unit 141, and generates a playback signal by performing binaural processing, including convolution of the personalized HRTF, on the audio signal.

[0077] Furthermore, the playback processing unit 142 adjusts the personalized HRTF as appropriate based on the adjustment values ​​supplied from the adjustment unit 144, and performs binaural processing using the adjusted personalized HRTF. The playback signal generated by the playback processing unit 142 is supplied to the output control unit 143.

[0078] The output control unit 143 outputs playback sound from the headphones 22 according to the signal supplied from the playback processing unit 142.

[0079] The adjustment unit 144 consists of a file information display unit 171, a sound quality adjustment unit 172, and a sound field adjustment unit 173.

[0080] The file information display unit 171 displays the contents of attribute information contained in the header of the personalized HRTF file supplied by the personalized HRTF file acquisition unit 141 on the adjustment screen. When adjusting the personalized HRTF data, the display 1A displays the adjustment screen, which is a GUI (Graphical User Interface) used for adjusting the personalized HRTF data.

[0081] The sound quality adjustment unit 172 displays information such as inverse correction data on the adjustment screen when adjusting the sound quality. The inverse correction data is included in the personalized HRTF data of the personalized HRTF file supplied by the personalized HRTF file acquisition unit 141. The sound quality adjustment unit 172 also displays the contents of the reference information included in the header portion of the personalized HRTF file on the adjustment screen.

[0082] The sound quality adjustment unit 172 acquires the adjustment value of the inverse correction data as the sound quality adjustment value in response to the producer's operation.

[0083] The sound field adjustment unit 173 displays information on the adjustment screen that represents the reverberation components based on the personalized HRTF from the speaker 23 to the ear when adjusting the sound field. The personalized HRTF from the speaker 23 to the ear is included in the personalized HRTF data of the personalized HRTF file supplied by the personalized HRTF file acquisition unit 141. The sound field adjustment unit 173 also displays the contents of the reference information included in the header of the personalized HRTF file on the adjustment screen.

[0084] The sound field adjustment unit 173 acquires the adjustment values ​​of the personalized HRTF from the speaker 23 to the ear as sound field adjustment values, according to the producer's operation.

[0085] Thus, the adjustment unit 144 functions as a presentation unit that displays the contents of attribute information, reference information, personalized HRTF data, etc., on the adjustment screen and presents them to the creator (user). The sound quality adjustment values ​​and sound field adjustment values ​​acquired by the adjustment unit 144 are supplied to the playback processing unit 142 and the recording unit 145.

[0086] The recording unit 145 records the adjustment values ​​supplied by the adjustment unit 144 in the header portion of the personal HRTF file supplied by the personal HRTF file acquisition unit 141.

[0087] Figure 10 shows an example of information recorded in a personalized HRTF file.

[0088] Personalized HRTF files record personalized HRTF data, and header information is recorded in the header section. As shown in the callout, the header information includes attribute information, reference information, and adjustment values.

[0089] Attribute information includes information indicating the measurement location, the user's name, and the model name of the headphones used for the measurement.

[0090] Information representing the measurement location is information representing the location of the measurement environment. In the example above, information about the movie theater used as the measurement environment is recorded as information representing the measurement location. For example, a producer adjusting a personalized HRTF can identify the measurement environment based on the information representing the measurement location.

[0091] The information representing the username indicates the creator who is editing using the personalized HRTF.

[0092] The information representing the model name of the headphones used for measurement includes information representing the manufacturer of the headphones 22, as well as identification information such as the model name.

[0093] The reference information includes reference information for sound quality adjustment and reference information for sound field adjustment.

[0094] Reference information for sound quality adjustment includes, for example, Split Freq and Limit Gain.

[0095] Split Freq represents the boundary frequency at which a constant correction value is applied. For bandwidths above Split Freq, a correction using a constant correction value is performed. Limit Gain represents the maximum value of the gain used for the correction value.

[0096] Reference information for sound field adjustment includes, for example, Gain, Start Point, and Length.

[0097] Gain represents the gain used as a correction value for reverberation components. Start Point represents the starting point of decay. Length represents the range over which a constant decay rate can be applied.

[0098] Each of the above pieces of information, along with adjustment values ​​consisting of sound quality adjustment values ​​and sound field adjustment values, is recorded as header information to form a personalized HRTF file. The personalized HRTF file, with the adjustment value information recorded in the header section, is provided to the playback device 31.

[0099] <Operation of the Information Processing Device> Here, we will explain the processing of the information processing device 1 having the above configuration.

[0100] • Personalized HRTF file generation process Referring to the flowchart in Figure 11, the process of generating a personalized HRTF file performed by the information processing device 1 in the measurement environment will be explained.

[0101] Here, we will explain that all the steps in Figure 11 are performed by the information processing device 1, but as appropriate, they may be performed by other equipment provided in the measurement environment. As mentioned above, the measurement of the personalized HRTF is performed with the producer, acting as a listener, sitting in a designated seat in a movie theater and with the microphone 21 attached to their ear canal.

[0102] In step S1, the output control unit 122 outputs playback sound from the movie theater's speaker 23.

[0103] In step S2, the HRTF acquisition unit 123 measures the personalized HRTF from the speaker 23 to the ear based on the sound collection results from the microphone 21. After the personalized HRTF from the speaker 23 to the ear has been measured, the creator puts on the headphones 22 so that they are placed over the ear to which the microphone 21 is attached.

[0104] In step S3, the output control unit 122 outputs playback sound from the headphones 22 worn by the creator.

[0105] In step S4, the HRTF acquisition unit 123 measures the personalized HRTF from the headphones 22 to the ear based on the sound collection results from the microphone 21.

[0106] In step S5, the HRTF data generation unit 124 generates personalized HRTF data that includes personalized HRTF data from headphones 22 to the ear and inverse correction data of the personalized HRTF data from speaker 23 to the ear.

[0107] In step S6, the reference information acquisition unit 125 acquires reference information based on the characteristics of the headphones 22 and the measurement environment.

[0108] In step S7, the HRTF file generation unit 126 generates a personalized HRTF file that records header information including reference information and personalized HRTF data.

[0109] In the above, the measurement of the personalized HRTF was performed using microphone 21, but the personalized HRTF from speaker 23 to the ear may also be obtained based on an ear image obtained by photographing the creator's ear. In this case, an inference model for personalized HRTF inference that has been generated in advance by machine learning or the like is used. The inference model for personalized HRTF inference is an inference model that takes an ear image as input and outputs the personalized HRTF.

[0110] • Personalized HRTF adjustment process Next, with reference to the flowchart in Figure 12, the personalized HRTF adjustment process performed by the information processing device 1 in the measurement environment will be explained.

[0111] In step S21, the file information display unit 171 performs file information display processing. This process reads the personalized HRTF file and displays the contents of its attribute information. The file information display processing will be described later with reference to the flowchart in Figure 13.

[0112] In step S22, the sound quality adjustment unit 172 performs sound quality adjustment processing. This process adjusts the inverse correction data, and the sound quality adjustment value is recorded in the personalized HRTF file. The sound quality adjustment processing will be described later with reference to the flowchart in Figure 15.

[0113] In step S23, the sound field adjustment unit 173 performs sound field adjustment processing. This processing adjusts the personalized HRTF data from the speaker 23 to the ear, and the sound field adjustment value is recorded in the personalized HRTF file. The sound field adjustment processing will be described later with reference to Figure 18.

[0114] • File information display processing Referring to the flowchart in Figure 13, the file information display process performed in step S21 of Figure 12 will be explained.

[0115] In step S31, the adjustment unit 144 reads the personalized HRTF file. By reading the personalized HRTF file, personalized HRTF data and inverse correction data from the speaker to the ear are acquired. Attribute information and reference information are also acquired.

[0116] In step S32, the file information display unit 171 displays information representing the measurement location based on the attribute information.

[0117] In step S33, the file information display unit 171 displays information representing the username based on the attribute information.

[0118] In step S34, the file information display unit 171 displays information representing the model name of the headphones 22 used for the measurement, based on the attribute information.

[0119] Figure 14 shows an example of how attribute information is displayed.

[0120] The screen shown in Figure 14 is displayed as the main screen of the adjustment screen used for adjusting personalized HRTF data. Item 201 at the top of the screen represents the personalized HRTF file to be adjusted. In the example in Figure 14, " / No Name / profiles / Username" is displayed as the personalized HRTF file to be adjusted.

[0121] Below item 201, image P1, which represents the measurement location, is displayed. To the right of image P1, area A1, which is the display area for attribute information, is formed. Area A1 displays three types of information: items 202 through 204.

[0122] Item 202 represents the name of the measurement location. In the example in Figure 14, "Room A" is displayed as the name of the measurement location.

[0123] Item 203 represents the username. In the example in Figure 14, "Username" is displayed as the username.

[0124] Item 204 represents the model name of the headphones used for the measurement. In the example in Figure 14, "Headphones" is displayed as the model name of the headphones.

[0125] The creator can view the displays in items 201 through 204 to identify the personalized HRTF file to be adjusted and to check information about the measurement environment.

[0126] After the attribute information is displayed in this manner, the process returns to step S21 in Figure 12, and the subsequent processing is carried out.

[0127] • Sound quality adjustment processing Referring to the flowchart in Figure 15, the sound quality adjustment process performed in step S22 of Figure 12 will be explained.

[0128] The sound quality adjustment process is initiated, for example, when the user is instructed to perform sound quality adjustments on the main screen shown in Figure 14. As described above, sound quality adjustments are performed by adjusting the inverse correction data, which represents the inverse characteristics of the personalized HRTF from the headphones 22 to the ear.

[0129] In step S41, the sound quality adjustment unit 172 displays information representing the inverse correction data.

[0130] In step S42, the sound quality adjustment unit 172 displays a reference line for the maximum correction amount that takes into account the device characteristics of the headphones 22, based on the reference information.

[0131] Figure 16 shows an example of the sound quality adjustment screen display.

[0132] As shown in Figure 16, the content of the reference information for sound quality adjustment is displayed at the top of the sound quality adjustment screen.

[0133] Item 211 is "Split Freq," which represents the boundary frequency to which a certain correction is applied. In the example in Figure 16, "11700" Hz is shown as the boundary frequency.

[0134] Item 212 is "Limit Gain," which represents the maximum value of the gain used for correction. In the example in Figure 16, "16.0" (dB) is displayed as the maximum gain for frequencies below 11700 Hz, and "-2.0" (dB) is displayed as the maximum gain for frequencies above 11700 Hz.

[0135] At the bottom of the screen, a reference line 213 is displayed along with a waveform 214 representing the inverse correction data. Waveform 214 represents the inverse correction data for the L channel and the inverse correction data for the R channel. The reference line 213 represents the maximum correction amount considering the device characteristics of the headphones 22. The contents of each piece of information are shown in Figure 17.

[0136] The creator can look at reference line 213 to determine what gain value is needed. The creator adjusts the reference information for sound quality adjustment, represented by "Split Freq" and "Limit Gain," by moving the display on waveform 214, for example.

[0137] Returning to the explanation of Figure 15, in step S43, the sound quality adjustment unit 172 adjusts the inverse correction data according to the producer's operation. The sound quality adjustment unit 172 sets the sound quality adjustment value, which is the adjustment value of the inverse correction data, according to the producer's operation. Playback using the adjusted inverse correction data with the sound quality adjustment value is performed as appropriate by the playback processing unit 142.

[0138] In step S44, the recording unit 145 records the sound quality adjustment value in the header section of the personalized HRTF file.

[0139] After the sound quality adjustment values ​​are recorded, the process returns to step S22 in Figure 12, and subsequent processing is carried out.

[0140] • Sound field adjustment processing Referring to the flowchart in Figure 18, the sound field adjustment process performed in step S23 of Figure 12 will be explained.

[0141] The sound field adjustment process is initiated, for example, when the user is instructed to perform sound field adjustments on the main screen shown in Figure 14.

[0142] In step S61, the sound field adjustment unit 173 displays information representing the reverberation components in room A based on the personalized HRTF from speaker 23 to the ear. Room A is, for example, a movie theater, which is the measurement environment.

[0143] In step S62, the sound field adjustment unit 173 displays a reference line for converting the reverberation components in room A to the reverberation components in room B, based on the reference information.

[0144] In step S63, the sound field adjustment unit 173 displays a reference line for converting the reverberation components in room A to the reverberation components in room C, based on the reference information.

[0145] Figure 19 shows an example of the sound field adjustment screen display.

[0146] As shown in Figure 19, the contents of the reference information for sound field adjustment are displayed at the top of the sound field adjustment screen. The contents of each piece of information are shown in Figure 20.

[0147] Item 221 is "Gain," which represents the gain used as a correction value for reverberation components. In the example in Figure 19, "-29" dB is displayed as the gain.

[0148] Item 222 is the "Start Point," which represents the starting position of the attenuation. In the example in Figure 19, the value "256" is displayed as the starting position of the attenuation.

[0149] Item 223 is "Length," which represents the range of application for a certain attenuation rate. In the example in Figure 19, "2k" is shown as the range of application for the attenuation rate.

[0150] At the bottom of the screen, reference lines 225 to 227 are displayed along with waveform 224, which represents the reverberation component. Reference line 225 represents the decay characteristics of the reverberation component in the measurement environment. Reference line 225 indicates the start position, end position, and gain of the reverberation component in the measurement environment.

[0151] Reference line 226 is a reference line for converting the reverberation components in the measurement environment to the reverberation components in "Room B," and represents the attenuation characteristics of "Room B." Reference line 227 is a reference line for converting the reverberation components in the measurement environment to the reverberation components in "Room C," and represents the attenuation characteristics of "Room C."

[0152] The creator can look at reference lines 226 and 227 to determine how much reverberation needs to be corrected. The creator adjusts the reference information for sound field adjustment, represented by "Gain," "Start Point," and "Length," by entering numerical values ​​in fields 221 through 223 or by moving the sliders displayed next to fields 221 through 223.

[0153] Returning to the explanation of Figure 18, in step S64, the sound field adjustment unit 173 adjusts the personalized HRTF from the speaker 23 to the ear according to the producer's operation. The sound field adjustment unit 173 sets the sound field adjustment value, which is the adjustment value of the personalized HRTF from the speaker 23 to the ear, according to the producer's operation. Playback using the adjusted personalized HRTF based on the sound field adjustment value is performed as appropriate by the playback processing unit 142.

[0154] In step S65, the recording unit 145 records the sound field adjustment value in the header section of the personalized HRTF file.

[0155] After the sound field adjustment values ​​are recorded, the process returns to step S23 in Figure 12, and subsequent processing takes place. The personalized HRTF file generated by this series of processes is then provided to the playback device 31.

[0156] <<4. Configuration and Operation of Playback Devices>> <Playback Equipment Configuration> Figure 21 is a block diagram showing an example of the functional configuration of the playback device 31.

[0157] In the playback device 31, the playback processing unit 251 is realized when a predetermined program is executed by the CPU of the PC that constitutes the playback device 31.

[0158] The playback processing unit 251 consists of an audio signal acquisition unit 261, a personalized HRTF file acquisition unit 262, an audio signal processing unit 263, and an output control unit 264. At least some of the components of the playback processing unit 251 may be implemented in other devices provided in the playback environment.

[0159] The audio signal acquisition unit 261 acquires, for example, the audio signal of the movie to be edited and outputs it to the audio signal processing unit 263.

[0160] The personalized HRTF file acquisition unit 262 acquires the personalized HRTF file provided by the information processing device 1 and outputs it to the audio signal processing unit 263.

[0161] The audio signal processing unit 263 reads personalized HRTF data from the personalized HRTF file supplied by the personalized HRTF file acquisition unit 262 and generates a playback signal by performing binaural processing on the audio signal supplied by the audio signal acquisition unit 261.

[0162] Furthermore, the audio signal processing unit 263 adjusts the personalized HRTF data as appropriate based on the adjustment values ​​included in the header of the personalized HRTF file, and performs binaural processing using the adjusted personalized HRTF. The playback signal generated by the audio signal processing unit 263 is supplied to the output control unit 264.

[0163] The output control unit 264 outputs playback sound from the headphones 32 according to the playback signal supplied from the audio signal processing unit 263.

[0164] <Operation of playback device> Referring to the flowchart in Figure 22, the playback process performed by the playback device 31 in the playback environment will be explained.

[0165] In step S81, the audio signal acquisition unit 261 acquires the audio signal of the movie.

[0166] In step S82, the personalized HRTF file acquisition unit 262 acquires the personalized HRTF file provided by the information processing device 1.

[0167] In step S83, the audio signal processing unit 263 adjusts the personalized HRTF from speaker 23 to the ear using sound field adjustment values. The personalized HRTF from speaker 23 to the ear is obtained from the personalized HRTF data of the personalized HRTF file, and the sound field adjustment values ​​are obtained from the header portion of the personalized HRTF file.

[0168] In step S84, the audio signal processing unit 263 adjusts the inverse correction data of the characteristics from the headphones 22 to the ear using sound quality adjustment values. The inverse correction data is obtained from the personalized HRTF data of the personalized HRTF file, and the sound quality adjustment values ​​are obtained from the header portion of the personalized HRTF file.

[0169] In step S85, the audio signal processing unit 263 corrects the adjusted personalized HRTF from the speaker 23 to the ear using the adjusted inverse correction data. Specifically, the correction is performed by superimposing the inverse characteristics of the adjusted personalized HRTF from the headphones 22 to the ear onto the adjusted personalized HRTF from the speaker 23 to the ear.

[0170] In step S86, the audio signal processing unit 263 performs binaural processing on the audio signal of the movie using the corrected personalized HRTF. The playback signal is generated by the binaural processing.

[0171] In step S87, the output control unit 264 outputs playback sound corresponding to the playback signal from the headphones 32.

[0172] As described above, film sound producers can adjust their personalized HRTF. By making it possible to adjust the personalized HRTF itself, it becomes possible to reproduce the sound quality and sound field required in the production of film sound and music.

[0173] <<5. Variation>> While the instructions assume the use of over-ear headphones as the audio output device, in-ear headphones (earphones) may also be used. Furthermore, speakers may be used as the audio output device instead of headphones.

[0174] Although the adjustment of the personalized HRTF is assumed to be performed in the measurement environment, the adjustment of the personalized HRTF may be performed in an environment different from the measurement environment. In this case, the creator adjusts the personalized HRTF data using headphones 22 brought back from the measurement environment.

[0175] Although the sound production system in Figure 1 is used for producing sound for movies, the sound production system in Figure 1 can be applied to various sound production systems, such as systems used for music production and systems used for producing sound for television programs.

[0176] The adjustment of personalized HRTF data may be performed not during the production of the content's audio, but when the audio is played back on a consumer device, as described above.

[0177] While it is assumed that information recorded in the form of HRTF (Head-Related Impulse Response), which is information in the frequency domain, is used as the head-related transfer function (HRTF) representing the sound transmission characteristics, it is also possible to use information recorded in the form of HRIR (Head-Related Impulse Response), which is information in the time domain.

[0178] • Example computer configuration The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up the software are installed from a program storage medium onto a computer that is built into dedicated hardware, or a general-purpose personal computer.

[0179] Figure 23 is a block diagram showing an example of the hardware configuration of a computer that executes the series of processes described above by a program. The information processing device 1 and the playback device 31 are made up of a PC having the same configuration as shown in Figure 23.

[0180] The CPU (Central Processing Unit) 501, ROM (Read Only Memory) 502, and RAM (Random Access Memory) 503 are interconnected by a bus 504.

[0181] An input / output interface 505 is further connected to the bus 504. An input unit 506 consisting of a keyboard, mouse, etc., and an output unit 507 consisting of a display, speakers, etc. are connected to the input / output interface 505. In addition, a storage unit 508 consisting of a hard disk, non-volatile memory, etc., a communication unit 509 consisting of a network interface, etc., and a drive 510 that drives removable media 511 are connected to the input / output interface 505.

[0182] In a computer configured as described above, the CPU 501 performs the aforementioned series of processes by loading, for example, a program stored in the memory unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 and executing it.

[0183] The program executed by the CPU 501 is recorded on removable media 511, for example, or provided via a wired or wireless transmission medium such as a local area network, the internet, or digital broadcasting, and installed in the storage unit 508.

[0184] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when they are called.

[0185] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are located in the same enclosure. Therefore, multiple devices housed in separate enclosures and connected via a network, and a single device containing multiple modules within a single enclosure, are both considered systems.

[0186] The effects described herein are illustrative and not limited to those described herein, and other effects may also occur.

[0187] The embodiments of this technology are not limited to those described above, and various modifications are possible without departing from the spirit of this technology.

[0188] For example, this technology can be configured as cloud computing, where a single function is shared and processed collaboratively by multiple devices via a network.

[0189] Furthermore, each step described in the flowchart above can be performed by a single device, or it can be divided and performed by multiple devices.

[0190] Furthermore, if a single step includes multiple processes, those processes can be executed by a single device or shared among multiple devices.

[0191] <Examples of configuration combinations> (1) The system includes an adjustment unit that adjusts the personalized transmission characteristics, which are the sound transmission characteristics in the measurement environment and are individualized for the user, according to the user's operation. Information processing device. (2) The system further comprises a presentation unit that presents to the user, along with the personalization characteristics, the content of the reference information that is referenced when adjusting the personalization characteristics. The information processing device described in (1) above. (3) The adjustment unit adjusts the personalized transmission characteristics measured based on the playback sound output from the output device worn by the user. The information processing device described in (2) above. (4) The display unit displays the contents of the reference information obtained based on the device characteristics of the output device. The information processing device described in (3) above. (5) The adjustment unit adjusts the personalized transmission characteristics measured based on the reproduced sound output from the speaker installed in the measurement environment. The information processing device described in any of (2) to (4) above. (6) The display unit displays the contents of the reference information obtained based on the reverberation characteristics of the sound in the measurement environment. The information processing device described in (5) above. (7) The display unit presents the content of the reference information, which represents the reverberation characteristics of sound in a specified space different from the measurement environment. The information processing device described in (6) above. (8) The aforementioned reference information is recorded in the header portion of the file on which the data of the personal communication characteristics is recorded. The information processing device described in any of (2) to (7) above. (9) The system further includes a recording unit that records adjustment values ​​corresponding to the user's operation in the header section. The information processing device described in (8) above. (10) The display unit displays the contents of attribute information, which includes information representing the location of the measurement environment, information representing the user who will use the personalized transmission characteristics in the playback environment, and information representing the output device worn by the user. The information processing apparatus described in (8) or (9) above. (11) The attribute information is recorded in the header section. The information processing device described in (10) above. (12) Information processing device, The sound transmission characteristics in the measurement environment, and the personalized transmission characteristics which are those transmission characteristics personalized to the user, are adjusted according to the user's operation. Information processing methods. (13) On the computer, The sound transmission characteristics in the measurement environment, and the personalized transmission characteristics which are those transmission characteristics personalized to the user, are adjusted according to the user's operation. A program to execute a process. (14) The system includes a playback unit that adjusts the personalized transmission characteristics, which are the sound transmission characteristics in the measurement environment and are personalized to the user, based on adjustment values ​​set by the user, and then reproduces the audio signal using the adjusted personalized transmission characteristics. playback equipment. (15) The playback unit adjusts a first personalized transmission characteristic measured based on the playback sound output from a speaker installed in the measurement environment, and a second personalized transmission characteristic measured based on the playback sound output from an output device worn by the user. The playback equipment described in (14) above. (16) The playback unit corrects the first personalized transmission characteristic by superimposing the inverse characteristic of the adjusted second personalized transmission characteristic onto the adjusted first personalized transmission characteristic, and then plays back the audio signal using the corrected first personalized transmission characteristic. The playback equipment described in (15) above. (17) The playback device, The sound transmission characteristics in the measurement environment, and the personalized transmission characteristics which are the transmission characteristics personalized to the user, are adjusted based on the adjustment values ​​set by the user. The audio signal is reproduced using the adjusted personalized transmission characteristics. How to play. (18) On the computer, The sound transmission characteristics in the measurement environment, and the personalized transmission characteristics which are the transmission characteristics personalized to the user, are adjusted based on the adjustment values ​​set by the user. The audio signal is reproduced using the adjusted personalized transmission characteristics. A program to execute a process. [Explanation of Symbols]

[0192] 1 Information Processing Device, 1A Display, 21 Microphone, 22 Headphones, 31 Playback Device, 31A Display, 32 Headphones, 101 Information Processing Unit, 111 File Generation Unit, 112 Adjustment Value Recording Unit, 121 Playback Processing Unit, 122 Output Control Unit, 123 HRTF Acquisition Unit, 124 HRTF Data Generation Unit, 125 Reference Information Acquisition Unit, 126 HRTF File Generation Unit, 141 Personalized HRTF File Acquisition Unit, 142 Playback Processing Unit, 143 Output Control Unit, 144 Adjustment Unit, 145 Recording Unit, 171 File Information Display Unit, 172 Sound Quality Adjustment Unit, 173 Sound Field Adjustment Unit, 251 Playback Processing Unit, 261 Audio Signal Acquisition Unit, 262 Personalized HRTF File Acquisition Unit, 263 Audio Signal Processing Unit, 264 Output control unit

Claims

1. An adjustment unit that adjusts a second personalized transmission characteristic, which is used to correct a first personalized transmission characteristic, which is a sound transmission characteristic personalized to the user in the measurement environment, according to the user's operation, A presentation unit presents to the user, along with the second personalization characteristics, information acquired based on the device characteristics of the output device worn by the user, as reference information to be referenced when adjusting the personalization characteristics. Equipped with, The second personalized transmission characteristic is measured based on the reproduced sound output from the output device. Information processing device.

2. The adjustment unit adjusts the second personalized transmission characteristic and the first personalized transmission characteristic, which is measured based on the reproduced sound output from the speaker installed in the measurement environment, according to the user's operation. The information processing apparatus according to claim 1.

3. The display unit presents to the user, as reference information, information acquired based on the reverberation characteristics of the sound in the measurement environment, along with the first personalized transmission characteristics. The information processing apparatus according to claim 2.

4. The display unit presents, as reference information, information representing the reverberation characteristics of sound in a specified space different from the measurement environment, along with the first personalized transmission characteristics. The information processing apparatus according to claim 3.

5. The aforementioned reference information is recorded in the header portion of the file containing the data for the second personalization characteristics. An information processing apparatus according to any one of claims 1 to 4.

6. The system further includes a recording unit that records adjustment values ​​corresponding to the user's operation in the header section. The information processing apparatus according to claim 5.

7. The display unit presents attribute information to the user, including information representing the location of the measurement environment, information representing the user who will use the first personalized transmission characteristics and the second personalized transmission characteristics in the playback environment, and information representing the output device. The information processing apparatus according to claim 5 or 6.

8. The attribute information is recorded in the header section. The information processing apparatus according to claim 7.

9. Used to correct a first personalized transmission characteristic, which is a sound transmission characteristic personalized to a user in a measurement environment, and to adjust a second personalized transmission characteristic, measured based on the reproduced sound output from an output device worn by the user, according to the user's operation, As reference information to be used when adjusting the personalized communication characteristics, information obtained based on the device characteristics of the output device is presented to the user along with the second personalized communication characteristics. Information processing methods including

10. On the computer, This method is used to correct a first personalized transmission characteristic, which is the sound transmission characteristic personalized to the user in the measurement environment, and adjusts a second personalized transmission characteristic, measured based on the reproduced sound output from an output device worn by the user, according to the user's operation. As reference information to be used when adjusting the personalized communication characteristics, information obtained based on the device characteristics of the output device is presented to the user along with the second personalized communication characteristics. A program that performs a process that includes the following.

Images