Information processing device, information processing method, and program

Abstract

The present technology relates to an information processing device, an information processing method, and a program with which it is possible to easily create high-quality content. An information processing device according to an embodiment of the present technology comprises a signal processing parameter generation unit that, in accordance with a change in the three-dimensional position of an audio object, generates a signal processing parameter corresponding to the post-change three-dimensional position, said signal processing parameter indicating content for signal processing to be performed on an audio signal of the audio object. The present technology can be applied to an information processing device used for work involving mixing 3D audio content.

Description

Information processing device, information processing method, and program

[0001] The present technology relates to an information processing device, an information processing method, and a program, and more particularly to an information processing device, an information processing method, and a program that enable high-quality content to be created.

[0002] There is known a technology for automatically determining the three-dimensional position, gain, etc. of object audio (see, for example, Patent Document 1). By using such a technology, users can easily create 3D audio content in a short amount of time.

[0003] International Publication No. 2020 / 066681

[0004] After determining the three-dimensional position, gain, etc. of an object, signal processing according to the three-dimensional position of the object may be applied to the audio signal itself to further improve the auditory sensation. If the three-dimensional position information of the object is further changed after signal processing has been applied to the audio signal, it is necessary to change the content of the signal processing applied to the audio signal again, which increases the work time.

[0005] The present technology has been developed in light of these circumstances, and makes it possible to easily create high-quality 3D audio content.

[0006] An information processing device according to one aspect of the present technology includes a signal processing parameter generation unit that generates signal processing parameters corresponding to a changed three-dimensional position of an audio object, the signal processing parameters indicating the content of signal processing for the audio signal of the audio object, in response to a change in the three-dimensional position of the audio object.

[0007] An information processing method or program according to one aspect of the present technology includes a step of generating, in response to a change in the three-dimensional position of an audio object, signal processing parameters corresponding to the changed three-dimensional position, which indicate the content of signal processing for the audio signal of the audio object.

[0008] In one aspect of the present technology, in response to a change in the three-dimensional position of an audio object, a signal processing parameter is generated that indicates the content of signal processing for the audio signal of the audio object and corresponds to the three-dimensional position after the change.

[0009] 1 is a diagram illustrating a software configuration for realizing a 3D audio content production system. FIG. 1 is a diagram illustrating a 3D audio content production plug-in. FIG. 1 is a diagram illustrating an example configuration of a signal processing parameter generation unit. FIG. 1 is a diagram illustrating an example of data included in a parameter interpolation map. FIG. 2 is a diagram illustrating point interpolation. FIG. 3 is a diagram illustrating areal interpolation. FIG. 4 is a diagram illustrating calculation of contribution. FIG. 5 is a diagram illustrating line interpolation. FIG. 6 is a diagram illustrating a GUI of a parameter interpolation map creation screen. FIG. 7 is a diagram illustrating a GUI when point interpolation is the interpolation pattern. FIG. 8 is a diagram illustrating a GUI when areal interpolation is the interpolation pattern. FIG. 9 is a diagram illustrating a GUI when applying a parameter interpolation map to an object. FIG. 10 is a diagram illustrating a GUI when line interpolation is the interpolation pattern. FIG. 11 is a diagram illustrating a GUI when applying a parameter interpolation map to a movement template. FIG. 12 is a diagram illustrating a GUI in which coordinate regions are three-dimensionally drawn. FIG. 13 is a flowchart of parameter interpolation map creation processing. FIG. 14 is a flowchart of signal processing parameter calculation processing. FIG. 15 is a diagram illustrating an example configuration of an information processing device. FIG. 16 is a diagram illustrating a 3D audio content production device.

[0010] Hereinafter, embodiments of the present technology will be described in the following order: 1. Overview of the present technology 2. Background 3. 3D audio content production system 4. Interpolation patterns of signal processing parameters (point interpolation, areal interpolation, linear interpolation) 5. Operation using GUI 6. Operation of signal processing parameter generation unit 7. Hardware configuration 8. Modified examples

[0011] <<1. Overview of the Present Technology>> The present technology makes it possible to automatically apply signal processing to audio signals according to the positions of audio objects (hereinafter also simply referred to as objects) when mixing content that uses 3D audio.

[0012] In particular, this technology associates the three-dimensional position information and gain (hereinafter referred to as metadata) of an object with signal processing parameters, which are parameters that represent the degree of signal processing to be applied to the object's audio signal, and automatically generates signal processing parameters in response to changes in the metadata.

[0013] In mixing 3D audio content, which is content that uses 3D object audio, adjusting the three-dimensional position of an object to a desired position is performed as metadata editing. This technology is used to support the production of 3D audio content by automatically generating signal processing parameters in response to, for example, a user adjusting the three-dimensional position of the object. For example, a mixing engineer who produces the 3D audio content is a user.

[0014] <1.1 Configuration of the Present Technology> The present technology includes two methods: a method for automatically generating signal processing parameters according to metadata, and a method for generating a parameter interpolation map using a GUI. The present technology is particularly configured as follows.

[0015] 1. Method for Automatically Generating Signal Processing Parameters 1.1 Based on a parameter interpolation map applied to each object by a user, signal processing parameters corresponding to the audio signal (audio data) of each object are generated. A parameter interpolation map is data describing the relationship between the position in three-dimensional space (three-dimensional position) where an object is located and the signal processing parameters of a given signal processing method. The parameter interpolation map is created in advance by the user as information for interpolating and generating signal processing parameters. 1.2 Signal processing parameters are generated based on not only three-dimensional position information of an object whose metadata is being edited or an object that is not being edited, but also information about the object, such as music information, object type, and waveform information features. 1.3 The signal processing parameter interpolation pattern may be point interpolation, areal interpolation, or linear interpolation. When the three-dimensional position of an object is edited, signal processing parameters corresponding to the edited three-dimensional position are generated based on the signal processing parameters of the interpolation points. Information about the interpolation points that serve as the basis for interpolating the signal processing parameters is included in the parameter interpolation map. 1.4 The parameter interpolation map includes information on the interpolation pattern, information on the signal processing method, interpolation parameters, and interpolation point information. 1.5 The user can select the parameter interpolation map to apply to an object. Furthermore, multiple parameter interpolation maps may be applied to one object. By applying a parameter interpolation map to an object, signal processing parameters are calculated in accordance with the interpolation content set in the parameter interpolation map in response to editing of metadata.

[0016] 2. Method for creating a parameter interpolation map using a GUI 2.1 The user can save, add, and edit parameter interpolation maps. 2.2 The interpolation pattern, signal processing method, interpolation parameters, and interpolation point information in the parameter interpolation map are determined by the user. 2.3 The parameter interpolation map is created using a screen that includes a display using a method for drawing a three-dimensional space as a 2D image or a method for drawing a three-dimensional space as a 3D image.

[0017] <<2. Background>> 3D audio differs from conventional 2-channel audio in that it offers a new musical experience in which sound can be heard from all directions, 360°. In particular, object audio, which is one format of 3D audio, allows for the expression of a variety of sounds by placing sound sources (objects) at any position in space.

[0018] To further popularize 3D audio, it is necessary to create a large amount of high-quality content. Here, mixing, that is, determining the three-dimensional position and gain of each object, becomes important. There are people called mixing engineers who specialize in mixing work.

[0019] A common method for producing 3D audio content is to convert existing 2-channel audio content into 3D audio content. In this case, the mixing engineer receives the existing 2-channel audio data in a state where it is separated into individual objects. For example, the mixing engineer receives audio signals for each object, such as a kick object, a bass object, and a vocal object.

[0020] Next, the mixing engineer listens to the entire content and the sound of each object, and analyzes the type of content, such as genre and melody, and the type of each object, such as instrument type, etc. The mixing engineer also analyzes the sound characteristics of each object, such as attack and duration.

[0021] The mixing engineer then determines the position and gain of each object in three-dimensional space based on the results of these analyses. Even for objects of the same instrument type, the appropriate three-dimensional position and gain will differ depending on the sound characteristics of the object, the genre of the music, etc.

[0022] Another technique used is to change the three-dimensional position and gain of an object over time within a song. After analyzing each track and the entire song according to the production flow described above, the three-dimensional position and gain can be changed over time based on the analysis results to change the auditory sensation and create excitement in the song.

[0023] After performing these production steps of varying the three-dimensional position and gain over time, a production technique may be employed to enhance the 3D audio effect by applying signal processing to the audio signal of the object itself, taking into account the three-dimensional position of the object, allowing the user to perceive various effects according to the three-dimensional position of the object.

[0024] <2.1 Contents and Effects of Signal Processing> The signal processing applied to the audio signal of an object has the following processing and effects.

[0025] (Signal Processing Example 1) The distance between the listening position and the localization position of the object is adjusted. In the rendering process when a user listens to 3D audio content, a head-related transfer function (HRTF) is convolved with the audio signal, so that each object included in the audio signal is localized outside the head of the user who is the listener, rather than inside the head. In other words, the distance between each object and the user becomes greater.

[0026] However, depending on the type of object, the user may find it preferable for the listening position and the object to be closer to each other.

[0027] In this case, signal processing is performed on the audio signal of a specific type of object, such that the inverse characteristics of the head-related transfer function are applied in advance, which makes the listener perceive the object as being closer to the user after the rendering process is performed.

[0028] (Signal Processing Example 2) In order to change the auditory sensation, editing is performed by moving an object up, down, left, and right in three-dimensional space. An object whose three-dimensional position changes over time is called a moving object.

[0029] Due to the characteristics of human hearing, it is often difficult to perceive the vertical movement of an object. Also, low-frequency sounds tend to be heard louder for objects at low positions, while high-frequency sounds tend to be heard louder for objects at high positions. For example, if low-frequency sounds are emphasized when an object is at a low position, it becomes easier to perceive the object as being at a lower position.

[0030] To make it easier to perceive the vertical movement of an object, equalizer processing may be applied to the audio signal in accordance with metadata indicating the object's movement path or the object's three-dimensional position.

[0031] (Signal Processing Example 3) Processing is performed to adjust the perceived distance between an object and a listener by adjusting the sound pressure or changing the type and intensity of reverb processing. Such adjustments to the sense of distance are also performed when mixing general music, not 3D audio. For example, if the three-dimensional position of an object is above the listener's head, a user may apply reverb processing that retains high-frequency sounds. In this way, the content of the reverb processing may be adjusted based on the three-dimensional position of the object.

[0032] 2.2 Effect on signal processing of objects other than the object being edited 3D audio content typically includes audio signals of multiple objects. The signal processing applied to the object being edited is adjusted based on the relationship between the object being edited and other objects.

[0033] For example, when the target object and other objects are close to each other, signal processing is performed to add equalizer processing to one of the objects so that the sound of the other object can be heard clearly. The level of equalizer processing is determined based on information such as the degree of spatial masking due to the timbre of the target object and other objects, the type of instrument, and priority. The level of equalizer processing is also determined according to the three-dimensional positions of the target object and other objects and the distance between them.

[0034] The bandwidth of the equalizer processing, the type of equalizer filter, etc. may be determined based on information such as the size of spatial masking of the target object and other objects, the type of instrument, the priority, the three-dimensional position, and the distance between the two objects, as described above, and other parameters related to the equalizer processing may also be determined.

[0035] The content of these signal processing steps for enhancing the sense of positioning is determined by engineers based on information such as the three-dimensional position of an object, and determining the content of such signal processing requires a high level of experience, knowledge, and time. Because the signal processing method and parameters applied to an audio signal are determined according to the three-dimensional position of the object, the signal processing method and parameters must be changed every time the three-dimensional position of the object is edited. Furthermore, multiple types of signal processing methods and parameters may be associated with a single object, and it would be time-consuming to change all of the signal processing methods and parameters every time the three-dimensional position of the object is edited.

[0036] In this technology, parameters of signal processing applied to an audio signal are associated with the three-dimensional position of an object. It is also possible to associate features extracted from the audio signal. This makes it possible to automatically calculate signal processing parameters when the three-dimensional position of an object is edited. This eliminates the need to repeatedly determine signal processing parameters, even if the three-dimensional position of an object is edited after determining the signal processing parameters.

[0037] If the task of setting signal processing parameters could be automated, 3D audio content could be produced in a shorter time, which could lead to further popularization of 3D audio.

[0038] 1 is a block diagram showing an example of the software configuration for realizing a 3D audio content production system. The software for realizing the 3D audio content production system is made up of a 3D audio content production plug-in 11, a music production tool 12, and a signal processing plug-in 13. The 3D audio content production system is realized by running the 3D audio content production plug-in 11, the music production tool 12, and the signal processing plug-in 13 on a computer such as a PC.

[0039] The music production tool 12 is software that controls the entire 3D audio content production system. The music production tool 12 is made up of software such as a DAW (Digital Audio Workstation). The music production tool 12 supplies audio signals to the 3D audio content production plug-in 11.

[0040] The 3D audio content production plug-in 11 is software for producing 3D audio content, which is provided as a plug-in for a music production tool 12, which is a DAW. Various processes relating to the production of 3D audio content are performed in response to user operations.

[0041] For example, the 3D audio content production plug-in 11 imports audio signals of each object from the music production tool 12, edits metadata, and generates signal processing parameters according to the metadata. The 3D audio content production plug-in 11 supplies the generated signal processing parameters to the signal processing plug-in 13. Audio signals to be subjected to signal processing using the signal processing parameters are also supplied from the 3D audio content production plug-in 11 to the signal processing plug-in 13 as appropriate.

[0042] The signal processing plug-in 13 is signal processing software provided as a plug-in for the music production tool 12. Signal processing plug-ins 13 are provided according to the type of signal processing, such as a plug-in that applies equalizer processing to audio signals, a plug-in that applies reverb processing, etc. Although one signal processing plug-in 13 is shown in Fig. 1, multiple signal processing plug-ins 13 may be provided as appropriate.

[0043] The signal processing plug-in 13 performs signal processing on the audio signal supplied from the 3D audio content production plug-in 11 based on the acquired signal processing parameters, and supplies the processed audio signal to the 3D audio content production plug-in 11 or the music production tool 12. The audio signal and metadata may be supplied from the 3D audio content production plug-in 11 to the signal processing plug-in 13 via the music production tool 12.

[0044] FIG. 2 is a block diagram showing an example of the configuration of the 3D audio content production plug-in 11. As shown in FIG.

[0045] The 3D audio content production plug-in 11 has, as its functional components, a signal processing parameter generation unit 21, a GUI control unit 22, and a 3D audio playback control unit 23.

[0046] The signal processing parameter generation unit 21 generates signal processing parameters according to the three-dimensional position of the object using a parameter interpolation map created in advance. Processing such as creating the parameter interpolation map is also performed by the signal processing parameter generation unit 21. Details of the functions of the signal processing parameter generation unit 21 will be described later. The signal processing parameters generated by the signal processing parameter generation unit 21 are supplied to the signal processing plug-in 13.

[0047] The GUI control unit 22 controls screen output to receive input from the user when controlling automatic mixing from parameter interpolation map creation to application to each object. For example, the GUI control unit 22 displays on the screen a GUI for applying an arbitrary parameter interpolation map to an object and receives input from the user. The input information is supplied to the signal processing parameter generation unit 21.

[0048] The 3D audio playback control unit 23 performs rendering on the mixed audio signal acquired from the music production tool 12 or the signal processing plug-in 13, and controls the playback of the audio signal.

[0049] FIG. 3 is a block diagram showing an example of the configuration of the signal processing parameter generating unit 21 shown in FIG.

[0050] The signal processing parameter generation unit 21 has, as functional components, an audio signal acquisition unit 31, a feature calculation unit 32, an interpolation pattern acquisition unit 33, a signal processing parameter acquisition unit 34, a metadata acquisition unit 35, and a signal processing parameter calculation unit 36. The signal processing parameter calculation unit 36 ​​further has a parameter interpolation map creation unit 37 and a calculation unit 38.

[0051] The audio signal acquisition unit 31 acquires the audio signals of each object from the music production tool 12, which is software that controls the entire 3D audio content production system, and supplies them to the feature calculation unit 32. The audio signals of various tracks used in producing 3D audio content are managed in the music production tool 12, and the audio signals of the tracks that make up each object are acquired by the audio signal acquisition unit 31.

[0052] The feature calculation unit 32 calculates feature amounts including information about the objects, such as music information indicating information about the instruments of the objects, object information indicating the types of the objects, and feature amounts of waveform information, from the audio signals acquired from the audio signal acquisition unit 31, and supplies the calculated feature amounts to the signal processing parameter calculation unit 36. The music information may be tempo, genre, or other information, and the type of the objects may be the type of instrument of each object or other information.

[0053] The interpolation pattern acquisition unit 33 acquires information related to the interpolation of signal processing parameters based on input from the user, and supplies the information to the signal processing parameter calculation unit 36. The information related to the interpolation of signal processing parameters includes information indicating an interpolation pattern for calculating signal processing parameters corresponding to the three-dimensional position of an object, interpolation parameters that are information indicating the interpolation strength and interpolation method, and position information of interpolation points that serve as reference points for interpolation. In other words, when the user uses a screen such as that described below to input how they want signal processing to be performed when an object is placed at each position in three-dimensional space, various information including information about the interpolation pattern is acquired by the interpolation pattern acquisition unit 33.

[0054] The signal processing parameter acquisition unit 34 acquires a signal processing method such as equalizer or reverb based on an input from the user, and supplies information indicating the signal processing method to the signal processing parameter calculation unit 36. The signal processing method may be a method other than equalizer or reverb, such as adding the inverse characteristics of HRTF.

[0055] Furthermore, the signal processing parameter acquisition unit 34 acquires signal processing parameters corresponding to the interpolation points for each signal processing method based on input from the user, and supplies the signal processing parameters to the signal processing parameter calculation unit 36. For example, if the signal processing method is an equalizer, the signal processing parameters include parameters indicating the center frequency, frequency width, gain, filter type, etc., and if the signal processing method is reverb, a parameter indicating the type of reverb is included.

[0056] When the user edits the metadata, the metadata acquisition unit 35 acquires the metadata changed by the editing and supplies it to the signal processing parameter calculation unit 36. For example, the user edits the metadata using a screen including a display showing a three-dimensional space in which objects are arranged.

[0057] The signal processing parameter calculation unit 36 ​​acquires the feature amounts, information relating to the interpolation of the signal processing parameters, information indicating the signal processing method, the signal processing parameters, and metadata supplied from each unit.

[0058] The parameter interpolation map creation unit 37 in the signal processing parameter calculation unit 36 ​​creates a parameter interpolation map based on information related to the interpolation of the signal processing parameters, information indicating the signal processing method, and the signal processing parameters. The parameter interpolation map is created in advance, for example, before the user edits the metadata.

[0059] FIG. 4 shows an example of data contained in the parameter interpolation map described above.

[0060] The parameter interpolation map includes information on the interpolation pattern, the signal processing method, the interpolation parameters, and the interpolation point information.

[0061] The interpolation pattern is information indicating an interpolation pattern for calculating signal processing parameters corresponding to the three-dimensional position of an object. Interpolation patterns include point interpolation, areal interpolation, linear interpolation, etc. Details of point interpolation, areal interpolation, and linear interpolation will be described later.

[0062] The signal processing method is information indicating the method of signal processing to be applied to the audio signal based on the three-dimensional position of the object. For example, signal processing such as applying an equalizer, reverb, or the inverse characteristics of HRTF may be used, but other methods may also be used.

[0063] The interpolation parameters are information indicating the interpolation strength and the interpolation method. The interpolation strength indicates the degree of interpolation and is expressed using a number from 0 to 100, for example. The interpolation method indicates the interpolation method, such as linear interpolation or quadratic interpolation. Other information used to interpolate and calculate signal processing parameters corresponding to any three-dimensional position may also be included in the interpolation parameters.

[0064] The interpolation point information includes the positions of the interpolation points and signal processing parameters. The positions of the interpolation points are set by the user in the three-dimensional space where the object is placed and serve as the basis for interpolation, and the signal processing parameters are variables that represent the details of processing in a certain signal processing method. The interpolation point information is generated by setting the signal processing parameters for the positions of the interpolation points.

[0065] Returning to the description of Fig. 3, the calculation section 38 in the signal processing parameter calculation section 36 uses the parameter interpolation map created in advance by the parameter interpolation map creation section 37 to calculate signal processing parameters corresponding to the three-dimensional position information of the object included in the metadata acquired by the metadata acquisition section 35. In other words, interpolation is performed to calculate signal processing parameters corresponding to the three-dimensional position of the object based on the interpolation pattern, interpolation parameters, position information of the interpolation points, signal processing method, and signal processing parameters included in the parameter interpolation map. By calculating the signal processing parameters by interpolating based on information about the interpolation points that have been set in advance, the calculation section 38 can calculate and generate signal processing parameters corresponding to the three-dimensional position indicated by the edited metadata even if the signal processing parameters corresponding to that position have not been set.

[0066] Furthermore, by allowing the user to input each piece of data contained in the parameter interpolation map, the user can freely set the method of calculating and interpolating the signal processing parameters.

[0067] In addition, at this time, the generation of signal processing parameters using the parameter interpolation map may be controlled according to not only the edited metadata but also the feature amounts. Furthermore, signal processing parameters using the parameter interpolation map may be generated according to the metadata and feature amounts of objects other than the target object. This makes it possible to set signal processing parameters that comprehensively take into account the metadata and feature amounts of the target object and other objects.

[0068] <<4. Interpolation Pattern of Signal Processing Parameter>> The signal processing parameter calculation unit 36 ​​stores the contents of each interpolation pattern, and an interpolation pattern according to an input from the user is used when creating a parameter interpolation map. For example, the user selects one of the following three types of interpolation patterns: (1) Parameter interpolation at any position between three-dimensional positions (point interpolation) (2) Parameter interpolation within any region including three-dimensional positions (area interpolation) (3) Parameter interpolation at any position on a movement path (line interpolation)

[0069] <4.1 Point Interpolation> Point interpolation is an interpolation pattern in which signal processing parameters set at each of a plurality of interpolation points are used to interpolate signal processing parameters at positions between interpolation points. The user sets the plurality of three-dimensional positions to be used as interpolation points and the signal processing parameters for each interpolation point when creating a parameter interpolation map.

[0070] The position in three-dimensional space where an object is placed is represented, for example, by "azimuth," which indicates the horizontal angle, "elevation," which indicates the vertical angle, and "radius," which indicates the distance (radius) from a reference position. The position of an object is represented using coordinates in such a polar coordinate system. Furthermore, the position in three-dimensional space where an object is placed may be represented not only by polar coordinates but also by Cartesian coordinates, which are x, y, and z coordinates. Cartesian coordinates may be used not only in FIG. 5 but also in the examples of FIGS. 6 to 15 .

[0071] FIG. 5 is a diagram illustrating an example of point interpolation.

[0072] Interpolation points P11 to P14 shown in Figure 5 are interpolation points whose three-dimensional positions have been set by the user. For example, coordinates are displayed with the horizontal axis representing azimuth and the vertical axis representing elevation, and the user sets each interpolation point using this display. Interpolation points can be added or deleted by the user.

[0073] Signal processing parameters are set and associated with the positions represented by the interpolation points. As shown in Fig. 5, the signal processing parameter values ​​corresponding to each interpolation point are set in such a way that, for example, the signal processing parameter a corresponding to interpolation point P11 has a value of 0.4, the signal processing parameter a corresponding to interpolation point P12 has a value of 0.2, the signal processing parameter a corresponding to interpolation point P13 has a value of 0.7, and the signal processing parameter a corresponding to interpolation point P14 has a value of 0.3.

[0074] A case will be described in which position P21 exists between the interpolation points as the three-dimensional position for which the signal processing parameters are to be calculated. When position P21 is set as the three-dimensional position of the edited object, position P21 becomes the three-dimensional position for which the signal processing parameters are to be calculated.

[0075] In this case, as indicated by the dashed lines, the signal processing parameter value corresponding to position P21 is calculated by referencing and interpolating the signal processing parameter values ​​at three or four surrounding interpolation points. At this time, the interpolation is performed based on the interpolation strength input by the user and the selected interpolation method. A known interpolation method between three or four points may be used, or another method may be used. In this way, it is possible to calculate signal processing parameters corresponding to any three-dimensional position of the object.

[0076] With point interpolation, signal processing parameters can be set for specific three-dimensional positions of interpolation points, so users with a specific image of the signal processing they want to perform can set the details of the interpolation. This allows users to reduce the amount of work required for production while maintaining the quality of the signal processing they apply.

[0077] <4.2 Areal Interpolation> Areal interpolation is an interpolation pattern that uses signal processing parameters set by the user for a certain interpolation point to interpolate signal processing parameters at positions within an area that includes the interpolation point. One interpolation point and an interpolation area, which is an area affected by the value of the signal processing parameter, are set when creating a parameter interpolation map.

[0078] FIG. 6 is a diagram illustrating an example of areal interpolation.

[0079] The concentric circles shown in Fig. 6 indicate interpolation areas. In the example of Fig. 6, a circle is used as the shape indicating the interpolation area. For example, coordinates are displayed with the horizontal axis representing azimuth and the vertical axis representing elevation, and the user sets the interpolation area using this display. Signal processing parameters are set for the interpolation point at the center of the interpolation area. Any value of the signal processing parameter is set outside the interpolation area. As shown in Fig. 6, for example, 0.3 is set as the value of signal processing parameter b corresponding to interpolation point P31. 0 is set as the value of signal processing parameter b outside the interpolation area.

[0080] The contribution of the interpolation parameter associated with interpolation point P31 is determined according to its distance from the center of the interpolation region, and the value of the signal processing parameter at a certain three-dimensional position within the interpolation region is calculated according to the contribution. The contribution refers to the degree to which the signal processing parameter set at the interpolation point affects the certain three-dimensional position. For example, the contribution is expressed as a value ranging from 0.0 to 1.0. The greater the contribution, the closer the value of the signal processing parameter at the certain three-dimensional position will be to the value of the signal processing parameter set at the interpolation point. For example, if the contribution at a certain three-dimensional position for which a signal processing parameter is to be calculated is 1.0, the value of the signal processing parameter set at interpolation point P31 itself will be applied to that three-dimensional position.

[0081] 7 is a diagram showing an example of calculation of the contribution degree according to the distance from the center of the interpolation area. The horizontal axis of Fig. 7 represents the distance from the interpolation point P31, and the vertical axis represents the contribution degree. The dashed-dotted line L11, dotted line L12, and solid line L13 each represent the contribution degree at each three-dimensional position according to the distance from the interpolation point P31.

[0082] The degree of contribution is determined by three parameters: the central parameter effective radius #11, the interpolation effective radius #12, and the steepness.

[0083] The central parameter effective radius #11 is a value indicating the radius of the range in which the signal processing parameters corresponding to the interpolation point located at the center of the interpolation area are directly reflected. The interpolation effective radius #12 is a value indicating the radius of the range in which the signal processing parameters corresponding to the interpolation point located at the center of the interpolation area are affected. The range between the central parameter effective radius #11 circle and the interpolation effective radius #12 circle (the range indicated by fine dot hatching in Figure 6) is the range in which the contribution changes.

[0084] The steepness is a value indicating how gently the interpolation is performed in the hatched range between the circle of the central parameter effective radius #11 and the circle of the interpolation effective radius #12. For example, the steepness is indicated by a value ranging from 0.0 to 1.0. The dashed-dotted line L11 indicates the change in contribution when the steepness is 0. In this case, the contribution changes sharply from 0 to 1 at a distance from the interpolation point P31 equal to the average length of the central parameter effective radius #11 and the interpolation effective radius #12. The dotted line L12 indicates the change in contribution when the steepness is any value in the range from 0.0 to 1.0. In this case, the interpolation is performed more gently as the steepness changes from 0 to 1. The solid line L13 indicates the change in contribution when the steepness is 1.0. In this case, the contribution changes linearly in the range between the circle with the central parameter effective radius #11 and the circle with the interpolation effective radius #12, and the value of the signal processing parameter calculated by interpolation also changes linearly. This interpolation method may be the method described above or any other method.

[0085] Although the interpolation area has been described as having a circular shape, other shapes such as an ellipse, a rectangle, or a triangle may also be used as the shape of the interpolation area.

[0086] By using areal interpolation, all three-dimensional positions included in the interpolation area can be specified as three-dimensional positions to which a certain signal processing parameter is to be associated. This eliminates the need to set many interpolation points when associating the same signal processing parameter with a wide area in three-dimensional space, leading to a reduction in work time.

[0087] <4.3 Linear Interpolation> Linear interpolation is an interpolation pattern that interpolates signal processing parameters at a certain position on the movement path of a moving object, which is an object whose metadata changes over time, using interpolation points set on the movement path. When creating 3D audio content, a user can save the movement path of a moving object as a movement template via a GUI. Examples of movement paths include vertical movement such as ascending or descending, horizontal movement such as rotating clockwise or counterclockwise, and floating movement that makes small movements up, down, left, and right. The metadata of the object changes according to this movement path. A parameter interpolation map with linear interpolation as the interpolation pattern is created for the movement template saved by the user.

[0088] 8 is a diagram showing an example of linear interpolation, in which the curved line shown in the center of the figure represents the movement path.

[0089] An interpolation point P41 is set as the start point of the movement path of the three-dimensional position indicated by the metadata, an interpolation point P42 is set as the end point, and interpolation points P43 and P44 are set as passing points, and a signal processing parameter is set for each interpolation point. As shown in Fig. 8, for example, 0.3 is set as the value of the signal processing parameter c corresponding to the interpolation point P41, and 1 is set as the value of the signal processing parameter c corresponding to the interpolation point P42. Furthermore, the user may be able to add or delete passing points on the movement path as desired.

[0090] The signal processing parameters corresponding to any position on the movement path are calculated by interpolating the signal processing parameters set at the positions of the interpolation points set before and after the any position. The interpolation method may be selected by the user from known interpolation methods such as linear interpolation and quadratic interpolation.

[0091] Because it is possible to interpolate signal processing parameters corresponding to any three-dimensional position on the movement path, when the section to which the movement template is applied is changed, the same signal processing parameters used in the section before the change are automatically set as the signal processing parameters to be used in the section after the change. Once a parameter interpolation map with linear interpolation as the interpolation pattern is created, there is no need to set signal processing parameters each time the section to which the moving object is applied is changed, which saves the effort required to create content using moving objects.

[0092] 2 displays a GUI on the screen, which is used by the user. Various GUIs, such as a GUI used to create a parameter interpolation map and a list of created parameter interpolation maps, are displayed by the GUI control unit 22. Various screens are displayed as appropriate using the functions of the music production tool 12.

[0093] <5.1 Creation of Parameter Interpolation Map> When the Interpolation Pattern is Point Interpolation Fig. 9 is a diagram showing an example of a display of a parameter interpolation map creation screen. The parameter interpolation map creation screen is a GUI used to create a parameter interpolation map. When an interpolation pattern of point interpolation is selected, the parameter interpolation map creation screen shown in Fig. 9 is displayed.

[0094] As shown in FIG. 9, a processing method selection area 61 is displayed on the left edge of the parameter interpolation map creation screen, and a tools area 62, an interpolation parameter setting area 63, and a signal processing parameter setting area 64 are displayed on the right side thereof.

[0095] The processing method selection area 61 is an area used to select a signal processing method. Various signal processing methods, including those implemented in the signal processing plug-in 13, such as EQ (equalizer), dynamic, imager, etc., are displayed in the processing method selection area 61. In the example of FIG. 9, "EQ (equalizer)" is displayed as the signal processing method selection result, and "EQ2" is selected from among them. The content set by the user using the display in the processing method selection area 61 is acquired by the signal processing parameter acquisition unit 34.

[0096] The tools area 62 is an area used to select a map creation tool. When the interpolation pattern is point interpolation, the tools area 62 displays information such as an icon indicating an interpolation point as a map creation tool.

[0097] The interpolation parameter setting area 63 is an area used to set interpolation parameters. In the example of Fig. 9, the interpolation strength (steepness) is set to "70" using the slide bar, and the interpolation method is set to "linear interpolation" using the pull-down menu. The contents set by the user using the display in the interpolation parameter setting area 63 are acquired by the interpolation pattern acquisition unit 33.

[0098] The signal processing parameter setting area 64 is an area used to set signal processing parameters to be set at interpolation points. A parameter window including a slide bar (not shown) used to set the signal processing parameters is displayed in the signal processing parameter setting area 64. When the signal processing method is an equalizer, tools such as a slide bar used to select each parameter such as the center frequency, frequency width, gain, and filter type are displayed in the parameter window. The contents set by the user using the display in the signal processing parameter setting area 64 are acquired by the signal processing parameter acquisition unit 34.

[0099] An editor area 65 is displayed to the right of the tools area 62, the interpolation parameter setting area 63, and the signal processing parameter setting area 64. The editor area 65 is an area used for editing interpolation points and interpolation areas. A rectangular coordinate area 71 is displayed within the editor area 65, and a new button 72 and a save button 73 are displayed at the bottom right of the coordinate area 71.

[0100] Coordinates indicating a three-dimensional space are displayed in the coordinate area 71. In the example of Fig. 9, coordinates are displayed that indicate the range of "azimuth," which is the horizontal angle, from -180° to 180°, and the range of "elevation," which is the vertical angle, from -90° to 90°, with the center of the coordinate area 71 as the origin. The New button 72 is a button that is operated when creating a new parameter interpolation map, and the Save button 73 is a button that is operated when saving a parameter interpolation map that is being created.

[0101] An example of a user operation when the interpolation pattern is point interpolation will be described using Fig. 10. In Fig. 10, parts corresponding to those in Fig. 9 are given the same reference numerals, and their description will be omitted as appropriate. The same applies to Figs. 11, 13, and 15, which will be described later.

[0102] The user selects any signal processing method (EQ, etc.) from the signal processing methods displayed in the processing method selection area 61. This signal processing method may be a function implemented in the music production tool 12, or may be a function implemented in the signal processing plug-in 13. The function for performing signal processing may also be implemented in the 3D audio content production plug-in 11. When the user selects a signal processing method, a coordinate area 71 is displayed in the editor area 65.

[0103] As indicated by arrow A11, the user drags and drops an icon representing an interpolation point from the tools area 62 to the coordinate area 71, thereby setting an interpolation point P51 as an interpolation point image in the coordinate area 71. In the example of FIG. 10 , four interpolation points are set in addition to the interpolation point P51. Note that the interpolation points may also be set by operations other than drag and drop. Furthermore, even after the interpolation points are set in the coordinate area 71, the positions of the interpolation points may be freely edited.

[0104] When the user clicks on interpolation point P51 from among the multiple interpolation points, the signal processing parameter setting area 64 is expanded (made operable) as indicated by arrow A12. The user inputs signal processing parameters using the display in the signal processing parameter setting area 64, thereby setting arbitrary signal processing parameters for the interpolation point P51, with the signal processing method set to "EQ2." This associates the signal processing parameters of the arbitrary signal processing method with the three-dimensional position of the interpolation point P51. At this time, the GUI of the signal processing plug-in 13 may be displayed in the signal processing parameter setting area 64.

[0105] The display of the interpolation parameter setting area 63 and the like are also used for setting the interpolation. For example, the user sets the steepness for the signal processing parameter interpolation between the interpolation points using the display of the interpolation parameter setting area 63. When the maximum value is set as the steepness of the interpolation parameter, the signal processing parameter is interpolated so as to change sharply, and when the minimum value is set, the signal processing parameter is interpolated by linear interpolation.

[0106] When the arrangement of the interpolation points, the setting of the signal processing parameters, and the setting of the interpolation parameters are completed, the save button 73 is pressed to save the parameter interpolation map associated with the settings.

[0107] When the Interpolation Pattern is Areal Interpolation FIG. 11 is a diagram showing another display example of the parameter interpolation map creation screen, which is a display when areal interpolation is selected as the interpolation pattern.

[0108] When areal interpolation is selected as the interpolation pattern, the display of the parameter interpolation map creation screen is basically the same as the display when point interpolation is selected. When areal interpolation is selected as the interpolation pattern, information such as icons used to select the shape of the interpolation region to be preset is displayed in the tools area 62.

[0109] The user can freely select the shape of the interpolation area. For example, if the user selects the "Circle, Ellipse" map creation tool and drags and drops it onto the coordinate area 71 in the editor area 65 as indicated by arrow A21, an image 81 of concentric circles representing the interpolation area is displayed within the coordinate area 71. The image 81 shown in FIG. 11 is composed of two elliptical images 81A and 81B, each centered on the interpolation point P61. The minor radius of image 81B is indicated by arrow #21, and the major radius is indicated by arrow #22. Note that image 81 may also be displayed by an operation other than drag-and-drop.

[0110] In image 81, the outer elliptical region of image 81A represents the region affected by the signal processing parameters corresponding to interpolation point P61. The inner elliptical region of image 81B represents the range that directly reflects the signal processing parameters corresponding to interpolation point P61. The region of interpolation effective radius #12 described with reference to Figure 7 corresponds to the region of image 81A, and the region of central parameter effective radius #11 corresponds to the region of image 81B.

[0111] The sizes of images 81A and 81B can be changed by dragging or the like. For example, if the user changes the radius of image 81B, the width of the range indicated by arrow #23, which is the difference between images 81A and 81B, changes. This allows interpolation parameters such as steepness to be changed in the range indicated by arrow #23, which is the range in which the contribution rate changes.

[0112] If the shape of the interpolation region is elliptical, the user can set the minor radius #21 and major radius #22 of the inner elliptical region using the display in the coordinate region 71. The minor radius #21 and major radius #22 on the GUI represent the central parameter effective radius #11. The user can also change the inclination of the ellipse by clicking the center arrow 82.

[0113] Applying a parameter interpolation map to an object Fig. 12 is a diagram showing an example of a screen used to select a created parameter interpolation map to apply to each object. As shown in Fig. 12, an object list 101 is displayed, which is a list showing the objects being handled in mixing and the waveform data of each object. In addition, a parameter interpolation map list 102 is displayed, which displays created parameter interpolation maps side by side.

[0114] 12, a parameter interpolation map whose application pattern is areal interpolation and a parameter interpolation map whose application pattern is point interpolation are displayed in the parameter interpolation map list 102. To the right of each created parameter interpolation map, the name of the parameter interpolation map is displayed, and below that, an apply button 111 and an edit button 112 are displayed. Below the parameter interpolation map list 102, a new button 113 is displayed.

[0115] In the parameter interpolation map list 102, an apply button 111 is operated when applying a parameter interpolation map to a specific object, and an edit button 112 is operated when editing a parameter interpolation map. Pressing the edit button 112 transitions to an edit screen (parameter interpolation map creation screen) such as those shown in Figures 9 to 11. A new button 113 is operated when creating a new parameter interpolation map.

[0116] As indicated by an arrow A31, a parameter interpolation map selected in the parameter interpolation map list 102 can be applied to a specific object in the object list 101 by dragging and dropping the selected parameter interpolation map onto the specific object. In other words, the parameter interpolation map is associated with the specific object. Note that the application process may be performed by an operation other than dragging and dropping.

[0117] Furthermore, when a specific object is selected in the object list 101, pressing the apply button 111 in the parameter interpolation map list 102 applies the parameter interpolation map to that specific object.

[0118] This allows interpolation using the applied parameter interpolation map according to the three-dimensional position of the object, and signal processing using the generated signal processing parameters can be applied to the audio signal of the object.

[0119] The name of the parameter interpolation map applied to an object is displayed in association with the object. In the example of Fig. 12, the name "Area_EQ" is displayed to the left of the object's waveform data. If no parameter interpolation map is applied to the object, "None" or similar is displayed.

[0120] The same parameter interpolation map may be applied to multiple different objects, or multiple different parameter interpolation maps may be applied to a single object, in which case multiple signal processing parameters are generated corresponding to the positions of the objects that have changed due to metadata editing.

[0121] 5.2 Creating a Parameter Interpolation Map (When the Interpolation Pattern is Linear Interpolation) Fig. 13 is a diagram showing an example of the display of a parameter interpolation map creation screen when an interpolation pattern of linear interpolation is selected. As described above, linear interpolation is an interpolation pattern that indicates the interpolation content of signal processing parameters at each interpolation point on the movement path of an object. The path created using the parameter interpolation map creation screen becomes the interpolation path of the signal processing parameters.

[0122] When the interpolation pattern of line interpolation is selected, the display of the parameter interpolation map creation screen is basically the same as the display when point interpolation or areal interpolation is selected.

[0123] When a movement path for the three-dimensional position of an object is selected, for example, from a list of movement templates (not shown), the movement path saved in the selected movement template is read and displayed as a dashed line image in the coordinate area 71 as shown in Fig. 13. A movement template is data in which the movement path for the three-dimensional position indicated by metadata is saved as a template. It is possible to select a movement template to apply to an object from the list of movement templates.

[0124] The user drags and drops an icon representing an interpolation point from the tools area 62 onto the movement path of the object's three-dimensional position, as indicated by arrow A41, to set interpolation points P71 to P73 on the movement path. Note that interpolation points may also be set by operations other than drag and drop. When the user clicks on interpolation point P72, a signal processing parameter setting area 64 is expanded, as indicated by arrow A42. The user sets signal processing parameters using the display in the signal processing parameter setting area 64, and any signal processing parameter of the signal processing method is set for interpolation point P72.

[0125] The user can also set the steepness (interpolation strength) for signal processing parameter interpolation between interpolation points using the display in the interpolation parameter setting area 63, or use a pull-down menu to select an arbitrary interpolation method from among the presets. The signal processing parameter corresponding to a three-dimensional position on the movement path is interpolated based on the signal processing parameters associated with two adjacent interpolation points and the selected interpolation method or interpolation strength.

[0126] When the user presses the save button 73, the parameter interpolation map created by the above-described method is set as the moving template.

[0127] When the user presses the New button 72, a new interpolation path of signal processing parameters can be created for an already saved moving template.

[0128] FIG. 14 is a diagram showing an example of a GUI when applying a parameter interpolation map to a movement template.

[0129] The template list tab 121 shown in FIG. 14 is a tab that displays a list of movement templates. The template list tab 121 in FIG. 14 displays a list of three movement templates, template1, template2, and template3, and further displays parameter interpolation map selection tabs 122-1 to 122-3 associated with each movement template. The parameter interpolation map selection tabs 122-1 to 122-3 are tabs for selecting a parameter interpolation map to edit from the parameter interpolation maps associated with each movement template. For example, the EQ1 parameter interpolation map is associated with the template1 movement template and the template2 movement template, and the EQ1, EQ2, and None parameter interpolation maps are associated with the template3 movement template. In this way, it is possible to apply multiple parameter interpolation maps to a single movement template.

[0130] Note that one change pattern representing the movement of an object is created and recorded as one movement template. For example, if a user sets the horizontal angle "azimuth," vertical angle "elevation," and radius "radius" of a specific object at each time using a GUI including a graph display shown on the left side of FIG. 14 to create a movement template, the newly created movement template is displayed in the template list tab 121.

[0131] The display of the template list tab 121 is used to edit the travel template. For example, when the user operates the insert button of the template list tab 121, an insert dialog 123 is displayed. When various items are entered in the insert dialog 123, the travel route is reflected in the set section and displayed as a graph, as indicated by the arrow 124.

[0132] The display of the template list tab 121 is also used to edit a parameter interpolation map. For example, when the user clicks on a parameter interpolation map selection tab 122-3 in the template list tab 121, the GUI display transitions to an editing screen for the parameter interpolation map associated with the moving template "template3," as indicated by the tip of arrow A51. The editing screen is the same as the parameter interpolation map creation screen described with reference to FIG. 13, and it is possible to set the signal processing method and the signal processing parameters to be applied in the same manner as described above.

[0133] Using the GUI described above, it is possible to apply a moving template in which a parameter interpolation map is set to any section of an audio signal of an object. When a moving template in which a parameter interpolation map is set is applied to any section, signal processing parameters are generated based on the parameter interpolation map set in the moving template, and signal processing is applied to the audio signal in that section. The section here refers to a period of time having a certain length, such as a four-bar unit. Even if the section of the audio signal to which the moving template is applied or its length is edited, signal processing is applied to the audio signal in the edited section.

[0134] 5.3 Modified Example of GUI Fig. 15 is a diagram showing another display example of the parameter interpolation map creation screen. In the coordinate area 71 of Fig. 15, a three-dimensional space used for arranging a movement path and interpolation points is displayed as a 3D image. In other words, three-dimensional drawing is used as a method for drawing the three-dimensional space.

[0135] The display of the parameter interpolation map creation screen shown in Fig. 15 is basically the same as the display of the parameter interpolation map creation screen in Fig. 9 etc., except that the method of drawing the three-dimensional space is different. In the parameter interpolation map creation screen in Fig. 9 etc., two-dimensional drawing is used as the method of drawing the three-dimensional space.

[0136] <<6. Operation of the Signal Processing Parameter Generator>> The parameter interpolation map creation process performed by the 3D audio content production plug-in 11 will be described with reference to the flowchart shown in FIG.

[0137] In step S11, the GUI control unit 22 displays a screen for creating a parameter interpolation map.

[0138] In step S12, the signal processing parameter acquisition unit 34 acquires an input of a signal processing method based on an input from the user on the parameter interpolation map screen.

[0139] In step S13, the interpolation pattern acquisition unit 33 acquires input of interpolation parameters based on input from the user on the parameter interpolation map creation screen.

[0140] In step S14, the interpolation pattern acquisition unit 33 acquires information about the selected interpolation points based on user input on the parameter interpolation map creation screen. The signal processing parameter acquisition unit 34 acquires signal processing parameters for the selected interpolation points. The parameter interpolation map creation unit 37 generates interpolation point information by setting the signal processing parameters for the positions of the selected interpolation points.

[0141] In step S15, the parameter interpolation map creating unit 37 creates and saves a parameter interpolation map based on the signal processing method, the interpolation parameters, the interpolation point information, and the interpolation pattern information.

[0142] The saved parameter interpolation map is applied to the object by the signal processing parameter calculation unit 36 ​​.

[0143] The process of calculating signal processing parameters using a parameter interpolation map applied to an object by the 3D audio content production plug-in 11 will be described with reference to the flowchart shown in FIG.

[0144] In step S21, the GUI control unit 22 receives an input of an object for which metadata is to be edited from the user via the GUI.

[0145] In step S22, the GUI control unit 22 receives the metadata edit from the user via the GUI, and then supplies the changed metadata to the metadata acquisition unit 35.

[0146] In step S23, the calculation unit 38 calculates signal processing parameters in accordance with the three-dimensional position indicated by the edited metadata, based on the parameter interpolation map applied to the object.

[0147] In step S24, the signal processing parameter generator 21 outputs the calculated signal processing parameters to the signal processing plug-in 13.

[0148] In step S25, the 3D audio playback control unit 23 acquires the processed audio signal from the signal processing plug-in 13.

[0149] In step S26, the 3D audio playback control unit 23 performs rendering using the processed audio signal, and outputs the playback sound.

[0150] The metadata is edited, for example, while 3D audio content is being played back. In response to the metadata edit, an audio signal that has been automatically subjected to signal processing is acquired and output as a playback sound, allowing the user to edit the metadata while listening to the playback sound of the audio signal after signal processing. Since there is no need to adjust signal processing parameters after editing the metadata, the user can efficiently edit the metadata, enabling the creation of high-quality 3D audio content.

[0151] <<7. Hardware Configuration>> FIG. 18 is a diagram showing an example configuration of an information processing device using the present technology.

[0152] 18 is constituted by, for example, a computer, etc. The information processing device 1 has an input unit 1001, a display unit 1002, a recording unit 1003, a communication unit 1004, an audio output unit 1005, and a control unit 1006.

[0153] The input unit 1001 is made up of input devices such as a mouse and a keyboard, and supplies signals according to user operations to the control unit 1006 .

[0154] The display unit 1002 comprises a display, and displays various images (screens) such as the display screen of the 3D audio production tool under the control of the control unit 1006. The recording unit 1003 records various data such as the audio signals of each object and each program in Figure 1, and supplies the recorded data to the control unit 1006 as necessary.

[0155] The communication unit 1004 communicates with external devices. For example, the communication unit 1004 receives audio signals of each object transmitted from an external device and supplies the signals to the control unit 1006, or transmits data supplied from the control unit 1006 to the external device.

[0156] The sound output unit 1005 is made up of a speaker or the like, and outputs sound based on the audio signal supplied from the control unit 1006 .

[0157] The control unit 1006 controls the overall operation of the information processing device 1. For example, the control unit 1006 executes a program recorded in the recording unit 1003, thereby causing the information processing device 1 to function as a 3D audio content production system.

[0158] <<8. Modifications>> Although the case where, in response to editing the three-dimensional position of an object, signal processing parameters are generated according to the three-dimensional position after the editing change has been described, in response to editing the gain of the object, signal processing parameters may also be generated according to the gain after the editing change. Furthermore, the created parameter interpolation map may be distributed to and shared with other users via a server or the like.

[0159] Modifications of Software Configuration Fig. 19 is a block diagram showing modifications of the software configuration. Fig. 19 shows the configuration of a 3D audio content production application, which is software that realizes a 3D audio content production system. The 3D audio content production application has, as its functional components, a music production unit 1101, a 3D audio content production unit 1102, and a signal processing unit 1103.

[0160] The music production unit 1101 has the same functions as the music production tool 12 in Fig. 1. The 3D audio content production unit 1102 has the same functions as the 3D audio content production plug-in 11 in Fig. 1. The signal processing unit 1103 has the same functions as the signal processing plug-in 13 in Fig. 1. In this way, it is possible to implement the functions of the 3D audio content production plug-in 11, music production tool 12, and signal processing plug-in 13 in a single program.

[0161] Others The above-described series of processes can be executed by hardware or software. When the series of processes are executed by software, the programs constituting the software are installed in a computer incorporated in dedicated hardware, or in the information processing device 1 shown in FIG. 18 .

[0162] The program to be installed is provided by being recorded on a removable medium such as an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.) or semiconductor memory. It may also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting. The program can be pre-installed in the ROM or recording unit 1003 that constitutes the control unit 1006.

[0163] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0164] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0165] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0166] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0167] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0168] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0169] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0170] <Examples of Combinations of Configurations> The present technology can also have the following configurations.

[0171] (1) An information processing device comprising: a signal processing parameter generation unit that generates, in response to a change in the three-dimensional position of an audio object, signal processing parameters that indicate content of signal processing for an audio signal of the audio object and that correspond to a changed three-dimensional position. (2) The information processing device described in (1), wherein the signal processing parameter generation unit generates the signal processing parameters that correspond to the changed three-dimensional position of the audio object based on parameter interpolation information created as information for interpolating and generating the signal processing parameters. (3) The information processing device described in (2), wherein the parameter interpolation information includes interpolation point information that associates positions of interpolation points serving as references for interpolation with the signal processing parameters, and interpolation parameters that indicate the degree of interpolation. (4) The information processing device described in (2) or (3), wherein the parameter interpolation information includes information on an interpolation pattern, and the interpolation pattern includes at least one of point interpolation, areal interpolation, and linear interpolation. (5) The information processing device described in (4), wherein the signal processing parameter generation unit interpolates the signal processing parameters that correspond to the changed three-dimensional position using the interpolation pattern selected by a user. (6) The information processing device according to any one of (3) to (5), wherein the signal processing parameter generation unit interpolates and generates the signal processing parameters corresponding to the three-dimensional position of the audio object after the change, based on a plurality of pieces of interpolation point information. (7) The information processing device according to any one of (2) to (6), wherein the signal processing parameter generation unit controls generation of the signal processing parameters using the parameter interpolation information, based on feature quantities representing features of the audio signal, including music information indicating information about the music of the audio object, object information indicating a type of the audio object, and waveform information of the audio object. (8) The information processing device according to any one of (2) to (7), wherein the signal processing parameter generation unit generates the signal processing parameters corresponding to the three-dimensional position after the change, based on the parameter interpolation information selected to be applied to the audio object.(9) The information processing device according to any of (2) to (8), wherein the signal processing parameter generation unit applies a plurality of pieces of parameter interpolation information to each of the audio objects to generate a plurality of signal processing parameters corresponding to the changed three-dimensional positions. (10) The information processing device according to any of (3) to (6), wherein the signal processing parameter generation unit accepts input of the signal processing parameter for an interpolation point selected from the plurality of interpolation points, and generates the parameter interpolation information by setting the input signal processing parameter for the position of the selected interpolation point. (11) The information processing device according to any of (3) to (10), further comprising a GUI control unit that controls display of a screen used for producing 3D audio content, including a creation screen used for creating the parameter interpolation information. (12) The information processing device according to (11), wherein the GUI control unit displays, on the creation screen, an image of a predetermined shape indicating a range in three-dimensional space to be interpolated for the signal processing parameters, and the signal processing parameter generation unit changes the interpolation parameter in response to an operation to deform the image. (13) The information processing device according to (11) or (12), wherein the GUI control unit displays an interpolation point image indicating the interpolation points on the creation screen, and the signal processing parameter generation unit sets the signal processing parameters for positions of the interpolation points indicated by the edited interpolation point image based on an operation to edit the positions of the interpolation point image. (14) The information processing device according to any of (11) to (13), wherein the three-dimensional position of the audio object changes in response to a user editing the three-dimensional position of the audio object using a screen displayed by the GUI control unit. (15) The information processing device according to any of (11) to (14), wherein the three-dimensional position of the audio object changes in accordance with a movement path set by a user using a screen displayed by the GUI control unit.(16) The information processing device according to (4) or (5), wherein the signal processing parameter generation unit generates the signal processing parameters corresponding to the three-dimensional position of the audio object whose three-dimensional position moves, based on the parameter interpolation information where the interpolation pattern is linear interpolation. (17) The information processing device according to (4), (5), or (16), wherein the signal processing parameter generation unit sets the parameter interpolation information where the interpolation pattern is linear interpolation for a movement template representing a movement path of the audio object, and when the movement template is applied to the audio object, generates the signal processing parameters based on the parameter interpolation information set for the movement template. (18) The information processing device according to any of (1) to (17), further comprising: a signal processing unit that performs signal processing on the audio signal of the audio object being played back, based on the signal processing parameters corresponding to the three-dimensional position after the change; and a playback control unit that outputs reproduced sound of the audio signal after the signal processing. (19) An information processing method, wherein the information processing device generates signal processing parameters corresponding to the three-dimensional position after the change, indicating content of signal processing for the audio signal of the audio object, in accordance with a change in the three-dimensional position of the audio object. (20) A program for causing a computer to execute a process of generating, in response to a change in the three-dimensional position of an audio object, signal processing parameters that indicate the content of signal processing for the audio signal of the audio object and correspond to the changed three-dimensional position.

[0172] REFERENCE SIGNS LIST 11 3D audio content production plug-in, 21 signal processing parameter generation unit, 31 audio signal acquisition unit, 32 feature amount calculation unit, 33 interpolation pattern acquisition unit, 34 signal processing parameter acquisition unit, 35 metadata acquisition unit, 36 signal processing parameter calculation unit

Claims

1. An information processing device having a signal processing parameter generation unit that generates signal processing parameters corresponding to a changed three-dimensional position of an audio object, indicating the content of signal processing for the audio signal of the audio object, in response to a change in the three-dimensional position of the audio object.

2. The information processing device according to claim 1, wherein the signal processing parameter generation unit generates the signal processing parameters corresponding to the three-dimensional position of the audio object after the change based on parameter interpolation information created as information for interpolating and generating the signal processing parameters.

3. The information processing device according to claim 2, wherein the parameter interpolation information includes interpolation point information associating the positions of interpolation points serving as the basis for interpolation with the signal processing parameters, and interpolation parameters representing the degree of interpolation.

4. The information processing device according to claim 3, wherein the parameter interpolation information includes information on an interpolation pattern, and the interpolation pattern includes at least one of point interpolation, areal interpolation, and line interpolation.

5. The information processing device according to claim 4, wherein the signal processing parameter generation unit interpolates the signal processing parameters corresponding to the changed three-dimensional position using the interpolation pattern selected by a user.

6. The information processing device according to claim 3, wherein the signal processing parameter generation unit interpolates and generates the signal processing parameters corresponding to the three-dimensional position of the audio object after the change based on the plurality of pieces of interpolation point information.

7. The information processing device according to claim 2, wherein the signal processing parameter generation unit controls the generation of the signal processing parameters using the parameter interpolation information based on features representing the characteristics of the audio signal, including music information indicating information about the music of the audio object, object information indicating the type of the audio object, and waveform information of the audio object.

8. The information processing device according to claim 2, wherein the signal processing parameter generation unit generates the signal processing parameters corresponding to the changed three-dimensional position based on the parameter interpolation information selected to be applied to the audio object.

9. The information processing device according to claim 2, wherein the signal processing parameter generation unit applies a plurality of pieces of parameter interpolation information to each of the audio objects, and generates a plurality of the signal processing parameters corresponding to the changed three-dimensional positions.

10. The information processing device according to claim 3, wherein the signal processing parameter generation unit receives input of the signal processing parameters for the interpolation points selected from the plurality of interpolation points, and generates the parameter interpolation information by setting the input signal processing parameters for the positions of the selected interpolation points.

11. The information processing device according to claim 3, further comprising a GUI control unit that controls the display of a screen used in producing 3D audio content, including a creation screen used to create the parameter interpolation information.

12. The information processing device according to claim 11, wherein the GUI control unit displays an image of a predetermined shape indicating a range in three-dimensional space to be interpolated for the signal processing parameters on the creation screen, and the signal processing parameter generation unit changes the interpolation parameters in response to an operation to transform the image.

13. The information processing device according to claim 11, wherein the GUI control unit displays an interpolation point image indicating the interpolation points on the creation screen, and the signal processing parameter generation unit sets the signal processing parameters for the positions of the interpolation points indicated by the edited interpolation point image based on an operation to edit the positions of the interpolation point image.

14. The information processing device according to claim 11, wherein the three-dimensional position of the audio object changes in response to a user editing the three-dimensional position of the audio object using a screen displayed by the GUI control unit.

15. The information processing device according to claim 11, wherein the three-dimensional position of the audio object changes according to a movement path set by the user using a screen displayed by the GUI control unit.

16. The information processing device according to claim 4, wherein the signal processing parameter generation unit generates the signal processing parameters corresponding to the three-dimensional position of the audio object whose three-dimensional position moves based on the parameter interpolation information in which the interpolation pattern is linear interpolation.

17. The information processing device of claim 4, wherein the signal processing parameter generation unit sets the parameter interpolation information, the interpolation pattern of which is linear interpolation, to a movement template representing a movement path of the audio object, and when the movement template is applied to the audio object, generates the signal processing parameter based on the parameter interpolation information set in the movement template.

18. The information processing device according to claim 1, further comprising: a signal processing unit that performs signal processing on the audio signal of the audio object being played back based on the signal processing parameters corresponding to the changed three-dimensional position; and a playback control unit that outputs the playback sound of the audio signal after signal processing.

19. An information processing method in which an information processing device generates, in response to a change in the three-dimensional position of an audio object, signal processing parameters corresponding to the changed three-dimensional position, which indicate the content of signal processing for the audio signal of the audio object.

20. A program for causing a computer to execute a process of generating signal processing parameters corresponding to a changed three-dimensional position of an audio object, which indicate the content of signal processing for the audio signal of the audio object, in accordance with a change in the three-dimensional position of the audio object.

Images