Information processing device and method, program, and information processing system

Abstract

To provide an information processing apparatus and method, a program, and an information processing system, that enable more efficient edit.SOLUTION: An information processing apparatus 11 includes: a control unit 23 configured to execute mute setting or solo setting for one or a plurality of sound sources in accordance with user's specification; and a display control unit 42 configured to set a display state of the sound sources to a first display state or a second display state. When the mute setting is executed, the display control unit 42 sets the display state of the sound source muted by the mute setting to the second display state. when the solo setting is executed, the display control unit sets the display state of the sound source except for the sound source soloed by the solo setting to the second display state.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present technology relates to an information processing device and method, a program, and an information processing system, and more particularly to an information processing device and method, a program, and an information processing system that enable more efficient editing. [Background technology]

[0002] In recent years, object-based audio technology has been attracting attention.

[0003] In object-based audio, object audio data is composed of a waveform signal for an audio object and meta-information indicating localization information of the audio object expressed by a relative position from a predetermined reference listening position.

[0004] Then, the waveform signal of the audio object is rendered into a signal with a desired number of channels by, for example, VBAP (Vector Based Amplitude Panning) based on the meta information and played back (see, for example, Non-Patent Document 1 and Non-Patent Document 2).

[0005] In object-based audio, audio objects can be positioned in various directions in three-dimensional space when creating audio content.

[0006] For example, Dolby Atoms Panner plus-in for Pro Tools (see, for example, Non-Patent Document 3) makes it possible to specify the position of an audio object on a 3D graphics user interface. With this technology, by specifying a position on an image of a virtual space displayed on the user interface as the position of the audio object, the sound image of the audio object can be localized in any direction in three-dimensional space.

[0007] On the other hand, the positioning of the sound image in conventional two-channel stereo is adjusted using a technique called panning. For example, the left-right position of the sound image can be determined by changing the proportion of the left and right channels for a specific audio track via a user interface. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] ISO / IEC 23008-3 Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 3: 3D audio [Non-patent document 2] Ville Pulkki, “Virtual Sound Source Positioning Using Vector Base Amplitude Panning”, Journal of AES, vol.45, no.6, pp.456-466, 1997 [Non-patent document 3] Dolby Laboratories, Inc., "Authoring for Dolby Atmos(R) Cinema Sound Manual," [online], [Retrieved August 1, 2018], Internet: < https: / / www.dolby.com / us / en / technologies / dolby-atmos / authoring-for-dolby-atmos-cinema-sound-manual.pdf > Summary of the Invention [Problem to be solved by the invention]

[0009] In object-based audio, editing is possible for each audio object, such as changing the spatial position of the audio object, that is, the sound image localization position, or adjusting the gain of the waveform signal of the audio object.

[0010] However, in the production of actual object-based audio content, a large number of audio objects are handled, and editing such as adjusting the positions and gains of these audio objects takes time.

[0011] For example, it is troublesome to specify a position in space for each audio object and determine the localization position of the sound image of each audio object.

[0012] For these reasons, there is a demand for a method that allows efficient editing of audio objects when producing audio content.

[0013] The present technology has been made in view of such circumstances, and makes it possible to perform editing more efficiently. [Means for solving the problem]

[0014] An information processing device according to a first aspect of the present technology includes a control unit that performs a mute setting or a solo setting for one or more sound sources in accordance with a user's specification, and a display control unit that sets the display state of the sound sources to either a first display state or a second display state, and when the mute setting is performed, the display control unit sets the display state of the sound sources that have been muted by the mute setting to the second display state, and when the solo setting is performed, sets the display state of the sound sources other than the sound source that has been soloed by the solo setting to the second display state.

[0015] An information processing method or program according to a first aspect of the present technology includes a step of performing a mute setting or a solo setting for one or more sound sources in accordance with a user's specification, and setting the display state of the sound sources to a first display state or a second display state, and when the mute setting is performed, the display state of the sound sources muted by the mute setting is set to the second display state, and when the solo setting is performed, the display state of the sound sources other than the sound source soloed by the solo setting is set to the second display state.

[0016] In a first aspect of the present technology, a mute setting or a solo setting is performed on one or more sound sources in response to a user's designation, and the display state of the sound sources is set to a first display state or a second display state. Also, when the mute setting is performed, the display state of the sound sources muted by the mute setting is set to the second display state, and when the solo setting is performed, the display state of the sound sources other than the sound source soloed by the solo setting is set to the second display state.

[0017] An information processing device according to a second aspect of the present technology controls reception of operation information regarding a user's operation on one or more sound sources, generates a setting control signal for controlling a mute setting or a solo setting for the sound sources in accordance with the operation information, generates a display control signal for changing the display state of the sound sources to a first display state or a second display state in accordance with the mute setting or the solo setting, and includes a control unit that controls transmission of the setting control signal and the display control signal, wherein when the mute setting is performed, the display state of the sound sources that have been muted by the mute setting is changed to the second display state, and when the solo setting is performed, the display state of the sound sources other than the sound source that has been soloed by the solo setting is changed to the second display state.

[0018] An information processing method according to a second aspect of the present technology includes the steps of receiving operation information regarding a user's operation on one or more sound sources, generating a setting control signal for controlling a mute setting or a solo setting for the sound sources in accordance with the operation information, generating a display control signal for changing the display state of the sound sources to a first display state or a second display state in accordance with the mute setting or the solo setting, and transmitting the setting control signal and the display control signal, wherein when the mute setting is performed, the display state of the sound sources that have been muted by the mute setting is set to the second display state, and when the solo setting is performed, the display state of the sound sources other than the sound source that has been soloed by the solo setting is set to the second display state.

[0019] In a second aspect of the present technology, operation information regarding an operation by a user on one or more sound sources is received, a setting control signal for controlling a mute setting or a solo setting for the sound sources is generated according to the operation information, a display control signal for changing a display state of the sound sources to a first display state or a second display state according to the mute setting or the solo setting is generated, and the setting control signal and the display control signal are transmitted. Also, when the mute setting is performed, the display state of the sound sources that have been muted by the mute setting is changed to the second display state, and when the solo setting is performed, the display state of the sound sources other than the sound source that has been soloed by the solo setting is changed to the second display state.

[0020] An information processing device according to a third aspect of the present technology includes a control unit that controls the transmission of operation information regarding a user's operation on one or more sound sources, controls the reception of setting control signals and display control signals transmitted in response to the transmission of the operation information, and performs a mute setting or a solo setting on the sound sources in response to the setting control signals, and a display control unit that sets the display state of the sound sources to a first display state or a second display state in response to the display control signals, wherein when the mute setting is performed, the display control unit sets the display state of the sound sources that have been muted by the mute setting to the second display state, and when the solo setting is performed, sets the display state of the sound sources other than the sound source that has been soloed by the solo setting to the second display state.

[0021] An information processing method according to a third aspect of the present technology includes the steps of transmitting operation information regarding a user's operation on one or more sound sources, receiving a setting control signal and a display control signal transmitted in response to the transmission of the operation information, performing a mute setting or a solo setting on the sound sources in response to the setting control signal, and setting the display state of the sound sources to a first display state or a second display state in response to the display control signal, wherein when the mute setting is performed, the display state of the sound sources muted by the mute setting is set to the second display state, and when the solo setting is performed, the display state of the sound sources other than the sound source soloed by the solo setting is set to the second display state.

[0022] In a third aspect of the present technology, operation information regarding an operation by a user on one or more sound sources is transmitted, a setting control signal and a display control signal transmitted in response to the transmission of the operation information are received, a mute setting or a solo setting is performed on the sound sources in response to the setting control signal, and the display states of the sound sources are set to a first display state or a second display state in response to the display control signal. Also, when the mute setting is performed, the display states of the sound sources muted by the mute setting are set to the second display state, and when the solo setting is performed, the display states of the sound sources other than the sound source soloed by the solo setting are set to the second display state.

[0023] An information processing system according to a fourth aspect of the present technology is an information processing system including the information processing device according to the second aspect of the present technology and the information processing device according to the third aspect. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 illustrates an example of the configuration of an information processing device. [Figure 2] FIG. 10 is a diagram illustrating an example of an editing screen. [Figure 3] FIG. 10 is a diagram showing an example of a POV image. [Figure 4] 10 is a flowchart illustrating a grouping process. [Figure 5] FIG. 10 is a diagram illustrating movement of grouped objects. [Figure 6] FIG. 10 is a diagram illustrating movement of grouped objects. [Figure 7] FIG. 10 is a diagram illustrating movement of grouped objects. [Figure 8] 10 is a flowchart illustrating an object movement process. [Figure 9] FIG. 1 is a diagram illustrating an L / R pair. [Figure 10] FIG. 1 is a diagram illustrating an L / R pair. [Figure 11] FIG. 1 is a diagram illustrating an L / R pair. [Figure 12] FIG. 1 is a diagram illustrating an L / R pair. [Figure 13] 10 is a flowchart illustrating a grouping process. [Figure 14] FIG. 10 is a diagram illustrating a change in object position information in units of an offset amount. [Figure 15] FIG. 10 is a diagram illustrating a change in object position information in units of an offset amount. [Figure 16] FIG. 10 is a diagram illustrating a change in object position information in units of an offset amount. [Figure 17] FIG. 10 is a diagram illustrating a change in object position information in units of an offset amount. [Figure 18] 10 is a flowchart illustrating an offset movement process. [Figure 19] 10A and 10B are diagrams illustrating an interpolation process for object position information. [Figure 20] 10A and 10B are diagrams illustrating an interpolation process for object position information. [Figure 21] 10A and 10B are diagrams illustrating an interpolation process for object position information. [Figure 22] 10 is a flowchart illustrating an interpolation method selection process. [Figure 23] FIG. 10 is a diagram illustrating an example of an editing screen. [Figure 24] FIG. 10 is a diagram showing an example of a POV image. [Figure 25] FIG. 10 is a diagram illustrating a mute setting and a solo setting. [Figure 26] FIG. 10 is a diagram illustrating a mute setting and a solo setting. [Figure 27] FIG. 10 is a diagram illustrating a mute setting and a solo setting. [Figure 28] 10 is a flowchart illustrating a setting process. [Figure 29] FIG. 10 is a diagram illustrating importing of an audio file. [Figure 30] FIG. 10 is a diagram showing an example of a track type selection screen. [Figure 31] FIG. 10 is a diagram illustrating an example of an editing screen. [Figure 32] FIG. 10 is a diagram showing an example of a track type selection screen. [Figure 33] FIG. 10 is a diagram illustrating an example of an editing screen. [Figure 34] FIG. 10 is a diagram showing an example of a POV image. [Figure 35] 10 is a flowchart illustrating an import process. [Figure 36] FIG. 1 illustrates an example of the configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0025] Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.

[0026] First Embodiment <Configuration example of information processing device> This technology enables more efficient editing by grouping multiple objects and changing their positions while maintaining their relative positions in three-dimensional space.

[0027] The object referred to here may be any object that can be assigned position information indicating its position in space, such as an audio object that is a sound source, or an image object that is a subject on an image.

[0028] In the following, a specific example will be described in which the object is an audio object, and the audio object will also be simply referred to as an object.

[0029] FIG. 1 is a diagram showing an example of the configuration of an embodiment of an information processing device to which the present technology is applied.

[0030] The information processing device 11 shown in FIG. 1 has an input unit 21, a recording unit 22, a control unit 23, a display unit 24, a communication unit 25, and a speaker unit 26.

[0031] The input unit 21 is made up of switches, buttons, a mouse, a keyboard, a touch panel superimposed on the display unit 24, and the like, and supplies a signal to the control unit 23 in response to an input operation by a user who is the creator of the content.

[0032] The recording unit 22 is formed of a non-volatile memory such as a hard disk, and records various data such as audio content data supplied from the control unit 23, and supplies the recorded data to the control unit 23. The recording unit 22 may be a removable recording medium that is detachable from the information processing device 11.

[0033] The control unit 23 is realized by, for example, a processor or the like, and controls the overall operation of the information processing device 11. The control unit 23 has a position determination unit 41 and a display control unit .

[0034] The position determination unit 41 determines the position of each object in space, i.e., the sound image localization position of the sound of each object, based on the signal supplied from the input unit 21. The display control unit 42 controls the display unit 24 to control the display of images, etc. on the display unit 24.

[0035] The display unit 24 is made up of, for example, a liquid crystal display panel, and displays various images and the like under the control of the display control unit 42 .

[0036] The communication unit 25 is formed of, for example, a communication interface, and communicates with external devices via a wired or wireless communication network such as the Internet. For example, the communication unit 25 receives data transmitted from an external device and supplies the data to the control unit 23, or transmits data supplied from the control unit 23 to an external device.

[0037] The speaker unit 26 is made up of speakers for each channel of a speaker system having a predetermined channel configuration, for example, and reproduces (outputs) the sound of the content based on the audio signal supplied from the control unit 23.

[0038] About grouping objects The information processing device 11 can function as an editing device that realizes editing of object-based audio content made up of object data of at least a plurality of objects.

[0039] The audio content data may include data other than object data, specifically, channel audio data consisting of audio signals for each channel.

[0040] Furthermore, although audio content may of course be standalone content such as music without accompanying video, etc., it is assumed here that audio content also has corresponding video content. That is, the audio signal of audio content is video data consisting of still images or moving images (video), that is, an audio signal accompanying the video data of video content. For example, if the video content is live video, the audio content corresponding to that video content would be the sound of the live video.

[0041] Each object data included in the audio content data is made up of an audio signal, which is a waveform signal of the sound of the object, and meta information of the object.

[0042] The meta information also includes object position information indicating the position of the object in a playback space, which may be a three-dimensional space, gain information indicating the gain of the object's audio signal, and priority information indicating the priority of the object.

[0043] Furthermore, in this embodiment, the object position information indicating the position of the object is represented by coordinates in a polar coordinate system based on the position of the listener listening to the sound of the audio content within the playback space (hereinafter also referred to as the listening position).

[0044] That is, the object position information consists of a horizontal angle, a vertical angle, and a radius. Note that, although an example in which the object position information is expressed in polar coordinates will be described here, the object position information is not limited to this and may be any type of information, such as absolute position information expressed in absolute coordinates.

[0045] The horizontal angle is the horizontal angle (Azimuth) that indicates the horizontal (left-right) position of the object as seen from the listening position, and the vertical angle is the vertical angle (Elevation) that indicates the vertical (up-down) position of the object as seen from the listening position. The radius is the distance (Radius) from the listening position to the object. Hereinafter, coordinates as object position information will be expressed as (Azimuth, Elevation, Radius).

[0046] For example, when audio content is played back, rendering is performed based on the audio signal of each object using VBAP or the like so that the sound image of the object is localized at the position indicated by the object position information.

[0047] Furthermore, when editing audio content, one piece of object data, i.e., the audio signal of one object, is generally treated as one audio track. In contrast, for channel audio data, the multiple audio signals that make up that channel audio data are treated as one audio track. Note that hereinafter, audio tracks will also be simply referred to as tracks.

[0048] Typically, audio content data includes object data for a large number of objects, such as tens or hundreds.

[0049] Therefore, the information processing device 11 is configured to be able to group a plurality of objects so that editing can be performed more efficiently when editing audio content. That is, the information processing device 11 is configured to be able to group a plurality of selected objects from a plurality of objects existing in the playback space so that the selected plurality of objects can be treated as one group.

[0050] In the information processing device 11, for a plurality of grouped objects, that is, a plurality of objects belonging to the same group, the object position information is changed while maintaining the relative positional relationship of those objects in the reproduction space.

[0051] In this way, when editing audio content, the information processing device 11 can edit object position information in units of groups, that is, specify (change) the sound image localization positions of objects. In this case, the number of operations for specifying object position information can be significantly reduced compared to when editing object position information for each object. Therefore, the information processing device 11 can edit audio content more efficiently and easily.

[0052] Although an example in which object position information is edited in units of groups will be described here, priority information and gain information may also be edited in units of groups.

[0053] In such a case, for example, when the priority information of a specific object is specified, the priority information of all other objects belonging to the same group as the specific object is also changed to the same value as the priority information of the specific object. Note that the priority information of these objects may be changed while maintaining the relative priorities of the objects belonging to the same group.

[0054] Furthermore, when the gain information of a specific object is specified, the gain information of all other objects belonging to the same group as the specific object is also changed, while maintaining the relative magnitude relationships of the gain information of all the objects belonging to the group.

[0055] The grouping of objects when editing audio content and the specification (change) of object position information of grouped objects will be described in more detail below.

[0056] For example, when editing audio content, the display control unit 42 causes the display unit 24 to display an editing screen, which displays the time waveform of the audio signal of each track, as the display screen of the content production tool. The display control unit 42 also causes the display unit 24 to display a POV image, which is a Point of View Shot from the listening position or a position near the listening position, as the display screen of the content production tool. The editing screen and the POV image may be displayed in different windows or in the same window.

[0057] The editing screen is a screen (image) for specifying or changing object position information, gain information, and priority information for each track of audio content, for example. The POV image is a 3D graphic image that simulates the playback space, that is, an image of the playback space as seen from the listener's listening position or a position near the listener.

[0058] For simplicity of explanation, it is assumed that audio content is edited, which is made up of object data of objects to which position information in the playback space, that is, object position information, has been assigned in advance.

[0059] As an example, the display control unit 42 causes the display unit 24 to display an edit screen ED11 shown in FIG.

[0060] In this example, the editing screen ED11 has, for each track, a track area in which information about the track is displayed, and a timeline area in which the time waveform of the audio signal for that track, object position information, gain information, and priority information are displayed.

[0061] Specifically, for example, in the diagram of the editing screen ED11, the area TR11 on the left side is the track area for one track, and the area TM11 located adjacent to the right side of the diagram of the area TR11 is the timeline area for the track corresponding to area TR11.

[0062] Each track area is also provided with a group display area, an object name display area, and a coordinate system selection area.

[0063] The group display area is an area where information indicating the track, that is, the group to which the object corresponding to the track belongs, is displayed.

[0064] For example, in area TR11, area GP11 on the left side of area TR11 is a group display area, and the letter (number) "1" in area GP11 indicates information indicating the group to which the object (track) belongs, i.e., the group ID. By looking at the group ID displayed in the group display area, the user can instantly understand the group to which the object belongs.

[0065] The information indicating the group, that is, the information for identifying the group, is not limited to the group ID represented by a number, but may be any other information such as text or color information.

[0066] Furthermore, on the editing screen ED11, the track areas of objects (tracks) belonging to the same group are displayed in the same color. For example, a color representing each group is predetermined, and when the user operates the input unit 21 to select (specify) a group of objects, the display control unit 42 displays the track area of ​​the object in the color representing the group selected for that object.

[0067] In the example of Fig. 2, the four track areas at the top of the editing screen ED11 are displayed in the same color, allowing the user to instantly understand that the four objects (tracks) corresponding to those track areas belong to the same group. Note that, hereinafter, a color set for a group consisting of multiple objects, that is, a color representing a group, will also be referred to as a group color.

[0068] The object name display area is an area where an object name that indicates the name of an object that is given to a track, that is, an object corresponding to a track, is displayed.

[0069] For example, in area TR11, area OB11 is an object name display area, and in this example, the characters "Kick" displayed in area OB11 are the object name. This object name "Kick" represents the bass drum that makes up the drum (drum set), that is, the so-called kick. Therefore, by seeing the object name "Kick," the user can instantly understand that the object is a kick.

[0070] In the following, when it is desired to clarify the object name of an object, the object name will be written after the word "object", for example, an object with the object name "Kick" will be written as object "Kick".

[0071] On the editing screen ED11, the group IDs of the objects whose object names "OH_L," "OH_R," and "Snare" are displayed in the object name display area are "1," the same as the group ID of the object "Kick."

[0072] The object "OH_L" is a sound object picked up by an overhead microphone placed on the left side above the drummer's head, the object "OH_R" is a sound object picked up by an overhead microphone placed on the right side above the drummer's head, and the object "Snare" is a snare drum that makes up the drums.

[0073] As such, the objects with the object names "Kick," "OH_L," "OH_R," and "Snare" constitute a drum, and therefore these objects are grouped together in the same group with a group ID of "1."

[0074] Normally, the relative positions of objects such as kick and snare drums that make up a drum set do not change. Therefore, if these objects are grouped together and the object position information is changed while maintaining the relative positions, simply changing the object position information of one object will allow the object position information of the other objects to be changed appropriately.

[0075] The coordinate system selection area is an area for selecting the coordinate system for object position information during editing. For example, the coordinate system selection area allows you to select any one of multiple coordinate systems using a drop-down list.

[0076] In the area TR11, the area PS11 is the coordinate system selection area, and in this example, the word "Polar" is displayed in the area PS11 to indicate the polar coordinate system, which is the selected coordinate system.

[0077] Note that an example in which the polar coordinate system is selected will be described here. However, for example, on the editing screen ED11, the object position information may be edited using the coordinates of the coordinate system selected in the coordinate system selection area, and then the object position information may be converted into coordinates expressed in the polar coordinate system and used as the object position information of the meta information, or the coordinates of the coordinate system selected in the coordinate system selection area may be used as the object position information of the meta information as is.

[0078] Furthermore, when specifying (selecting) a group of objects corresponding to a track, for example, the user operates the input unit 21 to display a group selection window GW11.

[0079] Specifically, for example, when specifying a group, the user selects the target track by specifying the group display area of ​​the desired track using a pointer or cursor, and a menu for grouping is displayed.

[0080] In the example of FIG. 2, a menu for grouping is displayed, which includes a menu item ME11 displaying the characters "Group" and a menu item ME12 displaying the characters "L / R pair."

[0081] Menu item ME11 is selected when displaying a group selection window GW11 for specifying the group ID of an object corresponding to a track selected by a pointer, cursor, etc. On the other hand, menu item ME12 is selected (operated) when forming an L / R pair, which will be described later, of objects corresponding to a track selected by a pointer, cursor, etc.

[0082] Here, since the menu item ME11 has been selected, a group selection window GW11 is displayed superimposed on the edit screen ED11.

[0083] The group selection window GW11 displays a plurality of group icons representing selectable groups, and a cursor CS11 for selecting one of the group icons.

[0084] In this example, the group icons are rectangular, and the group ID is displayed within the group icon. For example, group icon GA11 represents a group with a group ID of "1," and the group ID "1" is displayed within group icon GA11. Each group icon is also displayed in the group color.

[0085] The user operates the input unit 21 to move the cursor CS11 and select a desired group icon, thereby selecting a group to which the object corresponding to the track belongs.

[0086] Also, the display unit 24 displays, for example, the image shown in FIG. 3 as a POV image corresponding to the editing screen ED11.

[0087] In the example shown in Fig. 3, a POV image P11 is displayed in a specified window. The POV image P11 shows the walls of a room, which is the playback space as seen from slightly behind the listening position O, and a screen SC11 on which an image of video content is superimposed is placed in front of the listener in the room. The POV image P11 reproduces the playback space as seen from near the actual listening position O almost exactly as it is.

[0088] On the screen SC11, drums, electric guitar, and acoustic guitar, as well as the players of these instruments, are displayed as subjects within the video content.

[0089] In particular, in this example, the performers of each instrument are displayed on screen SC11: drummer PL11, electric guitar player PL12, first acoustic guitar player PL13, and second acoustic guitar player PL14.

[0090] Also displayed on the POV image P11 are marks representing objects, more specifically, object balls BL11 to BL19, which are marks representing the positions of the objects. In this example, these object balls BL11 to BL19 are located on the screen SC11.

[0091] Each object ball also displays a letter indicating the object name of the object corresponding to that object ball.

[0092] Specifically, for example, the object name "Kick" is displayed on the object ball BL11, and this object ball BL11 represents the object corresponding to the track in the area TR11 in Fig. 2, more specifically, the position of that object in the playback space. The object ball BL11 is displayed at the position indicated by the object position information of the object "Kick" on the POV image P11.

[0093] Furthermore, the object name "OH_L" is displayed on the object ball BL12, and it can be seen that this object ball BL12 represents the object "OH_L."

[0094] Similarly, the object name "OH_R" is displayed on the object ball BL13, and the object name "Snare" is displayed on the object ball BL14.

[0095] In the POV image P11, the object balls of objects belonging to the same group are displayed in the same color. In other words, the object balls of grouped objects are displayed in the group color of the group to which those objects belong.

[0096] Here, object balls BL11 to BL14 of the objects with object names "Kick," "OH_L," "OH_R," and "Snare," which belong to the group indicated by the group ID "1" on the editing screen ED11 shown in Fig. 2, are displayed in the same color. In particular, for these objects, object balls BL11 to BL14 and the track area on the editing screen ED11 are displayed in the group color of the group indicated by the group ID "1."

[0097] Therefore, the user can easily understand which objects belong to the same group on the editing screen ED11 and the POV image P11. Furthermore, the user can easily understand which object ball corresponds to which track between the editing screen ED11 and the POV image P11.

[0098] Furthermore, in FIG. 3, object balls BL15 to BL19, which are not particularly grouped, that is, objects that do not belong to a group, are displayed in a predetermined color, that is, a color different from any group color.

[0099] The user can specify the localization position of the sound image by operating the input unit 21 while viewing the editing screen ED11 or the POV image P11, inputting the coordinates of the object position information for each track, or directly manipulating the position of the object ball to move it, thereby enabling the user to easily determine (specify) the appropriate localization position of the sound image.

[0100] 3, the user can change the line of sight direction in the POV image P11 to any direction by operating the input unit 21. In this case, the display control unit 42 displays an image of the playback space in the changed line of sight direction as the POV image P11.

[0101] At this time, if the viewpoint position of the POV image P11 is set to a position near the listening position O, the listening position O is always displayed in the front area of ​​the POV image P11. This allows a user looking at the POV image P11 to easily understand the position from which the displayed POV image P11 is viewed, even if the viewpoint position is different from the listening position O.

[0102] 3, speakers are displayed on the POV image P11 on the front left and front right sides of the listening position O. These speakers are the speakers for each channel that make up the speaker system that the user envisions being used when playing audio content.

[0103] In this embodiment, an example has been described in which the group selection window GW11 is displayed on the edit screen ED11 and a group ID is specified for each track to group objects.

[0104] However, the user may operate the input unit 21 to display a group selection window with one or more object balls selected on the POV image P11, and group the objects by specifying a group ID.

[0105] Furthermore, multiple groups may be grouped together to form a large group consisting of these multiple groups. In such a case, for example, by changing the object position information of objects in units of large groups, it is possible to simultaneously change the object position information of each of the multiple objects belonging to the large group while maintaining the relative positional relationships between them.

[0106] Such large groups are particularly useful when you want to temporarily change the object position information of each object while maintaining the relative positional relationships of the objects in multiple groups. In this case, when the large group is no longer needed, you can ungroup the large group and perform subsequent editing on each individual group.

[0107] <Explanation of Grouping Process> Next, the operation performed by information processing device 11 when grouping the objects described above will be described. That is, the grouping process by information processing device 11 will be described below with reference to the flowchart in Fig. 4. It is assumed that the editing screen is already displayed on display unit 24 when the grouping process is started.

[0108] In step S11, the control unit 23 accepts designation of objects and groups to be grouped through an input operation on the input unit 21.

[0109] For example, the user specifies the objects to be grouped by operating the input unit 21 to specify (select) the group display area of ​​the track corresponding to the desired objects to be grouped on the editing screen ED11 shown in Fig. 2. The control unit 23 identifies the specified objects based on the signal supplied from the input unit 21.

[0110] Furthermore, the user designates a group by moving the cursor CS11 to designate a group icon in a group selection window GW11 that is displayed by designating a group display area.

[0111] At this time, the display control unit 42 of the control unit 23 displays the group selection window GW11 on the display unit 24 based on the signal supplied from the input unit 21, and the control unit 23 identifies the specified group based on the signal supplied from the input unit 21.

[0112] In step S12, the control unit 23 groups the objects specified in step S11 so that the objects belong to the groups specified in step S11, and generates group information.

[0113] For example, group information is information indicating which object belongs to which group, and is made up of a group ID and information indicating the object belonging to the group indicated by the group ID. Note that the information indicating the object may be an object ID or the like that identifies the object itself, or information indicating a track such as a track ID that indirectly identifies the object.

[0114] The control unit 23 supplies the generated group information to the recording unit 22 as necessary to record it. Note that if group information has already been recorded in the recording unit 22, the control unit 23 updates the group information of the specified group so that information indicating the newly specified object is added to the group information of the specified group.

[0115] By generating group information in this way, the objects are grouped.

[0116] In step S13, the display control unit 42 updates the display of the editing screen and POV image already displayed on the display unit 24 based on the newly generated or updated group information.

[0117] For example, the display control unit 42 controls the display unit 24 to display, among the track areas on the edit screen ED11 as shown in FIG. 2, the track areas of objects belonging to the same group in the group color of that group.

[0118] Similarly, the display control unit 42 controls the display unit 24 to display, among the object balls in the POV image P11, the object balls of objects belonging to the same group in the group color of that group, as shown in Fig. 3. This makes it easy to distinguish between objects belonging to the same group, i.e., objects that are highly related to each other.

[0119] The objects are grouped in this manner, and the display of the editing screen and POV image is updated accordingly, at which point the grouping process is completed.

[0120] As described above, the information processing device 11 groups objects so that an object designated by an input operation on the input unit 21 belongs to a designated group.

[0121] By grouping in this way, it becomes possible to edit object position information and the like in groups, which allows for more efficient editing.

[0122] Editing object location information Once the objects have been grouped, the information processing device 11 can edit information about the objects, such as object position information, for each group.

[0123] Specifically, for example, for a plurality of grouped objects, it is possible to change the object position information of each object while maintaining the relative positional relationship of the plurality of objects.

[0124] For example, it is assumed that the edit screen and POV image shown in Fig. 5 are displayed on the display unit 24. In Fig. 5, the same reference numerals are used to designate parts corresponding to those in Fig. 3, and the description thereof will be omitted as appropriate.

[0125] 5, an editing screen ED21 and a POV image P21 are displayed on the display unit 24. Note that, for ease of viewing, only a portion of the editing screen ED21 is shown here.

[0126] On the editing screen ED21, a track area and a timeline area are provided for each track, similar to the case shown in FIG.

[0127] That is, here, the track area and timeline area are displayed for the track of the vocal object with the object name "Vo" and the track of the electric guitar object with the object name "EG."

[0128] For example, the area TR21 is a track area for a vocal object track, and the area TM21 is a timeline area for a vocal object track.

[0129] In this example, in addition to area GP21, which is a group display area, area OB21, which is an object name display area, and area PS21, which is a coordinate system selection area, area TR21 also displays a track color display area TP21, a mute button MU21, and a solo button SL21.

[0130] Here, the track color display area TP21 is an area where a track color number is displayed. The track color number is information that indicates a track color that can be assigned to each track and is a color for identifying the track.

[0131] As will be described later, the information processing device 11 allows the user to select whether to display the object ball on the POV image in the group color or the track color.

[0132] Therefore, the user can specify a track color for each track by operating the track color display area on the editing screen ED21 using the input unit 21. That is, for example, the user can display a track color selection window similar to the group selection window GW11 shown in Fig. 2 and select a track color number from the track color selection window to select the track color of the track.

[0133] For example, the number "3" written in the track color display area TP21 indicates a track color number, and the track color display area TP21 is displayed in the track color indicated by that track color number.

[0134] Any track color can be selected for each track, and for example, different track colors can be selected (specified) for tracks corresponding to two objects that belong to the same group. Also, for example, the same track color can be selected for tracks corresponding to two objects that belong to different groups.

[0135] The mute button MU21 is a button that is operated when making a mute setting, which will be described later, and the solo button SL21 is a button that is operated when making a solo setting, which will be described later.

[0136] In addition, area TM21, which is the timeline area for a track of a vocal object, displays the time waveform L21 of the track, i.e., the audio signal of the object, and broken lines L22 to L24 representing the horizontal angle, vertical angle, and radius of the time series of the object.

[0137] In particular, points on the broken lines L22, L23, and L24 represent editing points that allow specification of the horizontal angle, vertical angle, and radius of the object position information at a certain time (timing) of the point. These editing points may be set to predetermined times, or may be set to times specified by the user. Furthermore, the user may be able to delete editing points.

[0138] Furthermore, when editing each track, the user can play back the sound of the rendered audio content and edit while listening to the played back sound, and the editing screen ED21 also displays a playback cursor TC21 that indicates the playback position of the sound of the audio content, i.e., the time currently being played. In the POV image P21, the object ball of each object is displayed based on the object position information of the time (timing) indicated by the playback cursor TC21.

[0139] In the example shown in Figure 5, the same group ID "3" is displayed in the group display area of ​​the tracks corresponding to the vocal and electric guitar objects, indicating that these objects belong to the same group.

[0140] Therefore, in the POV image P21, the object ball BL15 of the electric guitar object and the object ball BL16 of the vocal object are displayed in the same group color.

[0141] In the example shown in FIG. 5, the playback cursor TC21 is located at the time "13197."

[0142] At this time, the object position information of the vocal object is assumed to be coordinates (Azimuth, Elevation, Radius)=(-5.62078, 1.36393, 1), and the object position information of the electric guitar object is assumed to be coordinates (-3.57278, -3.79667, 1).

[0143] From the state shown in Fig. 5, for example, let us say that the user operates the input unit 21 to change the object position information of the vocal object at time "20227" as shown in Fig. 6. Note that in Fig. 6, the same reference numerals are used to designate parts that correspond to those in Fig. 5, and their explanation will be omitted where appropriate.

[0144] For example, the user instructs a change to the object position information by operating the input unit 21 to move the position of the editing point, move the object ball, or directly input the changed object position information. That is, the changed object position information is input.

[0145] In the example of FIG. 6, it is assumed that the user has designated coordinates (-22.5, 1.36393, 1) as the post-change object position information for the vocal object at time "20227".

[0146] Then, in response to the signal supplied from the input unit 21 in response to the user's operation, the position determination unit 41 determines the object position information of the vocal object at time "20227" to be the coordinates (-22.5, 1.36393, 1) specified by the user.

[0147] At the same time, the position determination unit 41 identifies other objects that belong to the same group as the vocal object whose object position information has been changed, by referring to the group information recorded in the recording unit 22. In this example, the electric guitar object is identified as an object in the same group as the vocal object.

[0148] The position determination unit 41 changes (determines) the object position information of the electric guitar object that belongs to the same group identified in this way so that the relative positional relationship with the vocal object is maintained. At this time, the object position information of the electric guitar object is determined based on the coordinates (-22.5, 1.36393, 1) that are the changed object position information of the vocal object.

[0149] Therefore, in this example, the object position information of the electric guitar object at time "20227" is set to coordinates (-20.452, -3.79667, 1).

[0150] When the object position information of the grouped objects is changed (determined) in this manner, the display control unit 42 controls the display unit 24 to move the object balls of those objects to the positions indicated by the changed object position information.

[0151] In the example shown in Figure 6, object ball BL16 of the vocal object and object ball BL15 of the electric guitar object, which belong to the same group, are moved to the right in the figure while maintaining the relative positional relationship between these objects.

[0152] Furthermore, from the state shown in Fig. 6, for example, suppose that the user operates the input unit 21 to change the object position information of the vocal object at time "27462" as shown in Fig. 7. Note that in Fig. 7, the same reference numerals are used to designate parts that correspond to those in Fig. 5, and their explanation will be omitted where appropriate.

[0153] In the example of FIG. 7, it is assumed that the user has designated coordinates (-56, 1.36393, 1) as the post-change object position information of the vocal object at time "27462".

[0154] Then, in response to a signal supplied from the input unit 21 in response to a user operation, the position determination unit 41 determines the object position information of the vocal object at time "27462" to be the coordinates (-56, 1.36393, 1) specified by the user.

[0155] At the same time, the position determination unit 41 changes (determines) the object position information of the electric guitar object that belongs to the same group as the vocal object so that the relative positional relationship with the vocal object is maintained.

[0156] Therefore, in this example, the object position information of the electric guitar object at time "27462" is set to coordinates (-53.952, -3.79667, 1).

[0157] When the object position information of the grouped objects is changed in this manner, the display control unit 42 controls the display unit 24 to move the object balls of those objects to the positions indicated by the changed object position information.

[0158] In the example shown in Figure 7, object ball BL16 of the vocal object and object ball BL15 of the electric guitar object, which belong to the same group, have been moved further to the right in the figure than in Figure 6, while maintaining the relative positional relationship between these objects.

[0159] In the examples of Figures 6 and 7, the user needs to input the changed object position information for the vocal object, but does not need to input the changed object position information for the electric guitar object that belongs to the same group as the vocal object.

[0160] In other words, by simply changing the object position information of one object, the object position information of all other objects belonging to the same group as that object is automatically changed all at once, without the user having to give any special instructions. In other words, the user does not have to go through the work of inputting and changing the object position information of all objects one by one. Moreover, the object position information can be changed appropriately while maintaining the relative positional relationships of those objects.

[0161] As described above, by changing the object position information of all objects belonging to the same group while maintaining their relative positional relationships, it is possible to more efficiently and simply edit the object position information.

[0162] 6 and 7, an example has been described in which when the object position information of a vocal object is changed, the object position information of an electric guitar object belonging to the same group is also changed accordingly.

[0163] However, conversely, when the user changes the object position information of the electric guitar object, the object position information of the vocal object is changed accordingly.

[0164] <Explanation of Object Movement Processing> Here, a description will be given of the processing that is performed when the object position information is changed and the position of the object in the reproduction space is moved as described with reference to Figures 5 to 7. That is, the object movement processing by the information processing device 11 will be described below with reference to the flowchart in Figure 8. Note that when this object movement processing is started, at least the editing screen is displayed on the display unit 24.

[0165] In step S41, the control unit 23 receives designation of an object for which the object position information is to be changed and the object position information of the object after the change.

[0166] For example, the user operates the input unit 21 to select a track area or the like on the editing screen to specify the object to be changed, and the control unit 23 identifies the specified object based on the signal supplied from the input unit 21.

[0167] In addition, for example, the user operates the input unit 21 to input information such as moving the position of the editing point of the horizontal angle, vertical angle, or radius that constitutes the object position information displayed in the timeline area of ​​the editing screen, thereby specifying the changed object position information.

[0168] In step S42, the control unit 23 refers to the group information recorded in the recording unit 22 and identifies objects that belong to the same group as the object designated in step S41.

[0169] In step S43, the position determination unit 41 changes (updates) the object position information of the specified object based on a signal supplied from the input unit 21 in response to an operation to specify the changed object position information.

[0170] Furthermore, in response to the change in the object position information of the specified object, the position determination unit 41 also changes the object position information of all other objects belonging to the same group identified in step S42. At this time, the object position information is changed so that the relative positional relationships of all objects belonging to the group are maintained (preserved).

[0171] In step S44, the display control unit 42 controls the display unit 24 to update the display of the editing screen and POV image displayed on the display unit 24 in accordance with the change in the object position information in step S43, and the object movement processing ends.

[0172] For example, the display control unit 42 updates the display of the horizontal angle, vertical angle, and radius position that constitute the object position information in the timeline area of ​​the editing screen, and moves the position of the object ball on the POV image. When the object position information is changed in this way, the object is moved in the playback space.

[0173] In this way, when changing the object position information of one object, the information processing device 11 changes not only the object but also the object position information of all other objects belonging to the same group as that object. At this time, the information processing device 11 changes the object position information of those objects so that the relative positional relationships of all objects belonging to the same group are maintained before and after the change.

[0174] In this way, editing can be performed more efficiently by simultaneously changing the object position information of objects belonging to the same group while maintaining the relative positional relationships of those objects.

[0175] <About the L / R pair> When grouping two objects that make up a pair, it is sometimes desirable to arrange the positions of the two objects in the playback space symmetrically with respect to a predetermined reference plane. The reference plane here is, for example, a median plane that includes a line parallel to the direction of the front as seen from the listening position O.

[0176] For example, when it comes to reverb components, i.e. ambience, there is a common demand for two ambiences to be paired as objects, and for these objects to be positioned symmetrically with respect to the reference plane.

[0177] Therefore, two objects that are to be placed symmetrically with respect to the reference plane may be designated as objects that make up an L / R pair.

[0178] Two objects that make up an L / R pair form one group. When a command is issued to change the object position information of one of the two objects, not only the object position information of one object but also the object position information of the other object is changed so that they are symmetrical with respect to the reference plane in the playback space.

[0179] Specifically, for example, when specifying objects to be grouped as objects that make up an L / R pair, the user performs an operation to specify menu item ME12 as shown in Fig. 9. Note that in Fig. 9, parts that correspond to those in Fig. 2 are given the same reference numerals, and their explanation will be omitted as appropriate.

[0180] FIG. 9 shows a part of the editing screen ED31 displayed on the display unit 24, and in this example, the editing screen ED31 displays a track area and a timeline area for each of two tracks.

[0181] For example, area TR31 is a track area for a track corresponding to an ambience object with an object name "Amb_L" that is located on the front left side as viewed from listening position O. Similarly, area TR32 is a track area for a track corresponding to an ambience object with an object name "Amb_R" that is located on the front right side as viewed from listening position O.

[0182] Furthermore, in FIG. 9, the area TR32, that is, the track corresponding to the object "Amb_R", is selected (specified), and menu items ME11 and ME12 and a group selection window GW11 are displayed.

[0183] In this state, when the user operates the input unit 21 to operate the menu item ME12 for designating an L / R pair, a check mark is displayed to the left of the text "L / R pair" in the menu item ME12. As a result, the object "Amb_R" is designated as an object that constitutes an L / R pair.

[0184] Also, in this example, the cursor CS11 has specified (selected) the group icon with group ID "9" in the group selection window GW11. Therefore, the object "Amb_R" belongs to the group with group ID "9" and is an object that constitutes an L / R pair.

[0185] In the example of FIG. 9, the group ID "9" is also displayed in the group display area in area TR31 for the track corresponding to object "Amb_L."

[0186] Therefore, it can be seen that the object "Amb_L" and the object "Amb_R" belong to a group with a group ID of "9" and are objects that make up an L / R pair.

[0187] In this way, if it is possible to specify for each object not only the group to which it belongs but also whether it is an L / R pair, the group information can also include an L / R pair flag, which is information indicating whether each object is an object that forms an L / R pair.

[0188] In such a case, for example, the group information includes a group ID, information indicating the objects belonging to the group, and an L / R pair flag.

[0189] For example, a value of "1" for the L / R pair flag indicates that the two objects belonging to the group are an L / R pair, and a value of "0" for the L / R pair flag indicates that the multiple objects belonging to the group are not an L / R pair.

[0190] In particular, a group corresponding to group information that includes an L / R pair flag with a value of "1" is always made up of two objects. In other words, only when two objects make up a group can those two objects be designated as an L / R pair. Therefore, being an L / R pair can be said to indicate one characteristic of a group.

[0191] When the objects "Amb_L" and "Amb_R" are set as an L / R pair as described above, the object position information of these objects is changed in response to a user operation, for example, as shown in Figures 10 to 12. Note that in Figures 10 to 12, parts corresponding to those in Figure 9 are given the same reference numerals, and their explanation will be omitted where appropriate.

[0192] For example, in the example shown in FIG. 10, an editing screen ED31 and a POV image P31 are displayed on the display unit 24.

[0193] On the editing screen ED31, an area TR31 that is the track area for the object "Amb_L" and an area TR32 that is the track area for the object "Amb_R" are displayed in the group color of the group to which these objects belong, whose group ID is "9." Also, on the timeline area on the editing screen ED31, the playback cursor TC31 is positioned at time "0."

[0194] In this state, it is assumed that the user operates the input unit 21 to specify coordinates (30,0,1) as the object position information of the object "Amb_L" at time "0".

[0195] The position determination unit 41 then determines the object position information of object "Amb_L" at time "0" to be coordinates (30,0,1). At the same time, the position determination unit 41 determines the object position information of object "Amb_R" at time "0" so that the position of object "Amb_R" in the playback space is symmetrical to the position of object "Amb_L" with respect to the reference plane. In other words, the object position information of object "Amb_R" is changed.

[0196] Here, the object position information of the object "Amb_R" at time "0" is set to coordinates (-30, 0, 1).

[0197] In this way, once the object position information of the objects "Amb_L" and "Amb_R", which belong to the same group and are an L / R pair, has been determined, the display control unit 42 updates the display of the POV image P31 based on the determined object position information.

[0198] Here, an object ball BL31 of the object "Amb_L" is displayed at a position corresponding to the coordinates (30,0,1) on the POV image P31.

[0199] The object name "Amb_L" is displayed on this object ball BL31, and the object ball BL31 is displayed in the group color of the group whose group ID is "9."

[0200] On the other hand, an object ball BL32 of the object "Amb_R" is displayed at a position corresponding to the coordinates (-30, 0, 1) on the POV image P31.

[0201] The object name "Amb_R" is displayed on this object ball BL32, and the object ball BL32 is displayed in the group color of the group whose group ID is "9."

[0202] In particular, here, a plane including the listening position O and a line parallel to the depth direction in the figure is used as the reference plane, and object ball BL31 and object ball BL32 are placed in positions symmetrical to the left and right of the reference plane.

[0203] Also, from the state shown in FIG. 10, assume that the user operates the input unit 21 as shown in FIG. 11 to specify coordinates (56.5, 0, 1) as the object position information of the object "Amb_L" at time "20000".

[0204] Then, the position determination unit 41 determines the object position information of the object "Amb_R" at time "20000" to be coordinates (-56.5,0,1) in accordance with the coordinates (56.5,0,1) as the object position information of the object "Amb_L".

[0205] Then, the display control unit 42 controls the display unit 24 based on the coordinates (56.5, 0, 1) and (−56.5, 0, 1) as the changed object position information, and updates the display of the POV image P31.

[0206] As a result, object ball BL31 is moved to a position corresponding to the coordinates (56.5, 0, 1) on the POV image P31, and object ball BL32 is moved to a position corresponding to the coordinates (-56.5, 0, 1) on the POV image P31. After the movement, object ball BL31 and object ball BL32 remain positioned symmetrically with respect to the reference plane, just like in FIG. 10.

[0207] Furthermore, from the state shown in FIG. 11, it is assumed that the user operates the input unit 21 as shown in FIG. 12 to specify coordinates (110, 25, 1) as the object position information of the object "Amb_L" at time "40000".

[0208] Then, the position determination unit 41 determines the object position information of the object "Amb_R" at time "40000" to be coordinates (-110, 25, 1) in accordance with the coordinates (110, 25, 1) as the object position information of the object "Amb_L".

[0209] Then, the display control unit 42 controls the display unit 24 based on the coordinates (110, 25, 1) and coordinates (−110, 25, 1) as the changed object position information, and updates the display of the POV image P31.

[0210] As a result, object ball BL31 is moved to a position corresponding to the coordinates (110, 25, 1) on the POV image P31, and object ball BL32 is moved to a position corresponding to the coordinates (-110, 25, 1) on the POV image P31. After the movement, these object balls BL31 and BL32 remain positioned symmetrically with respect to the reference plane, as in the cases of FIGS. 10 and 11.

[0211] Here, an example has been described in which when the object position information of object "Amb_L" of the L / R pair of object "Amb_L" and object "Amb_R" is specified, the object position information of object "Amb_R" is changed accordingly. However, conversely, when the object position information of object "Amb_R" is specified, the position determination unit 41 changes the object position information of object "Amb_L" accordingly.

[0212] As described above, when there are two objects in a group, the user can specify these two objects as an L / R pair. In other words, the user can set the L / R pair as a property of the group.

[0213] Once an L / R pair is set, simply changing the object position information of one of the objects in the L / R pair will automatically change the object position information of the other object without the user having to give any special instructions. Moreover, since the two objects in the L / R pair are positioned symmetrically with respect to the reference plane, the user can easily set symmetrical sound image positions.

[0214] <Explanation of Grouping Process> Here, a description will be given of the grouping process performed by the information processing device 11 when an L / R pair can be specified as described above. That is, hereinafter, the grouping process performed by the information processing device 11 will be described with reference to the flowchart in FIG.

[0215] When the grouping process is started, the process of step S71 is performed, but since the process of step S71 is the same as the process of step S11 in Fig. 4, a description thereof will be omitted. However, in step S71, the user appropriately specifies an L / R pair by operating a menu item for specifying the L / R pair on the editing screen.

[0216] In step S72, the control unit 23 determines, based on the signal supplied from the input unit 21, whether or not the number of objects designated as objects to be grouped is two.

[0217] If it is determined in step S72 that not two objects are to be grouped, that is, three or more objects are to be grouped, then the process proceeds to step S75.

[0218] On the other hand, if it is determined in step S72 that there are two objects, then in step S73 the control unit 23 determines whether or not the two objects to be grouped should be an L / R pair. For example, when grouping two objects, if the menu item ME12 shown in Fig. 9 is operated to specify an L / R pair, it is determined that the objects should be an L / R pair.

[0219] If it is determined in step S73 that the objects should be grouped as an L / R pair, then in step S74 the control unit 23 sets the value of the L / R pair flag of the group to which the two objects to be grouped belong to "1." That is, an L / R pair flag with a value of "1" is generated.

[0220] After the process of step S74 is performed, the process proceeds to step S76.

[0221] On the other hand, if it is determined in step S73 that an L / R pair is not to be formed, then the process proceeds to step S75.

[0222] If it is determined in step S73 that an L / R pair is not to be formed, or if it is determined in step S72 that the number of specified objects is not two, the process proceeds to step S75.

[0223] In step S75, the control unit 23 sets the value of the L / R pair flag of the group to which the objects to be grouped belong to to "0." That is, an L / R pair flag with a value of "0" is generated.

[0224] After the process of step S75 is performed, the process proceeds to step S76.

[0225] When the L / R pair flag is generated in step S74 or step S75, the processes of steps S76 and S77 are then performed, and the grouping process ends.

[0226] The processing of steps S76 and S77 is the same as the processing of steps S12 and S13 in Fig. 4, and therefore description thereof will be omitted. However, in step S76, control unit 23 generates group information consisting of a group ID, information indicating objects belonging to the group, and the L / R pair flag generated in step S74 or step S75, in response to the user's specifying operation in step S71.

[0227] As described above, the information processing device 11 performs grouping in response to an input operation on the input unit 21, and generates group information including an L / R pair flag.

[0228] By grouping in this way, object position information and the like can be edited more efficiently on a group basis. Furthermore, for an L / R pair of objects, the user can place the objects in symmetrical positions simply by specifying the position of one of the objects.

[0229] Furthermore, even when the grouping process described with reference to FIG. 13 is performed, when an instruction to change object position information is given, the same process as the object moving process described with reference to FIG. 8 is basically performed.

[0230] However, in this case, if the objects are objects that form an L / R pair, in step S43, the object position information of the two objects that make up the L / R pair is changed so that the two objects are bilaterally symmetrical with respect to the reference plane. That is, the object position information of the two objects is changed while maintaining the bilaterally symmetrical relationship between the two objects with respect to the reference plane. Therefore, even in this case, the user can perform editing more efficiently and simply.

[0231] <Simultaneous editing of object position information at multiple times> On the editing screen, the user can specify (change) the horizontal angle, vertical angle, and radius that make up the object position information for each time, that is, for each editing point.

[0232] Furthermore, when changing object position information, the information processing device 11 can select multiple editing points by specifying a change range that includes multiple editing points lined up in the time direction, and offset (change) the positions (coordinate values) of these multiple editing points simultaneously by a predetermined change amount.

[0233] In the following, the coordinate values ​​of multiple editing points included in the specified change range, i.e., the amount of change by which the horizontal angle, vertical angle, and radius are simultaneously changed with a single operation, will be specifically referred to as the “offset amount.” Additionally, the editing points included in the change range will also be specifically referred to as the “selected editing points.”

[0234] 14 to 17, a specific example will be described in which editing points at different times are simultaneously selected by specifying a change range, and the coordinate values ​​of the selected editing points are changed by offset amounts. Note that in Figures 14 to 17, corresponding parts are assigned the same reference numerals, and their description will be omitted where appropriate.

[0235] First, for example, as shown in FIG. 14, suppose that an area TR41, which is a track area, and an area TM41, which is a timeline area, are displayed for the track of the object "Amb_L" on the editing screen ED41 displayed on the display unit 24.

[0236] In FIG. 14, the polygonal lines L41, L42, and L43 in the area TM41, which is the timeline area, represent the horizontal angle, vertical angle, and radius of the object "Amb_L" in time series.

[0237] In particular, here, edit points EP41-1 to EP41-4 indicating the horizontal angles at times 20000, 25000, 30000, and 35000 are provided on broken line L41, which indicates the horizontal angles constituting the object position information. Note that hereinafter, when there is no need to particularly distinguish between edit points EP41-1 to EP41-4, they will also be simply referred to as edit points EP41.

[0238] Similarly, edit points EP42-1 to EP42-4 are provided on broken line L42, indicating the vertical angles at times 20000, 25000, 30000, and 35000, respectively. Note that hereinafter, when there is no need to particularly distinguish between edit points EP42-1 to EP42-4, they will also be simply referred to as edit points EP42.

[0239] Furthermore, edit points EP43-1 to EP43-4 are provided on the broken line L43, indicating radii at times 20000, 25000, 30000, and 35000, respectively. Note that hereinafter, when there is no need to particularly distinguish between edit points EP43-1 to EP43-4, they will also be simply referred to as edit points EP43.

[0240] Now, suppose the coordinates of object position information for object "Amb_L" at times "20000", "25000", "30000", and "35000" are (Azimuth, Elevation, Radius) = (56.5, 0, 1), (65.0, 0, 1), (35.0, 0, 1), and (90.0, 0, 1), respectively.

[0241] In this state, when the user operates the input unit 21, for example by performing an operation such as range selection using the mouse as the input unit 21, to select a range including multiple editing points as the change range, a frame W41 indicating the change range as shown in Figure 15 is displayed.

[0242] In this example, a range including four edit points EP42-1 to EP42-4 on the broken line L42 is enclosed by a frame W41, and the range enclosed by this frame W41 is designated as the change range.

[0243] It is possible to specify a range that includes only one editing point EP42 as the change range, or a range that includes editing points of different types (coordinate components), such as horizontal angles and vertical angles, etc. In other words, it is also possible to specify a range that includes multiple editing points EP41, EP42, and EP43 as the change range.

[0244] Alternatively, for example, when an editing point at a predetermined time is included within the change range, editing points of other coordinate components at the same time as that editing point may also be selected as being included within the change range.

[0245] Furthermore, the method of specifying the change range, i.e., the editing points to be included in the change range, may be any method, such as holding down the control key on the keyboard and operating the mouse to specify each editing point by clicking with the pointer, etc.

[0246] When the change range is specified, the display control unit 42 controls the display unit 24 to display, for example, an offset screen OF41 shown in FIG. 16 on the edit screen ED41.

[0247] In the example shown in FIG. 16, an offset screen OF41 is displayed superimposed on an area TM41 which is a timeline area of ​​an edit screen ED41.

[0248] This offset screen OF41 has an offset display area OFT41 that shows the offset amount when moving the position of the selected editing point in the time direction, that is, when changing the time of the selected editing point. The offset display area OFT41 displays the number "100" that shows the time offset amount of the selected editing point (hereinafter also referred to as the time offset amount).

[0249] Furthermore, at both ends of the offset display area OFT41 on the offset screen OF41, there are provided buttons BT41-1 and BT41-2 for moving the position of the selected editing point in the time direction by the time offset amount "100" at a time.

[0250] For example, each time the user operates the input unit 21 to press button BT41-1 once, the position of the selected editing point in the time direction moves toward the future by the time offset amount "100." In other words, the time of the object position information increases by the time offset amount "100."

[0251] Conversely, for example, each time the user operates the input unit 21 to press button BT41-2 once, the position of the selected editing point in the time direction moves backward by the time offset amount "100". In other words, the time of the object position information decreases by the time offset amount "100". Note that hereinafter, when there is no need to particularly distinguish between button BT41-1 and button BT41-2, they will also be simply referred to as button BT41.

[0252] The offset screen OF41 has an offset display area OFT42 that shows the amount of offset when changing the horizontal angle indicated by the selected editing point, that is, when moving the position of the selected editing point. The offset display area OFT42 displays the number "10" that indicates the amount of offset of the horizontal angle (hereinafter also referred to as the horizontal angle offset).

[0253] At both ends of the offset display area OFT42 on the offset screen OF41, there are provided buttons BT42-1 and BT42-2 for moving the horizontal angle, which is the value of the selected editing point, i.e., the vertical position of the selected editing point in the drawing, by the horizontal angle offset amount of "10".

[0254] For example, each time the user operates the input unit 21 to press button BT42-1 once, the position of the selected editing point moves upward in the drawing by the horizontal angle offset amount "10". In other words, the horizontal angle of the object position information increases by the horizontal angle offset amount "10".

[0255] Conversely, for example, each time the user operates the input unit 21 to press button BT42-2 once, the position of the selected editing point moves downward in the drawing by the horizontal angle offset amount "10". In other words, the horizontal angle of the object position information decreases by the horizontal angle offset amount "10". Note that hereinafter, when there is no need to particularly distinguish between button BT42-1 and button BT42-2, they will also be simply referred to as button BT42.

[0256] The offset screen OF41 has an offset display area OFT43 that shows the amount of offset when changing the vertical angle indicated by the selected editing point, that is, when moving the position of the selected editing point. The offset display area OFT43 displays the number "10" that indicates the amount of vertical angle offset (hereinafter also referred to as "vertical angle offset").

[0257] At both ends of the offset display area OFT43 on the offset screen OF41, there are provided buttons BT43-1 and BT43-2 for moving the vertical angle, which is the value of the selected editing point, i.e., the position of the selected editing point in the diagram in the up and down direction, by the vertical angle offset amount of "10".

[0258] For example, each time the user operates the input unit 21 to press button BT43-1 once, the position of the selected editing point moves upward in the drawing by the vertical angle offset amount "10". In other words, the vertical angle of the object position information increases by the vertical angle offset amount "10".

[0259] Conversely, for example, each time the user operates input unit 21 to press button BT43-2 once, the position of the selected editing point moves downward in the drawing by the vertical angle offset amount "10". In other words, the vertical angle of the object position information decreases by the vertical angle offset amount "10". Note that hereinafter, when there is no need to particularly distinguish between button BT43-1 and button BT43-2, they will also be simply referred to as button BT43.

[0260] The offset screen OF41 has an offset display area OFT44 that shows the amount of offset when changing the radius indicated by the selected editing point, that is, when moving the position of the selected editing point. The offset display area OFT44 displays the character "0.1" that indicates the amount of offset of the radius (hereinafter also referred to as "radius offset amount").

[0261] At both ends of the offset display area OFT44 on the offset screen OF41, there are provided buttons BT44-1 and BT44-2 for moving the radius, which is the value of the selected editing point, that is, the vertical position of the selected editing point in the drawing, by the radius offset amount of "0.1".

[0262] For example, each time the user operates the input unit 21 to press button BT44-1 once, the position of the selected editing point moves upward in the drawing by the radius offset amount "0.1". In other words, the radius of the object position information increases by the radius offset amount "0.1".

[0263] Conversely, for example, each time the user operates input unit 21 to press button BT44-2 once, the position of the selected editing point moves downward in the drawing by a radius offset amount of "0.1". In other words, the radius of the object position information decreases by a radius offset amount of "0.1". Note that hereinafter, when there is no need to particularly distinguish between button BT44-1 and button BT44-2, they will also be simply referred to as button BT44.

[0264] Furthermore, the user may operate the input unit 21 to change the numerical values ​​in the offset display areas OFT41 to OFT44, that is, the offset amounts, to any desired values.

[0265] As described above, when the range surrounded by the frame W41 is designated as the change range and the offset screen OF41 is displayed, the user operates the input unit 21 to operate buttons BT41, BT42, BT43, and BT44 provided on the offset screen OF41.

[0266] This allows the user to instruct a change in offset amount units for each component of the object position information. That is, the user can perform operations on the user interface, the offset screen OF41, to move the selected editing point relative to other editing points.

[0267] 15, that is, when the coordinates of the object position information at times "20000," "25000," "30000," and "35000" are (56.5,0,1), (65.0,0,1), (35.0,0,1), and (90.0,0,1), respectively, the user operates button BT43-1 five times. In other words, the user performs an operation to increase the vertical angle indicated by each of the four selected editing points EP42 by 50 degrees.

[0268] When such an operation is performed, the position determination unit 41 increases the vertical angle of the object position information of the object "Amb_L" corresponding to the selected editing point at times "20000", "25000", "30000", and "35000" by 50 based on the signal supplied from the input unit 21.

[0269] As a result, the coordinates of object position information for object "Amb_L" at times "20000", "25000", "30000", and "35000" are changed to (56.5,50,1), (65.0,50,1), (35.0,50,1), and (90.0,50,1).

[0270] In this example, the user can simultaneously change the object position information for four times by the vertical angle offset amount simply by operating button BT43.

[0271] When the object position information is changed in this way, the display control unit 42 controls the display unit 24 to update the display on the editing screen ED41. That is, the display control unit 42 updates the display on the editing screen ED41 so that the editing points EP42-1 to EP42-4 move upward in the figure as shown in Fig. 16 compared to the case shown in Fig. 15.

[0272] Furthermore, suppose that in this state the user operates button BT41-1 ten times, that is, the user performs an operation to increase the time of the selected editing point by 1000.

[0273] When such an operation is performed, the position determination unit 41 increments the time of the object position information of the object “Amb_L” corresponding to the selected editing point by 1000 based on the signal supplied from the input unit 21 .

[0274] In other words, the object position information of object "Amb_L", which was previously at times "20000", "25000", "30000", and "35000", is changed to object position information at times "21000", "26000", "31000", and "36000".

[0275] As a result, the coordinates of object "Amb_L" at times "21000", "26000", "31000", and "36000" are (56.5,50,1), (65.0,50,1), (35.0,50,1), and (90.0,50,1) as object position information.

[0276] At the same time, there will no longer be editing points at times "20000," "25000," "30000," and "35000," where there were previously editing points, and the object position information at those times will be determined by the interpolation process described below.

[0277] Here, only the vertical angle editing point EP42 is considered to be the selected editing point, but when the time of the editing point is changed, editing points EP41 and EP43, which are at the same time as editing point EP42, are also considered to be selected editing points, and the time of the object position information is changed.

[0278] When the object position information is changed in this way, the display control unit 42 controls the display unit 24 to update the display on the editing screen ED41. That is, the display control unit 42 updates the display on the editing screen ED41 so that the editing points EP41 to EP43 move further to the right in the figure than in the case shown in FIG. 16, as shown in FIG.

[0279] As described above, by making it possible to change multiple editing points included in the change range collectively by the offset amount, editing can be performed more easily and efficiently than when editing multiple object position information pieces at different times one by one.

[0280] In addition, when the object position information of one object at multiple times is changed collectively by the offset amount, if there are other objects that belong to the same group as that object, the object position information of the other objects at multiple times will also be changed.

[0281] For example, suppose that objects "Amb_L" and "Amb_R" belong to the same group, and an instruction is given to change the object position information of object "Amb_L" at time A1 and time A2 by an operation on the offset screen OF41.

[0282] In this case, the position determination unit 41 changes the object position information of the object "Amb_L" and the object "Amb_R" at time A1 and time A2 in units of the offset amount while maintaining the relative positional relationship between the object "Amb_L" and the object "Amb_R".

[0283] <Description of Offset Movement Processing> Next, an operation of the information processing device 11 when object position information for a plurality of different times is changed collectively and simultaneously by the operation on the offset screen described above will be described. That is, hereinafter, the offset movement processing by the information processing device 11 will be described with reference to the flowchart in FIG.

[0284] In step S101, the control unit 23 receives a designation of an object whose object position information is to be changed and a designation of the range of change for that object.

[0285] For example, the user specifies the range of change by directly specifying one or more editing points displayed in the timeline area of ​​the editing screen or by specifying an area including one or more editing points by operating the input unit 21. Based on the signal supplied from the input unit 21, the control unit 23 identifies the object specified as the object to be changed and the specified range of change for that object, that is, the selected editing points whose coordinate values ​​will be changed at the same time.

[0286] In step S102, the display control unit 42 controls the display unit 24 to superimpose and display the offset screen on the timeline area of ​​the editing screen displayed on the display unit 24. As a result, for example, the offset screen OF41 shown in Fig. 16 is displayed.

[0287] In step S103, the control unit 23 accepts an operation to change the position of the selected editing point by operating the offset screen, that is, an input of the amount of change in the coordinate value.

[0288] When the offset screen is displayed, the user operates the input unit 21 to input the amount of change by which the selected editing point is changed in units of offset amount. For example, in the example shown in Fig. 16, the user operates button BT41, button BT42, button BT43, or button BT44 to instruct a change in coordinate value.

[0289] In step S104, the position determination unit 41 simultaneously changes the values ​​of the selected editing points included in the change range of the specified object, i.e., the object position information, in units of the offset amount, based on the signal supplied from the input unit 21. In step S104, one or more pieces of object position information at each time are simultaneously changed by the change amount specified by the user, which is in units of the offset amount.

[0290] For example, in the state shown in Figure 15, if the button BT43-1 shown in Figure 16 is operated only once by the user, the position determination unit 41 increases the vertical angle that constitutes the object position information at the time corresponding to the selected editing point by 10 degrees.

[0291] In step S105, the control unit 23 determines whether or not the object to be changed belongs to a group, based on the object whose object position information is to be changed and the group information recorded in the recording unit 22. In other words, it is determined whether or not there is another object that belongs to the same group as the object to be changed.

[0292] If it is determined in step S105 that the object does not belong to a group, that is, if it is determined that there is no other object that belongs to the same group, the process proceeds to step S107.

[0293] On the other hand, if it is determined in step S105 that the object belongs to a group, that is, if there is another object that belongs to the same group, the process proceeds to step S106.

[0294] In step S106, the position determination unit 41 changes the object position information of all other objects that belong to the same group as the object to be changed. At this time, the position determination unit 41 changes the object position information of the other objects in offset amount units in accordance with the change in the object position information of the object to be changed, so that the relative positional relationships of all objects that belong to the group in the playback space are maintained. Note that if the object to be changed is an L / R pair object, the object position information of the other objects that form the L / R pair with respect to the object to be changed is changed so that the two objects that form the L / R pair are symmetrical with respect to the reference plane.

[0295] Once the object position information of the other objects has been changed, the process then proceeds to step S107.

[0296] If it is determined in step S105 that the object does not belong to a group, or if the process of step S106 is performed, then the process of step S107 is performed and the offset movement process ends. Note that the process of step S107 is the same as the process of step S44 in Fig. 8, so its description will be omitted.

[0297] In this way, the information processing device 11 simultaneously changes the object position information corresponding to one or more editing points included in the change range in units of offset amount. This reduces the number of operations required by the user compared to changing the positions of the editing points, i.e., the coordinate values, one by one, allowing for more efficient and simple editing.

[0298] Interpolation of object position information Incidentally, the information processing device 11 basically holds object position information, that is, meta information, for times when there are edit points, but does not hold meta information for times when there are no edit points.

[0299] However, when rendering audio content, object position information for all times is required. Therefore, the information processing device 11 uses interpolation processing to obtain object position information for times when there are no edit points when rendering audio content or when outputting audio content.

[0300] For example, as shown in FIG. 19, it is common to select two adjacent editing points and obtain the coordinate values ​​at each time between those editing points by linear interpolation.

[0301] In FIG. 19, on the editing screen ED51, for the track of the object with the object name "Vo", broken lines L51 to L53 are displayed which represent the horizontal angle, vertical angle, and radius which make up the time-series object position information.

[0302] For example, if we look at the time series horizontal angles indicated by broken line L51, the horizontal angle (object position information) at the time when editing point EP51-1 exists and the horizontal angle at the time when editing point EP51-2 adjacent to editing point EP51-1 exists are stored in information processing device 11.

[0303] On the other hand, the horizontal angles at the times between edit points EP51-1 and EP51-2 are not retained, so the horizontal angles at those times are calculated by linear interpolation based on the coordinate values ​​at edit points EP51-1 and EP51-2. Note that hereinafter, when there is no need to particularly distinguish between edit points EP51-1 and EP51-2, they will also be simply referred to as edit points EP51.

[0304] When linear interpolation is performed, it is assumed that the horizontal angle of the object changes at a constant speed, that is, the object moves at a constant angular velocity in the horizontal direction, between two edit points EP51. In other words, a linear change in the horizontal angle indicates that the object moves at a constant angular velocity in the horizontal direction.

[0305] However, since objects do not always move at a constant angular velocity, it would be convenient to be able to select a method of interpolation processing for object position information (interpolation method) from among multiple interpolation methods according to the movement pattern of the object.

[0306] Therefore, the information processing device 11 is configured to be able to select an interpolation method for each section between adjacent editing points for each component that constitutes the object position information.

[0307] Specifically, for example, as shown in FIG. 20, the user can operate the input unit 21 to display the interpolation method selection screen SG51 by performing operations such as selecting a section between two adjacent editing points in the timeline area of ​​the editing screen ED51.

[0308] 20, the same reference numerals are assigned to the parts corresponding to those in FIG. 19, and the description thereof will be omitted as appropriate. The operation for displaying the interpolation method selection screen SG51 may be any operation, such as a click operation.

[0309] In the example of FIG. 20, the section between edit points EP51-1 and EP51-2 is specified, and the interpolation method for the horizontal angle in that section can be selected on the interpolation method selection screen SG51.

[0310] Specifically, the interpolation method selection screen SG51 has menu items ME51 to ME54 that are operated when specifying each of four different interpolation methods, and the user specifies the interpolation method by selecting one of these menu items.

[0311] For example, menu item ME51 indicates linear interpolation, and menu item ME52 indicates cosine interpolation, which is an interpolation using a cosine function.

[0312] Menu item ME53 indicates an interpolation method that realizes a rectangular change in coordinate values, where the coordinate values ​​remain the same from the start of the section to just before the end, and then suddenly change just before the end of the section. Menu item ME54 indicates an interpolation method that realizes a rectangular change in coordinate values, where the coordinate values ​​suddenly change just after the start of the section to be interpolated, and then continue to remain the same until the end of the section.

[0313] Each menu item has a straight line, curve, or broken line that represents the change in coordinate value when interpolation is performed using the interpolation method corresponding to that menu item, allowing the user to intuitively understand the interpolation method just by looking at the menu item. For example, menu item ME52, which indicates cosine interpolation, has a cosine curve drawn, allowing the user to intuitively understand that the interpolation method is cosine interpolation.

[0314] The method of interpolation processing (interpolation method) is not limited to the method described with reference to FIG. 20, and any other method may be used, such as an interpolation method using a quadratic function or the like.

[0315] Also, when the section between editing points EP51-1 and EP51-2 is specified as shown in Figure 20, if the user selects (specifies) menu item ME52 by operating the input unit 21, the position determination unit 41 performs cosine interpolation according to the signal supplied from the input unit 21.

[0316] That is, based on the horizontal angle indicated by editing point EP51-1 and the horizontal angle indicated by editing point EP51-2, the position determination unit 41 calculates the horizontal angle at each time between editing point EP51-1 and editing point EP51-2 by cosine interpolation using a cosine function. Note that while the example described here is one in which only the horizontal angle of the object position information is cosine interpolated, it is also possible to perform cosine interpolation on the vertical angle and radius at the same time as the horizontal angle for a section in which cosine interpolation is performed. In other words, if one interpolation method, such as cosine interpolation, is specified for a section, the horizontal angle, vertical angle, and radius of the object position information for that section may be interpolated using the specified interpolation method.

[0317] When cosine interpolation is performed as the interpolation process in the above manner, the display on the editing screen ED51 is updated, for example, as shown in Fig. 21. In Fig. 21, the same reference numerals are used to designate parts corresponding to those in Fig. 19, and the description thereof will be omitted as appropriate.

[0318] In the example shown in FIG. 21, the section between editing points EP51-1 and EP51-2 where cosine interpolation has been performed is depicted as a cosine curve rather than a straight line.

[0319] If no particular interpolation method is specified, the coordinate values ​​between the editing points can be interpolated using an interpolation method determined in the initial settings, such as linear interpolation.

[0320] In this case, in a section where an interpolation method other than the default setting is selected, the line (straight line, curved line, or broken line) connecting two adjacent edit points may be displayed in a color different from the line in the section where the linear interpolation determined by the default setting is performed. Alternatively, the line connecting the edit points may be displayed in a different color depending on the selected interpolation method. This allows the user to instantly determine which interpolation method has been specified.

[0321] <Explanation of Interpolation Method Selection Process> Next, the operation of the information processing device 11 when the user selects an interpolation method for a section between editing points will be described. That is, the interpolation method selection process performed by the information processing device 11 will be described below with reference to the flowchart in Fig. 22. Note that when this interpolation method selection process starts, the editing screen is displayed on the display unit 24.

[0322] In step S131, the control unit 23 accepts the designation of two editing points displayed on the timeline area of ​​the editing screen.

[0323] For example, when selecting an interpolation method for a desired section, the user specifies the section for which the interpolation method is to be selected by specifying two editing points by operating the input unit 21. The control unit 23 identifies the editing points that are the start and end positions of the section for which the interpolation method is to be selected, based on a signal supplied from the input unit 21 in response to the user's operation.

[0324] In step S132, the display control unit 42 controls the display unit 24 to display an interpolation method selection screen superimposed on the timeline area of ​​the editing screen, thereby displaying, for example, the interpolation method selection screen SG51 shown in Fig. 20.

[0325] When the interpolation method selection screen is displayed, the user operates the input unit 21 to select (specify) a desired menu item on the interpolation method selection screen, thereby specifying an interpolation method.

[0326] In step S133, the control unit 23 selects an interpolation method for the section between the two editing points specified in step S131 based on a signal supplied from the input unit 21 in response to a user operation, and generates interpolation method designation information indicating the selection result. The control unit 23 supplies the interpolation method designation information thus generated to the recording unit 22.

[0327] In step S134, the recording unit 22 records the interpolation method designation information supplied from the control unit 23 as part of the data of the audio content.

[0328] Furthermore, when the interpolation method designation information is generated, the display control unit 42 controls the display unit 24 to update the display on the editing screen. As a result, for example, as shown in Fig. 21, the line of the section to be processed, i.e., the line connecting two editing points, is displayed in a shape and color corresponding to the interpolation method indicated by the interpolation method designation information.

[0329] Furthermore, object position information, more specifically, the horizontal angle, vertical angle, and radius components that make up the object position information, are interpolated at appropriate times, such as when rendering audio content.

[0330] That is, in step S135, the position determining unit 41 performs interpolation processing for each time for which object position information is not held, and generates object position information for all objects.

[0331] At this time, the position determination unit 41 performs interpolation processing for each component of the object position information using the interpolation method indicated by the interpolation method designation information recorded in the recording unit 22, based on the object position information at other times that has been stored.

[0332] When the object position information for each time is obtained by the interpolation process, the interpolation method selection process ends. After that, the audio content data is output or rendering is performed based on the audio content data as appropriate.

[0333] In this way, the information processing device 11 generates and records interpolation method designation information that indicates the interpolation method designated for each section for each component that constitutes the object position information. Then, the information processing device 11 performs interpolation processing using the interpolation method indicated by the interpolation method designation information to obtain the object position information at each time. In this way, it becomes possible to more accurately express the movement (motion) of an object. In other words, it is possible to increase the degree of freedom in expressing the movement of an object, and it becomes possible to express a variety of sound images.

[0334] <About track color display> Incidentally, in the example shown in FIG. 5, it has been explained that the track area of ​​the edit screen ED21 is provided with a track color display area for each track.

[0335] In this track color display area, a track color number is displayed, and each track color display area is displayed in a track color that is predetermined for that track color number.

[0336] As described above, the information processing device 11 allows the user to select whether to display the object ball on the POV image in the group color or the track color.

[0337] For example, if the object ball on the POV image is set to be displayed in the track color, at the timing when the display of the POV image is updated, such as step S13 in Figure 4 or step S44 in Figure 8, the display control unit 42 controls the display by the display unit 24 so that the object ball is displayed in the track color.

[0338] If track colors can be individually assigned to objects, i.e., tracks, the user can easily distinguish between tracks by looking at the track color. In particular, even when audio content has a large number of objects, the user can easily distinguish which object ball corresponds to which track.

[0339] 5 has been described as an example in which a track color display area and a group display area are displayed in each track area. However, if the object ball is set to be displayed in the track color, the track color display area may be displayed in the track area, but the group display area may not be displayed.

[0340] In such a case, for example, an edit screen ED61 shown in FIG.

[0341] In the example shown in FIG. 23, track areas of 11 tracks and timeline areas of these tracks are displayed on the editing screen ED61.

[0342] In particular, the track and timeline areas for each of the 11 objects with object names "Kick," "OH_L," "OH_R," "Snare," "Vo," "EG," "Cho," "AG1," "AG2," "Amb_L," and "Amb_R" are displayed.

[0343] The track area of ​​each object is provided with a track color display area, in which a track color number is displayed, and each track color display area is displayed in a track color that is predetermined for that track color number.

[0344] Specifically, for example, area TR61 is the track area for the track of object "Kick." Area TR61 contains area OB61, which is an object name display area, and track color display area TP61. The object name "Kick" is displayed in area OB61, and track color display area TP61 displays track color number "1." The entire area TR61, including track color display area TP61, is displayed in the track color determined for track color number "1."

[0345] 23, the track color number "1" is assigned to the four objects that make up the drums, whose object names are "Kick," "OH_L," "OH_R," and "Snare," more specifically, to the tracks of the four objects. Also, the track color number "3" is assigned to the object "Vo" that corresponds to the vocals of the electric guitar player and the object "EG" of that electric guitar.

[0346] The track color number "6" is assigned to the object "Cho" corresponding to the chorus performed by the acoustic guitar player and the acoustic guitar object "AG1."

[0347] Similarly, the track color number "22" is assigned to another acoustic guitar object "AG2." Furthermore, the track color number "9" is assigned to the ambience objects "Amb_L" and "Amb_R."

[0348] When an editing screen ED61 such as that shown in Fig. 23 is displayed, the display unit 24 displays, for example, a POV image P61 shown in Fig. 24. In Fig. 24, parts corresponding to those in Fig. 3 or 10 are given the same reference numerals, and descriptions thereof will be omitted as appropriate.

[0349] In Figure 24, object balls BL11 to BL14, which are the objects that make up the drum and have object names "Kick," "OH_L," "OH_R," and "Snare," are displayed in the track color "blue," which corresponds to track color number "1."

[0350] Additionally, the object ball BL15 of the object "EG" and the object ball BL16 of the object "Vo" are displayed in the track color "orange" corresponding to the track color number "3."

[0351] The object ball BL17 of the object "AG1" and the object ball BL18 of the object "Cho" are displayed in the track color "green" corresponding to the track color number "6", and the object ball BL19 of the object "AG2" is displayed in the track color "navy blue" corresponding to the track color number "22".

[0352] Furthermore, the object ball BL31 of the object "Amb_L" and the object ball BL32 of the object "Amb_R" are displayed in the track color "purple" corresponding to the track color number "9."

[0353] When displaying the POV image P61, the display control unit 42 displays the object ball of each object in the track color determined for the track color number based on the track color number designated (selected) for the track of each object.

[0354] By displaying the object balls of each object in the track color in this way, it is possible to easily determine which object ball corresponds to which object (track) even when there are a large number of objects.

[0355] Although the above description has been given of an example in which the object ball is displayed in the group color or the track color, the object ball may be displayed in the group color and the track color.

[0356] In such a case, for example, the display control unit 42 displays the center part of the object ball in the track color and the remaining part, i.e., the part of the object ball outside the part displayed in the track color, in the group color, so that the user can instantly determine which track the object corresponding to each object ball belongs to and which group that object belongs to.

[0357] In addition, the object ball may be displayed in a display format determined for information identifying the group, the track corresponding to the track color number, or a combination thereof, in addition to the color of the group or track, etc. Specifically, for example, the object ball may be displayed in a shape determined for the group.

[0358] <About mute and solo settings> As shown in FIG. 5, the edit screen is also provided with a mute button for making mute settings and a solo button for making solo settings.

[0359] The mute setting refers to muting the sound of a specified object when playing back audio content during editing of the audio content, i.e., not playing back (outputting) the sound of the object. Hereinafter, designating an object as one to be muted will also be referred to as turning on the mute setting, and the state in which the mute setting is on will also be referred to as the mute state.

[0360] In the information processing device 11, for an object that has been muted, the object ball of that object is hidden on the POV image. That is, the mute setting for the object is also reflected in the object ball on the POV image. Note that, when audio content data is output, the object data of the muted object may not be included in the audio content data.

[0361] In contrast, the solo setting refers to a setting that, when editing audio content, only the sound of a designated object is played (output) and the sounds of other objects are muted when the audio content is played back. Hereinafter, the designation of an object as an object whose sound is to be played back will also be referred to as "turning the solo setting on," and the state in which the solo setting is turned on will also be referred to as "the solo state."

[0362] In the information processing device 11, for an object that is set to a solo state, the object ball of that object is displayed on the POV image, and other objects that are not set to a solo state are hidden. That is, the solo setting for the object is also reflected in the object ball on the POV image. Note that when audio content data is output, only object data of objects that are set to a solo state may be included in the audio content data.

[0363] Furthermore, the mute setting and solo setting are configured so that when one setting is made, the other setting is disabled. That is, for example, when the mute setting is made, the solo setting is canceled, and conversely, when the solo setting is made, the mute setting is canceled.

[0364] In this way, by performing mute and solo settings, the object balls of muted objects for which no sound is played are hidden, and only the object balls of objects for which sound is played are displayed on the POV image, thereby improving usability.

[0365] That is, a muted object is an object that the user is not currently paying attention to, and an unmuted object is an object that the user is paying attention to.

[0366] Therefore, by displaying only the object balls of the objects that are not muted on the POV image, the user can easily grasp the transition of the position of the object of interest, etc. This improves the usability of the content production tool.

[0367] Here, specific examples of mute and solo settings will be described with reference to Figures 25 to 27. Note that in Figures 25 to 27, parts that correspond to those in Figures 5 or 24 are given the same reference numerals, and their explanation will be omitted where appropriate. Also, in Figures 25 to 27, parts that correspond to each other are given the same reference numerals, and their explanation will be omitted where appropriate.

[0368] For example, when neither solo nor mute settings have been made, object balls corresponding to all tracks are displayed in the POV image P71 as shown in Fig. 25. Note that in Fig. 25, only a part of the edit screen ED21 is displayed.

[0369] 25, the mute buttons for all object tracks, including the mute button MU21 for the track of object "Vo" and the mute button MU22 for the track of object "EG," are not operated. In other words, none of the objects are muted.

[0370] At the same time, the solo buttons for all object tracks, including the solo button SL21 for the track of the object "Vo" and the solo button SL22 for the track of the object "EG", are not operated. In other words, the solo state is not set for any object.

[0371] In this state, the object balls of all the objects are displayed in the POV image P71.

[0372] Here, the POV image P71 displays object balls BL11 to BL19, object ball BL31, and object ball BL32, each of which has the object names "Kick," "OH_L," "OH_R," "Snare," "EG," "Vo," "AG1," "Cho," "AG2," "Amb_L," and "Amb_R."

[0373] In this state, let us say that the user operates the input unit 21 to turn on the mute settings for the objects "Vo" and "EG" by clicking or otherwise operating the mute buttons MU21 and MU22 on the editing screen ED21. In other words, let us say that the objects "Vo" and "EG" are muted.

[0374] Then, on the edit screen ED21, as shown in FIG. 26, the operated mute buttons MU21 and MU22 are displayed in a color different from that before the operation.

[0375] For example, when a mute setting is made, the mute buttons of objects that are not muted are displayed in the same color as before the mute setting was made, and the mute buttons of objects that are muted are displayed in a different color than before the mute setting was made.

[0376] In the example of Fig. 26, the mute settings of the objects "Vo" and "EG" are turned on, and the mute settings of the other objects are not turned on. Therefore, the display control unit 42 controls the display unit 24 to update the display of the POV image P71 so that the POV image P71 shown on the right side of Fig. 26 is displayed on the display unit 24.

[0377] That is, in the example of Figure 26, in the POV image P71, the object ball BL15 of the object "EG" and the object ball BL16 of the object "Vo", which had been displayed until now but have been muted, are not displayed, i.e., are hidden.

[0378] In contrast, the object balls of the other objects that are not muted, namely, object balls BL11 to BL14, object balls BL17 to BL19, object ball BL31, and object ball BL32, remain displayed in the POV image P71.

[0379] 25, the user operates the input unit 21 to click or otherwise operate the solo buttons SL21 and SL22 on the editing screen ED21, thereby turning on the solo settings of the objects "Vo" and "EG." In other words, the objects "Vo" and "EG" are set to the solo state.

[0380] Then, on the edit screen ED21, as shown in FIG. 27, for example, the operated solo buttons SL21 and SL22 are displayed in a color different from that before the operation.

[0381] For example, when a solo setting is made, the solo button of an object whose solo setting is not turned on is displayed in the same color as before the solo setting was made, and the mute button of an object whose solo setting is turned on is displayed in a different color than before the solo setting was made.

[0382] In the example of Fig. 27, the solo settings of the objects "Vo" and "EG" are turned on, and the solo settings of the other objects are not turned on. Therefore, the display control unit 42 controls the display unit 24 to update the display of the POV image P71 so that the POV image P71 shown on the right side of Fig. 27 is displayed on the display unit 24.

[0383] That is, in the example of Figure 27, the solo setting is turned on in the POV image P71, and only the object ball BL15 and object ball BL16 corresponding to the objects "EG" and "Vo" that are in the solo state are displayed.

[0384] Therefore, the object balls of the other objects that have been displayed but are not in the solo state are erased and are no longer displayed. That is, in the POV image P71 of Fig. 27, object balls BL11 to BL14, object balls BL17 to BL19, object ball BL31, and object ball BL32 are not displayed.

[0385] In this way, by linking the object balls on the POV image with the mute and solo settings to display or hide them, the user can visually and easily understand which track corresponds to which object is in the mute or solo state, thereby improving usability.

[0386] <Explanation of the setting process> Next, the operation of the information processing device 11 when the user performs mute setting or solo setting will be described. That is, the setting process performed by the information processing device 11 will be described below with reference to the flowchart in Fig. 28. Note that when this setting process starts, an editing screen is displayed on the display unit 24.

[0387] In step S161, the control unit 23 determines, based on the signal supplied from the input unit 21, whether or not the mute button on the editing screen has been operated.

[0388] For example, when an operation such as a click is performed on the mute button MU21 or the mute button MU22 shown in FIG. 25, the control unit 23 determines that the mute button has been operated.

[0389] If it is determined in step S161 that the mute button has not been operated, the process of step S162 is not performed, and the process then proceeds to step S163.

[0390] On the other hand, if it is determined in step S161 that the mute button has been operated, then in step S162 the control unit 23 mutes the object (track) designated by the user's operation on the mute button.

[0391] Specifically, for example, as shown in Fig. 25, if the mute button MU21 is operated when the object "Vo" is not in the muted state, the control unit 23 mutes the object "Vo." Note that, for example, if the mute button MU21 is operated when the object "Vo" is in the muted state, the control unit 23 cancels the muted state of the object "Vo."

[0392] Once the mute setting has been performed in response to the operation of the mute button, the process proceeds to step S163.

[0393] If it is determined in step S161 that the mute button has not been operated, or if the process of step S162 is performed, the process of step S163 is performed.

[0394] In step S163, the control unit 23 determines whether or not a solo button on the editing screen has been operated, based on the signal supplied from the input unit 21. For example, the control unit 23 determines that a solo button has been operated when an operation such as a click is performed on the solo button SL21 or SL22 shown in FIG.

[0395] If it is determined in step S163 that the solo button has not been operated, the process of step S164 is not performed, and the process then proceeds to step S165.

[0396] On the other hand, if it is determined in step S163 that the solo button has been operated, then in step S164 the control unit 23 puts the object (track) designated by the user's operation on the solo button into a solo state.

[0397] Specifically, for example, as shown in Fig. 25, if the solo button SL21 is operated when the object "Vo" is not in the solo state, the control unit 23 puts the object "Vo" in the solo state. Note that, for example, if the solo button SL21 is operated when the object "Vo" is in the solo state, the control unit 23 cancels the solo state of the object "Vo."

[0398] Once solo settings have been made in accordance with the operation of the solo button in this way, the process then proceeds to step S165.

[0399] If the object to be muted or soloed belongs to a group, the control unit 23 may also set all other objects in the same group as that object to the muted or soloed state. In this case, the control unit 23 refers to the group information to determine whether the object to be processed belongs to a group, and then determines whether to perform the mute or solo setting on an object-by-object basis or on a group-by-group basis depending on the determination result.

[0400] If it is determined in step S163 that the solo button has not been operated or if the processing of step S164 is performed, in step S165 the display control unit 42 controls the display unit 24 in accordance with the mute setting or solo setting by the control unit 23, and updates the display of the editing screen and POV image.

[0401] Specifically, for example, if an object is muted by operating a mute button, the display control unit 42 changes the display format of the mute button in the track area of ​​the muted object on the editing screen, and also hides the object ball of the muted object on the POV image.

[0402] Once the mute setting or solo setting is reflected in the edit screen or POV image display in this way, the setting process ends.

[0403] In this way, the information processing device 11 performs mute setting or solo setting in response to the operation of the mute button or solo button, and reflects the setting contents on the edit screen and the display of the POV image. In this way, the user can easily understand which objects (tracks) are in the mute state or solo state, thereby improving usability.

[0404] About importing tracks Meanwhile, the information processing device 11 can import (take in) any audio signal file, that is, any audio file, as object data or channel audio data that constitutes audio content.

[0405] For example, the audio file to be imported may be an audio file recorded in the recording unit 22, an audio file received by the communication unit 25, or an audio file read from an external removable recording medium.

[0406] Specifically, for example, when importing an audio file recorded in the recording unit 22 as data constituting an audio content, the import can be performed by a drag-and-drop operation or the like as shown in FIG.

[0407] In the example shown in FIG. 29, an edit screen ED81 and a window WD81 displaying a list of audio files recorded in the recording unit 22 are displayed on the display unit 24.

[0408] For example, the user can issue a command to import an audio file by operating the input unit 21 to drag any audio file in the window WD81 and drop the audio file onto the editing screen ED81, as shown by arrow Q11. Note that the operation for specifying the audio file to import and issuing the command to import is not limited to a drag-and-drop operation, and may be any other operation, such as selecting (specifying) the desired audio file from a file menu.

[0409] When such a drag-and-drop operation is performed, the control unit 23 acquires the audio file designated by the user from the recording unit 22, and imports the acquired audio file as data constituting the audio content being edited.

[0410] In this example, a WAV format audio file with the file name "Congas.wav" is imported as audio content data.

[0411] For example, if the specified audio file is a monaural audio signal, i.e., a file of a one-channel audio signal, the control unit 23 may simply load the audio file as an audio signal constituting object data on the editing screen ED81. In other words, the control unit 23 may add the audio file as an audio signal of object data to the audio content data.

[0412] However, the specified audio file may contain multiple audio channels, such as two channels, i.e., a multi-channel file. In such cases, you must specify whether to import the specified audio file as object data for the number of channels, or as channel audio data.

[0413] Therefore, if the specified audio file is a multi-channel file, the display control unit 42 controls the display unit 24 to display, for example, a track type selection screen CO81 shown in FIG.

[0414] In the example shown in FIG. 30, the track type selection screen CO81 is provided with three buttons BT81 to BT83.

[0415] The button BT81 is operated when importing a specified audio file as object data, that is, as an object track.

[0416] The button BT82 is operated when importing the specified audio file as channel audio data, i.e., as a channel track, and the button BT83 is operated when canceling the import of the specified audio file.

[0417] Furthermore, the track type selection screen CO81 also displays a check box CB81 that is operated when importing a specified audio file as object data by adding object position information indicating a specific position.

[0418] In this example, the specified multi-channel file is a two-channel audio signal file, so the text message "set 2ch WAV(s) with L / R position (Azimuth +30 / -30)" is displayed to the right of checkbox CB81 in the illustration. The "L / R position (Azimuth +30 / -30)" text message indicates that horizontal angles of "30" and "-30" will be assigned as object position information. By looking at this display, the user can easily understand what kind of object position information will be assigned to the newly added object by importing.

[0419] In addition, the track type selection screen CO81 may also display check boxes or the like that allow the user to specify whether to import a specified audio file, i.e., the audio signals of multiple channels that make up a multi-channel file, as object data for multiple objects belonging to the same group.

[0420] Furthermore, for example, if the specified audio file is a multi-channel file consisting of two-channel audio signals, a check box may also be displayed on the track type selection screen CO81 that allows the user to specify whether to import those two-channel audio signals as L / R pair object data.

[0421] For example, suppose that the user operates the input unit 21 to operate (select) the button BT81 on the track type selection screen CO81 when no check mark is displayed in the check box CB81.

[0422] The control unit 23 then develops the specified audio file into a plurality of object tracks in accordance with the number of channels of the audio file.

[0423] Specifically, the control unit 23 reads out the audio signals of each channel constituting the specified multi-channel file from the recording unit 22 or the like based on the signal supplied from the input unit 21, and imports them as object data for each object. That is, each of the audio signals of the multiple channels is treated as an audio signal for each of the multiple objects. This generates new objects equal to the number of channels in the multi-channel file.

[0424] When the audio file is imported in this way, the display control unit 42 controls the display unit 24 in response to the execution of the import, and updates the display of the editing screen and POV image.

[0425] In the examples shown in Figures 29 and 30, when button BT81 is operated to import a two-channel audio file, the updated editing screen ED81 will look, for example, as shown in Figure 31. Note that in Figure 31, parts corresponding to those in Figure 29 are given the same reference numerals, and their explanation will be omitted where appropriate.

[0426] In this case, the audio file that was instructed to be imported, with the file name "Congas.wav", is a two-channel file, so the control unit 23 generates two objects, "Congas-0" and "Congas-1", through the import.

[0427] The display of the editing screen ED81 is updated so that a track area and a timeline area are provided for each track corresponding to these objects.

[0428] 31, the area TR81 and the area TM81 on the editing screen ED81 are the track area and the timeline area of ​​the track of the object "Congas-0." Similarly, the area TR82 and the area TM82 are the track area and the timeline area of ​​the track of the object "Congas-1."

[0429] If the specified audio file contains position information such as a sound image localization position, the position information may be used as object position information to generate meta information for the object.

[0430] On the other hand, if the specified audio file does not have position information, i.e., a position within the playback space, a predetermined position, such as a position in front of the listening position O, can be assigned as the position within the playback space of the object. In this case, the same object position information is assigned to each of multiple objects.

[0431] Additionally, the specified audio file may be a multi-channel file with a specific number of channels, such as 2 channels, 6 channels, or 8 channels.

[0432] In such a case, when the specified audio file is expanded as a track for multiple objects, it may be possible to reduce the editing effort by assigning a specific position to each of the multiple objects as an initial value.

[0433] For example, when an audio file is a two-channel file, the two-channel audio signals that make up the audio file are generally left and right channels, that is, an L channel audio signal and an R channel audio signal.

[0434] Therefore, coordinates (Azimuth, Elevation, Radius) = (30, 0, 1) and (-30, 0, 1), which are positions in a typical left / right (LR) channel arrangement, may be assigned to objects corresponding to these two channels as object position information indicating their positions in the playback space. The position indicated by these coordinates (30, 0, 1) and the position indicated by the coordinates (-30, 0, 1) are positions that are symmetrical left / right with respect to the above-mentioned reference plane in the playback space.

[0435] By assigning specific positions in this way, the user does not have to input object position information at each time for each object newly added by import.

[0436] Here, we will explain an example in which two objects added by importing a two-channel audio file are assigned positions indicated by coordinates (30,0,1) and (-30,0,1), respectively, but any other positions may be assigned.

[0437] Also, as in the case of two channels, when the audio file has six channels, the object position information of the six objects corresponding to those channels can be, for example, given coordinates (Azimuth, Elevation, Radius) = (30,0,1), (-30,0,1), (0,0,1), (0,-30,0), (110,0,1), and (-110,0,1).

[0438] Furthermore, when an audio file has eight channels, the object position information of the eight objects corresponding to those channels may be assigned coordinates such as (30,0,1), (-30,0,1), (0,0,1), (0,-30,0), (110,0,1), (-110,0,1), (30,30,1), and (-30,30,1).

[0439] A check box CB81 is provided on the track type selection screen CO81 so that object position information indicating a specific position in the playback space can be assigned as an initial value to an object newly added by import in this way.

[0440] For example, suppose an operation is performed to import a two-channel audio file with the file name "Congas.wav" as shown in Fig. 29. Then, when a track type selection screen CO81 is displayed on the display unit 24 in response to that operation, the user operates the input unit 21 to display a check mark in the check box CB81 as shown in Fig. 32, and then operates (selects) the button BT81. Note that in Fig. 32, parts corresponding to those in Fig. 30 are assigned the same reference numerals, and their description will be omitted as appropriate.

[0441] As shown in FIG. 32, when a check mark is displayed in the check box CB81 and then the button BT81 is operated, the control unit 23 expands the specified audio file into multiple object tracks according to the number of channels in the audio file.

[0442] In other words, just as when no check mark is displayed in the checkbox CB81 described above, the control unit 23 imports the audio signals of each channel that make up the specified two-channel audio file as the audio signals of each newly added object.

[0443] Furthermore, the position determination unit 41 assigns coordinates (30, 0, 1) as object position information to the object corresponding to the L channel of the two newly added objects. Similarly, the position determination unit 41 assigns coordinates (-30, 0, 1) as object position information to the object corresponding to the R channel of the two newly added objects.

[0444] When the audio file is imported in this way, the display control unit 42 controls the display unit 24 in response to the execution of the import, and updates the display of the editing screen and POV image.

[0445] In the examples shown in Figures 29 and 32, when button BT81 is operated to import a two-channel audio file, the updated editing screen and POV image will be as shown in Figures 33 and 34, respectively. Note that in Figure 33, parts corresponding to those in Figure 29 are given the same reference numerals, and their explanation will be omitted where appropriate.

[0446] In Figure 33, the audio file with the file name "Congas.wav" that was instructed to be imported is a two-channel file, so the control unit 23 generates two objects, object "Congas-L" and object "Congas-R", through the import.

[0447] The display of the editing screen ED81 is updated so that a track area and a timeline area are provided for each track corresponding to these objects.

[0448] That is, in Figure 33, the areas TR91 and TM91 of the editing screen ED81 are the track area and timeline area of ​​the track of the object "Congas-L", and in particular, the object position information of the object "Congas-L" at each time is coordinates (30,0,1).

[0449] Similarly, the area TR92 and the area TM92 are the track area and the timeline area of ​​the track of the object "Congas-R", and in particular, the object position information of the object "Congas-R" at each time is coordinates (-30, 0, 1).

[0450] Furthermore, the display control unit 42 causes the display unit 24 to display a POV image P91 shown in FIG. 34 as a POV image corresponding to the editing screen ED81 shown in FIG.

[0451] In Figure 34, object ball BL91 indicating the position of object "Congas-L" is placed on the front left side of the figure when viewed from listening position O, and object ball BL92 indicating the position of object "Congas-R" is placed on the front right side of the figure when viewed from listening position O.

[0452] If the audio file to be imported has a specific number of channels, the user can specify a specific position as the initial value for the object to be newly added by import, thereby reducing the time and effort required for the user to input object position information. This allows for more efficient and easier editing.

[0453] As described above, when importing an audio file, objects may be grouped or paired into L / R pairs.

[0454] <Import process explanation> Next, the operation of the information processing device 11 when importing the desired audio file described above, particularly when importing an audio file consisting of an audio signal that does not have a position in the reproduction space, will be described.

[0455] That is, the import processing by the information processing device 11 will be described below with reference to the flowchart in Fig. 35. This import processing starts when an import command is issued by an operation such as dragging and dropping a desired audio file as shown in Fig. 29, for example.

[0456] In step S191, the control unit 23 determines, based on the signal supplied from the input unit 21, whether or not the audio file instructed to be imported is a multi-channel file.

[0457] If it is determined in step S191 that the file is not a multi-channel file, that is, if an instruction to import a monaural audio file has been given, the process proceeds to step S192.

[0458] In step S192, the control unit 23 imports the specified audio file as one piece of object data.

[0459] For example, the control unit 23 imports one audio signal constituting a monaural audio file for which import is instructed as object data for one newly added object, i.e., the audio signal of the object. At this time, the control unit 23 appropriately adds object position information, gain information, priority information, etc. for a predetermined position to the audio signal to generate meta information, and generates object data consisting of the meta information and the audio signal.

[0460] Once the object data has been added in this way, the process then proceeds to step S199.

[0461] On the other hand, if it is determined in step S191 that the file is a multi-channel file, the display control unit 42 causes the display unit 24 to display a track type selection screen in step S193.

[0462] This causes the track type selection screen CO81 shown in Fig. 30 to be displayed. The user then operates the input unit 21 to appropriately operate the check boxes CB81, buttons BT81, and the like on the track type selection screen CO81.

[0463] In step S194, the control unit 23 determines whether or not to import the data as object data based on a signal supplied from the input unit 21 in response to a user's operation on the track type selection screen.

[0464] For example, when the button BT81 on the track type selection screen CO81 shown in FIG. 30 is operated, the control unit 23 determines in step S194 that the data is to be imported as object data.

[0465] If it is determined in step S194 that the data should not be imported as object data, that is, if the user has instructed to import the data as channel audio data, the process proceeds to step S195.

[0466] In step S195, the control unit 23 imports the specified audio file as one channel audio data. In this case, audio signals for each of the multiple channels are imported as one channel audio data, that is, one track data. After the channel audio data is added in this way, the process proceeds to step S199.

[0467] On the other hand, if it is determined in step S194 that the data is to be imported as object data, the process proceeds to step S196.

[0468] In step S196, the control unit 23 imports the designated audio file as object data of the number of objects corresponding to the number of channels of the audio file.

[0469] For example, the control unit 23 imports audio signals of multiple channels that make up the audio file for which import is instructed as audio signals that make up object data of multiple objects corresponding to those channels. That is, objects equal to the number of channels in the audio file are generated, and those objects are added to the audio content.

[0470] In step S197, the position determining unit 41 determines whether or not to assign a specific position within the reproduction space to the object generated in step S196.

[0471] For example, as shown in FIG. 32, when the button BT81 is operated in a state where a check mark is displayed in the check box CB81 on the track type selection screen CO81, it is determined in step S197 that a specific position is to be assigned.

[0472] If it is determined in step S197 that a specific position is not to be assigned, the process of step S198 is not performed, and the process then proceeds to step S199.

[0473] In this case, the position determination unit 41 assigns a predetermined position, such as a front position, within the reproduction space to the object newly added in the processing of step S196.

[0474] That is, the position determination unit 41 generates meta information including object position information indicating a predetermined position for each of the newly added objects, and generates object data consisting of the meta information and an audio signal. In this case, the same position is assigned to all of the newly added objects.

[0475] On the other hand, if it is determined in step S197 that a specific position should be assigned, in step S198 the position determination unit 41 assigns a specific position in the playback space to each of the objects newly added in the processing of step S196.

[0476] That is, for example, the position determination unit 41 generates meta information including object position information indicating a specific position that differs for each of the newly added multiple objects, and generates object data consisting of the meta information and an audio signal.

[0477] Specifically, for example, if two new objects are added, one object is assigned a position indicated by coordinates (30,0,1), as in the example above, and the other object is assigned a position indicated by coordinates (-30,0,1). In particular, each object is assigned a different position, such as a position that is symmetrical between the left and right. The specific position assigned to each object is the position determined for each channel of the audio file specified for import. In other words, each object is assigned a specific position according to the number of channels in the audio file to be imported.

[0478] By assigning specific positions in this way, the user does not have to input the object position information of each newly added object, making it easier to set the object position information, thereby improving editing efficiency.

[0479] When new objects are added by importing, the control unit 23 may group the objects. In this case, the grouping may be performed in accordance with a user instruction, or when multiple new objects are added simultaneously without any particular user instruction, the objects may be unconditionally grouped. Furthermore, when two newly added objects are added, the two objects may be grouped as an L / R pair in accordance with a user instruction or the like.

[0480] When grouping is performed, it can be said that the control unit 23 groups multiple objects that do not have a position in the playback space and assigns positions in the playback space to these grouped multiple objects.

[0481] In particular, when there are two objects to be grouped, the two objects can be assigned positions in the reproduction space so that they have a positional relationship that is symmetrical left and right with respect to a predetermined reference plane in the reproduction space.

[0482] Once a specific position is assigned to the object in step S198, the process then proceeds to step S199.

[0483] If the process of step S192, step S195, or step S198 has been performed, or if it is determined in step S197 that a specific position will not be assigned, the process of step S199 is performed.

[0484] In step S199, the display control unit 42 controls the display unit 24 in response to the import of the audio file, updates the display of the editing screen and POV image displayed on the display unit 24, and the import process ends.

[0485] For example, in step S199, the edit screen and POV image display are updated as shown in FIGS.

[0486] As described above, the information processing device 11 imports audio files and adds new object data and the like in accordance with the number of channels in the audio files and the user's operations on the track type selection screen.

[0487] By importing audio appropriately according to the number of channels in the audio file and the user's operations, the user's effort of inputting object position information, etc. is reduced, and editing can be performed more efficiently and easily.

[0488] <Example of computer configuration> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, for example, that can execute various functions by installing various programs.

[0489] FIG. 36 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes using a program.

[0490] In the computer, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503 are interconnected by a bus 504.

[0491] An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

[0492] The input unit 506 includes a keyboard, a mouse, a microphone, an image sensor, etc. The output unit 507 includes a display, a speaker, etc. The recording unit 508 includes a hard disk, a non-volatile memory, etc. The communication unit 509 includes a network interface, etc. The drive 510 drives a removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0493] In a computer configured as described above, the CPU 501 performs the above-described series of processes by, for example, loading a program recorded in the recording unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 and executing it.

[0494] The program executed by the computer (CPU 501) can be provided by being recorded on a removable recording medium 511 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0495] In a computer, a program can be installed in the recording unit 508 via the input / output interface 505 by inserting a removable recording medium 511 into the drive 510. The program can also be received by the communication unit 509 via a wired or wireless transmission medium and installed in the recording unit 508. Alternatively, the program can be installed in the ROM 502 or the recording unit 508 in advance.

[0496] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0497] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0498] For example, this technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0499] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by multiple devices.

[0500] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0501] Furthermore, the present technology can also be configured as follows.

[0502] (1) a control unit that selects and groups a plurality of objects that exist in a predetermined space, and changes the positions of the plurality of grouped objects while maintaining the relative positional relationship of the plurality of grouped objects in the space; Information processing device. (2) The control unit groups the plurality of objects that do not have positions in the space, and assigns positions in the space to the plurality of grouped objects. An information processing device according to (1). (3) When the two objects are grouped, the control unit changes the positions of the two objects while maintaining a relationship in which the two objects are symmetrical with respect to a predetermined plane in the space. An information processing device according to (1). (4) The control unit groups two of the objects that do not have positions in the space, and assigns positions in the space to the two grouped objects so that the two grouped objects have a positional relationship that is symmetrical with respect to a predetermined plane in the space. An information processing device according to (1). (5) The control unit groups the plurality of objects having positions in the space. An information processing device according to (1). (6) The control unit obtains, by interpolation processing, a position of the object at a time between the predetermined time and another time, based on a position of the object at the predetermined time and a position of the object at another time different from the predetermined time. An information processing device according to any one of (1) to (5). (7) The control unit performs the interpolation process using an interpolation method selected from a plurality of interpolation methods. (6) An information processing device according to the present invention. (8) When a plurality of different time positions of the object are selected and a change of the positions is instructed, the control unit simultaneously changes the positions of the selected plurality of time positions by a specified change amount. An information processing device according to any one of (1) to (7). (9) a display control unit that controls display of an image of the space in which the object is placed, with a predetermined position in the space as a viewpoint position; An information processing device according to any one of (1) to (8). (10) The display control unit displays the objects belonging to the same group in the same color on the image. (9) An information processing device according to (9). (11) The display control unit displays the object on the image in a color selected for the audio track corresponding to the object. (9) An information processing device according to (9). (12) The display control unit displays the object on the image in a color selected for the audio track corresponding to the object and a color determined for the group to which the object belongs. (9) An information processing device according to (9). (13) The display control unit displays only the designated object among the plurality of objects present in the space on the image. An information processing device according to any one of (9) to (12). (14) The object is an audio object An information processing device according to any one of (1) to (13). (15) The information processing device A plurality of objects existing in a predetermined space are selected and grouped, and the positions of the plurality of grouped objects are changed while maintaining the relative positional relationship of the plurality of grouped objects in the space. Information processing methods. (16) A plurality of objects existing in a predetermined space are selected and grouped, and the positions of the plurality of grouped objects are changed while maintaining the relative positional relationship of the plurality of grouped objects in the space. A program that causes a computer to execute a process that includes steps. [Explanation of symbols]

[0503] 11 information processing device, 21 input unit, 23 control unit, 24 display unit, 41 position determination unit, 42 display control unit

Claims

1. for a plurality of grouped objects existing in a predetermined space, changing positions of the plurality of grouped objects while maintaining a relative positional relationship with respect to a predetermined area in the space including a listening position, based on an operation on one of the plurality of grouped objects; In response to a user's designation, a mute setting or a solo setting is performed on the plurality of grouped objects, and when the mute setting is performed, a display state of the plurality of grouped objects that have been muted by the mute setting is changed, and when the solo setting is performed, a display state of the plurality of grouped objects that have been soloed by the solo setting is changed. Information processing methods.

2. Controlling the display state of the object to a first display state or a second display state; When the mute setting is performed, the display state of the grouped objects that have been muted by the mute setting is set to the second display state; When the solo setting is performed, the display state of the grouped objects that are set to the solo state by the solo setting is set to the first display state. The information processing method according to claim 1 .

3. When the solo setting is performed, the display state of the objects other than the plurality of grouped objects that are set to the solo state by the solo setting is set to the second display state. The information processing method according to claim 2 .

4. When the mute setting is performed, the display state of the object that is not in the muted state is set to the first display state. The information processing method according to claim 2 .

5. The grouped plurality of objects including the object designated by the user are muted. The information processing method according to claim 1 .

6. The grouped plurality of objects including the object designated by the user are set to the solo state. The information processing method according to claim 1 .

7. The first display state is a display state in which the object is displayed, and the second display state is a display state in which the object is not displayed. The information processing method according to claim 2 .

8. When one of the mute setting and the solo setting is set, the other setting is canceled. The information processing method according to any one of claims 1 to 7.

9. The object is an audio object The information processing method according to any one of claims 1 to 7.

10. The sound of the object that has been muted is muted, or the data of the object that has been muted is not output as content data. The information processing method according to claim 9.

11. Only the sound of the object that has been put into the solo state is reproduced, or only the data of the object that has been put into the solo state is output as content data. The information processing method according to claim 9.

12. a control unit that, for a plurality of grouped objects existing in a predetermined space, changes positions of the plurality of grouped objects while maintaining a relative positional relationship with respect to a predetermined area including a listening position in the space, based on an operation on one of the plurality of grouped objects, and performs a mute setting or a solo setting on the plurality of grouped objects in accordance with a user's instruction; a display control unit that, when the mute setting is performed, changes a display state of the plurality of grouped objects that have been set to a muted state by the mute setting, and, when the solo setting is performed, changes a display state of the plurality of grouped objects that have been set to a solo state by the solo setting; An information processing device comprising:

13. for a plurality of grouped objects existing in a predetermined space, changing positions of the plurality of grouped objects while maintaining a relative positional relationship with respect to a predetermined area in the space including a listening position, based on an operation on one of the plurality of grouped objects; In response to a user's designation, a mute setting or a solo setting is performed on the plurality of grouped objects, and when the mute setting is performed, a display state of the plurality of grouped objects that have been muted by the mute setting is changed, and when the solo setting is performed, a display state of the plurality of grouped objects that have been soloed by the solo setting is changed. A program that causes a computer to execute a process that includes steps.

14. For a plurality of grouped objects present in a predetermined space, control reception of first operation information regarding an operation on one of the plurality of grouped objects; generating a first setting control signal for changing positions of the plurality of objects while maintaining a relative positional relationship with respect to a predetermined area including a listening position in the space in response to the first operation information, and controlling transmission of the first setting control signal; Controlling reception of second operation information relating to a mute setting or solo setting of the plurality of grouped objects by a user; generating a second setting control signal for controlling a mute setting or a solo setting for the plurality of grouped objects in accordance with the second operation information, and generating a display control signal for changing a display state of the plurality of grouped objects in accordance with the mute setting or the solo setting; Controlling transmission of the second setting control signal and the display control signal A control unit is provided, When the mute setting is performed, the display state of the grouped objects that are muted by the mute setting is changed, When the solo setting is performed, the display state of the grouped objects that are set to the solo state by the solo setting is changed. Information processing device.

15. receiving first operation information regarding an operation on one of a plurality of grouped objects that exist in a predetermined space; generating a first setting control signal for changing positions of the plurality of objects while maintaining a relative positional relationship with respect to a predetermined area including a listening position in the space in response to the first operation information, and transmitting the first setting control signal; receiving second operation information relating to a mute setting or a solo setting of the plurality of grouped objects by a user; generating a second setting control signal for controlling a mute setting or a solo setting for the plurality of grouped objects in accordance with the second operation information, and generating a display control signal for changing a display state of the plurality of grouped objects in accordance with the mute setting or the solo setting; Transmitting the second setting control signal and the display control signal including the steps When the mute setting is performed, the display state of the grouped objects that are muted by the mute setting is changed, When the solo setting is performed, the display state of the grouped objects that are set to the solo state by the solo setting is changed. Information processing methods.

16. For a plurality of grouped objects present in a predetermined space, controlling transmission of first operation information regarding an operation on one of the plurality of grouped objects, controlling reception of a first setting control signal transmitted in response to the transmission of the first operation information, and changing positions of the plurality of objects in response to the first setting control signal while maintaining a relative positional relationship with respect to a predetermined area in the space including a listening position; Controlling transmission of second operation information relating to mute setting or solo setting of the plurality of grouped objects by a user, controlling reception of a second setting control signal and a display control signal transmitted in response to the transmission of the second operation information, and performing mute setting or solo setting of the plurality of grouped objects in response to the second setting control signal. A control unit; a display control unit that changes a display state of the object in response to the display control signal; Equipped with The display control unit When the mute setting is performed, changing the display state of the plurality of grouped objects that have been muted by the mute setting; When the solo setting is performed, the display state of the grouped objects that are set to the solo state by the solo setting is changed. Information processing device.

17. For a plurality of grouped objects present in a predetermined space, transmitting first operation information regarding an operation on one of the plurality of grouped objects; receiving a first setting control signal transmitted in response to the transmission of the first operation information; changing positions of the plurality of objects while maintaining a relative positional relationship with respect to a predetermined area in the space including a listening position in response to the first setting control signal; transmitting second operation information regarding a mute setting or a solo setting by a user for the plurality of grouped objects; receiving a second setting control signal and a display control signal transmitted in response to the transmission of the second operation information; performing a mute setting or a solo setting for the plurality of grouped objects in response to the second setting control signal; Changing the display state of the object in response to the display control signal including the steps When the mute setting is performed, changing the display state of the plurality of grouped objects that have been muted by the mute setting; When the solo setting is performed, the display state of the grouped objects that are set to the solo state by the solo setting is changed. Information processing methods.

18. An information processing system having a first information processing device and a second information processing device, The first information processing device for a plurality of grouped objects existing in a predetermined space, controlling transmission of first operation information regarding an operation on one of the plurality of grouped objects to the second information processing device, controlling reception of a first setting control signal transmitted from the second information processing device in response to the transmission of the first operation information, and changing positions of the plurality of objects in response to the first setting control signal while maintaining a relative positional relationship with respect to a predetermined area in the space including a listening position; Controlling transmission of second operation information relating to a mute setting or a solo setting of the plurality of grouped objects by a user to the second information processing device, controlling reception of a second setting control signal and a display control signal transmitted from the second information processing device in response to the transmission of the second operation information, and performing a mute setting or a solo setting of the plurality of grouped objects in response to the second setting control signal. a first control unit; a display control unit that changes a display state of the object in response to the display control signal; Equipped with The display control unit When the mute setting is performed, changing the display state of the plurality of grouped objects that have been muted by the mute setting; When the solo setting is performed, changing the display state of the plurality of grouped objects that are set to a solo state by the solo setting; The second information processing device controlling reception of the first operation information transmitted from the first information processing device, generating the first setting control signal in response to the first operation information, and controlling transmission of the first setting control signal to the first information processing device; Controlling reception of the second operation information transmitted from the first information processing device, generating the second setting control signal in response to the second operation information, generating the display control signal in response to the mute setting or the solo setting, and controlling transmission of the second setting control signal and the display control signal to the first information processing device. A second control unit is provided. Information processing system.

Images