Information Processing Method and Information Processing Apparatus

Abstract

To provide an information processing method and an information processing device for achieving the sound image localization taking into account a shape of a physical space.SOLUTION: An information processing method includes the steps of: accepting the setting of the first spatial information and the second spatial information; receiving a first sound image localization information showing a position where the sound image is localized in a first coordinate in the first spatial information; and converting the first sound image localization information into the second sound image localization information corresponding a second coordinate in the second spatial information.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] One embodiment of the present invention relates to an information processing method and an information processing apparatus.

Background Art

[0002] The audio adjustment desk of Patent Document 1 receives the coordinates of the audio localization point in a rectangular parallelepiped-shaped space. The audio adjustment desk of Patent Document 1 calculates the volume of the sound output from a plurality of speakers arranged in the space so that the sound image is localized at the received coordinates.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the physical space such as a concert hall is not limited to a rectangular parallelepiped shape. Therefore, even if a device such as the audio adjustment desk of Patent Document 1 receives the coordinates of the audio localization point in a rectangular parallelepiped space, it may not be able to localize the sound image at the position as intended by the user because it does not consider the coordinates of the physical space.

[0005] Therefore, one of the objects of one embodiment of the present invention is to provide an information processing method and an information processing apparatus that realize audio localization considering the shape of the physical space.

Means for Solving the Problems

[0006] An information processing method according to an embodiment of the present invention receives settings of first space information and second space information, receives first audio localization information indicating a position where a sound image is to be localized at a first coordinate in the first space information, and converts the first audio localization information into second audio localization information corresponding to a second coordinate in the second space information.

Advantages of the Invention

[0007] According to an embodiment of the present invention, it is possible to realize sound image localization considering the shape of a physical space.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Mode for Carrying Out the Invention

[0009] FIG. 1 is a block diagram showing the configuration of the information processing apparatus 1. The information processing apparatus 1 includes a communication unit 11, a processor 12, a RAM 13, a flash memory 14, a display 15, and a user I / F 16.

[0010] The information processing apparatus 1 is composed of a personal computer, a smartphone, a tablet computer, or the like. Also, an audio device such as an audio mixer is an example of the information processing apparatus.

[0011] The communication unit 11 communicates with other devices such as a server. The communication unit 11 has a wireless communication function such as Bluetooth (registered trademark) or Wi-Fi (registered trademark), and a wired communication function such as USB or LAN. The communication unit 11 acquires, for example, space information indicating the shape of a physical space such as a concert hall. The space information is information indicating two-dimensional or three-dimensional coordinates with a certain position as a reference point (origin). The space information is information including two-dimensional or three-dimensional coordinates such as CAD data indicating the shape of a physical space such as a concert hall.

[0012] The processor 12 consists of a CPU, a DSP, or a SoC (System on a Chip), etc. The processor 12 reads a program from the flash memory 14 which is a storage medium, and temporarily stores it in the RAM 13 to perform various operations. The processor 12 realizes functional components such as a space setting unit 141, an audio-visual localization information reception unit 142, and a conversion unit 143 according to the read program. Note that the program does not necessarily have to be stored in the flash memory 14. The processor 12 may, for example, download it from another device such as a server when necessary and temporarily store it in the RAM 13.

[0013] The display 15 consists of an LCD, etc. The display 15 displays, for example, an audio-visual localization setting screen as shown in FIG. 2.

[0014] The user I / F 16 is an example of an operation unit. The user I / F 16 consists of a mouse, a keyboard, or a touch panel, etc. The user I / F 16 receives the operations of the user. Note that the touch panel may be laminated on the display 15.

[0015] Referring to FIGS. 2 and 3, the audio-visual localization setting screen will be described. FIG. 2 is a diagram showing an example of the audio-visual localization setting screen displayed on the display 15. FIG. 3 is a flowchart showing the operation of the processor 12. The audio-visual localization setting screen shown in FIG. 2 is an example of a content editing screen. The user edits the audio-visual localization position of the sound source included in the content on the audio-visual localization setting screen.

[0016] The display 15 displays a logical space image 151 in a logical coordinate system and a physical space image 152 in a physical coordinate system. In this example, the display 15 displays the logical space image 151 in the upper left of the screen and the physical space image 152 in the upper right of the screen. Also, the display 15 displays a logical plane image 153 in the lower left of the screen and a physical plane image 154 in the lower right of the screen.

[0017] The logical space image 151 is, for example, in the shape of a rectangular parallelepiped. The logical plane image 153 corresponds to a view of the logical plane image 153 in plan view. The physical space image 152 is, for example, in the shape of an octagonal prism. The physical plane image 154 corresponds to a view of the physical space image 152 in plan view.

[0018] First, the space setting unit 141 of the processor 12 receives settings of first space information, which is information corresponding to a logical space, and second space information, which is information corresponding to a physical space such as a concert hall (S11).

[0019] The first space information is logical coordinates. The logical coordinates are, for example, normalized coordinates from 0 to 1. In the present embodiment, the space setting unit 141 receives settings of the space information of a rectangular parallelepiped as the first space information, but may also receive various space information such as a pyramid, a prism, a polyhedron, a cylinder, a cone, or a sphere. Further, the space setting unit 141 may receive information of a two-dimensional space. The two-dimensional space includes, for example, a polygon composed of straight lines, a circle composed of curves, or a composite shape composed of straight lines and curves.

[0020] The second space information is physical coordinates. The physical coordinates are two-dimensional or three-dimensional coordinates included in CAD data or the like indicating the shape of a physical space such as a concert hall. The space setting unit 141 of the processor 12 receives settings of the second space information, for example, by reading information including two-dimensional or three-dimensional coordinates such as CAD data from the flash memory 14.

[0021] Next, the space setting unit 141 generates the logical space image 151, the physical space image 152, the logical plane image 153, and the physical plane image 154, and displays them on the display 15 (S12). In the example of FIG. 2, the logical space image 151 is an image of an elevation view of a cube shape, and the logical plane image 153 is a square-shaped image. The physical space image 152 and the physical plane image 154 are images simulating an actual space such as a concert hall. The space setting unit 141 generates the physical space image 152 and the physical plane image 154 based on information including two-dimensional or three-dimensional coordinates such as CAD data.

[0022] Next, the sound image localization information receiving unit 142 of the processor 12 receives the speaker arrangement information or the sound image localization information (S13). The speaker arrangement information and the sound image localization information are each coordinates in a logical coordinate system and are examples of the first sound image localization information.

[0023] The user operates the user I / F 16 to edit the speaker arrangement information or the sound image localization information in the logical space image 151 or the logical plane image 153 shown in FIG. 2. For example, in the example of FIG. 2, the user arranges the center speaker 50C, the left speaker 50L, the right speaker 50R, the left rear speaker 50SL, and the right rear speaker 50SR in the logical space image 151 and the logical plane image 153. The center speaker 50C, the left speaker 50L, the right speaker 50R, the left rear speaker 50SL, and the right rear speaker 50SR are arranged in the middle in the height direction.

[0024] Assuming that the position of the upper left vertex in the logical plane image 153 is the origin, the coordinates of the left speaker 50L are (x, y) = (0, 0). The coordinates of the right speaker 50R are (x, y) = (1, 0). The coordinates of the center speaker 50C are (x, y) = (0.5, 0). The coordinates of the left rear speaker 50SL are (x, y) = (0, 1). The coordinates of the right rear speaker 50SR are (x, y) = (1, 1).

[0025] Also, in the example of FIG. 2, the user arranges the sound image localization position of the sound source 55 in the logical space image 151 and the logical plane image 153 to the left of the center (between the left end and the center). That is, the coordinates of the sound source 55 are (x, y) = (0.25, 0.5).

[0026] In the example of FIG. 2, the height direction coordinates of the center speaker 50C, the left speaker 50L, the right speaker 50R, the left rear speaker 50SL, the right rear speaker 50SR, and the sound source are all z = 0.5.

[0027] The audio-visual positioning information receiving unit 142 receives the speaker arrangement information or the sound source position information by receiving an operation of editing, for example, the speaker arrangement information of the user or the audio-visual positioning information of the sound source as shown in FIG. 2 (S13: Yes).

[0028] The conversion unit 143 performs coordinate conversion based on the received speaker arrangement information or sound source position information (S14).

[0029] FIG. 4 and FIG. 5 are diagrams for explaining the concept of coordinate conversion. The conversion unit 143 converts the speaker arrangement information and the sound source position information from the first coordinates of the first space information in the logical coordinate system to the second coordinates of the second space information in the physical coordinate system. In the example of FIG. 4, in the physical coordinate system, there are eight reference points 70A(x1, y1), 70B(x2, y2), 70C(x3, y3), 70D(x4, y4), 70E(x5, y5), 70F(x6, y6), 70G(x7, y7), 70H(x8, y8), and eight reference points 70A(0, 0), 70B(0.25, 0), 70C(0.75, 0), 70D(1, 0), 70E(0, 1), 70F(0.25, 1), 70G(0.75, 1), 70H(1, 1) in the logical coordinate system before conversion. The conversion unit 143 obtains the centroid G of the eight reference points in the logical coordinate system before conversion and the centroid G' of the eight reference points in the physical coordinate system after conversion, and generates a triangular mesh centered on these centroids. The conversion unit 143 converts the internal space of the triangle in the logical coordinate system and the internal space of the triangle in the physical coordinate system by a predetermined coordinate conversion. The conversion uses, for example, an affine transformation. The affine transformation is an example of a geometric transformation. The affine transformation represents the x coordinate (x') and y coordinate (y') after conversion as functions of the x coordinate (x) and y coordinate (y) before conversion, respectively. That is, the affine transformation performs coordinate conversion using the equations x' = ax + by + c and y' = dx + ey + f. From the coordinates of the three vertices of the triangle before conversion and the coordinates of the three vertices of the triangle after conversion, the coefficients a to f can be uniquely obtained. The conversion unit 143 obtains the affine transformation coefficients in the same way for all triangles, thereby converting from the first coordinates in the logical coordinate system to the second coordinates of the second space information in the physical coordinate system. Note that the coefficients a to f may be obtained by the least squares method.

[0030] Then, the conversion unit 143 converts the coordinates of the speaker arrangement information and the sound source position information using the obtained coefficients a to f. In FIG. 5, the conversion unit 143 converts the coordinates (x, y) in the logical coordinate system of the sound source 55 into the coordinates (x', y') in the physical coordinate system using the above formula.

[0031] As a result, the coordinates of the speaker arrangement information and the sound source position information are converted into second sound image localization information that matches the shape of the physical space. The processor 12 stores the second sound image localization information in, for example, the flash memory 14. Alternatively, the processor 12 transmits the second sound image localization information to another device such as an audio device via, for example, the communication unit 11. The audio device performs a process of localizing the sound image based on the received second sound image localization information. The audio device calculates the level balance of the sound signals output to a plurality of speakers so that the sound image of the sound source is localized at the specified position based on the speaker arrangement information and the sound source position information included in the second sound image localization information, and adjusts the level of the sound signals. Therefore, the information processing apparatus 1 of the present embodiment can realize sound image localization considering the shape of the physical space.

[0032] Note that the above mesh may be a polygon mesh other than a triangle or a combination thereof. For example, as shown in FIG. 9, the conversion unit 143 may generate a quadrilateral mesh and perform coordinate conversion. The conversion method is not limited to the above affine conversion. For example, the conversion unit 143 may convert the quadrilateral mesh based on the following formula and convert the coordinates (x, y) in the logical coordinate system of the sound source 55 into the coordinates (x', y') in the physical coordinate system (where x0, y0, x1, y1, x2, y2, x3, y3 are the coordinates of the conversion points, respectively). x’ = x0 + (x1 - x0)x + (x3 - x0)y + (x0 - x1 + x2 - x3)xy y’ = y0 + (y1 - y0)x + (y3 - y0)y + (y0 - y1 + y2 - y3)xy The conversion method may also be other geometric conversions such as, for example, an isometric mapping, a similarity transformation, or a projective transformation. For example, a projective transformation is represented by the equations x’=(ax + by + c) / (gx + hy + 1) and y’=(dx + ey + f) / (gx + hy + 1). The coefficients are obtained in the same manner as the above-described affine transformation. For example, the eight coefficients (a to h) constituting the projective transformation of a quadrilateral can be uniquely obtained by an eight simultaneous equations. Or, the coefficients may be obtained, for example, by the least squares method.

[0033] FIG. 6 is a block diagram showing the configuration of the information processing apparatus 1A according to Modification 1. FIG. 7 is a flowchart showing the operation of the information processing apparatus 1A. The same configurations, functions, and operations as those of the information processing apparatus 1 are denoted by the same reference numerals, and the description thereof is omitted.

[0034] The information processing apparatus 1A further includes an audio I / F 17. The audio I / F 17 includes an analog audio terminal or a digital audio terminal or the like. The processor 12 acquires a sound signal of a sound source via the audio I / F 17. Thereby, the processor 12 functions as a sound signal acquisition unit. Also, the sound signal may be acquired from an external device via the communication unit 11. Also, the sound signal may be stored in the flash memory 14.

[0035] The audio I / F 17 is connected to a center speaker 50C, a left speaker 50L, a right speaker 50R, a left rear speaker 50SL, and a right rear speaker 50SR installed in an actual space such as a concert hall.

[0036] The processor 12 includes a DSP. The processor 12 performs a predetermined signal processing on the sound signal. The processor 12 outputs the sound signal after the signal processing to the center speaker 50C, the left speaker 50L, the right speaker 50R, the left rear speaker 50SL, and the right rear speaker 50SR via the audio I / F 17.

[0037] The processor 12 further realizes the functional configuration of the sound source localization processing unit 144 by reading the program stored in the flash memory 14 into the RAM 13. The sound source localization processing unit 144 of the processor 12 performs a process of sound image localizing the sound signal to a position corresponding to the second sound image localization information based on the speaker arrangement information and the sound source position information (second sound image localization information) converted by the conversion unit 143 (S15). That is, the sound source localization processing unit 144 calculates the level balance of the sound signals output to the center speaker 50C, the left speaker 50L, the right speaker 50R, the left rear speaker 50SL, and the right rear speaker 50SR so that the sound image of the sound source is localized at the designated position based on the speaker arrangement information and the sound source position information included in the second sound image localization information, and adjusts the level of the sound signals. In this way, the information processing apparatus may perform sound image localization processing.

[0038] In FIGS. 2 to 5, the coordinate conversion in the two-dimensional space (plane) is shown. However, the information processing apparatus may perform coordinate conversion in the three-dimensional space. In this case, the converted coordinates x', y', z' are each represented as a function of x, y, z. The conversion unit 143 converts the speaker arrangement information and the sound source position information based on the function.

[0039] Also, the information in the three-dimensional space may be in a form including the planar coordinates (x, y) and information indicating a plurality of layers in the height direction.

[0040] FIG. 8 is a diagram showing the concept of layers. The user operates the user I / F 16 to edit the layers, the speaker arrangement information, or the sound image localization information of the sound source. In the example of FIG. 8, the user designates the heights of three layers arranged in the height direction. Also, the user designates the arrangement of the speakers or the sound image of the sound source in any of the designated layers. In the example of FIG. 8, the user designates the layers 151L1, 151L2, 151L3, 152L1, 152L2, and 152L3, and designates the arrangement of the speakers or the sound image of the sound source from these layers.

[0041] The conversion unit 143 converts the coordinates (x’, y’) on the plane of the physical coordinate system through geometric conversion as described above. The height coordinate is specified by the user. In the example of FIG. 8, for the layer 151L1 in the logical coordinate system, z = 1.0, for the layer 151L2, it is the coordinate of z = 0.5, and for the layer 151L3, z = 0. The layer 152L1 in the physical coordinate system corresponds to the highest position in the actual space, that is, the coordinates of the ceiling surface. The layer 152L3 in the physical coordinate system corresponds to the lowest position in the actual space, that is, the coordinates of the floor surface. The layer 152L2 is the coordinate between the coordinates of the ceiling surface and the floor surface. For example, when the sound source 55 is arranged in the layer 151L3 of the logical coordinate system, the conversion unit 143 obtains the height direction coordinate z3 of the layer 152L3 in the physical coordinate system as the height information of the second sound image localization information.

[0042] Also, either the speaker arrangement information or the sound source position information may specify the coordinates between multiple layers. For example, the speaker arrangement information may be specified by layers, and the sound source position information may specify a free position in the three-dimensional space. In this case, the conversion unit 143 obtains the sound source position information based on the height information of multiple layers. The conversion unit 143 obtains the coordinates of the physical coordinate system by, for example, linear interpolation. For example, when the sound source is between the layer 151L1 and the layer 151L2, the conversion unit 143 obtains the converted sound source coordinate z’ from the coordinate z of the sound source before conversion as follows.

[0043] z’=(z - z1)*(z’2 - z’1) / (z2 - z1)+z’1 Of course, the number of layers is not limited to three. The number of layers may be two or four or more.

[0044] The description of this embodiment should be considered illustrative in all respects and not restrictive. The scope of the present invention is shown not by the above-described embodiments but by the scope of the claims. Furthermore, the scope of the present invention includes the scope equivalent to the scope of the claims.

[0045] For example, the user may edit the speaker arrangement information and the sound image localization information of the sound source in the physical space image 152 or the physical plane image 154. In this case, the space setting unit 141 receives the space information in the physical coordinate system as the first space information and receives the space information in the logical coordinate system as the second space information. The conversion unit 143 converts the speaker arrangement information and the sound source position information (first sound image localization information) in the physical coordinate system into the speaker arrangement information and the sound source position information (second sound image localization information) in the logical coordinate system.

[0046] Note that the number of sound sources is not limited to one. FIG. 10 is a diagram showing an example of a sound image localization setting screen displayed on the display 15 when editing the sound image localization position information of a plurality of sound sources 55A and 55B. In the example of FIG. 10, the information processing apparatus 1 or the information processing apparatus 1A displays the logical plane image 153 and the physical plane image 154, but actually also displays the logical space image 151 and the physical space image 152. The operation of the information processing apparatus 1 is the same as the flowchart shown in FIG. 3, and the operation of the information processing apparatus 1A is the same as the flowchart shown in FIG. 7.

[0047] In this example, the user arranges the sound image localization positions of the sound sources 55A and 55B in the logical plane image 153 and the physical plane image 154. The coordinates of the sound source 55A are (x1, y1) = (0.25, 0.5). The coordinates of the sound source 55B are (x2, y2) = (0.25, 0.25).

[0048] The user edits the sound sources 55A and 55B arranged in the logical plane image 153 or the physical plane image 154 respectively. For example, the user changes the sound sources 55A and 55B arranged in the physical plane image 154 to different positions respectively. The conversion unit 143 converts the sound source position coordinates (first sound image localization information) in the physical coordinate system of the changed sound sources 55A and 55B into the sound source position information (second sound image localization information) in the logical coordinate system respectively.

[0049] Further, the first audio-visual localization information may be defined as a group including a plurality of audio-visuals. FIGS. 11(A) and 11(B) are diagrams showing an example of an audio-visual localization setting screen displayed on the display 15 when editing the audio-visual localization information of a plurality of sound sources 55A and 55B.

[0050] In this case, the sound sources 55A and 55B are defined as the same group. And the first audio localization information is defined as the group including the sound sources 55A and 55B. Further, the second audio localization information is also defined as the group including the sound sources 55A and 55B. The user edits either the sound source 55A or the sound source 55B arranged on the logical plane image 153 or the physical plane image 154. The audio localization information reception unit 142 changes the first audio localization information while maintaining the relative positional relationship of each of a plurality of audios included in the same group. For example, as shown in FIG. 11(A), the user changes the sound source 55A arranged on the logical plane image 153 from the coordinates (x1, y1) = (0.25, 0.5) to the coordinates (x1, y1) = (0.75, 0.75). The audio localization information reception unit 142 changes the coordinates of the sound source 55B while maintaining the relative positional relationship between the sound sources 55A and 55B included in the same group. The coordinates of the sound source 55A are (x1, y1) = (0.25, 0.5). The coordinates of the sound source 55B are (x2, y2) = (0.25, 0.25). In this case, the relative position is (x1 - x2, y1 - y2) = (0, 0.25). Therefore, the audio localization information reception unit 142 changes the coordinates of the sound source 55B to (x2, y2) = (0.75, 0.5). The display 15 displays the sound sources 55A and 55B on the logical plane image 153 according to the changed coordinates of the sound sources 55A and 55B. Then, the conversion unit 143 converts the sound source position coordinates (the first audio localization information) in the logical coordinate system of the changed sound sources 55A and 55B into the sound source position information (the second audio localization information) in the physical coordinate system respectively. Alternatively, the conversion unit 143 may convert the coordinates of the changed sound source 55A and the relative position in the logical coordinate system into the physical coordinate system respectively. In this case, the conversion unit 143 may obtain the position of the sound source 55B in the physical coordinate system based on the coordinates of the sound source 55A in the physical coordinate system and the coordinates of the relative position. Thereafter, as shown in FIG. 11(B), the display 15 changes the positions of the sound sources 55A and 55B in the physical plane image 154.

[0051] The user may change either the sound source 55A or the sound source 55B arranged on the physical plane image 154. For example, when the user changes the sound source 55A arranged on the physical plane image 154, the sound image localization information reception unit 142 changes the coordinates of the sound source 55B while maintaining the relative positional relationship between the sound source 55A and the sound source 55B included in the same group. The display 15 displays the sound sources 55A and 55B on the physical plane image 154 according to the coordinates of the sound sources 55A and 55B after the change. Then, the conversion unit 143 converts the sound source position coordinates (first sound image localization information) in the physical coordinate system of the sound sources 55A and 55B after the change into the sound source position information (second sound image localization information) in the logical coordinate system. Thereafter, the display 15 changes the positions of the sound sources 55A and 55B on the logical plane image 153.

[0052] Note that the display 15 may display, for example, the representative point of the group. The user may collectively change the positions of the sound sources 55A and 55B by changing the position of the representative point of the group. Also in this case, the sound image localization information reception unit 142 changes the coordinates of each sound source while maintaining the relative positional relationship between the sound source 55A and the sound source 55B included in the same group.

[0053] Next, an example of receiving the designation of the space information (third space information) in another physical coordinate system within the physical plane image 154 will be described. FIG. 12 is a diagram showing an example of the sound image localization setting screen displayed on the display 15 of the information processing apparatus 1 or the information processing apparatus 1A. For ease of explanation, the display of the speakers is omitted in FIG. 12. Also in this case, the operation of the information processing apparatus 1 is the same as the flowchart shown in FIG. 3, and the operation of the information processing apparatus 1A is the same as the flowchart shown in FIG. 7.

[0054] In this example, the display 15 further displays a physical plane image 155 within the physical plane image 154. The physical plane image 155 corresponds to third spatial information different from the first spatial information corresponding to the physical plane image 154. The third spatial information is also physical coordinates. The space setting unit 141 accepts the setting of the third spatial information by reading information including two-dimensional or three-dimensional coordinates such as CAD data from the flash memory 14 in the operation of S11 shown in FIGS. 3 and 7. In this way, the space setting unit 141 accepts the specification of the third spatial information within the first spatial information displayed on the display 15. The audio-visual localization information reception unit 142 receives the position of the audio-visual in the logical plane image 153 or the physical plane image 154. The conversion unit 143 converts the sound source position coordinates in the physical coordinate system into the sound source position coordinates in the logical coordinate system, or converts the sound source position coordinates in the logical coordinate system into the sound source position coordinates in the physical coordinate system.

[0055] Also, the audio-visual localization information reception unit 142 receives the change in the position of the audio-visual in the logical plane image 153 or the physical plane image 154. For example, as shown in FIG. 13(A), the user designates the position of the sound source 55 arranged in the physical plane image 154 to the upper right end of the physical plane image 155.

[0056] The display 15 displays the position of the sound source 55 after the change in the physical plane image 154. In the example of FIG. 13(A), the display 15 displays the position of the sound source 55 at the upper right end of the physical plane image 155 displayed within the physical plane image 154.

[0057] The conversion unit 143 converts the sound source position coordinates (first audio-visual localization information) in the physical coordinate system into the sound source position coordinates (second audio-visual localization information) in the logical coordinate system by means of an affine transformation or the like. In the above-described embodiment, the conversion unit 143 performed the conversion between the physical coordinates corresponding to the physical plane image 154 and the logical coordinates corresponding to the logical plane image 153. On the other hand, in the example of FIG. 13(A), the third spatial information in the physical coordinate system corresponds to the second spatial information in the logical coordinate system. For example, the coordinates of the lower left end of the physical plane image 155 correspond to the coordinates (x, y) = (0, 0) in the logical coordinate system, the coordinates of the upper left end of the physical plane image 155 correspond to the coordinates (x, y) = (0, 1) in the logical coordinate system, the coordinates of the lower right end of the physical plane image 155 correspond to the coordinates (x, y) = (1, 0) in the logical coordinate system, and the coordinates of the upper right end of the physical plane image 155 correspond to the coordinates (x, y) = (1, 1) in the logical coordinate system.

[0058] The conversion unit 143 obtains the sound source position coordinates in the logical coordinate system based on the third spatial information of the physical plane image 155 and the second spatial information of the logical plane image 153. That is, the conversion unit 143 converts the physical coordinates corresponding to the physical plane image 155 into the logical coordinates corresponding to the logical plane image 153.

[0059] In the example of FIG. 13(A), the sound source 55 is located at the upper right end of the physical plane image 155. Therefore, the sound source position coordinates of the sound source 55 in the logical coordinate system are (x, y) = (1, 1). The display 15 displays the position of the sound source 55 at the sound source position coordinates in the logical coordinate system obtained by the conversion unit 143.

[0060] In this way, the conversion unit 143 converts the first audio-visual localization information in the physical coordinate system into the second audio-visual localization information in the logical coordinate system based on the third spatial information and the second spatial information.

[0061] Note that, as shown in FIG. 14(A), the user can also specify the position of the sound source 55 outside the physical plane image 155 within the physical plane image 154. The coordinates of the end portion of the physical plane image 155 correspond to the coordinates of the end portion of the logical plane image 153. Therefore, when the sound source 55 is located outside the physical plane image 155, the conversion unit 143 corresponds at least either the x coordinate or the y coordinate of the coordinates of the sound source 55 in the logical coordinate system to 0 or 1 corresponding to the end portion of the logical plane image 153. In the example of FIG. 14(B), since both the x coordinate and the y coordinate of the coordinates of the sound source 55 are specified outside the physical plane image 155, even if the user changes the position of the sound source 55 further upward and rightward from the upper right end portion of the physical plane image 155, the position of the sound source 55 in the logical plane image 153 remains unchanged at (x, y) = (1, 1).

[0062] The same also applies to a plurality of sound sources defined in the same group. When at least one of the plurality of sound sources defined in the same group is specified outside the physical plane image 155, the conversion unit 143 corresponds at least either the x coordinate or the y coordinate of the coordinates of the sound source 55 in the logical coordinate system to 0 or 1 for the sound source specified outside. Then, for other sound sources other than the sound source specified outside, which are defined as the same group as the sound source specified outside, the coordinates in the physical coordinate system are changed so as to have the relative positional relationship to be maintained in the group with the coordinates of the sound source specified outside in the specified physical coordinate system.

[0063] As described above, the user edits the sound source 55 arranged in the logical plane image 153 or the physical plane image 154. When the sound source 55 is located outside the physical plane image 155 as shown in Fig. 15(A), the user may change the position of the sound source 55 in the logical plane image 153. As described above, the physical plane image 155 in the physical coordinate system corresponds to the logical plane image 153 in the logical coordinate system. When the sound source 55 is located outside the physical plane image 155 as shown in Fig. 15(A) and the user changes the position of the sound source 55 in the logical plane image 153, if the conversion unit 143 converts the coordinates in the logical coordinate system into the coordinates in the physical coordinate system, the position of the sound source 55 in the physical coordinate system will instantaneously move from outside the physical plane image 155 to inside the physical plane image 155. Since the audio device performs the process of localizing the sound image based on the position of the sound source 55 in the physical coordinate system, when the position of the sound source 55 in the physical coordinate system instantaneously moves from outside the physical plane image 155 to inside the physical plane image 155, the localization position of the sound image changes abruptly.

[0064] Therefore, the information processing device 1 or the information processing device 1A performs the operations shown in the flowchart of Fig. 16. When the information processing device 1 or the information processing device 1A receives a change in the position of the sound source 55 in the logical coordinate system while the sound source 55 is located outside the physical plane image 155, it performs the operations shown in Fig. 16. First, the conversion unit 143 obtains the position of the sound source 55 in the physical coordinate system (the sound source position coordinates in the physical coordinate system (the first sound image localization information)) (S31). Specifically, the conversion unit 143 obtains the relative positions of the sound source 55 before and after the movement in the logical coordinate system. Then, the conversion unit 143 converts the relative position into the relative position in the physical coordinate system to obtain the position of the sound source 55 after the movement in the physical coordinate system. In this case, the conversion unit 143 may associate the physical coordinate system corresponding to the physical plane image 155 with the logical coordinate system and convert the relative position in the logical coordinate system into the relative position in the physical coordinate system. Alternatively, the conversion unit 143 may associate the physical coordinate system corresponding to the physical plane image 154 with the logical coordinate system and convert the relative position in the logical coordinate system into the relative position in the physical coordinate system.

[0065] Further, the conversion unit 143 may associate the physical coordinate system corresponding to the physical plane image 154 with the logical coordinate system, convert the coordinates of the sound source 55 after movement in the logical coordinate system into the coordinates after movement in the physical coordinate system, and obtain the position of the sound source 55 after movement in the physical coordinate system.

[0066] Then, as shown in Fig. 15(A), the display 15 first displays the position of the sound source 55 after movement in the physical plane image 154 of the physical coordinate system (S32). At this time, it is preferable that the display 15 displays the position of the sound source 55 after movement in the logical coordinate system in a light color or with a dotted line as a temporary position.

[0067] Thereafter, the conversion unit 143 converts the coordinates of the sound source 55 in the physical coordinate system after movement into the coordinates in the logical coordinate system (the sound source position coordinates in the logical coordinate system (second sound image localization information)) (S33). When the coordinates of the sound source 55 in the physical coordinate system are within the physical plane image 155, the coordinates of the sound source 55 in the logical coordinate system are within the logical plane image 153. On the other hand, when the coordinates of the sound source 55 in the physical coordinate system are outside the physical plane image 155, the conversion unit 143 corresponds the coordinates of the sound source 55 in the logical coordinate system to 0 or 1 which are the coordinates at the end of the logical coordinate system. In this case, as shown in Fig. 15(B), the sound source 55 in the logical coordinate system remains located at the end of the logical plane image 153.

[0068] As described above, even when the position of the sound source 55 is outside the physical plane image 155 and the position of the sound source 55 in the logical plane image 153 is changed, the position of the sound source 55 in the physical coordinate system does not change abruptly. The sound image localization position does not change abruptly. Note that the operation shown in Fig. 16 is not limited to the case where the sound source 55 is located outside the physical plane image 155 and the position of the sound source 55 is changed in the logical coordinate system, and the operation may always be performed when the position of the sound source 55 is changed in the logical coordinate system.

[0069] (Other examples) Images of the logical coordinate system (logical space image 151 and logical plane image 153) and images of the physical coordinate system (physical space image 152 and physical plane image 154) may be displayed on separate and different devices. For example, the image of the logical coordinate system may be displayed on information processing device 1, and the image of the physical coordinate system may be displayed on information processing device 1A. In this case, information processing device 1 and information processing device 1A may simply transmit and receive information indicating the spatial information and the coordinates of the sound source to each other and share it.

[0070] In FIGS. 10 to 15, the display and coordinate conversion of the sound source in the two-dimensional space (plane) are shown. However, the display and coordinate conversion of the sound source in the three-dimensional space may also be performed. Further, as shown in FIG. 8, the information in the three-dimensional space may be in a form including the plane coordinates (x, y) and information indicating a plurality of layers in the height direction.

Explanation of Signs

[0071] 1, 1A... Information processing devices 11... Communication unit 12... Processor 13... RAM 14... Flash memory 15... Display 16... User I / F 17... Audio I / F 50C... Center speaker 50L... Left speaker 50R... Right speaker 50SL... Left rear speaker 50SR... Right rear speaker 55... Sound source 70A, 70B, 70C, 70D, 70E, 70F, 70G, 70H... Reference points 141... Space setting unit 142... Sound image localization information reception unit 143... Conversion unit 144... Localization processing unit 151... Logical space image 152... Physical space image 151L1, 151L2, 151L3, 152L1, 152L2, 152L3... Layers 153... Logical plane image 154…Physical plane image

Claims

1. Receiving settings of first spatial information of information corresponding to one of a logical space or a physical space, and second spatial information of information corresponding to the other of the logical space or the physical space, Receiving first audio-visual localization information indicating a position for localizing an audio-visual at a first coordinate in the first spatial information, Converting the first audio-visual localization information into second audio-visual localization information corresponding to a second coordinate in the second spatial information, Characterized in that the logical space and the physical space have different shapes, An information processing method, Displaying the first spatial information, the second spatial information, the first audio-visual localization information, and the second audio-visual localization information, Receiving a specification of third spatial information corresponding to a physical space within the displayed first spatial information, An information processing method.

2. Acquiring an audio signal, Localizing the audio-visual at a position corresponding to the second audio-visual localization information for the audio signal, The information processing method according to claim 1.

3. The first spatial information and the second spatial information include information of a two-dimensional space, The information processing method according to claim 1 or claim 2.

4. The first spatial information and the second spatial information include information of a three-dimensional space, The information processing method according to claim 1 or claim 2.

5. The information of the three-dimensional space includes a planar coordinate and information indicating a plurality of layers in the height direction, The information processing method according to claim 4.

6. Converting the first audio-visual localization information into the second audio-visual localization information based on the height information of the plurality of layers, The information processing method according to claim 5.

7. The first audio-visual localization information is defined as a group including a plurality of audio-visuals, When receiving a change in the position of at least any one of the plurality of audio-visuals, changing the first audio-visual localization information while maintaining the relative positional relationship of each of the plurality of audio-visuals included in the group, Converting the first audio-visual localization information of each of the plurality of audio-visuals included in the group into the second audio-visual localization information, The information processing method according to any one of claims 1 to 6.

8. Receiving a change in the position of the audio-visual in the second spatial information, Based on the received change, obtaining the first audio-visual localization information, and then converting the first audio-visual localization information into the second audio-visual localization information based on the third spatial information and the second spatial information, The information processing method according to any one of claims 1 to 7.

9. A space setting unit that receives settings of first space information of information corresponding to one of a logical space or a physical space and second space information of information corresponding to the other of the logical space or the physical space; An audio-visual localization information receiving unit that receives first audio-visual localization information indicating a position for localizing audio-visual at a first coordinate in the first space information; A conversion unit that converts the first audio-visual localization information into second audio-visual localization information corresponding to a second coordinate in the second space information; A display that displays the first space information, the second space information, the first audio-visual localization information, and the second audio-visual localization information; An operation unit that receives an operation of a user who designates the first audio-visual localization information; Comprising; The logical space and the physical space have different shapes; The operation unit receives a designation of third space information in the first space information displayed on the display; An information processing apparatus.

10. A sound signal acquisition unit that acquires a sound signal; A localization processing unit that localizes the audio-visual at a position corresponding to the second audio-visual localization information with the sound signal; The information processing apparatus according to claim 9, comprising.

11. The first space information and the second space information include information of a two-dimensional space; The information processing apparatus according to claim 9 or claim 10.

12. The first space information and the second space information include information of a three-dimensional space; The information processing apparatus according to claim 9 or claim 10.

13. The information of the three-dimensional space includes plane coordinates and information indicating a plurality of layers in the height direction; The information processing apparatus according to claim 12.

14. The conversion unit converts the first audio-visual localization information into the second audio-visual localization information based on the height information of the plurality of layers; The information processing apparatus according to claim 13.

15. The first audio-visual localization information is defined as a group including a plurality of audio-visuals; When the audio-visual localization information receiving unit receives a change in the position of at least any one of the plurality of audio-visuals, the first audio-visual localization information is changed while maintaining the relative positional relationship of each of the plurality of audio-visuals included in the group; The conversion unit converts each of the first audio-visual localization information of the plurality of audio-visuals included in the group into the second audio-visual localization information; The information processing apparatus according to any one of claims 9 to 14.

16. The operation unit receives a change in the position of the audio-visual in the second space information; Based on the received change, the conversion unit obtains the first audio-visual localization information, and then converts the first audio-visual localization information into second audio-visual localization information based on the third spatial information and the second spatial information. The information processing apparatus according to any one of claims 9 to 15.

Images