Information processing method and information processing device

Abstract

To provide an information processing method and an information processing device that realize sound image localization taking into account the shape of a physical space.SOLUTION: An information processing method includes accepting settings of first spatial information, which is information corresponding to one of a logical space or a physical space, and second spatial information, which is information corresponding to the other of the logical space or the physical space, accepting first sound image localization information indicating a position at which a sound image is to be localized at a first coordinate in the first spatial information, and converting the first sound image localization information into second sound image localization information corresponding to a second coordinate in the second spatial information, the logical space and the physical space having different shapes.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] One embodiment of the present invention relates to an information processing method and an information processing apparatus.

Background Art

[0002] The audio adjustment console of Patent Document 1 receives the coordinates of the audio localization point in a rectangular parallelepiped-shaped space. The audio adjustment console of Patent Document 1 calculates the volume of the sound output from a plurality of speakers arranged in the space so that the sound image is localized at the received coordinates.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the physical space such as a concert hall is not limited to a rectangular parallelepiped shape. Therefore, even if a device such as the audio adjustment console of Patent Document 1 receives the coordinates of the audio localization point in a rectangular parallelepiped space, it may not be able to localize the sound image at the position as intended by the user because it does not consider the coordinates of the physical space.

[0005] Therefore, one of the objects of one embodiment of the present invention is to provide an information processing method and an information processing apparatus that realize audio localization considering the shape of the physical space.

Means for Solving the Problems

[0006] An information processing method according to an embodiment of the present invention receives setting of first spatial information of information corresponding to one of a logical space or a physical space and second spatial information of information corresponding to the other of the logical space or the physical space, receives first audio-visual localization information indicating a position where an audio-visual is to be localized at a first coordinate in the first spatial information, and converts the first audio-visual localization information into second audio-visual localization information corresponding to a second coordinate in the second spatial information, wherein the logical space and the physical space have different shapes.

Effect of the Invention

[0007] According to an embodiment of the present invention, it is possible to realize audio-visual localization considering the shape of a physical space.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

DETAILED DESCRIPTION OF THE INVENTION

[0009] FIG. 1 is a block diagram showing the configuration of the information processing apparatus 1. The information processing apparatus 1 includes a communication unit 11, a processor 12, a RAM 13, a flash memory 14, a display 15, and a user I / F 16.

[0010] The information processing apparatus 1 is composed of a personal computer, a smartphone, a tablet computer, or the like. Also, an audio device such as an audio mixer is an example of the information processing apparatus.

[0011] The communication unit 11 communicates with other devices such as a server. The communication unit 11 has a wireless communication function such as Bluetooth (registered trademark) or Wi-Fi (registered trademark), and a wired communication function such as USB or LAN. The communication unit 11 acquires, for example, space information indicating the shape of a physical space such as a concert hall. The space information is information indicating two-dimensional or three-dimensional coordinates with a certain position as a reference point (origin). The space information is information including two-dimensional or three-dimensional coordinates such as CAD data indicating the shape of a physical space such as a concert hall.

[0012] The processor 12 is composed of a CPU, a DSP, or a SoC (System on a Chip), etc. The processor 12 reads a program from the flash memory 14 which is a storage medium, and temporarily stores it in the RAM 13, thereby performing various operations. The processor 12 realizes functional components such as a space setting unit 141, an audio-visual localization information receiving unit 142, and a conversion unit 143 according to the read program. Note that the program does not necessarily have to be stored in the flash memory 14. The processor 12 may, for example, download it from another device such as a server when necessary and temporarily store it in the RAM 13.

[0013] The display 15 is composed of an LCD, etc. The display 15 displays, for example, an audio-visual localization setting screen as shown in FIG. 2.

[0014] The user I / F 16 is an example of an operation unit. The user I / F 16 is composed of a mouse, a keyboard, or a touch panel, etc. The user I / F 16 receives the operations of the user. Note that the touch panel may be laminated on the display 15.

[0015] With reference to FIGS. 2 and 3, the audio-visual localization setting screen will be described. FIG. 2 is a diagram showing an example of the audio-visual localization setting screen displayed on the display 15. FIG. 3 is a flowchart showing the operations of the processor 12. The audio-visual localization setting screen shown in FIG. 2 is an example of a content editing screen. The user edits the audio-visual localization positions of the sound sources included in the content on the audio-visual localization setting screen.

[0016] The display 15 displays a logical space image 151 in a logical coordinate system and a physical space image 152 in a physical coordinate system. In this example, the display 15 displays the logical space image 151 at the upper left of the screen and the physical space image 152 at the upper right of the screen. Also, the display 15 displays a logical plane image 153 at the lower left of the screen and a physical plane image 154 at the lower right of the screen.

[0017] The logical space image 151 is, for example, in the shape of a rectangular parallelepiped. The logical plane image 153 corresponds to a view of the logical plane image 153 in plan view. The physical space image 152 is, for example, in the shape of an octagonal prism. The physical plane image 154 corresponds to a view of the physical space image 152 in plan view.

[0018] First, the space setting unit 141 of the processor 12 receives settings of first space information, which is information corresponding to a logical space, and second space information, which is information corresponding to a physical space such as a concert hall (S11).

[0019] The first space information is logical coordinates. The logical coordinates are, for example, normalized coordinates from 0 to 1. In the present embodiment, the space setting unit 141 receives settings of the space information of a rectangular parallelepiped as the first space information, but may also receive various space information such as a pyramid, a prism, a polyhedron, a cylinder, a cone, or a sphere. Further, the space setting unit 141 may receive information of a two-dimensional space. The two-dimensional space includes, for example, a polygon composed of straight lines, a circle composed of curves, or a composite shape composed of straight lines and curves.

[0020] The second space information is physical coordinates. The physical coordinates are two-dimensional or three-dimensional coordinates included in CAD data or the like indicating the shape of a physical space such as a concert hall. The space setting unit 141 of the processor 12 receives settings of the second space information, for example, by reading information including two-dimensional or three-dimensional coordinates such as CAD data from the flash memory 14.

[0021] Next, the space setting unit 141 generates the logical space image 151, the physical space image 152, the logical plane image 153, and the physical plane image 154, and displays them on the display 15 (S12). In the example of FIG. 2, the logical space image 151 is an image of an elevation view of a cube shape, and the logical plane image 153 is a square-shaped image. The physical space image 152 and the physical plane image 154 are images simulating an actual space such as a concert hall. The space setting unit 141 generates the physical space image 152 and the physical plane image 154 based on information including two-dimensional or three-dimensional coordinates such as CAD data.

[0022] Next, the sound image localization information receiving unit 142 of the processor 12 receives the speaker arrangement information or the sound image localization information (S13). The speaker arrangement information and the sound image localization information are each coordinates in a logical coordinate system and are an example of the first sound image localization information.

[0023] The user operates the user I / F 16 to edit the speaker arrangement information or the sound image localization information in the logical space image 151 or the logical plane image 153 shown in FIG. 2. For example, in the example of FIG. 2, the user arranges the center speaker 50C, the left speaker 50L, the right speaker 50R, the left rear speaker 50SL, and the right rear speaker 50SR in the logical space image 151 and the logical plane image 153. The center speaker 50C, the left speaker 50L, the right speaker 50R, the left rear speaker 50SL, and the right rear speaker 50SR are arranged in the middle in the height direction.

[0024] Assuming that the position of the upper left vertex in the logical plane image 153 is the origin, the coordinates of the left speaker 50L are (x, y) = (0, 0). The coordinates of the right speaker 50R are (x, y) = (1, 0). The coordinates of the center speaker 50C are (x, y) = (0.5, 0). The coordinates of the left rear speaker 50SL are (x, y) = (0, 1). The coordinates of the right rear speaker 50SR are (x, y) = (1, 1).

[0025] Also, in the example of FIG. 2, the user arranges the sound image localization position of the sound source 55 in the logical space image 151 and the logical plane image 153 on the left side (between the left end and the center) of the center. That is, the coordinates of the sound source 55 are (x, y) = (0.25, 0.5).

[0026] In the example of FIG. 2, the coordinates in the height direction of the center speaker 50C, the left speaker 50L, the right speaker 50R, the left rear speaker 50SL, the right rear speaker 50SR, and the sound source are all z = 0.5.

[0027] The audio-visual positioning information reception unit 142 receives the speaker arrangement information or the sound source position information by receiving, for example, an operation of editing the speaker arrangement information of the user or the audio-visual positioning information of the sound source as shown in FIG. 2 (S13: Yes).

[0028] The conversion unit 143 performs coordinate conversion based on the received speaker arrangement information or sound source position information (S14).

[0029] FIG. 4 and FIG. 5 are diagrams for explaining the concept of coordinate conversion. The conversion unit 143 converts the speaker arrangement information and the sound source position information from the first coordinates of the first space information in the logical coordinate system to the second coordinates of the second space information in the physical coordinate system. In the example of FIG. 4, in the physical coordinate system, there are eight reference points 70A(x1,y1), 70B(x2,y2), 70C(x3,y3), 70D(x4,y4), 70E(x5,y5), 70F(x6,y6), 70G(x7,y7), 70H(x8,y8), and eight reference points 70A(0,0), 70B(0.25,0), 70C(0.75,0), 70D(1,0), 70E(0,1), 70F(0.25,1), 70G(0.75,1), 70H(1,1) in the logical coordinate system before conversion. The conversion unit 143 obtains the centroid G of the eight reference points in the logical coordinate system before conversion and the centroid G' of the eight reference points in the physical coordinate system after conversion, and generates a triangular mesh centered on these centroids. The conversion unit 143 converts the internal space of the triangle in the logical coordinate system and the internal space of the triangle in the physical coordinate system by a predetermined coordinate conversion. The conversion uses, for example, an affine transformation. The affine transformation is an example of a geometric transformation. The affine transformation represents the x-coordinate (x') and y-coordinate (y') after conversion as functions of the x-coordinate (x) and y-coordinate (y) before conversion, respectively. That is, the affine transformation performs coordinate conversion using the equations x' = ax + by + c and y' = dx + ey + f. From the coordinates of the three vertices of the triangle before conversion and the coordinates of the three vertices of the triangle after conversion, the coefficients a to f can be uniquely determined. The conversion unit 143 obtains the affine transformation coefficients in the same way for all triangles, thereby converting from the first coordinates in the logical coordinate system to the second coordinates of the second space information in the physical coordinate system. Note that the coefficients a to f may be obtained by the least squares method.

[0030] Then, the conversion unit 143 converts the coordinates of the speaker arrangement information and the sound source position information using the obtained coefficients a to f. In FIG. 5, the conversion unit 143 converts the coordinates (x, y) in the logical coordinate system of the sound source 55 into the coordinates (x', y') in the physical coordinate system using the above formula.

[0031] As a result, the coordinates of the speaker arrangement information and the sound source position information are converted into second sound image localization information that matches the shape of the physical space. The processor 12 stores the second sound image localization information in, for example, the flash memory 14. Alternatively, the processor 12 transmits the second sound image localization information to another device such as an audio device via, for example, the communication unit 11. The audio device performs a process of localizing the sound image based on the received second sound image localization information. The audio device calculates the level balance of the sound signals output to the plurality of speakers so that the sound image of the sound source is localized at the specified position based on the speaker arrangement information and the sound source position information included in the second sound image localization information, and adjusts the levels of the sound signals. Therefore, the information processing apparatus 1 of the present embodiment can realize sound image localization considering the shape of the physical space.

[0032] Note that the above mesh may be a polygon mesh other than a triangle or a combination thereof. For example, as shown in FIG. 9, the conversion unit 143 may generate a quadrilateral mesh and perform coordinate conversion. The conversion method is not limited to the above affine conversion. For example, the conversion unit 143 may convert the quadrilateral mesh based on the following formula and convert the coordinates (x, y) in the logical coordinate system of the sound source 55 into the coordinates (x', y') in the physical coordinate system (where x0, y0, x1, y1, x2, y2, x3, y3 are the coordinates of the conversion points, respectively). x’ = x0 + (x1 - x0)x + (x3 - x0)y + (x0 - x1 + x2 - x3)xy y’ = y0 + (y1 - y0)x + (y3 - y0)y + (y0 - y1 + y2 - y3)xy The conversion method may also be other geometric conversions such as, for example, an isometric mapping, a similarity transformation, or a projective transformation. For example, a projective transformation is represented by the equations x’=(ax+by+c) / (gx+hy+1) and y’=(dx+ey+f) / (gx+hy+1). The coefficients are obtained in the same manner as the above-described affine transformation. For example, the eight coefficients (a to h) constituting the projective transformation of a quadrilateral can be uniquely obtained by an eight simultaneous equations. Or, the coefficients may be obtained, for example, by the least squares method.

[0033] FIG. 6 is a block diagram showing the configuration of the information processing apparatus 1A according to Modification 1. FIG. 7 is a flowchart showing the operation of the information processing apparatus 1A. The same configurations, functions, and operations as those of the information processing apparatus 1 are denoted by the same reference numerals, and the description thereof is omitted.

[0034] The information processing apparatus 1A further includes an audio I / F 17. The audio I / F 17 includes an analog audio terminal or a digital audio terminal or the like. The processor 12 acquires a sound signal of a sound source via the audio I / F 17. Thereby, the processor 12 functions as a sound signal acquisition unit. Further, the sound signal may be acquired from an external device via the communication unit 11. Further, the sound signal may be stored in the flash memory 14.

[0035] The audio I / F 17 is connected to a center speaker 50C, a left speaker 50L, a right speaker 50R, a left rear speaker 50SL, and a right rear speaker 50SR installed in an actual space such as a concert hall.

[0036] The processor 12 includes a DSP. The processor 12 performs predetermined signal processing on the sound signal. The processor 12 outputs the sound signal after the signal processing to the center speaker 50C, the left speaker 50L, the right speaker 50R, the left rear speaker 50SL, and the right rear speaker 50SR via the audio I / F 17.

[0037] The processor 12 further realizes the functional configuration of the sound localization processing unit 144 by reading the program stored in the flash memory 14 into the RAM 13. The sound localization processing unit 144 of the processor 12 performs a process of sound image localizing an audio signal to a position corresponding to the second sound image localization information based on the speaker arrangement information and the sound source position information (the second sound image localization information) converted by the conversion unit 143 (S15). That is, the sound localization processing unit 144 calculates the level balance of the audio signals output to the center speaker 50C, the left speaker 50L, the right speaker 50R, the left rear speaker 50SL, and the right rear speaker 50SR so that the sound image of the sound source is localized at the specified position based on the speaker arrangement information and the sound source position information included in the second sound image localization information, and adjusts the level of the audio signal. In this way, the information processing apparatus may perform sound image localization processing.

[0038] In FIGS. 2 to 5, coordinate conversion in a two-dimensional space (plane) is shown. However, the information processing apparatus may perform coordinate conversion in a three-dimensional space. In this case, the converted coordinates x', y', z' are each represented as a function of x, y, z. The conversion unit 143 converts the speaker arrangement information and the sound source position information based on the function.

[0039] Further, the information in the three-dimensional space may be in a form including the planar coordinates (x, y) and information indicating a plurality of layers in the height direction.

[0040] FIG. 8 is a diagram showing the concept of layers. The user operates the user I / F 16 to edit the layers, the speaker arrangement information, or the sound image localization information of the sound source. In the example of FIG. 8, the user designates the heights of three layers arranged in the height direction. Further, the user designates the arrangement of the speakers or the sound image of the sound source in any of the designated layers. In the example of FIG. 8, the user designates the layers 151L1, 151L2, 151L3, 152L1, 152L2, and 152L3 and designates the arrangement of the speakers or the sound image of the sound source from these layers.

[0041] The conversion unit 143 converts the coordinates (x’, y’) on the plane of the physical coordinate system by geometric conversion as described above. The coordinate of the height is specified by the user. In the example of FIG. 8, the layer 151L1 in the logical coordinate system has z = 1.0, the layer 151L2 has the coordinate of z = 0.5, and the layer 151L3 has z = 0. The layer 152L1 in the physical coordinate system corresponds to the highest position in the actual space, that is, the coordinate of the ceiling surface. The layer 152L3 in the physical coordinate system corresponds to the lowest position in the actual space, that is, the coordinate of the floor surface. The layer 152L2 is the coordinate between the coordinate of the ceiling surface and the coordinate of the floor surface. For example, when the sound source 55 is arranged in the layer 151L3 of the logical coordinate system, the conversion unit 143 obtains the coordinate z3 in the height direction of the layer 152L3 in the physical coordinate system as the height information of the second sound image localization information.

[0042] In addition, either the speaker arrangement information or the sound source position information may specify the coordinates between a plurality of layers. For example, the speaker arrangement information may be specified by layers, and the sound source position information may specify a free position in the three-dimensional space. In this case, the conversion unit 143 obtains the sound source position information based on the height information of a plurality of layers. The conversion unit 143 obtains the coordinates of the physical coordinate system by, for example, linear interpolation. For example, when the sound source is between the layer 151L1 and the layer 151L2, the conversion unit 143 obtains the converted sound source coordinate z’ from the coordinate z of the sound source before conversion as follows.

[0043] z’=(z - z1)*(z’2 - z’1) / (z2 - z1)+z’1 Of course, the number of layers is not limited to three. The number of layers may be two or four or more.

[0044] The description of this embodiment should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is shown not by the above-described embodiment but by the scope of the claims. Further, the scope of the present invention includes the scope equivalent to the scope of the claims.

[0045] For example, the user may edit the speaker arrangement information and the sound image localization information of the sound source in the physical space image 152 or the physical plane image 154. In this case, the space setting unit 141 receives the space information in the physical coordinate system as the first space information and the space information in the logical coordinate system as the second space information. The conversion unit 143 converts the speaker arrangement information and the sound source position information (first sound image localization information) in the physical coordinate system into the speaker arrangement information and the sound source position information (second sound image localization information) in the logical coordinate system.

[0046] Note that the number of sound sources is not limited to one. FIG. 10 is a diagram showing an example of a sound image localization setting screen displayed on the display 15 when editing the sound image localization position information of a plurality of sound sources 55A and 55B. In the example of FIG. 10, the information processing device 1 or the information processing device 1A displays the logical plane image 153 and the physical plane image 154, but actually also displays the logical space image 151 and the physical space image 152. The operation of the information processing device 1 is the same as the flowchart shown in FIG. 3, and the operation of the information processing device 1A is the same as the flowchart shown in FIG. 7.

[0047] In this example, the user arranges the sound image localization positions of the sound sources 55A and 55B in the logical plane image 153 and the physical plane image 154. The coordinates of the sound source 55A are (x1, y1) = (0.25, 0.5). The coordinates of the sound source 55B are (x2, y2) = (0.25, 0.25).

[0048] The user edits the sound sources 55A and 55B arranged in the logical plane image 153 or the physical plane image 154 respectively. For example, the user changes the sound sources 55A and 55B arranged in the physical plane image 154 to different positions respectively. The conversion unit 143 converts the sound source position coordinates (first sound image localization information) in the physical coordinate system of the changed sound sources 55A and 55B into the sound source position information (second sound image localization information) in the logical coordinate system respectively.

[0049] Further, the first audio-visual localization information may be defined as a group including a plurality of audio-visuals. FIGS. 11(A) and 11(B) are diagrams showing an example of an audio-visual localization setting screen displayed on the display 15 when editing the audio-visual localization information of a plurality of sound sources 55A and 55B.

[0050] In this case, sound sources 55A and 55B are defined as belonging to the same group. The first audio localization information is defined as a group that includes sound sources 55A and 55B. Furthermore, the second audio localization information is also defined as a group that includes sound sources 55A and 55B. The user edits either sound source 55A or sound source 55B arranged on the logical plane image 153 or the physical plane image 154. The audio localization information reception unit 142 changes the first audio localization information while maintaining the relative positional relationship of each of a plurality of audio signals included in the same group. For example, as shown in FIG. 11(A), the user changes the coordinates of sound source 55A arranged on the logical plane image 153 from (x1, y1) = (0.25, 0.5) to (x1, y1) = (0.75, 0.75). The audio localization information reception unit 142 changes the coordinates of sound source 55B while maintaining the relative positional relationship between sound sources 55A and 55B included in the same group. The coordinates of sound source 55A are (x1, y1) = (0.25, 0.5). The coordinates of sound source 55B are (x2, y2) = (0.25, 0.25). In this case, the relative position is (x1 - x2, y1 - y2) = (0, 0.25). Therefore, the audio localization information reception unit 142 changes the coordinates of sound source 55B to (x2, y2) = (0.75, 0.5). The display 15 displays sound sources 55A and 55B on the logical plane image 153 according to the changed coordinates of sound sources 55A and 55B. Then, the conversion unit 143 converts the sound source position coordinates (first audio localization information) in the logical coordinate system of the changed sound sources 55A and 55B into sound source position information (second audio localization information) in the physical coordinate system, respectively. Alternatively, the conversion unit 143 may convert the coordinates of the changed sound source 55A and the relative position in the logical coordinate system into the physical coordinate system, respectively. In this case, the conversion unit 143 may obtain the position of sound source 55B in the physical coordinate system based on the coordinates of sound source 55A and the coordinates of the relative position in the physical coordinate system. Thereafter, as shown in FIG. 11(B), the display 15 changes the positions of sound sources 55A and 55B in the physical plane image 154.

[0051] The user may change either the sound source 55A or the sound source 55B arranged on the physical plane image 154. For example, when the user changes the sound source 55A arranged on the physical plane image 154, the sound image localization information receiving unit 142 changes the coordinates of the sound source 55B while maintaining the relative positional relationship between the sound source 55A and the sound source 55B included in the same group. The display 15 displays the sound sources 55A and 55B on the physical plane image 154 according to the coordinates of the changed sound sources 55A and 55B. Then, the conversion unit 143 converts the sound source position coordinates (first sound image localization information) in the physical coordinate system of the changed sound sources 55A and 55B into the sound source position information (second sound image localization information) in the logical coordinate system. Thereafter, the display 15 changes the positions of the sound sources 55A and 55B in the logical plane image 153.

[0052] Note that the display 15 may display, for example, the representative point of the group. The user may collectively change the positions of the sound sources 55A and 55B by changing the position of the representative point of the group. Also in this case, the sound image localization information receiving unit 142 changes the coordinates of each sound source while maintaining the relative positional relationship between the sound source 55A and the sound source 55B included in the same group.

[0053] Next, an example of receiving the designation of the space information (third space information) in another physical coordinate system within the physical plane image 154 will be described. FIG. 12 is a diagram showing an example of the sound image localization setting screen displayed on the display 15 of the information processing apparatus 1 or the information processing apparatus 1A. For ease of explanation, the display of the speaker is omitted in FIG. 12. Also in this case, the operation of the information processing apparatus 1 is the same as the flowchart shown in FIG. 3, and the operation of the information processing apparatus 1A is the same as the flowchart shown in FIG. 7.

[0054] In this example, the display 15 further displays a physical plane image 155 within the physical plane image 154. The physical plane image 155 corresponds to third spatial information different from the first spatial information corresponding to the physical plane image 154. The third spatial information is also physical coordinates. The space setting unit 141 receives the setting of the third spatial information by reading information including two-dimensional or three-dimensional coordinates such as CAD data from the flash memory 14 in the operation of S11 shown in FIGS. 3 and 7. In this way, the space setting unit 141 receives the specification of the third spatial information within the first spatial information displayed on the display 15. The audio-visual localization information reception unit 142 receives the position of the audio-visual in the logical plane image 153 or the physical plane image 154. The conversion unit 143 converts the sound source position coordinates in the physical coordinate system into the sound source position coordinates in the logical coordinate system, or converts the sound source position coordinates in the logical coordinate system into the sound source position coordinates in the physical coordinate system.

[0055] Also, the audio-visual localization information reception unit 142 receives the change in the position of the audio-visual in the logical plane image 153 or the physical plane image 154. For example, as shown in FIG. 13(A), the user designates the position of the sound source 55 arranged in the physical plane image 154 to the upper right end of the physical plane image 155.

[0056] The display 15 displays the position of the sound source 55 after the change in the physical plane image 154. In the example of FIG. 13(A), the display 15 displays the position of the sound source 55 at the upper right end of the physical plane image 155 displayed within the physical plane image 154.

[0057] The conversion unit 143 converts the sound source position coordinates (first audio-visual localization information) in the physical coordinate system into the sound source position coordinates (second audio-visual localization information) in the logical coordinate system by means of an affine transformation or the like. In the above-described embodiment, the conversion unit 143 performed the conversion between the physical coordinates corresponding to the physical plane image 154 and the logical coordinates corresponding to the logical plane image 153. On the other hand, in the example of FIG. 13(A), the third spatial information in the physical coordinate system corresponds to the second spatial information in the logical coordinate system. For example, the coordinates of the lower left end of the physical plane image 155 correspond to the coordinates (x, y) = (0, 0) in the logical coordinate system, the coordinates of the upper left end of the physical plane image 155 correspond to the coordinates (x, y) = (0, 1) in the logical coordinate system, the coordinates of the lower right end of the physical plane image 155 correspond to the coordinates (x, y) = (1, 0) in the logical coordinate system, and the coordinates of the upper right end of the physical plane image 155 correspond to the coordinates (x, y) = (1, 1) in the logical coordinate system.

[0058] The conversion unit 143 obtains the sound source position coordinates in the logical coordinate system based on the third spatial information of the physical plane image 155 and the second spatial information of the logical plane image 153. That is, the conversion unit 143 converts the physical coordinates corresponding to the physical plane image 155 into the logical coordinates corresponding to the logical plane image 153.

[0059] In the example of FIG. 13(A), the sound source 55 is located at the upper right end of the physical plane image 155. Therefore, the sound source position coordinates of the sound source 55 in the logical coordinate system are (x, y) = (1, 1). The display 15 displays the position of the sound source 55 at the sound source position coordinates in the logical coordinate system obtained by the conversion unit 143.

[0060] In this way, the conversion unit 143 converts the first audio-visual localization information in the physical coordinate system into the second audio-visual localization information in the logical coordinate system based on the third spatial information and the second spatial information.

[0061] Note that, as shown in FIG. 14(A), the user can also specify the position of the sound source 55 outside the physical plane image 155 within the physical plane image 154. The coordinates of the end portion of the physical plane image 155 correspond to the coordinates of the end portion of the logical plane image 153. Therefore, when the sound source 55 is located outside the physical plane image 155, the conversion unit 143 makes at least either the x-coordinate or the y-coordinate of the coordinates of the sound source 55 in the logical coordinate system correspond to 0 or 1 corresponding to the end portion of the logical plane image 153. In the example of FIG. 14(B), since both the x-coordinate and the y-coordinate of the coordinates of the sound source 55 are specified outside the physical plane image 155, even if the user changes the position of the sound source 55 further upward and rightward from the upper right end portion of the physical plane image 155, the position of the sound source 55 in the logical plane image 153 remains unchanged at (x, y) = (1, 1).

[0062] The same also applies to a plurality of sound sources defined in the same group. When at least one of the plurality of sound sources defined in the same group is specified outside the physical plane image 155, the conversion unit 143 makes at least either the x-coordinate or the y-coordinate of the coordinates of the sound source 55 in the logical coordinate system correspond to 0 or 1 for the sound source specified outside. Then, for other sound sources other than the sound source specified outside, which are defined as the same group as the sound source specified outside, the coordinates in the physical coordinate system are changed so as to have the relative positional relationship to be maintained in the group with the coordinates of the sound source specified outside in the specified physical coordinate system.

[0063] As described above, the user edits the sound source 55 arranged in the logical plane image 153 or the physical plane image 154. When the sound source 55 is located outside the physical plane image 155 as shown in FIG. 15(A), the user may change the position of the sound source 55 in the logical plane image 153. As described above, the physical plane image 155 in the physical coordinate system corresponds to the logical plane image 153 in the logical coordinate system. When the sound source 55 is located outside the physical plane image 155 as shown in FIG. 15(A) and the user changes the position of the sound source 55 in the logical plane image 153, if the conversion unit 143 converts the coordinates in the logical coordinate system into the coordinates in the physical coordinate system, the position of the sound source 55 in the physical coordinate system will instantaneously move from outside the physical plane image 155 to inside the physical plane image 155. Since the audio device performs the process of localizing the sound image based on the position of the sound source 55 in the physical coordinate system, when the position of the sound source 55 in the physical coordinate system instantaneously moves from outside the physical plane image 155 to inside the physical plane image 155, the localization position of the sound image changes abruptly.

[0064] Therefore, the information processing device 1 or the information processing device 1A performs the operations shown in the flowchart of FIG. 16. When the information processing device 1 or the information processing device 1A receives a change in the position of the sound source 55 in the logical coordinate system while the sound source 55 is located outside the physical plane image 155, it performs the operations shown in FIG. 16. First, the conversion unit 143 obtains the position of the sound source 55 in the physical coordinate system (the sound source position coordinates in the physical coordinate system (the first sound image localization information)) (S31). Specifically, the conversion unit 143 obtains the relative positions of the sound source 55 before and after the movement in the logical coordinate system. Then, the conversion unit 143 converts the relative position into the relative position in the physical coordinate system to obtain the position of the sound source 55 after the movement in the physical coordinate system. In this case, the conversion unit 143 may associate the physical coordinate system corresponding to the physical plane image 155 with the logical coordinate system and convert the relative position in the logical coordinate system into the relative position in the physical coordinate system. Alternatively, the conversion unit 143 may associate the physical coordinate system corresponding to the physical plane image 154 with the logical coordinate system and convert the relative position in the logical coordinate system into the relative position in the physical coordinate system.

[0065] Further, the conversion unit 143 may associate the physical coordinate system corresponding to the physical plane image 154 with the logical coordinate system, convert the coordinates of the sound source 55 after movement in the logical coordinate system into the coordinates after movement in the physical coordinate system, and obtain the position of the sound source 55 after movement in the physical coordinate system.

[0066] Then, as shown in FIG. 15(A), the display 15 first displays the position of the sound source 55 after movement in the physical plane image 154 of the physical coordinate system (S32). At this time, it is preferable that the display 15 displays the position of the sound source 55 after movement in the logical coordinate system in a light color or with a dotted line as a temporary position.

[0067] Thereafter, the conversion unit 143 converts the coordinates of the sound source 55 in the physical coordinate system after movement into the coordinates in the logical coordinate system (the sound source position coordinates in the logical coordinate system (second sound image localization information)) (S33). When the coordinates of the sound source 55 in the physical coordinate system are within the physical plane image 155, the coordinates of the sound source 55 in the logical coordinate system are within the logical plane image 153. On the other hand, when the coordinates of the sound source 55 in the physical coordinate system are outside the physical plane image 155, the conversion unit 143 corresponds the coordinates of the sound source 55 in the logical coordinate system to 0 or 1 which are the coordinates of the end of the logical coordinate system. In this case, as shown in FIG. 15(B), the sound source 55 in the logical coordinate system remains located at the end of the logical plane image 153.

[0068] As described above, when the sound source 55 is located outside the physical plane image 155 and the position of the sound source 55 in the logical plane image 153 is changed, the position of the sound source 55 in the physical coordinate system does not change abruptly. The sound image localization position does not change abruptly. Note that the operation shown in FIG. 16 is not limited to the case where the sound source 55 is located outside the physical plane image 155 and the position of the sound source 55 is changed in the logical coordinate system, and the operation may always be performed when the position of the sound source 55 is changed in the logical coordinate system.

[0069] (Other examples) Images of the logical coordinate system (logical space image 151 and logical plane image 153) and images of the physical coordinate system (physical space image 152 and physical plane image 154) may be displayed on separate and different devices. For example, the image of the logical coordinate system may be displayed on information processing device 1, and the image of the physical coordinate system may be displayed on information processing device 1A. In this case, information processing device 1 and information processing device 1A may simply transmit and receive information indicating the spatial information and the coordinates of the sound source to each other and share it.

[0070] In FIGS. 10 to 15, the display and coordinate conversion of the sound source in the two-dimensional space (plane) are shown. However, the display and coordinate conversion of the sound source in the three-dimensional space may also be performed. Further, as shown in FIG. 8, the information in the three-dimensional space may be in a form including the plane coordinates (x, y) and information indicating a plurality of layers in the height direction.

Explanation of Signs

[0071] 1, 1A... Information processing devices 11... Communication unit 12... Processor 13... RAM 14... Flash memory 15... Display 16... User I / F 17... Audio I / F 50C... Center speaker 50L... Left speaker 50R... Right speaker 50SL... Left rear speaker 50SR... Right rear speaker 55... Sound source 70A, 70B, 70C, 70D, 70E, 70F, 70G, 70H... Reference points 141... Space setting unit 142... Sound image localization information reception unit 143... Conversion unit 144... Localization processing unit 151... Logical space image 152... Physical space image 151L1, 151L2, 151L3, 152L1, 152L2, 152L3... Layers 153... Logical plane image 154…Physical plane image

Claims

1. Accepting settings of first spatial information corresponding to one of the logical space and the physical space, and second spatial information corresponding to the other of the logical space and the physical space; receiving first sound image localization information indicating a position at which a sound image is to be localized in a first coordinate in the first spatial information; converting the first sound image localization information into second sound image localization information corresponding to a second coordinate in the second spatial information; The logical space and the physical space have different shapes. Information processing methods.

2. Acquire a sound signal, localizing the sound image of the sound signal at a position corresponding to the second sound image localization information; The information processing method according to claim 1 .

3. the first spatial information and the second spatial information include information of a two-dimensional space; 3. The information processing method according to claim 1.

4. the first spatial information and the second spatial information include information of a three-dimensional space; 3. The information processing method according to claim 1.

5. The information on the three-dimensional space includes planar coordinates and information indicating a plurality of layers in a height direction. The information processing method according to claim 4.

6. converting the first sound image localization information into the second sound image localization information based on height information of the plurality of layers; The information processing method according to claim 5.

7. The first sound image localization information is defined as a group including a plurality of sound images, when a change in the position of at least one of the plurality of sound images is received, the first sound image localization information is changed while maintaining a relative positional relationship between the plurality of sound images included in the group; converting the first sound image localization information of each of the plurality of sound images included in the group into the second sound image localization information; The information processing method according to any one of claims 1 to 6.

8. a space setting unit that receives settings of first space information corresponding to one of the logical space and the physical space, and second space information corresponding to the other of the logical space and the physical space; a sound image localization information receiving unit that receives first sound image localization information indicating a position where a sound image is to be localized in a first coordinate in the first spatial information; a conversion unit that converts the first sound image localization information into second sound image localization information corresponding to second coordinates in the second spatial information; Equipped with The logical space and the physical space have different shapes. Information processing device.

9. A sound signal acquisition unit that acquires a sound signal; a localization processing unit that localizes the sound image of the sound signal at a position corresponding to the second sound image localization information; The information processing device according to claim 8 .

10. the first spatial information and the second spatial information include information of a two-dimensional space; 10. The information processing device according to claim 8 or 9.

11. the first spatial information and the second spatial information include information of a three-dimensional space; 10. The information processing device according to claim 8.

12. The information on the three-dimensional space includes planar coordinates and information indicating a plurality of layers in a height direction. The information processing device according to claim 11.

13. The conversion unit converts the first sound image localization information into the second sound image localization information based on height information of the plurality of layers. The information processing device according to claim 12.

14. The first sound image localization information is defined as a group including a plurality of sound images, the sound image localization information receiving unit, when receiving a change in the position of at least one of the plurality of sound images, changes the first sound image localization information while maintaining a relative positional relationship between the plurality of sound images included in the group; The conversion unit converts the first sound image localization information of each of the plurality of sound images included in the group into the second sound image localization information. The information processing device according to claim 8 .

15. a display that displays the first spatial information, the second spatial information, the first sound image localization information, and the second sound image localization information; an operation unit that receives an operation by a user to specify the first sound image localization information; The information processing device according to claim 8 , further comprising:

16. The operation unit accepts a change in the position of the sound image in the second spatial information, The conversion unit obtains the first sound image localization information based on the received change, and then converts the first sound image localization information into second sound image localization information based on the third spatial information and the second spatial information. The information processing device according to claim 15.

Images