Signal processing apparatus, control method of signal processing apparatus, and program

The signal processing device addresses discrete sound source placement issues by calculating and controlling sound attenuation based on virtual viewpoint positions, reducing audible discomfort through strategic volume management.

JP2024165115AInactive Publication Date: 2024-11-28CANON KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023080993
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-16
Publication Date
2024-11-28
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The placement of sound sources corresponding to collected sound signals becomes discrete due to limited microphone setups, causing variations in sound pressure based on the virtual viewpoint position, leading to audible discomfort.

Method used

A signal processing device that calculates sound attenuation amounts for each virtual sound source based on the virtual viewpoint and sound source positions, switching attenuation methods depending on the virtual viewpoint's position, using distance and panning techniques to control sound volume changes.

Benefits of technology

Generates virtual viewpoint sound with reduced auditory discomfort by ensuring linear volume changes as the virtual viewpoint moves, minimizing fluctuations in sound pressure.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024165115000001_ABST
    Figure 2024165115000001_ABST
Patent Text Reader

Abstract

To generate sound corresponding to a virtual viewpoint without causing acoustic discomfort.SOLUTION: A signal processing apparatus generates sound corresponding to a virtual viewpoint on the basis of sound data of multiple virtual sound sources corresponding to multiple sound collection signals, respectively, the signal processing apparatus including: calculation means which calculates the amount of attenuation of the sound related to each of the virtual sound sources, on the basis of a position of the virtual viewpoint and positions of the virtual sound sources set in advance; attenuation processing means which performs attenuation processing on the sound data on the basis of the calculated amount of attenuation; and generation means which generates sound corresponding to the virtual viewpoint on the basis of the attenuated sound data. The method of calculating the amount of attenuation is switched according to the position of the virtual viewpoint with respect to the virtual sound sources.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a signal processing device, a control method for a signal processing device, and a program. [Background technology]

[0002] A technology that generates an audio signal corresponding to a specified virtual viewpoint using multiple pickup signals obtained by multiple microphones has been attracting attention. Patent Document 1 discloses a mixing device that provides surround sound to a listener by performing mixing processing on sounds picked up by multiple microphones installed at different positions according to the installation positions of the microphones and the input listening position. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-223771 A Summary of the Invention [Problem to be solved by the invention]

[0004] When a limited number of microphones are used to collect sound in a region to be collected, the arrangement of sound sources corresponding to collected signals becomes discrete, and the obtained sound pressure varies depending on the position of the virtual viewpoint, which may cause an uncomfortable feeling to the ear. Patent Document 1 does not disclose a method for solving the uncomfortable feeling to the ear caused by the arrangement of sound sources corresponding to collected signals becoming discrete due to the constraints of such a sound collection environment, and the obtained sound pressure varies depending on the position of the listening point. Therefore, the object of the present invention is to make it possible to generate sound corresponding to a virtual viewpoint without an uncomfortable feeling to the ear. [Means for solving the problem]

[0005] The signal processing device of the present invention is a signal processing device that generates sound corresponding to a virtual viewpoint based on sound data of a plurality of virtual sound sources corresponding respectively to a plurality of picked-up signals, and is characterized in that it has a calculation means that calculates an amount of sound attenuation for each virtual sound source based on a set position of the virtual viewpoint and the position of the virtual sound source, an attenuation processing means that performs an attenuation process on the sound data based on the calculated attenuation amount, and a generation means that generates sound corresponding to the virtual viewpoint based on the attenuation-processed sound data, and the calculation means switches a calculation method of the attenuation amount depending on the position of the virtual viewpoint relative to the virtual sound source. Effect of the Invention

[0006] According to the present invention, it is possible to generate sound that corresponds to a virtual viewpoint and does not give an audible sense of discomfort. [Brief description of the drawings]

[0007] [Figure 1] 2 is a diagram illustrating an example of a functional configuration of a signal processing device according to the first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a signal processing device. [Diagram 3] FIG. 1 is a diagram for explaining the arrangement of a virtual sound source. [Figure 4] FIG. 11 is a diagram for explaining reduction of audible discomfort. [Diagram 5] 4 is a flowchart showing an example of processing by the signal processing device in the first embodiment. [Figure 6] FIG. 11 is a diagram for explaining sound generation in the second embodiment. [Figure 7] FIG. 11 is a diagram illustrating an example of the functional configuration of a signal processing device according to a second embodiment. [Figure 8] 10 is a flowchart showing an example of processing by the signal processing device in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment described below does not limit the present invention. Although a plurality of features are described in the embodiment, not all of these features are necessarily essential configurations for the invention, and the plurality of features may be combined arbitrarily. Also, the same reference numerals are used for the same or similar configurations, and duplicated explanations are omitted.

[0009] <Embodiment 1> In this embodiment, a description will be given taking keirin as an example. In keirin, a racing bicycle that goes around a predetermined course in a velodrome is called a pist, and the sound generated by the running of the pist is called pist running sound. The description will be centered on an example in which virtual viewpoint sound is generated with the pist running sound as the sound collection target. However, the sound collection target is not limited to the pist running sound, and may also be the cheers of the spectators in the stands. Note that the subject to which the present invention is applied is not limited to keirin, and may be, for example, other competitions or music concerts.

[0010] 1 is a block diagram showing an example of the functional configuration of a signal processing device in embodiment 1. The signal processing device in this embodiment includes an input unit 101, a storage unit 102, an attenuation processing unit 103, a surround processing unit 104, an output unit 105, a boundary setting unit 106, a virtual sound source position setting unit 107, a virtual viewpoint setting unit 108, and an attenuation amount calculation unit 109.

[0011] The input unit 101 receives a collected signal from the sound collection unit 110, and outputs acoustic data based on the collected signal to the storage unit 102. The sound collection unit 110 is composed of, for example, M microphones from sound collection unit 110-1 to sound collection unit 110-M. Hereinafter, when there is no particular need to distinguish between the sound collection units, they will be simply referred to as sound collection unit 110. In this embodiment, the sound collection unit 110 is arranged in a bicycle racing track to collect the sound of the piste running, and outputs the collected sound signal to the input unit 101.

[0012] The storage unit 102 stores the acoustic data input from the input unit 101. The attenuation processing unit 103 acquires the acoustic data stored in the storage unit 102, and performs attenuation processing on the acquired acoustic data based on an attenuation amount calculated by a method described later. The attenuation processing unit 103 outputs the attenuated acoustic data to the surround processing unit 104. The surround processing unit 104 performs surround processing on the acoustic data input from the attenuation processing unit 103 based on virtual sound source position information and virtual viewpoint information. The surround processing unit 104 outputs the surround-processed acoustic data to the output unit 105 as virtual viewpoint acoustic data. The surround processing unit 104 is an example of a generating means.

[0013] The output unit 105 generates a virtual viewpoint audio signal for playback based on the virtual viewpoint audio data input from the surround processing unit 104, and outputs the generated signal to the outside. The virtual viewpoint audio signal may be, for example, an audio signal obtained by D / A conversion (digital-to-analog conversion) of the virtual viewpoint audio data. Also, for example, the virtual viewpoint audio data may be converted into an existing surround format or binaural signal. Also, for example, the output unit 105 may include an audio device capable of playing and outputting sound (for example, a surround speaker system having multiple speakers or headphones), and may play back sound based on the virtual viewpoint audio signal.

[0014] The boundary setting unit 106 sets boundary information used in the attenuation calculation process described later in the attenuation calculation unit 109. The virtual sound source position setting unit 107 acquires virtual sound source position information for arranging a virtual sound source (virtual sound source) corresponding to each piece of acoustic data in a virtual space, and sets the acquired virtual sound source position information in the attenuation calculation unit 109 and the surround processing unit 104. The virtual viewpoint setting unit 108 acquires the position and direction of a virtual viewpoint (virtual viewpoint) in the virtual space as virtual viewpoint information, and sets the acquired virtual viewpoint information in the attenuation calculation unit 109 and the surround processing unit 104.

[0015] The attenuation amount calculation unit 109 calculates the attenuation amount of each virtual sound source with respect to the virtual viewpoint based on each piece of information set by the boundary setting unit 106, the virtual sound source position setting unit 107, and the virtual viewpoint setting unit 108, and outputs the calculated attenuation amount to the attenuation processing unit 103. That is, the attenuation amount calculation unit 109 calculates the attenuation amount to be applied to the acoustic data of each virtual sound source according to the virtual viewpoint based on the boundary information, the virtual sound source position information, and the virtual viewpoint information.

[0016] 1 shows an example in which the sound collection unit 110 is directly connected to the input unit 101, but the present invention is not limited to this configuration. For example, acoustic data based on sound collection by the sound collection unit 110 may be stored in a storage device (not shown) connectable to a signal processing device, and the signal processing device may acquire the acoustic data from the storage device.

[0017] 2 is a block diagram showing an example of a hardware configuration of a signal processing device in this embodiment. The signal processing device in this embodiment has a CPU 201, a ROM 202, a RAM 203, an auxiliary storage device 204, a display unit 205, an audio output unit 206, an operation unit 207, a communication I / F 208, and a bus 209. The CPU 201, the ROM 202, the RAM 203, the auxiliary storage device 204, the display unit 205, the audio output unit 206, the operation unit 207, and the communication I / F 208 are connected via the bus 209 so as to be able to communicate with each other.

[0018] 1 by controlling the entire signal processing device using programs (computer programs) and data stored in ROM 202 and RAM 203. For example, CPU 101 reads and executes programs stored in ROM 202 or the like to perform processing according to each flowchart described below. Note that one or more pieces of dedicated hardware different from CPU 201 may be provided, and at least a part of the processing by CPU 201 may be performed by the dedicated hardware. An example of the dedicated hardware is an ASIC (Application Specific Integrated Circuit).

[0019] The ROM (Read Only Memory) 202 stores various programs, various data, etc. The ROM 203 stores, for example, programs related to the control of the signal processing device by the CPU 101, etc. The RAM (Random Access Memory) 203 temporarily stores programs and data supplied from an auxiliary storage device 204, etc., and data supplied from the outside via a communication I / F 208, etc. The auxiliary storage device 204 is composed of, for example, a hard disk drive (HDD) or a solid state drive (SSD), etc., and stores various data such as audio signals and setting information.

[0020] The display unit 205 is composed of, for example, a liquid crystal display or LEDs, and displays a GUI (Graphical User Interface) and the like for the user to operate the system. The audio output unit 206 is composed of, for example, a speaker or headphones, and outputs a generated audio signal. The operation unit 207 is composed of, for example, a keyboard, a mouse, a joystick, a touch panel, and the like, and inputs various instructions to the CPU 201 in response to operations by the user. The CPU 201 operates as a display control unit that controls the display unit 205, and as an operation control unit that controls the operation unit 207.

[0021] The communication I / F (interface) 208 is used for communication with an external device. For example, when the signal processing device is connected to an external device by wire, a communication cable is connected to the communication I / F 208. Also, for example, when the signal processing device has a function of wirelessly communicating with an external device, the communication I / F 208 includes an antenna. The bus 209 connects each unit to transmit information.

[0022] The generation of sound corresponding to a virtual viewpoint (virtual viewpoint sound) in this embodiment will be described. Virtual viewpoint sound is sound heard at a virtual viewpoint, and is sound corresponding to a virtual viewpoint image, for example. FIG. 3 is a diagram for explaining a velodrome that is a sound collection target in this embodiment, and an example of the arrangement of virtual sound sources for generating virtual viewpoint sound. As shown in FIG. 3, velodrome 300 is composed of spectator seats 302 and bank 303 along which the pistes run during races, and an inner ring line 304 and an outer ring line 305 are drawn within bank 303. The pistes run mainly between inner ring line 304 and outer ring line 305 or in the vicinity thereof, and sound collection units 110 composed of multiple microphones are installed at various locations within velodrome 300 so as to collect the sound of the pistes running generated in the area where the pistes run.

[0023] Here, the position and directional direction of the sound collection unit 110 are described in a coordinate system corresponding to the velodrome 300 where sound collection is performed. For example, the origin of the global coordinate system is set at the center 301 of the velodrome 300, which is the space where the sound collection unit 110 is located, the x-axis and y-axis are set parallel and perpendicular to the straight part of the bank 303, and the z-axis is set vertically upward perpendicular to these axes. The positions and directional directions of the sound collection units 110-1 to 110-M are expressed by the coordinate values ​​in the coordinate system defined in this way. However, the format of the sound collection area information and the sound collection unit 110 information is not limited to this. In this example, the area where the pistes run is set as the sound collection target area in the first quadrant of the xy plane centered on the center 301.

[0024] In the following, a virtual point sound source corresponding to each sound data is processed as a virtual sound source 306. The virtual sound source 306 is composed of virtual sound sources 306-1 to 306-M arranged corresponding to each sound pickup signal (sound data). Hereinafter, when each virtual sound source is not particularly distinguished, it is simply written as a virtual sound source 306. For example, for a sound pickup signal picked up by the sound pickup unit 110-1, the virtual sound source 306-1 is arranged in a virtual space, and is processed as a sound emitted from the arranged location. At this time, each position of the virtual sound source 306 arranged in the virtual space is regarded as a virtual sound source position. For example, the attenuation processing unit 103 and the surround processing unit 104 process the sound data based on the sound pickup signal by the sound pickup unit 110-1 as sound data output from the virtual sound source 306-1 located at 306-1. The same applies to the sound pickup units 110-2 to 110-5. In a process described below, the attenuation processing unit 103 and the surround processing unit 104 process sound data based on a signal picked up by the sound collection unit 110 based on a virtual sound source position and a virtual viewpoint corresponding to the sound data, thereby generating virtual viewpoint sound. The virtual viewpoint can be set, for example, at the center 301 of the bicycle racing track 300, and it is also possible to change the position and direction of the virtual viewpoint so as to follow the running pistes from behind.

[0025] It is generally known that the attenuation of a sound from a point sound source when the sound reaches a point away from the point sound source is inversely proportional to the square of the distance from the point sound source. In this embodiment, the attenuation of the sound from each virtual sound source when the sound reaches the virtual viewpoint is basically calculated by distance attenuation.

[0026] By arranging the virtual sound sources 306 densely in the sound collection target area, it is possible to generate virtual viewpoint sound with little audible discomfort. However, in an actual sound collection site, restrictions arise on the number and installation positions of the sound collection units 110, i.e., microphones, due to various reasons such as spoiling the aesthetics, interfering with the competition, lack of physical installation space, and limited number of equipment. In this example, due to the above-mentioned restrictions, the sound collection units 110 are arranged discretely in the sound collection target area as shown in FIG. 3. Accordingly, the arrangement of the virtual sound sources 306 is also similarly discrete in the sound collection target area.

[0027] Here, for example, in the case of a virtual viewpoint that tracks a pistol running along the virtual sound source 306 from the virtual sound source 306-1 to the virtual sound source 306-5 at a constant distance, it is natural that the volume of the pistol running sound at the virtual viewpoint will not change much. However, when the virtual sound sources are located discretely, the volume obtained at the virtual viewpoint changes as the virtual viewpoint moves due to the above-mentioned characteristics of distance attenuation, and an uncomfortable feeling is generated in the auditory sense. For example, when the virtual viewpoint moves from the virtual sound source 306-1 to the virtual sound source 306-2, the volume decreases as the virtual sound source approaches the midpoint between the virtual sound source 306-1 and the virtual sound source 306-2. After that, the volume increases as the virtual sound source approaches the midpoint between the virtual sound source 306-1 and the virtual sound source 306-2. Similarly, the volume increases and decreases repeatedly as the virtual viewpoint moves between the other virtual sound sources, which causes an uncomfortable feeling in the auditory sense.

[0028] A method for reducing this audible discomfort will be described with reference to Fig. 4. The two virtual sound sources (the closest virtual sound source and the second closest virtual sound source) closest (shortest) to a virtual viewpoint at a certain point in time are assumed to be virtual sound source 306-1 and virtual sound source 306-2, and a straight line connecting virtual sound source 306-1 and virtual sound source 306-2 is assumed to be line 401.

[0029] Here, for example, when the virtual viewpoint moves along the straight line 401 in the vicinity of the straight line 401, if the attenuation of the sounds of the virtual sound sources 306-1 and 306-2 is controlled by distance attenuation according to the distance from the virtual viewpoint to the virtual sound source, the above-mentioned strange sensation is likely to occur. Therefore, the region where the strange sensation is likely to occur when the attenuation of the sounds is controlled by distance attenuation is set as the panning region 402. Furthermore, the panning region 402 is set to have a cylindrical shape with the straight line 401 as the central axis, and the radius of the panning region 402 is set as the panning boundary 403. The radius of the panning region 402 corresponds to the second threshold value. Moreover, the panning boundary 403 is an example of the second boundary. The attenuation of the sounds of the virtual sound sources 306-1 and 306-2 relative to the virtual viewpoint 407 in the panning region 402 is calculated by panning the virtual sound sources 306-1 and 306-2. This makes it possible to linearly change the volume of the virtual sound source in response to the movement of the virtual viewpoint, thereby reducing the sense of discomfort felt by the listener in the virtual viewpoint sound.

[0030] On the other hand, when the distance from the virtual viewpoint to the virtual sound source is sufficiently far, controlling the amount of sound attenuation by distance attenuation according to the distance does not easily cause an uncomfortable feeling in the auditory sense. Therefore, the area in which the amount of sound attenuation of the virtual sound source is calculated by distance attenuation is set as the distance attenuation area 406. The distance attenuation area 406 is set as an area outside the distance attenuation boundary 405 with respect to the straight line 401, that is, an area in which the distance to the straight line 401 is greater than the distance between the straight line 401 and the distance attenuation boundary 405. The distance between the straight line 401 and the distance attenuation boundary 405 corresponds to a first threshold value. The distance attenuation boundary 405 is an example of a first boundary. The amount of sound attenuation of the virtual sound source 306-1 and the virtual sound source 306-2 with respect to the virtual viewpoint 409 in the distance attenuation area 406 is calculated by distance attenuation according to the distance to the virtual sound source.

[0031] The mixing area 404 is an area for suppressing a sudden change in the attenuation amount of the virtual sound source, and is an area surrounded by the panning boundary 403 and the distance attenuation boundary 405. In other words, the area between the panning boundary 403 and the distance attenuation boundary 405 is the mixing area 404. The attenuation amount of the sound of the virtual sound source 306-1 and the virtual sound source 306-2 with respect to the virtual viewpoint 408 in the mixing area 404 is calculated by adding the attenuation amount calculated by panning and the attenuation amount calculated by distance attenuation at a predetermined ratio. The ratio of the addition is changed, for example, in proportion to the distance from the straight line 401 to the virtual viewpoint, and the weight of the attenuation amount due to panning is increased if the virtual viewpoint 408 is close to the panning boundary 403, and the weight of the distance attenuation is increased if the virtual viewpoint 408 is close to the distance attenuation boundary 405.

[0032] Fig. 5 is a flowchart showing an example of a virtual viewpoint sound generation process of the signal processing device in the first embodiment. The process shown in Fig. 5 is started, for example, when a sound signal collected by the sound collection unit 110 is input to the signal processing device and an instruction to start a process related to the generation of a virtual viewpoint sound signal is received by the signal processing device. The instruction to start the process may be given by a user operation via the operation unit 207 of the signal processing device, or an instruction may be input from another device. Note that the timing to start the execution of the process shown in Fig. 5 is not limited to the above-mentioned timing.

[0033] 5 may be executed in parallel with the sound collection by the sound collection unit 110, or may be executed after the sound collection by the sound collection unit 110 is completed. The process shown in Fig. 5 is realized by the CPU 201 expanding a program stored in the ROM 202 or the like into the RAM 203 and executing it. Note that at least a part of the process shown in Fig. 5 may be executed by one or more dedicated hardware different from the CPU 201.

[0034] In step S501, the virtual sound source position setting unit 107 accepts a user operation, acquires a virtual sound source position corresponding to each of the multiple acoustic data, and sets the acquired virtual sound source position in the attenuation amount calculation unit 109 and the surround processing unit 104. The virtual sound source position may be input manually. Also, for example, a straight line connecting each microphone constituting the sound collection unit 110 and the inner sphere line 304 closest to the microphone is obtained, and the midpoint of the intersection of the straight line with the inner sphere line 304 and the outer sphere line 305 may be automatically set as the virtual sound source position. Hereinafter, this midpoint is also referred to as the "midpoint of the inner sphere line 304 and the outer sphere line 305".

[0035] In step S502, the boundary setting unit 106 accepts a user operation, acquires boundary information, and sets the acquired boundary information in the attenuation calculation unit 109. The boundary information is information indicating a boundary for switching a method for calculating an attenuation amount of a virtual sound source in a virtual viewpoint, and specifically, is information indicating the panning boundary 403 and the distance attenuation boundary 405 described above. The boundary information may be input manually. Also, for example, the distance from the midpoint between the inner sphere line 304 and the outer sphere line 305 to the outer sphere line 305 may be set as the panning boundary 403, and the distance attenuation boundary 405 may be set automatically by multiplying the panning boundary 403 by two.

[0036] In step S503, the input unit 101 receives the input of a plurality of collected sound signals by the sound collection unit 110, and acquires a plurality of acoustic data based on the inputted plurality of collected sound signals. The acquired plurality of acoustic data are stored in the storage unit 102.

[0037] In step S504, the virtual viewpoint setting unit 108 accepts a user operation, acquires virtual viewpoint information in the space in which the bicycle racing track 300 exists, and sets the acquired virtual viewpoint information in the attenuation calculation unit 109 and the surround processing unit 104. The virtual viewpoint information includes a virtual viewpoint position and a virtual viewpoint direction, and the user can arbitrarily specify the virtual viewpoint. In this embodiment, it is assumed that a virtual listening position and listening direction for sound are determined based on the virtual viewpoint position and virtual viewpoint direction of the virtual viewpoint information.

[0038] The order of executing the processes in steps S501 to S504 described above is not limited to the example shown in FIG. 5 and can be arbitrarily changed. In addition, some or all of the processes in steps S501 to S504 may be executed in parallel.

[0039] In step S505, the attenuation calculation unit 109 selects acoustic data to be panned from among the multiple acoustic data acquired and stored in step S503. Based on the virtual sound source position corresponding to each acoustic data, the attenuation calculation unit 109 selects acoustic data of two virtual sound sources (the shortest virtual sound source and the next shortest virtual sound source) from the side with the shortest distance from the virtual viewpoint to the virtual sound source position, and sets them as panning targets. That is, the IDs of the acoustic data corresponding to the virtual sound source positions are set as panning target 1 and panning target 2 in the order of the shortest distance between the virtual sound source and the virtual sound source position. In the example shown in FIG. 4, panning target 1 for the virtual viewpoint 407 is the virtual sound source 306-1, and panning target 2 is the virtual sound source 306-2.

[0040] However, when the attenuation calculation unit 109 determines that the virtual viewpoint is outside the panning target 1 and the panning target 2, it determines that there is no sound data to be panned, and sets values ​​indicating invalidity to the panning target 1 and the panning target 2. For example, in the example shown in FIG. 4, the virtual viewpoint 410 is determined to be outside the panning target 1 and the panning target 2, although the two virtual sound sources from the short distance side are the virtual sound sources 306-1 and 306-2. Specifically, in FIG. 4, the foot of a perpendicular line drawn from the virtual viewpoint 410 to a straight line 411 passing through the virtual sound sources 306-1 and 306-2 is set to a point 412. When the sum of the distance from this point 412 to the virtual sound source 306-1 and the distance from the point 412 to the virtual sound source 306-2 exceeds the length of the straight line 401, the attenuation calculation unit 109 determines that the virtual viewpoint 410 is outside the panning target 1 and the panning target 2. In other words, if the distance from point 412 to virtual sound source 306-1 or 306-2 exceeds the distance between virtual sound source 306-1 and virtual sound source 306-2, attenuation calculation unit 109 determines that virtual viewpoint 410 is outside panning target 1 and panning target 2.

[0041] In step S506, the attenuation amount calculation unit 109 calculates the panning amount to be applied to the sound data of the panning target. Here, the panning amount to be applied to the sound data of the panning target 1 is set as P1, and the panning amount to be applied to the sound data of the panning target 2 is set as P2. When the IDs of the sound data valid for the panning target 1 and the panning target 2 are set, the attenuation amount calculation unit 109 sets the panning amount of the sound data corresponding to the panning target 1 and the panning target 2, respectively. Specifically, as shown in FIG. 4, the position of the panning target 1 is set as point A, the position of the panning target 2 is set as point B, and the foot of the perpendicular line drawn from the virtual viewpoint V to the straight line 401 is set as point H. Also, the distance from point A to point B is set as AB, the distance from point A to point H is set as AH, and the distance from point B to point H is set as BH. At this time, the attenuation amount calculation unit 109 calculates the panning amount P1 by P1=AH / AB, and calculates the panning amount P2 by P2=1-P1(=BH / AB). When values ​​indicating invalidity are set for panning target 1 and panning target 2, attenuation amount calculation section 109 sets 0 to panning amounts P1 and P2.

[0042] The processes from step S507 to step S512 are performed on each of the multiple acoustic data acquired in step S503 and corresponding to the multiple sound collection units 110. Here, it is assumed that the process is repeatedly performed on the acoustic data to be processed while changing the acoustic data ID, but the processes on the multiple acoustic data may be performed in parallel.

[0043] In step S507, the attenuation calculation unit 109 judges whether or not the attenuation amount of the target sound data is calculated only by distance attenuation. If the attenuation calculation unit 109 judges that the attenuation amount of the sound data is not calculated only by distance attenuation (NO in step S507), the process of step S508 is executed. On the other hand, if the attenuation calculation unit 109 judges that the attenuation amount of the sound data is calculated only by distance attenuation (YES in step S507), the process of step S509 is executed. Specifically, if the sound source data ID of the attenuation calculation target matches the sound data ID set as panning target 1 or panning target 2 and the virtual viewpoint is in the panning area 402 or the mixed area 404, the process of step S508 is executed. Otherwise, the attenuation amount is calculated only by distance attenuation and the process of step S509 is executed.

[0044] In step S508, the attenuation calculation unit 109 judges whether or not the attenuation of the target sound data is calculated only by panning. If the attenuation calculation unit 109 judges that the attenuation of the sound data is calculated only by panning (YES in step S508), the process of step S510 is executed. On the other hand, if the attenuation calculation unit 109 judges that the attenuation of the sound data is not calculated only by panning, that is, that the attenuation is calculated by a mixture of distance attenuation and panning (NO in step S508), the process of step S511 is executed. Specifically, the attenuation calculation unit 109 judges whether the virtual viewpoint is in the panning region 402 or the mixed region 404. If the virtual viewpoint is in the panning region 402, the process of step S510 is executed assuming that the attenuation is calculated only by panning. If the virtual viewpoint is in the mixed region 404, the process of step S511 is executed assuming that the attenuation is calculated by a mixture of distance attenuation and panning.

[0045] In the processes of steps S509, S510, and S511, the attenuation calculation unit 109 sets coefficients for calculating the amount of attenuation according to each condition. That is, the attenuation calculation unit 109 sets a distance attenuation coefficient α and a panning coefficient β used to calculate the amount of attenuation. In step S509, the attenuation calculation unit 109 calculates the amount of attenuation based only on the distance attenuation, so that the distance attenuation coefficient α is set to 1 and the panning coefficient β is set to 0. In step S510, the attenuation calculation unit 109 calculates the attenuation amount based on only panning, so that the distance attenuation coefficient α is set to 0 and the panning coefficient β is set to 1.

[0046] In step S511, the attenuation amount calculation unit 109 calculates the attenuation amount by mixing distance attenuation and panning, so that the distance attenuation coefficient α and the panning coefficient β are set according to the position of the virtual viewpoint in the mixed area 404. Specifically, as shown in FIG. 4, the distance from the virtual viewpoint V to the point H is VH, the distance from the straight line 401 to the panning boundary 403 is D1, and the distance from the straight line 401 to the distance attenuation boundary 405 is D2. At this time, the attenuation amount calculation unit 109 determines and sets the distance attenuation coefficient α and the panning coefficient β by α=(VH-D1) / (D2-D1) and β=1-α(=(D2-VH) / (D2-D1)). Note that the calculation method of the distance attenuation coefficient α and the panning coefficient β is not limited to this, and for example, weighting may be performed separately according to the position of the virtual viewpoint.

[0047] In step S512, the attenuation calculation unit 109 calculates the attenuation to be applied to each piece of sound data. The attenuation is calculated by adding together the distance attenuation multiplied by the distance attenuation coefficient α and the panning amount multiplied by the panning coefficient β, and is expressed as Attenuation = α * Distance Attenuation + β * Panning Amount. Note that the attenuation calculation unit 109 calculates the distance attenuation based on the distance from the virtual sound source position corresponding to the sound data for which the attenuation is to be calculated to the virtual viewpoint. For example, if the bicycle racing track 300 is outdoors, then the directional characteristics of the virtual sound source with respect to the virtual viewpoint are Q, pi is π, and the distance from the virtual sound source to the virtual viewpoint is r, then the distance attenuation is Distance Attenuation = 10log(Q / (4πr 2 )) In addition, if the bicycle racing track 300 is indoors, and the indoor constant is R, the distance attenuation can be calculated as distance attenuation = 10log(Q / (4πr 2)+(4 / R)). However, the method of calculating the distance attenuation is not limited to the above, and for example, the sound absorption coefficient of the wall may also be taken into consideration. Furthermore, as for the panning amount, if the sound data ID of the object for which the attenuation amount is to be calculated matches the sound data ID of panning object 1, a panning amount P1 is set, and if it matches the sound data ID of panning object 2, a panning amount P2 is set. In all other cases, the panning amount is set to 0.

[0048] The above-mentioned processes of steps S507 to S512 are performed for all sound data, and when the calculation of the amount of attenuation of each sound data is completed, the process of step S513 is performed. In step S513, the attenuation amount calculation unit 109 outputs the attenuation amount of each sound data calculated in step S512 to the attenuation processing unit 103.

[0049] In step S514, the attenuation processing unit 103 acquires the acoustic data from the storage unit 102, and performs attenuation processing for each acoustic data based on the attenuation amount of each acoustic data acquired in step S513. Then, the attenuation processing unit 103 outputs the attenuated acoustic data to the surround processing unit 104.

[0050] In step S515, the surround processing unit 104 performs surround processing on the sound data input from the attenuation processing unit 103 based on the virtual sound source position and the virtual viewpoint, and outputs the surround-processed sound data to the output unit 105 as virtual viewpoint sound data. The surround processing is performed using a known panning technique such as VBAP (Vector Base Amplitude Panning). Here, speaker layout information required for surround processing by VBAP is acquired from the storage unit 102 or a user operation via the operation unit 207. The panning technique is not limited to VBAP, and MDAP (Multiple-Direction Amplitude Panning) or the like may be used. In addition, when the reproducing audio device is configured with headphones, the surround-processed sound data may be binaurally processed, or may be directly binaurally processed according to the position of the virtual sound source relative to the virtual viewpoint without surround processing by a known panning technique.

[0051] In step S516, the output unit 105 generates a virtual viewpoint sound signal for reproduction based on the virtual viewpoint sound data input from the surround processing unit 104, and outputs the generated signal to the outside.

[0052] According to this embodiment, for the two virtual sound sources located closer to the virtual viewpoint than the virtual sound source position, the attenuation amount for the two virtual sound sources is controlled according to the distance from the straight line connecting the two virtual sound sources to the virtual viewpoint. In this way, by controlling the attenuation amount of the virtual sound source based on the positional relationship between the virtual viewpoint and the virtual sound source, when the virtual viewpoint moves near the virtual sound source, the volume change of the virtual sound source with respect to the movement of the virtual viewpoint becomes linear, and the sense of strangeness in the auditory sense can be reduced. Therefore, it is possible to generate virtual viewpoint sound that does not sound strange in the auditory sense.

[0053] <Embodiment 2> In the second embodiment, an example will be described in which panning between two pieces of sound data from the side of the virtual viewpoint to the virtual sound source position is not suitable by taking into account the position of the sound collection target in the selection of sound data to be panned. For example, in FIG. 6, the two virtual sound sources close to the virtual viewpoint 601 are the virtual sound source 602-1 and the virtual sound source 602-3. In this case, when the panning target is selected as described above, the panning target is the virtual sound source 602-1 and the virtual sound source 602-3. However, when the sound collection target 603 moves along the trajectory 604, the panning target is set to the virtual sound source 602-1 and the virtual sound source 602-2, so that the audible discomfort in the virtual viewpoint sound can be reduced. Therefore, in the second embodiment, the position of the sound collection target 603 is added to the conditions for selecting sound data to be panned, so that in the example shown in FIG. 6, the virtual sound source 602-2 and the virtual sound source 602-3 are selected as the panning target.

[0054] Fig. 7 is a block diagram showing an example of the functional configuration of a signal processing device in embodiment 2. In Fig. 7, components having the same functions as those shown in Fig. 1 are given the same reference numerals, and duplicated explanations will be omitted. The signal processing device includes an input unit 101, a storage unit 102, an attenuation processing unit 103, a surround processing unit 104, an output unit 105, a boundary setting unit 106, a virtual sound source position setting unit 107, a virtual viewpoint setting unit 108, an attenuation amount calculation unit 109, and a sound collection target position setting unit 701.

[0055] The sound collection target position setting unit 701 acquires the position of the sound collection target, and sets the acquired sound collection target position information in the attenuation amount calculation unit 109. The position of the sound collection target is acquired, for example, using a global positioning system (GPS) or a known tracking technology. In the second embodiment, the attenuation amount calculation unit 109 selects sound data to be panned from among a plurality of sound data, based on the sound collection target position information set by the sound collection target position setting unit 701 in addition to the virtual sound source position information set by the virtual sound source position setting unit 107. The hardware configuration of the signal processing device in the second embodiment is similar to the hardware configuration of the signal processing device in the first embodiment shown in FIG.

[0056] Fig. 8 is a flowchart showing an example of a virtual viewpoint sound generation process of the signal processing device in the second embodiment. The process shown in Fig. 8 is started, for example, when a sound signal collected by the sound collection unit 110 is input to the signal processing device and an instruction to start a process related to the generation of a virtual viewpoint sound signal is received by the signal processing device. The process shown in Fig. 8 may be executed in parallel with the sound collection by the sound collection unit 110, or may be executed after the sound collection by the sound collection unit 110 is completed. The process shown in Fig. 8 is realized by the CPU 201 expanding a program stored in the ROM 202 or the like into the RAM 203 and executing it. At least a part of the process shown in Fig. 8 may be executed by one or more dedicated hardware different from the CPU 201.

[0057] In step S801, the virtual sound source position setting unit 107 accepts a user operation, acquires a virtual sound source position corresponding to a plurality of sound data, and sets the acquired virtual sound source position in the attenuation amount calculation unit 109 and the surround processing unit 104. In step S802, the boundary setting unit 106 accepts a user operation, acquires boundary information, and sets the acquired boundary information in the attenuation amount calculation unit 109. The processing in steps S801 and S802 is similar to the processing in steps S501 and S502 in the processing shown in FIG.

[0058] In step S803, the sound collection target position setting unit 701 accepts a user operation, acquires sound collection target position information, and sets the acquired sound collection target position information in the attenuation amount calculation unit 109. In step S804, the input unit 101 accepts the input of a plurality of collected sound signals by the sound collection unit 110, acquires a plurality of pieces of acoustic data based on the inputted plurality of collected sound signals, and stores the same in the storage unit . In step S805, the virtual viewpoint setting unit 108 accepts a user operation, acquires virtual viewpoint information, and sets the acquired virtual viewpoint information in the attenuation amount calculation unit 109 and the surround processing unit 104. The processing in steps S804 and S805 is similar to the processing in steps S503 and S504 in the processing shown in FIG.

[0059] The order in which the processes in steps S801 to S805 described above are executed is not limited to the example shown in FIG. 8 and can be arbitrarily changed. In addition, some or all of the processes in steps S801 to S805 may be executed in parallel.

[0060] In step S806, the attenuation calculation unit 109 selects the sound data to be panned from among the multiple sound data acquired and stored in step S804, in the same manner as in step S305 in FIG. 5 in the first embodiment. Furthermore, the attenuation calculation unit 109 changes the panning target according to the sound collection target position information for the panning target 1 and the panning target 2 obtained by the same processing as in the first embodiment. Specifically, in the example shown in FIG. 6, it is assumed that the IDs of the sound data of the panning target 1 and the panning target 2 obtained by the same processing as in the first embodiment are the virtual sound source 602-1 and the virtual sound source 602-3, respectively. The attenuation calculation unit 109 searches for a virtual sound source whose distance from the virtual viewpoint 601 is next to the panning target 1 and the panning target 2, and it is assumed that the virtual sound source is the virtual sound source 602-2. The attenuation calculation unit 109 determines whether or not there is a sound collection target between the virtual sound source whose distance from the virtual viewpoint 601 is the shortest and the third closest virtual sound source, and changes the panning target 2 according to the result. That is, in this example, when the sound collection target 603 is between the virtual sound source 602-1 and the virtual sound source 602-2, the panning target 2 is changed from the virtual sound source 602-3 to the virtual sound source 602-2.

[0061] In step S807, the attenuation amount calculation unit 109 calculates the panning amount to be applied to the sound data to be panned. The process of this step S807 is similar to the process of step S506 in the process shown in FIG. 5 in the first embodiment, so a detailed description will be omitted.

[0062] The processes from step S808 to step S813 are performed for each of the plurality of acoustic data acquired in step S804. In step S808, the attenuation calculation unit 109 determines whether or not to calculate the attenuation of the target acoustic data based only on distance attenuation. If the attenuation calculation unit 109 determines not to calculate the attenuation based only on distance attenuation (NO), the process of step S809 is executed, and if it determines to calculate the attenuation based only on distance attenuation (YES), the process of step S810 is executed.

[0063] In step S809, the attenuation calculation unit 109 determines whether or not to calculate the attenuation of the target acoustic data by only panning. If the attenuation calculation unit 109 determines to calculate the attenuation by only panning (YES), the process of step S811 is executed, and if it determines not to calculate the attenuation by only panning (NO), the process of step S812 is executed.

[0064] In step S810, the attenuation calculation unit 109 calculates the attenuation amount based only on the distance attenuation, so that the distance attenuation coefficient α is set to 1 and the panning coefficient β is set to 0. In step S811, the attenuation calculation unit 109 calculates the amount of attenuation based on only panning, so that the distance attenuation coefficient α is set to 0 and the panning coefficient β is set to 1. In step S812, the attenuation amount calculation unit 109 calculates the amount of attenuation by combining distance attenuation and panning, so that a distance attenuation coefficient α and a panning coefficient β are set according to the position of the virtual viewpoint. In step S813, the attenuation calculation unit 109 calculates the attenuation to be applied to each piece of acoustic data.

[0065] The processing in steps S808 to S813 is similar to the processing in steps S507 to S512 in the processing shown in FIG. 5 in the first embodiment, and therefore a detailed description thereof will be omitted. The processes of steps S808 to S813 are performed for all acoustic data, and when the calculation of the amount of attenuation of each acoustic data is completed, the process of step S814 is performed.

[0066] In step S814, the attenuation amount calculation unit 109 outputs the attenuation amount of each sound data calculated in step S813 to the attenuation processing unit 103. In step S815, the attenuation processing unit 103 performs attenuation processing on each piece of acoustic data acquired from the memory unit 102 based on the attenuation amount of each piece of acoustic data obtained in step S513, and outputs the attenuated acoustic data to the surround processing unit 104. In step S816, the surround processing unit 104 performs surround processing on the sound data input from the attenuation processing unit 103 based on the virtual sound source position and the virtual viewpoint, and outputs the surround-processed sound data to the output unit 105 as virtual viewpoint sound data. In step S817, the output unit 105 generates a virtual viewpoint sound signal for reproduction based on the virtual viewpoint sound data input from the surround processing unit 104, and outputs the generated signal to the outside. The process of steps S814 to S817 is similar to the process of steps S513 to S516 in the process shown in FIG. 5 in the first embodiment, and therefore a detailed description thereof will be omitted.

[0067] According to the second embodiment, two virtual sound sources to be panned are selected based on the trajectory (position) of the sound collection target, and the attenuation amount for the two virtual sound sources is controlled based on the positional relationship between the two virtual sound sources and the virtual sound source. By controlling in this way, when the virtual viewpoint moves near the virtual sound source, the volume change of the virtual sound source corresponding to the position of the sound collection target becomes linear, reducing the sense of incongruity in hearing, and it is possible to generate virtual viewpoint sound that does not sound uncongruent in hearing.

[0068] <Other embodiments of the present invention> The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiment is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0069] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0070] The disclosure of the present embodiment includes the following configurations, methods, etc. (Configuration 1) A signal processing device that generates a sound corresponding to a virtual viewpoint based on sound data of a plurality of virtual sound sources corresponding to a plurality of collected sound signals, a calculation means for calculating an amount of sound attenuation for each virtual sound source based on the set positions of the virtual viewpoint and the set positions of the virtual sound sources; an attenuation processing means for performing an attenuation process on the acoustic data based on the calculated attenuation amount; a generating means for generating a sound corresponding to the virtual viewpoint based on the attenuation-processed sound data, The signal processing device according to claim 1, wherein the calculation means switches a method for calculating the amount of attenuation depending on a position of the virtual viewpoint with respect to the virtual sound source. (Configuration 2) The signal processing device according to configuration 1, wherein the calculation means switches a calculation method of the attenuation amount for two virtual sound sources, a first virtual sound source and a second virtual sound source, among the plurality of virtual sound sources, depending on a position of the virtual viewpoint relative to the virtual sound source. (Configuration 3) 3. The signal processing device according to configuration 2, wherein the first virtual sound source and the second virtual sound source are virtual sound sources whose distance from a virtual viewpoint to the virtual sound source is shorter than that of the other virtual sound sources. (Configuration 4) The calculation means is When the virtual viewpoint is in a first region outside a first boundary, the attenuation amount is calculated based on distance attenuation; When the virtual viewpoint is in a second region within a second boundary different from the first boundary, the attenuation amount is calculated by panning; The signal processing device according to configuration 2 or 3, wherein when the virtual viewpoint is in a third region between the first boundary and the second boundary, the amount of attenuation is calculated by at least one of distance attenuation and panning. (Configuration 5) The signal processing device according to configuration 4, characterized in that when the virtual viewpoint is in the third area, the calculation means calculates the attenuation amount by mixing an attenuation amount due to distance attenuation and an attenuation amount due to panning in a ratio according to a distance of the virtual viewpoint to the first virtual sound source and the second virtual sound source. (Configuration 6) The calculation means calculates a distance from a straight line connecting the first virtual sound source and the second virtual sound source to the virtual viewpoint as follows: If the distance is longer than the first threshold, the attenuation amount is calculated based on distance attenuation; If the time is shorter than the second threshold, the attenuation is calculated by panning; 4. The signal processing device according to configuration 2 or 3, wherein when the distance is shorter than the first threshold and longer than the second threshold, the amount of attenuation is calculated based on at least one of distance attenuation and panning. (Configuration 7) The signal processing device according to configuration 6, characterized in that when a distance from a straight line connecting the first virtual sound source and the second virtual sound source to the virtual viewpoint is shorter than the first threshold value and longer than the second threshold value, the calculation means calculates the attenuation amount by mixing an attenuation amount due to distance attenuation and an attenuation amount due to panning in a ratio according to the distance. (Configuration 8) The signal processing device according to any one of configurations 2 to 7, wherein the calculation means calculates the attenuation amount for the remaining virtual sound sources excluding the first virtual sound source and the second virtual sound source based on distance attenuation. (Configuration 9) The signal processing device according to any one of configurations 2 to 8, wherein the calculation means changes one of the two virtual sound sources for which the calculation method of the attenuation amount is switched based on the position of a sound collection target. (Configuration 10) a boundary setting means for setting boundary information for switching the attenuation amount calculation method; 10. The signal processing device according to any one of configurations 1 to 9, wherein the calculation means calculates the amount of attenuation based on the position of the virtual viewpoint, the position of the virtual sound source, and the boundary information. (Method 1) A method for controlling a signal processing device that generates a sound corresponding to a virtual viewpoint based on sound data of a plurality of virtual sound sources corresponding to a plurality of picked-up signals, the method comprising the steps of: A calculation step of calculating an amount of sound attenuation for each virtual sound source based on the set positions of the virtual viewpoint and the set positions of the virtual sound source; an attenuation processing step of performing an attenuation process on the acoustic data based on the calculated attenuation amount; a generating step of generating a sound corresponding to the virtual viewpoint based on the attenuation-processed sound data, A control method for a signal processing device, wherein in the calculation step, a method for calculating the amount of attenuation is switched depending on a position of the virtual viewpoint with respect to the virtual sound source. (Program 1) A computer of a signal processing device that generates sound corresponding to a virtual viewpoint based on sound data of a plurality of virtual sound sources corresponding to a plurality of collected sound signals, a calculation step of calculating an amount of sound attenuation for each virtual sound source based on the set positions of the virtual viewpoint and the set positions of the virtual sound source; an attenuation processing step of performing an attenuation process on the acoustic data based on the calculated attenuation amount; a generating step of generating sound corresponding to the virtual viewpoint based on the attenuation-processed sound data; The calculation step switches a method for calculating the amount of attenuation depending on a position of the virtual viewpoint with respect to the virtual sound source. [Explanation of symbols]

[0071] 101: Input unit 102: Storage unit 103: Attenuation processing unit 104: Surround processing unit 105: Output unit 106: Boundary setting unit 107: Virtual sound source position setting unit 108: Virtual viewpoint setting unit 109: Attenuation amount calculation unit 701: Sound collection target position setting unit

Claims

1. A signal processing device that generates a sound corresponding to a virtual viewpoint based on sound data of a plurality of virtual sound sources corresponding to a plurality of collected sound signals, a calculation means for calculating an amount of sound attenuation for each virtual sound source based on the set positions of the virtual viewpoint and the set positions of the virtual sound sources; an attenuation processing means for performing an attenuation process on the acoustic data based on the calculated attenuation amount; a generating means for generating a sound corresponding to the virtual viewpoint based on the attenuation-processed sound data, The signal processing device according to claim 1, wherein the calculation means switches a method for calculating the amount of attenuation depending on a position of the virtual viewpoint with respect to the virtual sound source.

2. The signal processing device according to claim 1, characterized in that the calculation means switches a method of calculating the amount of attenuation for two virtual sound sources, a first virtual sound source and a second virtual sound source, among the plurality of virtual sound sources, depending on a position of the virtual viewpoint relative to the virtual sound source.

3. The signal processing device according to claim 2 , wherein the first virtual sound source and the second virtual sound source are virtual sound sources whose distance from a virtual viewpoint to the virtual sound source is shorter than that of the other virtual sound sources.

4. The calculation means is When the virtual viewpoint is in a first region outside a first boundary, the attenuation amount is calculated based on distance attenuation; When the virtual viewpoint is in a second region within a second boundary different from the first boundary, the amount of attenuation is calculated by panning; 3. The signal processing device according to claim 2, wherein when the virtual viewpoint is in a third region between the first boundary and the second boundary, the amount of attenuation is calculated by at least one of distance attenuation and panning.

5. The signal processing device according to claim 4, characterized in that, when the virtual viewpoint is in the third area, the calculation means calculates the attenuation amount by mixing an attenuation amount due to distance attenuation and an attenuation amount due to panning in a ratio corresponding to a distance of the virtual viewpoint to the first virtual sound source and the second virtual sound source.

6. The calculation means calculates a distance from a straight line connecting the first virtual sound source and the second virtual sound source to the virtual viewpoint as follows: If the distance is longer than the first threshold, the attenuation amount is calculated based on distance attenuation; If the time is shorter than the second threshold, the attenuation is calculated by panning; 3. The signal processing apparatus according to claim 2, wherein when the distance is shorter than the first threshold value and longer than the second threshold value, the amount of attenuation is calculated based on at least one of distance attenuation and panning.

7. The signal processing device according to claim 6, characterized in that, when a distance from a straight line connecting the first virtual sound source and the second virtual sound source to the virtual viewpoint is shorter than the first threshold value and longer than the second threshold value, the calculation means calculates the attenuation amount by mixing an attenuation amount due to distance attenuation and an attenuation amount due to panning in a ratio corresponding to the distance.

8. 3. The signal processing apparatus according to claim 2, wherein the calculation means calculates the amount of attenuation for the remaining virtual sound sources excluding the first virtual sound source and the second virtual sound source based on distance attenuation.

9. 3. The signal processing device according to claim 2, wherein the calculation means changes one of the two virtual sound sources for which the calculation method of the amount of attenuation is switched based on a position of a sound collection target.

10. a boundary setting means for setting boundary information for switching the attenuation amount calculation method; 2 . The signal processing device according to claim 1 , wherein the calculation means calculates the amount of attenuation based on the position of the virtual viewpoint, the position of the virtual sound source, and the boundary information.

11. A method for controlling a signal processing device that generates a sound corresponding to a virtual viewpoint based on sound data of a plurality of virtual sound sources corresponding to a plurality of picked-up sound signals, the method comprising the steps of: A calculation step of calculating an amount of sound attenuation for each virtual sound source based on the set positions of the virtual viewpoint and the set positions of the virtual sound source; an attenuation processing step of performing an attenuation process on the acoustic data based on the calculated attenuation amount; a generating step of generating a sound corresponding to the virtual viewpoint based on the attenuation-processed sound data, A control method for a signal processing device, wherein in the calculation step, a method for calculating the amount of attenuation is switched depending on a position of the virtual viewpoint with respect to the virtual sound source.

12. A computer of a signal processing device that generates sound corresponding to a virtual viewpoint based on sound data of a plurality of virtual sound sources corresponding to a plurality of collected sound signals, a calculation step of calculating an amount of sound attenuation for each virtual sound source based on the set positions of the virtual viewpoint and the set positions of the virtual sound source; an attenuation processing step of performing an attenuation process on the acoustic data based on the calculated attenuation amount; a generating step of generating sound corresponding to the virtual viewpoint based on the attenuation-processed sound data; The calculation step switches a method for calculating the amount of attenuation depending on a position of the virtual viewpoint with respect to the virtual sound source.

Citation Information

Patent Citations

  • Surround sound mixing apparatus and program for the same

    JP2005223771A