A method and system for realizing metaverse immersive audio

Abstract

The present invention provides a method and system for realizing immersive audio in a metaverse, and the method comprises: step 1: constructing a multi-channel microphone array, and collecting a first sound signal in an environment in real time based on the multi-channel microphone array; step 2: performing signal processing on the first sound signal to obtain a second sound signal, and analyzing the second sound signal to determine the position distribution state, and encoding the second sound signal at the same time; step 3: decoding the second sound signal in the first virtual space of the metaverse, and performing spatial audio rendering on the decoded second sound signal according to the position distribution state to generate a second virtual space; step 4: constructing an interactive management terminal, and interactively managing the user terminal and the second virtual space based on the interactive management terminal. A highly realistic immersive sound effect with a sense of space and orientation is realized in the metaverse. This will bring users a more realistic and rich virtual environment experience.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for realizing metaverse immersive audio. Background Art

[0002] At present, with the rapid development of metaverse technology, users have higher and higher requirements for sensory experience in virtual environments; the rise of the metaverse concept has brought people a new imagination space and experience possibilities. In this virtual world, the feeling of being in the scene is crucial, and the sound system, as a key element in creating atmosphere and enhancing immersion, has great development potential;

[0003] However, in traditional sound systems, it is often difficult to fully simulate the sound effects in the real environment, and it is impossible to meet the requirements of high authenticity and spatial sense of sound in the metaverse, nor to achieve highly realistic immersive sound effects with spatial sense and orientation in the metaverse, which is not conducive to ensuring user experience.

[0004] Therefore, in order to overcome the above-mentioned technical problems, the present invention provides a method and system for implementing metaverse immersive audio. Summary of the invention

[0005] The present invention provides a method and system for realizing immersive audio in the metaverse, which can effectively guarantee the quality of the sound signal by constructing a multi-channel microphone array and acquiring the first sound signal in the environment in real time and performing signal processing to obtain the second sound signal, thereby facilitating accurate analysis of the second sound signal and determining the position distribution state of the second sound signal; by encoding and decoding the second sound signal, the high fidelity of the sound during transmission can be effectively guaranteed, and the spatial audio rendering in the first virtual space of the metaverse can be realized according to the position distribution state through the second sound signal, thereby improving the user experience, and by constructing an interactive management terminal, the user's interaction and immersive experience can be effectively realized; based on the above, a highly realistic immersive sound effect with a sense of space and orientation is realized in the metaverse. This will bring users a more realistic and rich virtual environment experience.

[0006] A method for realizing metaverse immersive audio, comprising:

[0007] Step 1: construct a multi-channel microphone array, and collect a first sound signal in the environment in real time based on the multi-channel microphone array;

[0008] Step 2: Processing the first sound signal to obtain a second sound signal, analyzing the second sound signal to determine the position distribution state, and encoding the second sound signal;

[0009] Step 3: Decode the second sound signal in the first virtual space of the metaverse, and perform spatial audio rendering on the decoded second sound signal according to the position distribution state to generate a second virtual space;

[0010] Step 4: Construct an interactive management terminal, and perform interactive management between the user terminal and the second virtual space based on the interactive management terminal.

[0011] Preferably, a method for implementing Metaverse immersive audio, in step 1, constructing a multi-channel microphone array, comprising:

[0012] Obtaining the distribution space of the sound to be collected;

[0013] Analyze the distribution space of the sound to be collected to obtain the spatial characteristics of the distribution space, and divide the distribution space according to the spatial characteristics of the distribution space to obtain multiple sub-distribution spaces;

[0014] Get the microphone's sound reception strength;

[0015] Determine the optimal distance between the multiple microphones according to the sound pickup strength of the microphones;

[0016] Sorting the positions in each sub-distribution space according to the optimal distances between the plurality of microphones, and obtaining a sub-microphone array in each sub-distribution space according to the sorting result;

[0017] The sub-microphone arrays in the multiple sub-distribution spaces are integrated according to the positional relationship between each sub-distribution space to generate a multi-channel microphone array.

[0018] Preferably, a method for realizing metaverse immersive sound analyzes the distribution space of the collected sound to obtain the spatial characteristics of the distribution space, and divides the distribution space according to the spatial characteristics of the distribution space to obtain multiple sub-distribution spaces, including:

[0019] Read the distribution space of the sound to be collected, and determine the three-dimensional coordinate points of the edge contour of the distribution space;

[0020] Collecting multiple sub-three-dimensional coordinate point sets corresponding to the same quadrant coordinate value in the edge contour three-dimensional coordinate points, and determining the contour plane of the distribution space according to the multiple sub-three-dimensional coordinate point sets;

[0021] Determine a planar relationship between each contour plane and the remaining contour planes based on the sub-three-dimensional coordinate point sets corresponding to the plurality of contour planes;

[0022] Dividing the distribution space according to a preset space division condition and based on the planar relationship between each contour plane and the remaining contour planes;

[0023] A plurality of sub-distribution spaces are obtained based on the partitioning result.

[0024] Preferably, a method for implementing a metaverse immersive sound system determines an optimal distance between multiple microphones according to the sound pickup strength of the microphones, including:

[0025] Obtaining the position point of the user in the distribution space, and when constructing the microphone array, obtaining the initial intensity value corresponding to the current microphone based on the position point of the user in the distribution space;

[0026] Set the initial distance between the current microphone and the adjacent microphones in advance;

[0027] Get the real-time sound reception strength value of the current microphone at the initial distance;

[0028] Compare the real-time sound reception intensity value with the initial intensity value to determine whether the initial distance needs to be adjusted;

[0029] If the real-time sound reception intensity value is equal to or greater than the initial intensity value, it is determined that the initial distance does not need to be adjusted, and the initial distance is used as the optimal distance between the multiple microphones;

[0030] If the real-time sound reception intensity value is less than the initial intensity value, the initial distance is adjusted until the real-time sound reception intensity value is equal to or greater than the initial intensity value, and at the same time, the target adjustment distance is obtained;

[0031] An adjustment direction of a target adjustment distance is obtained, and optimal distances between the plurality of microphones are calculated according to the adjustment direction, the target adjustment distance, and the initial distance.

[0032] Preferably, a method for implementing the Metaverse immersive audio system, in step 1, real-time acquisition of a first sound signal in an environment includes:

[0033] Configure the interface of the multi-channel microphone array;

[0034] Based on the interface configuration result, the microphone array is controlled to collect the first sound signal in the environment in real time.

[0035] Preferably, a method for implementing a metaverse immersive sound system, in step 2, performing signal processing on the first sound signal to obtain a second sound signal, includes:

[0036] Reading the signal characteristics of the first sound signal, and at the same time, obtaining the standard signal characteristics;

[0037] Matching the signal characteristics of the first sound signal with the standard signal characteristics to determine the difference characteristics between the signal characteristics of the first sound signal and the standard signal characteristics;

[0038] Determining a data processing dimension for processing the first sound signal according to the difference feature;

[0039] Match the data processing dimension in a preset sound signal processing library to determine the data processing algorithm corresponding to each data processing dimension;

[0040] The data processing algorithms corresponding to the multiple data processing dimensions are encapsulated, and the first sound signal is processed based on the encapsulation result to obtain the second sound signal.

[0041] Preferably, a method for implementing a metaverse immersive sound system, in step 2, analyzing the second sound signal to determine the position distribution state, and encoding the second sound signal, including:

[0042] Positioning the second sound signal according to the multi-channel microphone array, and determining difference information of the second sound signal when arriving at different microphones;

[0043] Determine the time delay between the second sound signal reaching different microphones according to the difference information;

[0044] Read the position distribution relationship of each microphone in the multi-channel microphone array;

[0045] Analyze the time delay between the second sound signal reaching different microphones and the position distribution relationship of each microphone in the multi-channel microphone array to determine the position distribution state, wherein the position distribution state includes: the sound source and direction of the second sound signal;

[0046] At the same time, a preset encoding algorithm is retrieved, and the second sound signal is encoded according to the preset encoding algorithm.

[0047] Preferably, a method for implementing immersive audio in the metaverse, in step 3, decoding is performed in the first virtual space of the metaverse, and spatial audio rendering is performed on the decoded second sound signal according to the position distribution state to generate the second virtual space, including:

[0048] Acquire a coding rule for encoding the second sound signal in the first virtual space of the metaverse, and decode the second sound signal accordingly based on the coding rule;

[0049] The decoded second sound signal is simulated and processed in the first virtual space according to the position distribution state, and at the same time, a signal transmission algorithm is retrieved;

[0050] Construct an audio adaptation model based on the signal transmission algorithm;

[0051] Obtaining the user's current position data and head orientation data, and inputting the user's current position data and head orientation data into the audio adaptive model for analysis, and outputting sound simulation parameters;

[0052] Audio configuration is performed in the first virtual space based on the sound simulation parameters, spatial audio rendering of the first virtual space is completed based on the audio configuration, and a second virtual space is generated based on the spatial audio rendering result.

[0053] Preferably, a method for implementing a metaverse immersive audio system, in step 4, constructing an interactive management terminal, and performing interactive management between a user terminal and a second virtual space based on the interactive management terminal, including:

[0054] Constructing an interactive management terminal based on the user terminal and the second virtual space;

[0055] When the user terminal generates a parameter adjustment request, the interactive management terminal parses the parameter adjustment request to determine the adjustment type and the parameter value corresponding to the adjustment type;

[0056] The interactive management terminal controls the second virtual space to adjust parameters based on the adjustment type and the parameter value corresponding to the adjustment type, thereby completing the interactive management between the user terminal and the second virtual space.

[0057] A system for realizing a metaverse immersive sound system, comprising:

[0058] A sound collection module is used to construct a multi-channel microphone array and collect the first sound signal in the environment in real time based on the multi-channel microphone array;

[0059] An audio processing module, used for performing signal processing on the first sound signal to obtain a second sound signal, analyzing the second sound signal to determine a position distribution state, and encoding the second sound signal;

[0060] A spatial audio rendering module is used to decode the second sound signal in the first virtual space of the metaverse, and perform spatial audio rendering on the decoded second sound signal according to a position distribution state to generate a second virtual space;

[0061] The interaction management module is used to construct an interaction management terminal and perform interaction management between the user terminal and the second virtual space based on the interaction management terminal.

[0062] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.

[0063] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0065] Figure 1 This is a flow chart of a method for implementing metaverse immersive audio in an embodiment of the present invention;

[0066] Figure 2 This is a flowchart of step 1 in a method for implementing metaverse immersive audio in an embodiment of the present invention;

[0067] Figure 3 A system structure diagram for implementing a metaverse immersive audio system in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0069] Embodiment 1:

[0070] This embodiment provides a method for implementing the metaverse immersive audio system. Figure 1 As shown, including:

[0071] Step 1: construct a multi-channel microphone array, and collect a first sound signal in the environment in real time based on the multi-channel microphone array;

[0072] Step 2: Processing the first sound signal to obtain a second sound signal, analyzing the second sound signal to determine the position distribution state, and encoding the second sound signal;

[0073] Step 3: Decode the second sound signal in the first virtual space of the metaverse, and perform spatial audio rendering on the decoded second sound signal according to the position distribution state to generate a second virtual space;

[0074] Step 4: Construct an interactive management terminal, and perform interactive management between the user terminal and the second virtual space based on the interactive management terminal.

[0075] In this embodiment, the multi-channel microphone array can be constructed in the distribution space of the sound to be collected based on the spatial characteristics of the distribution space of the sound to be collected and the sound collection strength of the microphone.

[0076] In this embodiment, the first sound signal may be based on a sound signal in a real-time acquisition environment.

[0077] In this embodiment, the signal processing of the first sound signal may include: performing noise reduction, echo cancellation, audio gain and other processing on the first sound signal to obtain the second sound signal in order to ensure the sound quality.

[0078] In this embodiment, the high-fidelity audio coding and decoding technology is used to encode and decode the second sound signal in order to ensure high fidelity of the sound signal during transmission.

[0079] In this embodiment, the first virtual space may be the initial virtual space in the metaverse, and the second virtual space may be generated by performing spatial audio rendering on the second sound signal according to a position distribution state, wherein the position distribution state of the sound signal may include: the source and direction of the sound, and through the sound source localization algorithm, the direction and distance of the sound source are accurately determined, and the directional transmission of the sound signal is achieved in combination with the user's position and head orientation.

[0080] In this embodiment, spatial audio rendering can be used to effectively simulate the characteristics of sound in three-dimensional space, such as position, direction, distance, and movement, based on the position distribution state.

[0081] In this embodiment, the interactive management terminal is constructed to provide a user-friendly interactive interface, facilitate users to set parameters and adjust sound effects, support seamless connection with other metaverse devices, and realize multi-sensory interaction and immersive experience.

[0082] The working principle and beneficial effects of the above technical solution are: by constructing a multi-channel microphone array and collecting the first sound signal in the environment in real time and performing signal processing, the second sound signal can be obtained to effectively ensure the quality of the sound signal, which is conducive to accurate analysis of the second sound signal and determination of the position distribution state of the second sound signal; by encoding and decoding the second sound signal, the high fidelity of the sound during transmission can be effectively guaranteed, and the spatial audio rendering in the first virtual space of the metaverse can be realized according to the position distribution state through the second sound signal, thereby improving the user experience, and by constructing an interactive management terminal, the user's interaction and immersive experience can be effectively realized; based on the above, a highly realistic immersive sound effect with a sense of space and orientation is realized in the metaverse. This will bring users a more realistic and rich virtual environment experience.

[0083] Embodiment 2:

[0084] Based on Example 1, this example provides a method for implementing the metaverse immersive audio system. Figure 2 As shown, in step 1, a multi-channel microphone array is constructed, including:

[0085] S101: Obtaining the distribution space of the sound to be collected;

[0086] S102: Analyze the distribution space of the sound to be collected to obtain spatial characteristics of the distribution space, and divide the distribution space according to the spatial characteristics of the distribution space to obtain multiple sub-distribution spaces;

[0087] S103: Obtaining the sound reception strength of the microphone;

[0088] S104: Determine the optimal distance between the multiple microphones according to the sound reception strength of the microphones;

[0089] S105: sorting the positions in each sub-distribution space according to the optimal distances between the multiple microphones, and obtaining a sub-microphone array in each sub-distribution space according to the sorting result;

[0090] S106: Integrate the sub-microphone arrays in the multiple sub-distribution spaces according to the positional relationship between each sub-distribution space to generate a multi-channel microphone array.

[0091] In this embodiment, the spatial characteristics of the distribution space represent the distribution conditions of each plane in the distribution space, and the corresponding sub-distribution space such as a cuboid or a cube is determined according to the distribution conditions of each plane.

[0092] In this embodiment, the positions of the microphones are sorted in each sub-distribution space according to the optimal distances between the multiple microphones, that is, one-to-one ranking is achieved in each sub-distribution space according to the optimal distances between the microphones, and the microphones are distributed accordingly in each sub-distribution space, thereby obtaining a sub-microphone array.

[0093] The beneficial effects of the above technical solution are: by conducting a reasonable and effective analysis of the distribution space, the sub-distribution space can be acquired, and by determining the optimal distance between multiple microphones, the effectiveness of the microphone position distribution in each sub-distribution space can be effectively guaranteed, thereby ensuring the effectiveness and accuracy of generating a multi-channel microphone array, thereby laying the foundation for realizing metaverse immersive audio.

[0094] Embodiment 3:

[0095] On the basis of Example 2, this embodiment provides a method for realizing metaverse immersive sound, analyzing the distribution space of the collected sound to obtain the spatial characteristics of the distribution space, and dividing the distribution space according to the spatial characteristics of the distribution space to obtain multiple sub-distribution spaces, including:

[0096] Read the distribution space of the sound to be collected, and determine the three-dimensional coordinate points of the edge contour of the distribution space;

[0097] Collecting multiple sub-three-dimensional coordinate point sets corresponding to the same quadrant coordinate value in the edge contour three-dimensional coordinate points, and determining the contour plane of the distribution space according to the multiple sub-three-dimensional coordinate point sets;

[0098] Determine a planar relationship between each contour plane and the remaining contour planes based on the sub-three-dimensional coordinate point sets corresponding to the plurality of contour planes;

[0099] Dividing the distribution space according to a preset space division condition and based on the planar relationship between each contour plane and the remaining contour planes;

[0100] A plurality of sub-distribution spaces are obtained based on the partitioning result.

[0101] In this embodiment, one sub-three-dimensional coordinate point set corresponds to one contour plane.

[0102] In this embodiment, the plane relationship may include: a parallel relationship and an intersection relationship between the current contour plane and the remaining contour planes. If there is a target contour plane among the remaining contour planes that is neither parallel nor intersecting with the current contour plane, it is determined that there is no relationship between the current contour plane and the target contour plane.

[0103] In this embodiment, the preset space division conditions may include: the shape of the space divided in the distribution space, for example, if it is a rectangular parallelepiped: the first contour plane parallel to the current contour plane is of the same size, and the second contour plane intersecting with the current contour plane is of the same size as the third contour plane parallel to the second contour plane, and the third contour plane intersects with the current contour plane and the first contour plane respectively, then the space formed by the current contour plane, the first contour plane, the second contour plane and the third contour plane is the sub-distribution space.

[0104] The beneficial effects of the above technical solution are: by conducting a reasonable, effective and detailed analysis of the distribution space, the sub-distribution space can be acquired, thereby ensuring the accuracy and effectiveness of the division of the distribution space, realizing the automatic division of the distribution space, and thus improving the efficiency of microphone arrangement.

[0105] Embodiment 4:

[0106] Based on Example 2, this embodiment provides a method for implementing a metaverse immersive sound system, which determines the optimal distance between multiple microphones according to the sound reception strength of the microphones, including:

[0107] Obtaining the position point of the user in the distribution space, and when constructing the microphone array, obtaining the initial intensity value corresponding to the current microphone based on the position point of the user in the distribution space;

[0108] Set the initial distance between the current microphone and the adjacent microphones in advance;

[0109] Get the real-time sound reception strength value of the current microphone at the initial distance;

[0110] Compare the real-time sound reception intensity value with the initial intensity value to determine whether the initial distance needs to be adjusted;

[0111] If the real-time sound reception intensity value is equal to or greater than the initial intensity value, it is determined that the initial distance does not need to be adjusted, and the initial distance is used as the optimal distance between the multiple microphones;

[0112] If the real-time sound reception intensity value is less than the initial intensity value, the initial distance is adjusted until the real-time sound reception intensity value is equal to or greater than the initial intensity value, and at the same time, the target adjustment distance is obtained;

[0113] An adjustment direction of a target adjustment distance is obtained, and optimal distances between the plurality of microphones are calculated according to the adjustment direction, the target adjustment distance, and the initial distance.

[0114] In this embodiment, the optimal distance is the sum of the absolute values ​​of the initial distance and the target adjustment distance, which is determined by the adjustment direction. If the adjustment direction is to adjust to the right based on the initial distance, the optimal distance = initial distance + target adjustment distance; if the adjustment direction is to adjust to the left based on the initial distance, the optimal distance = initial distance - target adjustment distance.

[0115] In this embodiment, the initial strength value may be set in advance.

[0116] The beneficial effect of the above technical solution is: by determining the initial distance between the current microphone and the adjacent microphone, and reading the real-time sound reception intensity of the current microphone, the real-time sound reception intensity is effectively used to determine the optimal distance based on the initial distance, thereby improving the effectiveness and accuracy of obtaining the optimal distance, and ensuring the effectiveness of the multi-channel microphone array distribution.

[0117] Embodiment 5:

[0118] Based on Example 1, this example provides a method for implementing Metaverse immersive audio. In step 1, a first sound signal in an environment is collected in real time, including:

[0119] Configure the interface of the multi-channel microphone array;

[0120] Based on the interface configuration result, the microphone array is controlled to collect the first sound signal in the environment in real time.

[0121] In this embodiment, the interface configuration may be used to configure parameters of the multi-channel microphone array, including operating current, volume, and other parameters of the multi-channel microphone array.

[0122] The beneficial effect of the above technical solution is that it ensures the effectiveness of real-time collection of the first sound signal.

[0123] Embodiment 6:

[0124] Based on Example 1, this embodiment provides a method for implementing a metaverse immersive sound system. In step 2, the first sound signal is processed to obtain a second sound signal, including:

[0125] Reading the signal characteristics of the first sound signal, and at the same time, obtaining the standard signal characteristics;

[0126] Matching the signal characteristics of the first sound signal with the standard signal characteristics to determine the difference characteristics between the signal characteristics of the first sound signal and the standard signal characteristics;

[0127] Determining a data processing dimension for processing the first sound signal according to the difference feature;

[0128] Match the data processing dimension in a preset sound signal processing library to determine the data processing algorithm corresponding to each data processing dimension;

[0129] The data processing algorithms corresponding to the multiple data processing dimensions are encapsulated, and the first sound signal is processed based on the encapsulation result to obtain the second sound signal.

[0130] In this embodiment, the first sound signal may be a sound signal that has not been processed.

[0131] In this embodiment, the second sound signal may be a sound signal obtained by processing the first sound signal.

[0132] In this embodiment, the signal characteristics may include characteristics such as the signal frequency of the first sound signal, noise characteristics, and an audio echo signal.

[0133] In this embodiment, the standard signal feature may be a sound signal without noise, echo, etc., which can realize direct analysis of the signal and is free of interference.

[0134] In this embodiment, the data processing dimension may be a dimension that needs to be processed for the sound signal, specifically including: noise, echo, etc.

[0135] The beneficial effect of the above technical solution is: by determining the difference characteristics of processing the first sound signal, the data processing dimension is effectively acquired, and then by encapsulating the corresponding data processing algorithm and realizing signal processing of the first sound signal according to the encapsulation result, targeted signal processing of the first sound signal is effectively realized, which ensures the accuracy of obtaining the second sound signal, and also ensures the signal processing efficiency of the first sound signal, and at the same time, ensures the sound quality.

[0136] Embodiment 7:

[0137] Based on Example 1, this embodiment provides a method for implementing a metaverse immersive sound system. In step 2, the second sound signal is analyzed to determine a position distribution state, and at the same time, the second sound signal is encoded, including:

[0138] Positioning the second sound signal according to the multi-channel microphone array, and determining difference information of the second sound signal when arriving at different microphones;

[0139] Determine the time delay between the second sound signal reaching different microphones according to the difference information;

[0140] Read the position distribution relationship of each microphone in the multi-channel microphone array;

[0141] Analyze the time delay between the second sound signal reaching different microphones and the position distribution relationship of each microphone in the multi-channel microphone array to determine the position distribution state, wherein the position distribution state includes: the sound source and direction of the second sound signal;

[0142] At the same time, a preset encoding algorithm is retrieved, and the second sound signal is encoded according to the preset encoding algorithm.

[0143] In this embodiment, the difference information may include: sound difference, phase difference, sound intensity, etc.

[0144] In this embodiment, based on the analysis of the time delay between the second sound signal reaching different microphones and the position distribution relationship of each microphone in the multi-channel microphone array, the position distribution state can be determined, for example, by using a TDOA algorithm, by using known microphone positions and combining time delays (i.e., time delays) to determine the geometric relationship of the second sound signal, and then determine the source and direction of the second sound signal.

[0145] In this embodiment, the preset encoding algorithm may be set in advance and used to implement encoding of the second sound signal.

[0146] The working principle and beneficial effects of the above technical solution are: the second sound signal is positioned through a multi-channel microphone array, so as to effectively determine the difference information of the second sound signal when it arrives at different microphones, so as to effectively realize the determination of the time delay between the second sound signal reaching different microphones, and then analyze the relationship between the time delay between the second sound signal reaching different microphones and the position distribution of each microphone in the multi-channel microphone array, so as to effectively determine the position distribution state, thereby improving the effectiveness and accuracy of the determination of the position distribution state, and realizing the encoding of the second sound signal through a preset encoding algorithm, which can effectively ensure the high fidelity of the sound during the transmission process.

[0147] Embodiment 8:

[0148] Based on Example 1, this embodiment provides a method for implementing immersive audio in the metaverse. In step 3, decoding is performed in the first virtual space of the metaverse, and spatial audio rendering is performed on the decoded second sound signal according to the position distribution state to generate a second virtual space, including:

[0149] Acquire a coding rule for encoding the second sound signal in the first virtual space of the metaverse, and decode the second sound signal accordingly based on the coding rule;

[0150] The decoded second sound signal is simulated and processed in the first virtual space according to the position distribution state, and at the same time, a signal transmission algorithm is retrieved;

[0151] Construct an audio adaptation model based on the signal transmission algorithm;

[0152] Obtaining the user's current position data and head orientation data, and inputting the user's current position data and head orientation data into the audio adaptive model for analysis, and outputting sound simulation parameters;

[0153] Audio configuration is performed in the first virtual space based on the sound simulation parameters, spatial audio rendering of the first virtual space is completed based on the audio configuration, and a second virtual space is generated based on the spatial audio rendering result.

[0154] In this embodiment, the signal transmission algorithm constructs the audio adaptive model, wherein the signal transmission algorithm may be, for example, a HRTF transmission function, and the audio adaptive model is constructed by learning and training the signal transmission algorithm.

[0155] In this embodiment, the sound simulation parameters are determined by constructing an audio-only model. The sound simulation parameters can be determined by determining the user's current position data and head orientation data, and then analyzing and determining them through an audio adaptive model, including parameters such as the frequency response of the audio reaching the human ear.

[0156] In this embodiment, audio configuration is performed in the first virtual space based on sound simulation parameters, and spatial audio rendering of the first virtual space is completed based on the audio configuration. This can be achieved by simulating the frequency response differences when the sound reaches the human ear, thereby creating a feeling that the sound comes from different directions; using reverberation effects to simulate sound reflection and attenuation in different spatial environments; and adjusting audio parameters in real time according to the movement trajectory and speed of the sound source to enhance the dynamic effect.

[0157] The working principle and beneficial effects of the above technical solution are: by decoding the second sound signal accordingly, and then simulating the decoded second sound signal in the first virtual space according to the position distribution state, the effective simulation of the first virtual space can be effectively realized, thereby providing data support for building the second virtual space, and by calling the signal transmission algorithm, the audio adaptive model can be effectively constructed, and by obtaining the user's current position data and head orientation data, and inputting the user's current position data and head orientation data into the audio adaptive model for analysis, the sound simulation parameters can be effectively output, and by performing audio configuration and spatial audio rendering in the first virtual space with the sound simulation parameters, the generation of the second virtual space is realized, thereby improving the user experience, and by constructing an interactive management terminal, the user's interaction and immersive experience can be effectively realized.

[0158] Embodiment 9:

[0159] Based on Example 1, this embodiment provides a method for implementing a metaverse immersive audio system. In step 4, an interactive management terminal is constructed, and interactive management of a user terminal and a second virtual space is performed based on the interactive management terminal, including:

[0160] Constructing an interactive management terminal based on the user terminal and the second virtual space;

[0161] When the user terminal generates a parameter adjustment request, the interactive management terminal parses the parameter adjustment request to determine the adjustment type and the parameter value corresponding to the adjustment type;

[0162] The interactive management terminal controls the second virtual space to adjust parameters based on the adjustment type and the parameter value corresponding to the adjustment type, thereby completing the interactive management between the user terminal and the second virtual space.

[0163] In this embodiment, the adjustment type is, for example, an adjustment of the volume, sound effects, etc. of the sound when the second virtual space performs sound simulation, and the parameter value is a specific adjustment value of the corresponding adjustment type.

[0164] The working principle and beneficial effect of the above technical solution are: by constructing an interactive management terminal between the user terminal and the second virtual space, when the user terminal generates a parameter adjustment request, the parameters of the second virtual space can be effectively adjusted, making it convenient for users to set parameters and adjust sound effects.

[0165] Embodiment 10:

[0166] This embodiment provides a system for implementing a metaverse immersive audio system. Figure 3 As shown, including:

[0167] A sound collection module is used to construct a multi-channel microphone array and collect the first sound signal in the environment in real time based on the multi-channel microphone array;

[0168] An audio processing module, used for performing signal processing on the first sound signal to obtain a second sound signal, analyzing the second sound signal to determine a position distribution state, and encoding the second sound signal;

[0169] A spatial audio rendering module is used to decode the second sound signal in the first virtual space of the metaverse, and perform spatial audio rendering on the decoded second sound signal according to a position distribution state to generate a second virtual space;

[0170] The interaction management module is used to construct an interaction management terminal and perform interaction management between the user terminal and the second virtual space based on the interaction management terminal.

[0171] The working principle and beneficial effects of the above technical solution are: by constructing a multi-channel microphone array and collecting the first sound signal in the environment in real time and performing signal processing, the second sound signal can be obtained to effectively ensure the quality of the sound signal, which is conducive to accurate analysis of the second sound signal and determination of the position distribution state of the second sound signal; by encoding and decoding the second sound signal, the high fidelity of the sound during transmission can be effectively guaranteed, and the spatial audio rendering in the first virtual space of the metaverse can be realized according to the position distribution state through the second sound signal, thereby improving the user experience, and by constructing an interactive management terminal, the user's interaction and immersive experience can be effectively realized; based on the above, a highly realistic immersive sound effect with a sense of space and orientation is realized in the metaverse. This will bring users a more realistic and rich virtual environment experience.

[0172] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for implementing metaverse immersive audio, characterized in that: include: Step 1: construct a multi-channel microphone array, and collect a first sound signal in the environment in real time based on the multi-channel microphone array; Step 2: Processing the first sound signal to obtain a second sound signal, analyzing the second sound signal to determine the position distribution state, and encoding the second sound signal; Step 3: Decode the second sound signal in the first virtual space of the metaverse, and perform spatial audio rendering on the decoded second sound signal according to the position distribution state to generate a second virtual space; Step 4: construct an interaction management terminal, and perform interaction management between the user terminal and the second virtual space based on the interaction management terminal; In step 1, a multi-channel microphone array is constructed, including: Obtaining the distribution space of the sound to be collected; Analyze the distribution space of the sound to be collected to obtain the spatial characteristics of the distribution space, and divide the distribution space according to the spatial characteristics of the distribution space to obtain multiple sub-distribution spaces; Get the microphone's sound reception strength; Determine the optimal distance between the multiple microphones according to the sound pickup strength of the microphones; Sorting the positions in each sub-distribution space according to the optimal distances between the plurality of microphones, and obtaining a sub-microphone array in each sub-distribution space according to the sorting result; The sub-microphone arrays in the multiple sub-distribution spaces are integrated according to the positional relationship between each sub-distribution space to generate a multi-channel microphone array; The distribution space of the sound to be collected is analyzed to obtain the spatial characteristics of the distribution space, and the distribution space is divided according to the spatial characteristics of the distribution space to obtain multiple sub-distribution spaces, including: Read the distribution space of the sound to be collected, and determine the three-dimensional coordinate points of the edge contour of the distribution space; Collecting multiple sub-three-dimensional coordinate point sets corresponding to the same quadrant coordinate value in the edge contour three-dimensional coordinate points, and determining the contour plane of the distribution space according to the multiple sub-three-dimensional coordinate point sets; Determine a planar relationship between each contour plane and the remaining contour planes based on the sub-three-dimensional coordinate point sets corresponding to the plurality of contour planes; Dividing the distribution space according to a preset space division condition and based on the planar relationship between each contour plane and the remaining contour planes; Obtain multiple sub-distribution spaces based on the partitioning result; Determine the optimal distance between multiple microphones based on the sound pickup strength of the microphones, including: Obtaining the position point of the user in the distribution space, and when constructing the microphone array, obtaining the initial intensity value corresponding to the current microphone based on the position point of the user in the distribution space; Set the initial distance between the current microphone and the adjacent microphones in advance; Get the real-time sound reception strength value of the current microphone at the initial distance; Compare the real-time sound reception intensity value with the initial intensity value to determine whether the initial distance needs to be adjusted; If the real-time sound reception intensity value is equal to or greater than the initial intensity value, it is determined that the initial distance does not need to be adjusted, and the initial distance is used as the optimal distance between the multiple microphones; If the real-time sound reception intensity value is less than the initial intensity value, the initial distance is adjusted until the real-time sound reception intensity value is equal to or greater than the initial intensity value, and at the same time, the target adjustment distance is obtained; Obtaining an adjustment direction of a target adjustment distance, and calculating an optimal distance between the plurality of microphones according to the adjustment direction, the target adjustment distance, and the initial distance; In step 2, signal processing is performed on the first sound signal to obtain a second sound signal, including: Reading the signal characteristics of the first sound signal, and at the same time, obtaining the standard signal characteristics; Matching the signal characteristics of the first sound signal with the standard signal characteristics to determine the difference characteristics between the signal characteristics of the first sound signal and the standard signal characteristics; Determining a data processing dimension for processing the first sound signal according to the difference feature; Match the data processing dimension in a preset sound signal processing library to determine the data processing algorithm corresponding to each data processing dimension; encapsulate the data processing algorithms corresponding to the multiple data processing dimensions, and perform signal processing on the first sound signal based on the encapsulation result to obtain a second sound signal; In step 3, decoding is performed in the first virtual space of the metaverse, and spatial audio rendering is performed on the decoded second sound signal according to the position distribution state to generate a second virtual space, including: Acquire a coding rule for encoding the second sound signal in the first virtual space of the metaverse, and decode the second sound signal accordingly based on the coding rule; The decoded second sound signal is simulated and processed in the first virtual space according to the position distribution state, and at the same time, a signal transmission algorithm is retrieved; An audio adaptive model is constructed according to a signal transmission algorithm, wherein the signal transmission algorithm is trained by learning the signal transmission algorithm through a HRTF transmission function to construct an audio adaptive model; Obtaining the user's current position data and head orientation data, and inputting the user's current position data and head orientation data into the audio adaptive model for analysis, and outputting sound simulation parameters, wherein the sound simulation parameters are determined by determining the user's current position data and head orientation data and then analyzing and determining them through the audio adaptive model, including: frequency corresponding parameters of the audio reaching the human ear; Audio configuration is performed in the first virtual space based on sound simulation parameters, spatial audio rendering of the first virtual space is completed based on the audio configuration, and a second virtual space is generated based on the spatial audio rendering result, wherein audio configuration is performed in the first virtual space based on sound simulation parameters, and spatial audio rendering of the first virtual space is completed based on the audio configuration by simulating the frequency response difference when the sound reaches the human ear, thereby creating a feeling that the sound comes from different directions, using reverberation effects to simulate sound reflection and attenuation in different spatial environments, and adjusting audio parameters in real time according to the movement trajectory and speed of the sound source to enhance the dynamic effect.

2. The method for realizing the metaverse immersive audio according to claim 1, characterized in that: In step 1, collecting a first sound signal in an environment in real time includes: Configure the interface of the multi-channel microphone array; Based on the interface configuration result, the microphone array is controlled to collect the first sound signal in the environment in real time.

3. The method for realizing the metaverse immersive audio according to claim 1, characterized in that: In step 2, the second sound signal is analyzed to determine the position distribution state, and the second sound signal is encoded, including: Positioning the second sound signal according to the multi-channel microphone array, and determining difference information of the second sound signal when arriving at different microphones; Determine the time delay between the second sound signal reaching different microphones according to the difference information; Read the position distribution relationship of each microphone in the multi-channel microphone array; Analyze the time delay between the second sound signal reaching different microphones and the position distribution relationship of each microphone in the multi-channel microphone array to determine the position distribution state, wherein the position distribution state includes: the sound source and direction of the second sound signal; At the same time, a preset encoding algorithm is retrieved, and the second sound signal is encoded according to the preset encoding algorithm.

4. The method for realizing the metaverse immersive audio according to claim 1, characterized in that: In step 4, an interaction management terminal is constructed, and interaction management between the user terminal and the second virtual space is performed based on the interaction management terminal, including: Constructing an interactive management terminal based on the user terminal and the second virtual space; When the user terminal generates a parameter adjustment request, the interactive management terminal parses the parameter adjustment request to determine the adjustment type and the parameter value corresponding to the adjustment type; The interactive management terminal controls the second virtual space to adjust parameters based on the adjustment type and the parameter value corresponding to the adjustment type, thereby completing the interactive management between the user terminal and the second virtual space.

5. A system for implementing a metaverse immersive sound system according to a method for implementing a metaverse immersive sound system according to claim 1, characterized in that: include: A sound collection module is used to construct a multi-channel microphone array and collect the first sound signal in the environment in real time based on the multi-channel microphone array; An audio processing module, used for performing signal processing on the first sound signal to obtain a second sound signal, analyzing the second sound signal to determine a position distribution state, and encoding the second sound signal; A spatial audio rendering module is used to decode the second sound signal in the first virtual space of the metaverse, and perform spatial audio rendering on the decoded second sound signal according to a position distribution state to generate a second virtual space; The interaction management module is used to construct an interaction management terminal and perform interaction management between the user terminal and the second virtual space based on the interaction management terminal.

Images