An audio orientation system and method for the metaverse space

Abstract

The present invention provides a metaverse sound directional system and method. The system includes: a collection module for collecting ambient sound in the metaverse and simultaneously tracking the user's position in the metaverse to obtain the user's position trajectory in the metaverse; an analysis module for analyzing the ambient sound to obtain the sound characteristics of the ambient sound; a solution determination module for constructing a real-time directional propagation solution for directional propagation of the ambient sound based on the sound characteristics of the ambient sound and the user's position trajectory in the metaverse; and a directional transmission module for controlling the metaverse sound equipment to output the ambient sound in a directional manner according to the real-time directional propagation solution for directional propagation of the ambient sound. This system effectively ensures the spatial and directional sense of sound in the metaverse, enhancing the user's highly realistic sound experience in the metaverse.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a sound orientation system and method for a metaverse space. Background Art

[0002] At present, with the rapid development of metaverse technology, the user's immersive experience in the virtual environment has become particularly important. As an important part of the sensory experience, the quality and effect of audio directly affect the user's immersion.

[0003] However, traditional audio equipment cannot adjust the sound direction in real time based on the user's position and head orientation, resulting in a lack of spatial and directional sound effects, which cannot meet the demand for highly realistic audio experiences in the metaverse.

[0004] Therefore, the present invention provides a metaverse sound orientation system and method. Summary of the Invention

[0005] The present invention provides a metaverse space sound directional system and method, which is used to collect the ambient sound of the metaverse space, obtain the user's position trajectory in the metaverse space, and analyze the ambient sound to effectively obtain the sound characteristics of the ambient sound, and then determine the directional propagation plan for the ambient sound based on the sound characteristics of the ambient sound and the user's position trajectory in the metaverse space, thereby effectively realizing the dynamic directional output of the ambient sound by controlling the audio of the metaverse space based on the propagation plan, effectively ensuring the spatial sense and orientation sense of the sound in the metaverse space, and improving the user's experience of highly realistic sound in the metaverse space.

[0006] A sound directional system for a metaverse space, comprising:

[0007] The collection module is used to collect ambient sounds in the metaverse space and, at the same time, track the user's position in the metaverse space to obtain the user's position trajectory in the metaverse space;

[0008] An analysis module is used to analyze the ambient sound and obtain the sound characteristics of the ambient sound;

[0009] A scheme determination module is used to construct a real-time directional propagation scheme for directional propagation of ambient sounds based on the sound characteristics of the ambient sounds and the position trajectory of the user in the metaverse space;

[0010] The directional transmission module is used to control the audio equipment in the metaverse space to output the ambient sound in a directional manner according to a real-time directional propagation scheme for directional propagation of the ambient sound.

[0011] Preferably, a sound directional system for the metaverse space, a collection module, comprises:

[0012] A sound collection unit, used to obtain a microphone array in the metaverse space and collect ambient sounds in the metaverse space based on the microphone array;

[0013] The position tracking unit is used to locate the user in the metaverse space and track the user's position trajectory in the metaverse space.

[0014] Preferably, a sound directional system for metaverse space, a sound collecting unit, comprises:

[0015] The sound processing subunit is configured to obtain the ambient sound signal collected by each microphone in the microphone array after completing the collection of the ambient sound;

[0016] a signal reading subunit, configured to read the ambient sound signal, determine a first signal set corresponding to the ambient sound signal collected by each microphone, a signal type of the first signal set, and a signal strength corresponding to each signal type;

[0017] a first filtering subunit, configured to read the first signal set collected by each microphone, perform type similarity comparison on the first signal set collected by each microphone, and filter multiple different signal types in the first signal set according to the comparison result to obtain a second signal set;

[0018] The signal reading subunit is further configured to read the signal strength corresponding to each signal type in the second signal set;

[0019] The second filtering subunit is configured to:

[0020] obtaining a signal filtering threshold, and at the same time, comparing the signal strength corresponding to each signal type in the second signal set with the signal filtering threshold;

[0021] When there is a signal in the second signal set with a strength less than the signal filtering threshold, the corresponding signal type is marked in the second signal set;

[0022] Filtering the marked target signal types in the second signal set to obtain a third signal set;

[0023] The target pure ambient sound acquisition subunit is configured to obtain the target pure ambient sound based on the third signal set.

[0024] Preferably, a metaverse space sound orientation system, a position tracking unit, comprises:

[0025] Image acquisition subunit, used for:

[0026] Based on collecting a target image of the user in the metaverse space, determining a positioning key point of the user in the metaverse space, and first marking the positioning key point in the target image;

[0027] The positioning key points include: facial contour points of the user in the target image and foot contour points of the user in the target image;

[0028] a positioning factor generating subunit, configured to generate a first positioning factor based on the facial contour points of the user in the target image, and simultaneously generate a second positioning factor based on the foot contour points of the user;

[0029] Positioning subunit, used for:

[0030] Constructing three-dimensional coordinates in the metaverse space, and determining in real time a first three-dimensional coordinate point set of the user in the metaverse space based on the first positioning factor;

[0031] At the same time, based on the second positioning factor, a second three-dimensional coordinate point set of the user's face in the metaverse space is determined in real time;

[0032] The position trajectory acquisition subunit is used to track the first three-dimensional coordinate point set and the second three-dimensional coordinate point set in the metaverse space, and perform second marking to obtain the user's position trajectory in the metaverse space.

[0033] Preferably, a metaverse sound orientation system and analysis module include:

[0034] A first analysis unit is used to perform a first analysis on the ambient sound to obtain target position information of the ambient sound in the metaverse space;

[0035] The second analysis unit is used to perform a second analysis on the ambient sound to determine the sound type of the ambient sound.

[0036] Preferably, a sound directional system for a metaverse space, the first analysis unit includes:

[0037] An information acquisition subunit, used to obtain the three-dimensional coordinate information of each microphone in the metaverse space of the microphone array, and at the same time, read the time point when the ambient sound in the metaverse space reaches each microphone;

[0038] The target distance acquisition subunit is used to obtain the propagation speed of the ambient sound in the metaverse space, and determine the target distance from the sound source of the ambient sound to different microphones based on the propagation speed of the ambient sound in the metaverse space and the time point when the ambient sound in the metaverse space reaches each microphone;

[0039] A target angle acquisition subunit is used to calculate the target angle of the sound source of the ambient sound relative to each microphone based on the target distance from the sound source of the ambient sound to different microphones;

[0040] The target position information acquisition subunit is used to determine the target position information of the sound source of the ambient sound in the metaverse space based on the target distance from the sound source of the ambient sound to different microphones and the target angle of the sound source of the ambient sound relative to each microphone, combined with the three-dimensional coordinate information of each microphone in the metaverse space.

[0041] Preferably, a sound directional system for a metaverse space, the second analysis unit, comprises:

[0042] A frequency recognition subunit is used to identify the ambient sound and determine the target sound frequency of the ambient sound;

[0043] A sound category library retrieving subunit, configured to retrieve a sound category library, wherein the sound category library contains sound frequencies of multiple different scenarios;

[0044] A similarity matching subunit is used to input the target sound frequency of the environmental sound into the sound category library and perform similarity matching with the sound frequencies of different categories to obtain the target similarity;

[0045] The sound type acquisition subunit is used to obtain a similarity threshold, locate the target type corresponding to the target similarity equal to or greater than the similarity threshold in the sound type library, and use the target type as the sound type of the environmental sound.

[0046] Preferably, a sound directional system for a metaverse space, a solution determination module, includes:

[0047] A position point determination unit is used to read the first position point of the sound source of the ambient sound in the metaverse space in the sound feature, and at the same time, read the position trajectory of the user in the metaverse space to determine the initial position point of the user in the metaverse space and the head orientation of the user in the metaverse space;

[0048] The initial information acquisition unit is used to:

[0049] Determining an initial relative distance between a sound source of ambient sound and the user in the metaverse space based on the first position point and the initial position point;

[0050] Determining an initial relative direction between a sound source of ambient sound and the user in the metaverse space based on the first location point and the head orientation;

[0051] Optimal information acquisition unit for:

[0052] Read the sound type of the ambient sound in the sound characteristics, and obtain the playback requirements that match the sound type in the preset requirements library according to the sound type;

[0053] Determine the optimal sound reading direction and distance for the user in the metaverse space based on playback requirements;

[0054] Correction unit for:

[0055] Performing a first correction on the initial relative direction according to the optimal sound reading direction, and performing a second correction on the initial relative distance according to the optimal sound reading distance;

[0056] According to the first correction result and the second correction result, the target audio device is determined in the metaverse space, and the real-time directional propagation scheme generation unit is used to

[0057] Performing parameter configuration on the target audio device based on the first correction result and the second correction result, and generating an editable directional transmission plan based on the parameter configuration result;

[0058] When the user's position in the metaverse space changes, parameters are edited in the editable directional transmission scheme in real time according to the user's position trajectory to generate a real-time directional transmission scheme for directional propagation of environmental sounds.

[0059] Preferably, a metaverse sound directional system and a directional transmission module include:

[0060] A program reading unit is used to read a real-time directional propagation program for directional propagation of ambient sound and locate the corresponding target audio equipment according to the reading result;

[0061] An instruction generating unit, configured to obtain a device number of a target audio device and generate a control instruction according to the device number of the target audio device;

[0062] The control unit is used to control the target audio equipment in the metaverse space to output the ambient sound in a directional manner based on the control instructions.

[0063] A method for sound orientation in a metaverse space, comprising:

[0064] Step 1: Collect ambient sounds in the metaverse space and simultaneously track the user's position in the metaverse space to obtain the user's position trajectory in the metaverse space;

[0065] Step 2: Analyze the ambient sound to obtain the sound characteristics of the ambient sound;

[0066] Step 3: Determine a directional propagation plan for the ambient sound based on the sound characteristics of the ambient sound and the user's position trajectory in the metaverse space;

[0067] Step 4: Control the audio system of the metaverse space to dynamically and directional output the ambient sound according to the directional propagation scheme of the ambient sound.

[0068] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0069] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0071] Figure 1 This is a structural diagram of a sound directional system for a metaverse space according to an embodiment of the present invention;

[0072] Figure 2 This is a structural diagram of a directional transmission module in a metaverse sound directional system according to an embodiment of the present invention;

[0073] Figure 3 This is a flow chart of a method for sound orientation in a metaverse space according to an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0075] Example 1:

[0076] This embodiment provides a sound directional system for the metaverse space, such as Figure 1 As shown, including:

[0077] The collection module is used to collect ambient sounds in the metaverse space and, at the same time, track the user's position in the metaverse space to obtain the user's position trajectory in the metaverse space;

[0078] An analysis module is used to analyze the ambient sound and obtain the sound characteristics of the ambient sound;

[0079] The scheme determination module determines the directional propagation scheme for the ambient sound based on the sound characteristics of the ambient sound and the user's position trajectory in the metaverse space;

[0080] The directional transmission module is used to control the dynamic directional output of the ambient sound by the audio system of the metaverse space according to the directional propagation scheme of the ambient sound.

[0081] In this embodiment, the ambient sound may be a virtual sound in the metaverse space.

[0082] In this embodiment, the position trajectory can be the user's movement status in the metaverse space, specifically including the user's position distribution in the metaverse space and the user's head orientation at different positions.

[0083] In this embodiment, the sound features may include: the direction of the ambient sound, the sound type of the ambient sound, wherein the sound type may be a specific sound type, such as: music surround sound, waterfall sound, game sound, etc.

[0084] In this embodiment, the position propagation scheme can be determined based on the sound characteristics of the ambient sound and the user's position trajectory in the metaverse space, and is used to achieve dynamic directional output of the ambient sound based on the user in the metaverse space, thereby improving the user's experience in the metaverse space.

[0085] In this embodiment, the system is implemented by a multi-channel microphone array, a high-performance audio processor, and a headset or speaker array. The microphone array is used to collect ambient sound in real time, the audio processor is responsible for processing and directional transmission of sound signals, and the headset or speaker array (i.e., audio equipment) is used to play sound.

[0086] The working principle and beneficial effects of the above technical solution are: by collecting the ambient sounds of the metaverse space, obtaining the user's position trajectory in the metaverse space, and analyzing the ambient sounds, the sound characteristics of the ambient sounds are effectively obtained, and then the directional propagation plan for the ambient sounds is determined based on the sound characteristics of the ambient sounds and the user's position trajectory in the metaverse space, thereby effectively realizing the dynamic directional output of the ambient sounds by the audio in the metaverse space based on the propagation plan, effectively ensuring the spatial sense and orientation sense of the sounds in the metaverse space, and improving the user's experience of highly realistic sound in the metaverse space.

[0087] Example 2:

[0088] Based on Example 1, this embodiment provides a sound directional system for the metaverse space, and a collection module, including:

[0089] A sound collection unit, used to obtain a microphone array in the metaverse space and collect ambient sounds in the metaverse space based on the microphone array;

[0090] The position tracking unit is used to locate the user in the metaverse space and track the user's position trajectory in the metaverse space.

[0091] The working principle and beneficial effects of the above technical solution are: the microphone array can effectively realize the collection of environmental sounds in the metaverse space, thereby improving the effectiveness and comprehensiveness of the collection of environmental sounds. By determining the positioning and tracking of users in the metaverse space, the user's position trajectory in the metaverse space can be accurately obtained, thereby laying a data foundation for determining the directional propagation plan.

[0092] Example 3:

[0093] Based on Example 2, this embodiment provides a metaverse sound directional system, a sound collection unit, including:

[0094] The sound processing subunit is configured to obtain the ambient sound signal collected by each microphone in the microphone array after completing the collection of the ambient sound;

[0095] a signal reading subunit, configured to read the ambient sound signal, determine a first signal set corresponding to the ambient sound signal collected by each microphone, a signal type of the first signal set, and a signal strength corresponding to each signal type;

[0096] a first filtering subunit, configured to read the first signal set collected by each microphone, perform type similarity comparison on the first signal set collected by each microphone, and filter multiple different signal types in the first signal set according to the comparison result to obtain a second signal set;

[0097] The signal reading subunit is further configured to read the signal strength corresponding to each signal type in the second signal set;

[0098] The second filtering subunit is configured to:

[0099] obtaining a signal filtering threshold, and at the same time, comparing the signal strength corresponding to each signal type in the second signal set with the signal filtering threshold;

[0100] When there is a signal in the second signal set with a strength less than the signal filtering threshold, the corresponding signal type is marked in the second signal set;

[0101] Filtering the marked target signal types in the second signal set to obtain a third signal set;

[0102] The target pure ambient sound acquisition subunit is configured to obtain the target pure ambient sound based on the third signal set.

[0103] In this embodiment, the different signal types in the first signal set are filtered so that each microphone finally obtains the second signal set. For example, there are microphone 1, microphone 2, and microphone 3 in the microphone array. The signal types of the first signal set of the ambient sound signal measured by microphone 1 are A, B, C, and H, and the signal types of the first signal set of the ambient sound signal measured by microphone 2 are A, B, C, and D; the signal types of the first signal set of the ambient sound signal measured by microphone 3 are A, B, and C; the different signal types of microphone 1, microphone 2, and microphone 3 are H and D, so the signal types corresponding to H and D are filtered to obtain signal types A, B, and C, and the signal types A, B, and C are the second signal set.

[0104] In this embodiment, the signal filtering threshold may be pre-set and used to measure whether the signal strength in the second signal set meets the filtering condition (ie, the signal strength in the second signal set is less than the signal filtering threshold).

[0105] In this embodiment, the third signal set may be a signal type set obtained by filtering the marked target signal types in the second signal set.

[0106] The working principle and beneficial effects of the above technical solution are: by obtaining the ambient sound signal collected by each microphone in the microphone array, the reading based on the ambient sound signal is effectively realized, and then the first signal set corresponding to the ambient sound signal is determined, by determining the signal type in the first signal set, and then by performing type similarity comparison in the first signal set collected by each microphone, the different signal types in the first signal set are effectively identified, and then effective filtering of the first signal is achieved to obtain the second signal set, and by determining the signal filtering threshold, the signal types that are smaller than the signal filtering threshold in each signal type in the second signal set can be effectively eliminated, and then the third signal set is effectively obtained to determine the target pure ambient sound, effectively realize effective processing of the ambient sound signal, and improve the quality of the obtained ambient sound signal.

[0107] Example 4:

[0108] Based on Example 2, this embodiment provides a metaverse space sound orientation system and a position tracking unit, including:

[0109] Image acquisition subunit, used for:

[0110] Based on collecting a target image of the user in the metaverse space, determining a positioning key point of the user in the metaverse space, and first marking the positioning key point in the target image;

[0111] The positioning key points include: facial contour points of the user in the target image and foot contour points of the user in the target image;

[0112] a positioning factor generating subunit, configured to generate a first positioning factor based on the facial contour points of the user in the target image, and simultaneously generate a second positioning factor based on the foot contour points of the user;

[0113] Positioning subunit, used for:

[0114] Constructing three-dimensional coordinates in the metaverse space, and determining in real time a first three-dimensional coordinate point set of the user in the metaverse space based on the first positioning factor;

[0115] At the same time, based on the second positioning factor, a second three-dimensional coordinate point set of the user's face in the metaverse space is determined in real time;

[0116] The position trajectory acquisition subunit is used to track the first three-dimensional coordinate point set and the second three-dimensional coordinate point set in the metaverse space, and perform second marking to obtain the user's position trajectory in the metaverse space.

[0117] In this embodiment, the positioning key points may include: facial contour points of the user in the target image and foot contour points of the user in the target image.

[0118] In this embodiment, the first positioning factor and the second positioning factor can be used as a positioning mechanism generated based on the user's facial contour points and the user's foot contour points in the target image to achieve effective tracking of the user in the metaverse.

[0119] The working principle and beneficial effects of the above technical solution are: by determining the positioning key points and performing a first marking on the positioning key points, the facial contour points of the user in the target image and the foot contour points of the user in the target image are effectively determined, thereby generating a first positioning factor and a second positioning factor based on the facial contour points of the user in the target image and the foot contour points of the user in the target image, and effectively realizing the positioning of the user in the three-dimensional coordinates in the metaverse space through the first positioning factor and the second positioning factor, and then effectively obtaining the first three-dimensional coordinate point set and the second three-dimensional coordinate point set, and performing a second marking on the first three-dimensional coordinate point set and the second three-dimensional coordinate point set, thereby effectively realizing the user's position trajectory in the metaverse space, thereby improving the effectiveness and accuracy of determining the user's position trajectory in the metaverse space.

[0120] Example 5:

[0121] Based on Example 1, this embodiment provides a metaverse sound orientation system, an analysis module, including:

[0122] A first analysis unit is used to perform a first analysis on the ambient sound to obtain target position information of the ambient sound in the metaverse space;

[0123] The second analysis unit is used to perform a second analysis on the ambient sound to determine the sound type of the ambient sound.

[0124] In this embodiment, the second analysis may be an identification analysis of the ambient sound, in order to determine the sound type of the ambient sound.

[0125] The working principle and beneficial effects of the above technical solution are: by performing the first analysis and the second analysis on the ambient sound respectively, the target position information of the ambient sound in the metaverse space and the sound type of the ambient sound can be accurately and effectively determined, which provides convenience for the dynamic directional output of the ambient sound by the audio system of the metaverse space.

[0126] Example 6:

[0127] Based on Example 5, this embodiment provides a metaverse sound orientation system, wherein the first analysis unit includes:

[0128] An information acquisition subunit, used to obtain the three-dimensional coordinate information of each microphone in the metaverse space of the microphone array, and at the same time, read the time point when the ambient sound in the metaverse space reaches each microphone;

[0129] The target distance acquisition subunit is used to obtain the propagation speed of the ambient sound in the metaverse space, and determine the target distance from the sound source of the ambient sound to different microphones based on the propagation speed of the ambient sound in the metaverse space and the time point when the ambient sound in the metaverse space reaches each microphone;

[0130] A target angle acquisition subunit is used to calculate the target angle of the sound source of the ambient sound relative to each microphone based on the target distance from the sound source of the ambient sound to different microphones;

[0131] The target position information acquisition subunit is used to determine the target position information of the sound source of the ambient sound in the metaverse space based on the target distance from the sound source of the ambient sound to different microphones and the target angle of the sound source of the ambient sound relative to each microphone, combined with the three-dimensional coordinate information of each microphone in the metaverse space.

[0132] In this embodiment, the three-dimensional coordinate information may be the actual position distribution of each microphone in the metaverse space.

[0133] In this embodiment, the target distance is information representing the actual distance from the sound source of the ambient sound to different microphones.

[0134] In this embodiment, the target angle may be the angle between the sound source representing the ambient sound and different microphones relative to a reference direction, wherein the reference direction is known in advance.

[0135] The working principle and beneficial effects of the above technical solution are: by determining the position information of each microphone in the microphone array in the metaverse space, support is provided for determining the target position information of the sound source of the ambient sound in the metaverse space; secondly, by obtaining the time point when the ambient sound reaches each microphone in the metaverse space and the corresponding propagation speed, the target distance of the sound source of the ambient sound to different microphones and the target angle of the sound source of the ambient sound relative to each microphone are accurately and effectively determined according to the time point and propagation speed; finally, the target distance of the sound source of the ambient sound to different microphones, the target angle of the sound source of the ambient sound relative to each microphone and the three-dimensional coordinate information of each microphone in the metaverse space are comprehensively analyzed to achieve accurate and effective determination of the target position information of the sound source of the ambient sound in the metaverse space, thereby facilitating dynamic directional output of the ambient sound.

[0136] Example 7:

[0137] Based on Example 5, this embodiment provides a metaverse sound orientation system, wherein the second analysis unit includes:

[0138] A frequency recognition subunit is used to identify the ambient sound and determine the target sound frequency of the ambient sound;

[0139] A sound category library retrieving subunit, configured to retrieve a sound category library, wherein the sound category library contains sound frequencies of multiple different scenarios;

[0140] A similarity matching subunit is used to input the target sound frequency of the environmental sound into the sound category library and perform similarity matching with the sound frequencies of different categories to obtain the target similarity;

[0141] The sound type acquisition subunit is used to obtain a similarity threshold, locate the target type corresponding to the target similarity equal to or greater than the similarity threshold in the sound type library, and use the target type as the sound type of the environmental sound.

[0142] In this embodiment, the target sound frequency may be the number of vibrations per second of the ambient sound in the metaverse space, which is a characteristic used to distinguish different types of sounds.

[0143] In this embodiment, the target similarity is used to characterize the matching degree between the target sound frequency of the current ambient sound and the sound frequencies of different scenes in the sound class library, in order to determine the sound type of the current ambient sound based on the matching degree.

[0144] In this embodiment, the similarity threshold is set in advance and is the minimum standard for measuring whether the current target similarity meets the similarity matching requirement, and can be adjusted.

[0145] In this embodiment, the target category may be a sound type in the sound category library that corresponds to the current environmental sound.

[0146] The working principle and beneficial effects of the above technical solution are: by accurately and effectively determining the target sound frequency of the ambient sound, and performing similarity matching between the determined target sound frequency and the sound frequency of different scenes in the sound library, the sound type of the ambient sound can be accurately and effectively determined based on the similarity matching results, which provides a basis for determining the sound characteristics of the ambient sound, thereby facilitating the effective formulation of a directional propagation plan for the ambient sound.

[0147] Example 8:

[0148] Based on Example 1, this embodiment provides a metaverse sound orientation system, and a solution determination module, including:

[0149] A position point determination unit is used to read the first position point of the sound source of the ambient sound in the metaverse space in the sound feature, and at the same time, read the position trajectory of the user in the metaverse space to determine the initial position point of the user in the metaverse space and the head orientation of the user in the metaverse space;

[0150] The initial information acquisition unit is used to:

[0151] Determining an initial relative distance between a sound source of ambient sound and the user in the metaverse space based on the first position point and the initial position point;

[0152] Determining an initial relative direction between a sound source of ambient sound and the user in the metaverse space based on the first location point and the head orientation;

[0153] Optimal information acquisition unit for:

[0154] Read the sound type of the ambient sound in the sound characteristics, and obtain the playback requirements that match the sound type in the preset requirements library according to the sound type;

[0155] Determine the optimal sound reading direction and distance for the user in the metaverse space based on playback requirements;

[0156] Correction unit for:

[0157] Performing a first correction on the initial relative direction according to the optimal sound reading direction, and performing a second correction on the initial relative distance according to the optimal sound reading distance;

[0158] According to the first correction result and the second correction result, the target audio device is determined in the metaverse space, and the real-time directional propagation scheme generation unit is used to

[0159] Performing parameter configuration on the target audio device based on the first correction result and the second correction result, and generating an editable directional transmission plan based on the parameter configuration result;

[0160] When the user's position in the metaverse space changes, parameters are edited in the editable directional transmission scheme in real time according to the user's position trajectory to generate a real-time directional transmission scheme for directional propagation of environmental sounds.

[0161] In this embodiment, the first correction may be to determine the direction of the sound source of the ambient sound, that is, to determine the sound source direction of the audio device, that is, the direction parameter for directional transmission of the ambient sound;

[0162] The second correction may be performed by correcting the relative distance between the source of the ambient sound and the user in the metaverse space, that is, determining the distance of directional transmission of the ambient sound;

[0163] Therefore, the best audio device (i.e., the target audio device) is accurately located through the first correction result and the second correction result, and the best sound experience effect of the user at the corresponding position point in the metaverse space is achieved through the best audio device.

[0164] In this embodiment, multiple audio devices at different locations are distributed in the metaverse space, for example, audio devices corresponding to the sound sources of ambient sounds.

[0165] In this embodiment, the first location point may be the specific location of the ambient sound in the metaverse space.

[0166] In this embodiment, the initial location point may be the specific location of the user in the metaverse space determined based on the user's location trajectory.

[0167] In this embodiment, the initial relative distance may be the relative distance between the sound source and the user determined based on the first position point and the position point of the user in the metaverse space.

[0168] In this embodiment, the initial relative direction may represent the directional relationship between the sound source and the user.

[0169] In this embodiment, the preset requirement library is known in advance and is used to store the playback requirements corresponding to different sound types, including the playback method and the working parameters of the audio.

[0170] In this embodiment, the optimal sound reading direction and the optimal sound reading distance may be the sound propagation direction and the sound propagation distance corresponding to the current situation determined according to the playback requirements.

[0171] In this embodiment, the parameter configuration may be to adjust the operating parameters of the target audio device, including the operating power of the target audio device and the direction of sound propagation.

[0172] In this embodiment, the editable directional transmission scheme can be generated based on the first correction, the second correction and the parameter configuration results. It is a specific control scheme for playing ambient sounds and can be adjusted according to demand.

[0173] The working principle and beneficial effects of the above technical solution are: by determining the first position point of the sound source of the ambient sound in the metaverse space and the position trajectory of the user in the metaverse space, the initial position point of the user in the metaverse space and the head orientation of the user in the metaverse space are accurately and effectively determined; secondly, the initial relative distance and initial relative direction between the sound source of the ambient sound and the user are accurately and effectively determined through the first position point, the initial position point and the head orientation; finally, through the sound characteristics of the ambient sound, the playback requirements of the current ambient sound are effectively obtained through the playback requirements of different sound types stored in the preset requirement library, and the optimal sound reading direction and optimal sound reading distance of the user in the metaverse space are determined according to the playback requirements, so that the determined initial relative direction and initial relative distance are corrected respectively according to the optimal sound reading direction and the optimal sound reading distance; finally, the real-time directional propagation scheme of the ambient sound is accurately and effectively determined according to the correction results, effectively ensuring the spatial sense and orientation sense of the sound in the metaverse space, and improving the user's highly realistic sound experience in the metaverse space.

[0174] Example 9:

[0175] Based on Example 1, this embodiment provides a space sound positioning system of the metaverse, such as Figure 2 As shown, the directional transmission module includes:

[0176] A program reading unit is used to read a real-time directional propagation program for directional propagation of ambient sound and locate the corresponding target audio equipment according to the reading result;

[0177] An instruction generating unit, configured to obtain a device number of a target audio device and generate a control instruction according to the device number of the target audio device;

[0178] The control unit is used to control the target audio equipment in the metaverse space to output the ambient sound in a directional manner based on the control instructions.

[0179] In this embodiment, the device number may be used to distinguish different target audio devices and is set in advance, thereby ensuring the accuracy of generating the control instruction and the effectiveness of the control.

[0180] The working principle and beneficial effects of the above technical solution are: by reading the ambient sound in a direction from the propagation plan, and then effectively locating the corresponding target audio device according to the reading result, by determining the device number of the target audio device, the effectiveness of generating the control instruction can be guaranteed, and then the target audio device in the metaverse space can be accurately controlled through the control instruction to output the ambient sound in a direction, thereby ensuring the effectiveness and accuracy of the directional output of the ambient sound.

[0181] Example 10:

[0182] This embodiment provides a method for sound orientation in a metaverse space, such as Figure 3 As shown, including:

[0183] Step 1: Collect ambient sounds in the metaverse space and simultaneously track the user's position in the metaverse space to obtain the user's position trajectory in the metaverse space;

[0184] Step 2: Analyze the ambient sound to obtain the sound characteristics of the ambient sound;

[0185] Step 3: Determine a directional propagation plan for the ambient sound based on the sound characteristics of the ambient sound and the user's position trajectory in the metaverse space;

[0186] Step 4: Control the audio system of the metaverse space to dynamically and directional output the ambient sound according to the directional propagation scheme of the ambient sound.

[0187] The working principle and beneficial effects of the above technical solution are: by collecting the ambient sounds of the metaverse space, obtaining the user's position trajectory in the metaverse space, and analyzing the ambient sounds, the sound characteristics of the ambient sounds are effectively obtained, and then the directional propagation plan for the ambient sounds is determined based on the sound characteristics of the ambient sounds and the user's position trajectory in the metaverse space, thereby effectively realizing the dynamic directional output of the ambient sounds by the audio in the metaverse space based on the propagation plan, effectively ensuring the spatial sense and orientation sense of the sounds in the metaverse space, and improving the user's experience of highly realistic sound in the metaverse space.

[0188] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A sound directional system for the metaverse, characterized in that: include: The collection module is used to collect the ambient sound in the metaverse space and, at the same time, track the user's position in the metaverse space to obtain the user's position trajectory in the metaverse space; An analysis module is used to analyze the ambient sound and obtain the sound characteristics of the ambient sound; A scheme determination module is used to construct a real-time directional propagation scheme for directional propagation of ambient sounds based on the sound characteristics of the ambient sounds and the position trajectory of the user in the metaverse space; A directional transmission module is used to control the audio equipment in the metaverse space to output the ambient sound in a directional manner according to a real-time directional propagation scheme for directional propagation of the ambient sound; Solution determination module, including: A position point determination unit is used to read the first position point of the sound source of the ambient sound in the metaverse space in the sound feature, and at the same time, read the position trajectory of the user in the metaverse space to determine the initial position point of the user in the metaverse space and the head orientation of the user in the metaverse space; The initial information acquisition unit is used to: Determining an initial relative distance between a sound source of ambient sound and the user in the metaverse space based on the first position point and the initial position point; Determining an initial relative direction between a sound source of ambient sound and the user in the metaverse space based on the first location point and the head orientation; Optimal information acquisition unit for: Read the sound type of the ambient sound in the sound characteristics, and obtain the playback requirements that match the sound type in the preset requirements library according to the sound type; Determine the optimal sound reading direction and distance for the user in the metaverse space based on playback requirements; Correction unit for: Performing a first correction on the initial relative direction according to the optimal sound reading direction, and performing a second correction on the initial relative distance according to the optimal sound reading distance; Determine the target audio device in the metaverse space based on the first correction result and the second correction result, Real-time directional propagation scheme generation unit, used for Performing parameter configuration on the target audio device based on the first correction result and the second correction result, and generating an editable directional transmission plan based on the parameter configuration result; When the user's position in the metaverse space changes, parameters are edited in the editable directional transmission plan in real time according to the user's position trajectory to generate a real-time directional transmission plan for directional propagation of environmental sounds.

2. The sound directional system for the metaverse according to claim 1, characterized in that: Collection modules, including: A sound collection unit, used to obtain a microphone array in the metaverse space and collect ambient sounds in the metaverse space based on the microphone array; The position tracking unit is used to locate the user in the metaverse space and track the user's position trajectory in the metaverse space.

3. The sound directional system for the metaverse according to claim 2, characterized in that: A sound collecting unit, comprising: The sound processing subunit is configured to obtain the ambient sound signal collected by each microphone in the microphone array after completing the collection of the ambient sound; a signal reading subunit, configured to read the ambient sound signal, determine a first signal set corresponding to the ambient sound signal collected by each microphone, a signal type of the first signal set, and a signal strength corresponding to each signal type; a first filtering subunit, configured to read the first signal set collected by each microphone, perform type similarity comparison on the first signal set collected by each microphone, and filter multiple different signal types in the first signal set according to the comparison result to obtain a second signal set; The signal reading subunit is further configured to read the signal strength corresponding to each signal type in the second signal set; The second filtering subunit is configured to: obtaining a signal filtering threshold, and at the same time, comparing the signal strength corresponding to each signal type in the second signal set with the signal filtering threshold; When there is a signal in the second signal set with a strength less than the signal filtering threshold, the corresponding signal type is marked in the second signal set; Filtering the marked target signal types in the second signal set to obtain a third signal set; The target pure ambient sound acquisition subunit is configured to obtain the target pure ambient sound based on the third signal set.

4. The sound directional system for the metaverse according to claim 2, characterized in that: Position tracking unit, including: Image acquisition subunit, used for: Based on collecting a target image of the user in the metaverse space, determining a positioning key point of the user in the metaverse space, and first marking the positioning key point in the target image; The positioning key points include: facial contour points of the user in the target image and foot contour points of the user in the target image; a positioning factor generating subunit, configured to generate a first positioning factor based on the facial contour points of the user in the target image, and simultaneously generate a second positioning factor based on the foot contour points of the user; Positioning subunit, used for: Constructing three-dimensional coordinates in the metaverse space, and determining in real time a first three-dimensional coordinate point set of the user in the metaverse space based on the first positioning factor; At the same time, based on the second positioning factor, a second three-dimensional coordinate point set of the user's face in the metaverse space is determined in real time; The position trajectory acquisition subunit is used to track the first three-dimensional coordinate point set and the second three-dimensional coordinate point set in the metaverse space, and perform second marking to obtain the user's position trajectory in the metaverse space.

5. The sound directional system for the metaverse according to claim 1, characterized in that: Analysis modules, including: A first analysis unit is used to perform a first analysis on the ambient sound to obtain target position information of the ambient sound in the metaverse space; The second analysis unit is used to perform a second analysis on the ambient sound to determine the sound type of the ambient sound.

6. The sound directional system for the metaverse according to claim 5, characterized in that: The first analysis unit includes: An information acquisition subunit, used to obtain the three-dimensional coordinate information of each microphone in the metaverse space of the microphone array, and at the same time, read the time point when the ambient sound in the metaverse space reaches each microphone; The target distance acquisition subunit is used to obtain the propagation speed of the ambient sound in the metaverse space, and determine the target distance from the sound source of the ambient sound to different microphones based on the propagation speed of the ambient sound in the metaverse space and the time point when the ambient sound in the metaverse space reaches each microphone; A target angle acquisition subunit is used to calculate the target angle of the sound source of the ambient sound relative to each microphone based on the target distance from the sound source of the ambient sound to different microphones; The target position information acquisition subunit is used to determine the target position information of the sound source of the ambient sound in the metaverse space based on the target distance from the sound source of the ambient sound to different microphones and the target angle of the sound source of the ambient sound relative to each microphone, combined with the three-dimensional coordinate information of each microphone in the metaverse space.

7. The sound directional system for metaverse space according to claim 5, characterized in that: The second analysis unit includes: A frequency recognition subunit is used to identify the ambient sound and determine the target sound frequency of the ambient sound; A sound category library retrieving subunit, configured to retrieve a sound category library, wherein the sound category library contains sound frequencies of multiple different scenarios; A similarity matching subunit is used to input the target sound frequency of the environmental sound into the sound category library and perform similarity matching with the sound frequencies of different categories to obtain the target similarity; The sound type acquisition subunit is used to obtain a similarity threshold, locate the target type corresponding to the target similarity equal to or greater than the similarity threshold in the sound type library, and use the target type as the sound type of the environmental sound.

8. The metaverse sound directional system according to claim 1, characterized in that: Directional transmission module, including: A program reading unit is used to read a real-time directional propagation program for directional propagation of ambient sound and locate the corresponding target audio equipment according to the reading result; An instruction generating unit, configured to obtain a device number of a target audio device and generate a control instruction according to the device number of the target audio device; The control unit is used to control the target audio equipment in the metaverse space to output the ambient sound in a directional manner based on the control instructions.

9. A method for sound directionality in metaverse space, characterized in that: include: Step 1: Collect ambient sounds in the metaverse space and simultaneously track the user's position in the metaverse space to obtain the user's position trajectory in the metaverse space; Step 2: Analyze the ambient sound to obtain the sound characteristics of the ambient sound; Step 3: Determine a directional propagation plan for the ambient sound based on the sound characteristics of the ambient sound and the user's position trajectory in the metaverse space; Step 4: Control the sound system of the metaverse space to dynamically and directional output the ambient sound according to the directional propagation scheme for the ambient sound; Step 3 includes: Read the first location of the sound source of the ambient sound in the metaverse space in the sound signature. At the same time, read the user's location trajectory in the metaverse space to determine the user's initial location in the metaverse space and the user's head orientation in the metaverse space. Determining an initial relative distance between a sound source of ambient sound and the user in the metaverse space based on the first position point and the initial position point; Determining an initial relative direction between a sound source of ambient sound and the user in the metaverse space based on the first location point and the head orientation; Read the sound type of the ambient sound in the sound characteristics, and obtain the playback requirements that match the sound type in the preset requirements library according to the sound type; Determine the optimal sound reading direction and distance for the user in the metaverse space based on playback requirements; Performing a first correction on the initial relative direction according to the optimal sound reading direction, and performing a second correction on the initial relative distance according to the optimal sound reading distance; Determine the target audio device in the metaverse space based on the first correction result and the second correction result, Performing parameter configuration on the target audio device based on the first correction result and the second correction result, and generating an editable directional transmission plan based on the parameter configuration result; When the user's position in the metaverse space changes, parameters are edited in the editable directional transmission plan in real time according to the user's position trajectory to generate a real-time directional transmission plan for directional propagation of environmental sounds.

Images