Spatial audio generation method, device, audio equipment and storage medium

Abstract

The embodiments of the present invention relate to a method, apparatus, audio device and storage medium for generating spatial audio, which are applied to an audio device, wherein a target sensor is provided on the audio device, and the method comprises: obtaining the target coordinates after the head of the target object is rotated through the target sensor; determining the angle information of the corresponding speaker according to the target coordinates; determining the head-related impulse response HRIR data matching the angle information; and performing convolution processing based on the HRIR data and the decompressed audio data to generate spatial audio. In this way, it is possible to avoid a poor audio experience due to the fixed position of the speaker when the head of the target object is rotated. Determining the HRIR data matching the angle information in the space, and generating spatial audio based on the HRIR data and the decompressed audio data, can provide the target object with an audio experience in different orientations.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of spatial audio technology, and in particular, to a spatial audio generation method, apparatus, audio equipment, and storage medium. Background Art

[0002] With the popularization of 5G network transmission and high-quality rendering technologies such as HDR (High Dynamic Range Imaging) and Dolby Vision, higher requirements are placed on audio rendering. Currently, when users wear headphones to listen to music or other audio, when the user's head turns, the audio heard will also change (for example, the sound becomes smaller, etc.), resulting in a poor user experience. Summary of the invention

[0003] In view of this, in order to solve the above technical problems or at least partially solve the above technical problems, embodiments of the present invention provide a spatial audio generation method, apparatus, audio device and storage medium.

[0004] In a first aspect, an embodiment of the present invention provides a spatial audio generation method, which is applied to an audio device, wherein a target sensor is provided on the audio device, and the method includes:

[0005] Acquiring the target coordinates of the target object after the head is rotated by the target sensor;

[0006] Determining angle information of a corresponding speaker according to the target coordinates;

[0007] Determining head-related impulse response (HRIR) data matching the angle information;

[0008] Convolution processing is performed based on the HRIR data and the decompressed audio data to generate spatial audio.

[0009] In a possible implementation manner, acquiring the target coordinates of the target object after the head is rotated by the target sensor includes:

[0010] Acquiring the initial coordinates of the target object's head before rotation through the target sensor;

[0011] Collecting rotation information of the target object's head through the target sensor;

[0012] The target coordinates of the target object after the head is rotated are determined according to the rotation information and the initial coordinates.

[0013] In a possible implementation manner, the rotation information includes at least one of the following rotation angle information: horizontal rotation angle information, up and down rotation angle information, left and right deflection angle information;

[0014] In the case where the rotation information includes a rotation angle information, determining the target coordinates after the head of the target object is rotated according to the rotation information and the initial coordinates includes:

[0015] Determine the initial rotation axis coordinates of the target object's head before rotation corresponding to the rotation angle information;

[0016] Determining the target rotation axis coordinates after the head of the target object is rotated according to the rotation angle information and the initial rotation axis coordinates;

[0017] The target rotation axis coordinates, initial coordinates, and rotation angle information are input into a preset rotation coordinate calculation model to obtain the target coordinates after the head of the target object is rotated.

[0018] In a possible implementation manner, when the rotation information includes two rotation angle information, the two rotation angle information include first rotation angle information and second rotation angle information;

[0019] The step of determining the target coordinates of the target object after the head is rotated according to the rotation information and the initial coordinates further includes:

[0020] Determine a first initial rotation axis coordinate of the target object's head before rotation corresponding to the first rotation angle information;

[0021] Determine, according to the first rotation angle information and the first initial rotation axis coordinate, a first target rotation axis coordinate after the head of the target object is rotated;

[0022] Inputting the first target rotation axis coordinate, the initial coordinate, and the first rotation angle information into the rotation coordinate calculation model to obtain the first intermediate coordinate after the head of the target object is rotated;

[0023] Determine a second initial rotation axis coordinate of the target object's head before rotation corresponding to the second rotation angle information;

[0024] Determine the second target rotation axis coordinates after the head of the target object is rotated according to the second rotation angle information and the second initial rotation axis coordinates;

[0025] The second target rotation axis coordinates, the first intermediate coordinates, and the second rotation angle information are input into the rotation coordinate calculation model to obtain the target coordinates after the head of the target object is rotated.

[0026] In a possible implementation manner, when the rotation information includes three rotation angle information, the three rotation angle information include third rotation angle information, fourth rotation angle information and fifth rotation angle information;

[0027] The step of determining the target coordinates of the target object after the head is rotated according to the rotation information and the initial coordinates further includes:

[0028] Determine a third initial rotation axis coordinate of the target object's head before rotation corresponding to the third rotation angle information;

[0029] Determine the third target rotation axis coordinate after the head of the target object is rotated according to the third rotation angle information and the third initial rotation axis coordinate;

[0030] Inputting the third target rotation axis coordinate, the initial coordinate, and the third rotation angle information into the rotation coordinate calculation model to obtain the second intermediate coordinate after the head of the target object is rotated;

[0031] Determine a fourth initial rotation axis coordinate of the target object's head before rotation corresponding to the fourth rotation angle information;

[0032] Determine, according to the fourth rotation angle information and the fourth initial rotation axis coordinate, a fourth target rotation axis coordinate after the head of the target object is rotated;

[0033] Inputting the fourth target rotation axis coordinate, the second intermediate coordinate, and the fourth rotation angle information into the rotation coordinate calculation model to obtain the third intermediate coordinate after the head of the target object is rotated;

[0034] Determine the fifth initial rotation axis coordinates of the target object's head before rotation corresponding to the fifth rotation angle information;

[0035] Determine, according to the fifth rotation angle information and the fifth initial rotation axis coordinate, the fifth target rotation axis coordinate after the head of the target object is rotated;

[0036] The fifth target rotation axis coordinate, the third intermediate coordinate, and the fifth rotation angle information are input into the rotation coordinate calculation model to obtain the target coordinate after the head of the target object is rotated.

[0037] In a possible implementation manner, determining angle information of a corresponding speaker according to the target coordinates includes:

[0038] The target coordinates are input into a preset angle calculation formula to obtain the angle information of the target object's head rotating in the left-handed coordinate system. The angle calculation formula is as follows:

[0039]

[0040] Among them, θ and is the angle information of the target object's head rotation in the left-handed coordinate system, and (x′, y′, z′) is the target coordinates of the target object's head after rotation;

[0041] The angle information is determined to be the angle information of the corresponding speaker.

[0042] In a possible implementation manner, determining the HRIR data matching the angle information includes:

[0043] For each preset HRIR data in a preset HRIR database, determining preset angle information of the preset HRIR data;

[0044] Determining a difference between the preset angle information and the angle information;

[0045] When the difference is less than or equal to a preset difference threshold, it is determined that the HRIR data corresponding to the difference is the HRIR data matching the angle information.

[0046] In a second aspect, an embodiment of the present invention provides a spatial audio generation device, the device comprising:

[0047] A coordinate acquisition module, used to acquire the target coordinates of the target object after the head is rotated through the target sensor;

[0048] An angle information determination module, used to determine the angle information of the corresponding speaker according to the target coordinates;

[0049] An HRIR data determination module, used to determine head-related impulse response HRIR data matching the angle information;

[0050] The audio generation module is used to perform convolution processing based on the HRIR data and the decompressed audio data to generate spatial audio.

[0051] In a third aspect, an embodiment of the present invention provides an audio device, including: a target sensor, a processor, and a memory;

[0052] The target sensor is used to collect the target coordinates after the head of the target object rotates;

[0053] The processor is configured to execute the spatial audio generation program stored in the memory to implement the spatial audio generation method according to any one of the first aspects.

[0054] In a fourth aspect, an embodiment of the present invention provides a storage medium, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the spatial audio generation method described in any one of the first aspects.

[0055] The technical solution provided by the embodiment of the present invention obtains the target coordinates of the target object after the head is rotated through the target sensor, determines the angle information of the corresponding speaker according to the target coordinates, determines the head-related impulse response HRIR data matching the angle information, and performs convolution processing based on the HRIR data and the decompressed audio data to generate spatial audio. When the head of the target object sends a rotation, the rotated target coordinates can be determined, and then the angle information of the speaker corresponding to the target coordinates can be determined, so as to avoid poor audio experience due to the fixed position of the speaker when the head of the target object sends a rotation. Determine the HRIR data in the space that matches the angle information, and finally generate spatial audio based on the HRIR data and the decompressed audio data, which can provide the target object with audio experience in different orientations. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flowchart of an embodiment of a spatial audio generation method provided by an embodiment of the present invention;

[0057] Figure 2 An example of a left-handed coordinate system provided by an embodiment of the present invention;

[0058] Figure 3 A flowchart of another spatial audio generation method provided by an embodiment of the present invention;

[0059] Figure 4 Another example of a left-handed coordinate system provided by an embodiment of the present invention;

[0060] Figure 5 A flow chart of an embodiment of a method for determining target coordinates provided by an embodiment of the present invention;

[0061] Figure 6 A flowchart of another target coordinate determination method provided by an embodiment of the present invention;

[0062] Figure 7 A flow chart of another target coordinate determination method provided by an embodiment of the present invention;

[0063] Figure 8 A block diagram of an embodiment of a spatial audio generation device provided by an embodiment of the present invention;

[0064] Fig. 9 A schematic diagram of the structure of an audio device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] The spatial audio generation method provided by the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings, and the embodiments do not constitute a limitation on the embodiments of the present invention.

[0067] See also Figure 1 , is a flow chart of an embodiment of a spatial audio generation method provided by an embodiment of the present invention. As an embodiment, the process is applied to an audio device, and a target sensor is provided on the audio device.

[0068] like Figure 1 As shown, the process may include the following steps:

[0069] Step 101: Acquire the target coordinates of the target object after the head is rotated through the target sensor.

[0070] The target sensor is used to collect the target coordinates after the target object's head rotates, and it can be a position sensor, a spatial audio sensor, etc., which is not limited in the embodiment of the present invention. In addition, the embodiment of the present invention does not limit the type and quantity of the target sensor.

[0071] The target object may be the user of the current audio device. Here, the audio device may be an audio playback accessory adapted to the electronic device, such as headphones, speakers, etc., which is not limited in the embodiments of the present invention. The rotation refers to the rotation of the head in space, which may be an up and down rotation (e.g., lowering the head, raising the head, etc.), a left and right rotation (e.g., looking to the left, looking to the right, etc.), or a left and right deflection (e.g., turning the head to the left, turning the head to the right, etc.), which is not limited in the embodiments of the present invention.

[0072] In practice, when the target object is listening to audio, the body or head may rotate for some reason, while the position of the audio playback device remains fixed, so the audio heard by the target object changes, for example, the sound becomes smaller or more chaotic, etc. The above audio can be music, film and television sound, recording, etc.

[0073] In one embodiment, a spatial coordinate system may be established based on the position of the head of the target object. In this way, the target coordinates of the head of the target object in the spatial coordinate system after the head of the target object rotates can be clearly expressed. In this way, the target coordinates of the head of the target object after the rotation can be obtained.

[0074] The above-mentioned spatial coordinate system can be a left-handed coordinate system, for example, Figure 2 The figure shows an example of a left-handed coordinate system provided by an embodiment of the present invention. Specifically, the left-handed coordinate system can be established with the facial orientation of the target object's head as the Z axis, the direction to the right of the head as the X axis, and the direction to the top of the head as the Y axis. It should be noted that the above is only an example of establishing a left-handed coordinate system. In practice, other methods can also be used to establish a left-handed coordinate system, and the embodiment of the present invention does not limit this.

[0075] Step 102: Determine the angle information of the corresponding speaker according to the target coordinates.

[0076] The above-mentioned speaker represents an audio playback device for the audio currently listened to by the target object, which may be a mobile phone speaker, a sound box, etc., and the embodiment of the present invention does not limit this.

[0077] In one embodiment, after the target object's head rotates, in order to avoid a poor audio experience caused by the rotation of the head, after obtaining the target coordinates after the target object's head rotates, the angle information of the speaker corresponding to the target object's head rotation can be determined based on the target coordinates.

[0078] Optionally, the target coordinates may be substituted into a preset angle information calculation formula to calculate the angle information of the corresponding speaker.

[0079] Optionally, the target coordinates may be input into a preset angle training model to obtain angle information output by the angle training model, and the angle information is the angle information of the corresponding speaker.

[0080] Step 103: Determine HRIR (head-related impulse response) data that matches the angle information.

[0081] The above HRIR is the pre-collected spatial transfer function of the direct sound reaching the two ears. Usually, in order to obtain the transfer function of a linear system, a pulse signal is used as the input of the system, and then the output signal of the system is recorded.

[0082] It can be seen from the above description that after determining the angle information of the speaker corresponding to the rotation of the target object's head, the HRIR data corresponding to the angle can be recorded in advance at different angles in the spatial coordinate system, so as to determine the HRIR data matching the angle information in the spatial coordinate system.

[0083] For example, assuming that the angle information of the XOZ plane in the left-handed coordinate system is 45 degrees, and assuming that the first HRIR data exists at a position of 45 degrees of the XOZ plane in the left-handed coordinate system, then the HRIR data matching the angle information can be determined as the first HRIR data.

[0084] As for how to determine the HRIR data that matches the angle information, it can be found in the following Figure 3 It is described in detail in , and will not be described in detail here.

[0085] Step 104: Perform convolution processing based on the HRIR data and the decompressed audio data to generate spatial audio.

[0086] As can be seen from the description in the above step 103, the matching HRIR data can be determined based on the angle information. Therefore, in order to generate spatial audio, in this step 104, the HRIR data and the decompressed audio data can be convolved by a preset convolution algorithm to obtain the convolved spatial audio.

[0087] So far, completed Figure 1 A description of the process shown.

[0088] pass Figure 1 It can be seen from the process shown that in the technical solution of the present invention, the target coordinates after the rotation of the target object's head are obtained by the target sensor, and the angle information of the corresponding speaker is determined according to the target coordinates, and the head-related impulse response HRIR data matching the angle information is determined, and convolution processing is performed based on the HRIR data and the decompressed audio data to generate spatial audio. When the target object's head sends a rotation, the rotated target coordinates can be determined, and then the angle information of the speaker corresponding to the target coordinates can be determined, so as to avoid poor audio experience due to the fixed position of the speaker when the target object's head sends a rotation. Determine the HRIR data in the space that matches the angle information, and finally generate spatial audio based on the HRIR data and the decompressed audio data, which can provide the target object with audio experience in different orientations.

[0089] See also Figure 3 , is a flow chart of another embodiment of a spatial audio generation method provided by an embodiment of the present invention. Figure 3 The process shown in the above Figure 1 Based on the process shown, how to generate spatial audio is described in detail. The following steps may be included:

[0090] Step 301: Acquire the initial coordinates of the target object's head before rotation through the target sensor.

[0091] Step 302: Collect rotation information of the target object's head through a target sensor, where the rotation information includes at least one of the following rotation angle information: horizontal rotation angle information, up and down rotation angle information, and left and right deflection angle information.

[0092] Step 303: Determine the target coordinates of the target object's head after rotation according to the rotation information and the initial coordinates.

[0093] The following is a unified description of steps 301 to 303:

[0094] The above initial coordinates represent the coordinates of the target object in the spatial coordinate system when listening to the audio before the head rotates, which may be the coordinates of the corresponding speaker before the rotation, and the embodiment of the present invention does not limit this.

[0095] In practice, a target sensor is usually provided in an audio device. For example, when the audio device is a headset, the target object's initial position information, i.e., the initial coordinates, can be obtained through the target sensor when the target object wears the headset. When the target object's head rotates, the rotation information of the target object's head can be detected by the target sensor provided in the headset. The above rotation information may include, but is not limited to: rotation direction, rotation angle, etc. Moreover, the rotation of the target object's head may include rotation in one or more directions, so the rotation information may include one or more of the following: horizontal rotation angle information, up and down rotation angle information, left and right deflection angle information.

[0096] Among them, Figure 4 FIG. 2 is another example of a left-handed coordinate system provided by an embodiment of the present invention. Figure 4 As shown, it can be determined that the above-mentioned horizontal angle rotation information is the rotation angle information occurring in the XOZ plane, and the positive direction of rotation may be counterclockwise, that is, direction 401; the above-mentioned up and down rotation angle information is the rotation angle information occurring in the YOZ plane, and the positive direction of rotation may be clockwise, that is, direction 402; the above-mentioned left and right deflection angle information is the rotation angle information occurring in the XOY plane, and the positive direction of rotation may be counterclockwise, that is, direction 403.

[0097] It should be noted that the above is only the target object head in Figure 4 The illustrated exemplary description of the rotation angle information and the positive direction of rotation occurring in the left-handed coordinate system, in practice, the rotation angle information and the positive direction of rotation may also be in other forms, which are not limited in the embodiments of the present invention.

[0098] Furthermore, the rotation information and the initial coordinates may be substituted into a preset coordinate calculation formula to determine the target coordinates after the head of the target object is rotated.

[0099] In one embodiment, when the rotation information includes a rotation angle information, the specific implementation of determining the target coordinates after the head of the target object is rotated according to the rotation information and the initial coordinates can be seen in Figure 5 , is a flow chart of an embodiment of a method for determining target coordinates provided by an embodiment of the present invention. Figure 5 As shown, the process may include the following steps:

[0100] Step 501: Determine the initial rotation axis coordinates of the target object's head before rotation corresponding to the rotation angle information.

[0101] Step 502: Determine the target rotation axis coordinates after the head of the target object is rotated according to the rotation angle information and the initial rotation axis coordinates.

[0102] Step 503: input the target rotation axis coordinates, initial coordinates, and rotation angle information into a preset rotation coordinate calculation model to obtain the target coordinates after the target object's head is rotated.

[0103] The following is a unified description of steps 501 to 503:

[0104] The above rotation angle information is any one of horizontal rotation angle information, up-down rotation angle information, and left-right deflection angle information, and the embodiment of the present invention does not limit this.

[0105] The above-mentioned rotation axis is a geometric straight line on which the rotationally symmetric action is performed. For example, if point A rotates from the Z axis to the X axis, then the corresponding rotation axis is the Y axis. The above-mentioned rotation angle information is the angle information of rotation around the rotation axis, which can be any angle from 0 to 360 degrees. Correspondingly, the above-mentioned initial rotation axis coordinates can be any coordinate on the X axis, any coordinate on the Y axis, or any coordinate on the Z axis. The specific coordinate values ​​of the initial rotation axis coordinates are not limited by the embodiments of the present invention.

[0106] Optionally, the initial rotation axis coordinates and the rotation angle information may be substituted into a preset rotation axis coordinate calculation formula to calculate the target rotation axis coordinates after the head of the target object is rotated. Furthermore, the target rotation axis coordinates, the initial coordinates, and the rotation angle information may be input into a preset rotation coordinate calculation model to obtain the target coordinates after the head of the target object is rotated output by the rotation coordinate calculation model.

[0107] The above-mentioned rotation coordinate calculation model is used to calculate the target coordinates after the head of the target object is rotated. It can be a rotation coordinate calculation formula, and the embodiment of the present invention does not limit this.

[0108] For example, suppose Figure 4Taking the left-handed coordinate system shown in the figure as an example, the initial coordinates of the target object's head before rotation are (0, 0, a). The target object rotates 45 degrees to the left in the XOZ plane. It can be determined that before the target object's head rotates, the rotation axis is the Y axis, and the initial rotation axis coordinates are (0, n y , 0). After the target object's head rotates, the rotation axis is still the Y axis. Therefore, the target rotation axis coordinate after the target object's head rotates can be determined to be (0, n y , 0).

[0109] Furthermore, the target rotation axis coordinates, initial coordinates, and rotation angle information may be input into the rotation coordinate calculation model illustrated in the following formula (I), and the target coordinates after the target object is rotated may be obtained as follows:

[0110]

[0111]

[0112] In the above formula (1), (n x , n y , n z ) is the target rotation axis coordinate, where n x is the coordinate of the rotation axis on the X axis, n y is the coordinate of the rotation axis on the Y axis, n z is the coordinate of the rotation axis on the Z axis, α is the rotation angle information, (x, y, z) is the initial coordinate of the target object's head before rotation, and (x′, y′, z′) is the target coordinate of the target object's head after rotation.

[0113] So far, completed Figure 5 A description of the process shown.

[0114] In the technical solution of the present invention, by determining the initial rotation axis coordinates of the target object's head before rotation corresponding to the rotation angle information, the target rotation axis coordinates after the target object's head is rotated are determined according to the rotation angle information and the initial rotation axis coordinates. The target rotation axis coordinates, the initial coordinates, and the rotation angle information are input into a preset rotation coordinate calculation model to obtain the target coordinates after the target object's head is rotated. In this way, the target coordinates after the target object's head is rotated can be obtained when the rotation information includes a rotation angle information.

[0115] In another embodiment, when the rotation information includes two rotation angle information, the two rotation angle information include first rotation angle information and second rotation angle information. Figure 6 , is a flow chart of another target coordinate determination method provided by an embodiment of the present invention. Figure 6As shown, the process may include the following steps:

[0116] Step 601: Determine the first initial rotation axis coordinates of the target object's head before rotation corresponding to the first rotation angle information.

[0117] Step 602: Determine the first target rotation axis coordinates after the head of the target object is rotated according to the first rotation angle information and the first initial rotation axis coordinates.

[0118] Step 603: input the first target rotation axis coordinates, initial coordinates, and first rotation angle information into a rotation coordinate calculation model to obtain a first intermediate coordinate after the target object's head is rotated.

[0119] Step 604: Determine the second initial rotation axis coordinates of the target object's head before rotation, corresponding to the second rotation angle information.

[0120] Step 605: Determine the second target rotation axis coordinates after the head of the target object is rotated according to the second rotation angle information and the second initial rotation axis coordinates.

[0121] Step 606: input the second target rotation axis coordinates, the first intermediate coordinates, and the second rotation angle information into a rotation coordinate calculation model to obtain the target coordinates after the target object's head is rotated.

[0122] The following is a unified description of steps 601 to 606:

[0123] The first rotation angle information and the second rotation angle information are any two rotation angle information among the horizontal rotation angle information, the up-down rotation angle information, and the left-right deflection angle information included in the rotation information, and the embodiment of the present invention does not limit this.

[0124] The first initial rotation axis coordinates are the coordinates of the rotation axis corresponding to the target object's head before it is rotated based on the first rotation angle information. The second initial rotation axis coordinates are the coordinates of the rotation axis corresponding to the target object's head after it is rotated based on the first rotation angle information and before it is rotated based on the second rotation angle information.

[0125] By the above Figure 5 From the description of the process of the illustrated embodiment, it can be seen that the first target rotation axis coordinates, initial coordinates, and first rotation angle information can be input into the rotation coordinate calculation model to obtain the first intermediate coordinates after the target object's head is rotated. Here, the first intermediate coordinates are the coordinates in the left-hand coordinate system after the target object's head is rotated by the first rotation angle information.

[0126] Further, after obtaining the first intermediate coordinates, the rotation axis of the target object's head after rotating the second rotation angle information can be re-determined, and the second target rotation axis coordinates after rotation can be determined. The first intermediate coordinates are then used as the initial coordinates before the target object's head is rotated, and the second target rotation axis coordinates, the first intermediate coordinates, and the second rotation angle information are input into the above-mentioned rotation coordinate calculation model to obtain the target coordinates after the target object's head is rotated output by the rotation coordinate calculation model. Here, the target coordinates are the final coordinates of the target object after rotating the first rotation angle information and the second rotation angle information.

[0127] For example, suppose Figure 4 Taking the left-handed coordinate system shown in the figure as an example, the initial coordinates of the target object's head before rotation are (0, 0, a), the first rotation angle information is a horizontal rotation of 90 degrees to the left, and the second rotation angle information is a downward rotation of 90 degrees. It can be determined that before the target object's head rotates, the rotation axis is the Y axis, and the initial rotation axis coordinates are (0, n y , 0). After the target object's head rotates the first rotation angle information, the rotation axis is still the Y axis, so the first target rotation axis coordinate after the target object's head rotates can be determined as: (0, n y , 0).

[0128] Afterwards, the first target rotation axis coordinates, initial coordinates, and first rotation angle information can be input into the rotation coordinate calculation model exemplified by the above formula (I) to obtain the first intermediate coordinates (-a, 0, 0) after the target object's head is rotated.

[0129] Furthermore, before the target object's head rotates the second rotation angle information, the initial rotation axis coordinates are determined to be: (0, n y , 0), after the target object's head rotates the second angle information, the rotation axis is the Z axis. Then, the coordinates of the second target rotation axis can be determined as: (0, 0, n z ). Then, the second target rotation axis coordinate, the first intermediate coordinate, and the second rotation angle information are input into the rotation coordinate calculation model exemplified by the above formula (I), and the target coordinate after the rotation of the target object's head is obtained as (0, 0, -a).

[0130] It should be noted that the above is merely an exemplary description of the sequence of calculating the target coordinates according to the first rotation angle information and the second rotation angle information, and the embodiment of the present invention does not limit this.

[0131] So far, completed Figure 6 A description of the process shown.

[0132] In the technical solution of the present invention, by determining the first initial rotation axis coordinates of the target object's head before rotation corresponding to the first rotation angle information, the first target rotation axis coordinates after the target object's head is rotated are determined according to the first rotation angle information and the first initial rotation axis coordinates, and the first target rotation axis coordinates, the initial coordinates, and the first rotation angle information are input into the rotation coordinate calculation model to obtain the first intermediate coordinates after the target object's head is rotated. Afterwards, the second initial rotation axis coordinates of the target object's head before rotation corresponding to the second rotation angle information are determined. According to the second rotation angle information and the second initial rotation axis coordinates, the second target rotation axis coordinates after the target object's head is rotated are determined. Finally, the second target rotation axis coordinates, the first intermediate coordinates, and the second rotation angle information can be input into the rotation coordinate calculation model to obtain the target coordinates after the target object's head is rotated. In this way, when the rotation information includes two rotation angle information, that is, when the target object's head rotates in two directions, the target coordinates after the target object's head is rotated can be determined.

[0133] In another embodiment, when the rotation information includes three rotation angle information, the three rotation angle information includes third rotation angle information, fourth rotation angle information and fifth rotation angle information; the specific implementation of determining the target coordinates after the head of the target object is rotated according to the rotation information and the initial coordinates can be seen in Figure 7 , is a flow chart of another target coordinate determination method provided by an embodiment of the present invention, such as Figure 7 , the process may include:

[0134] Step 701: Determine the third initial rotation axis coordinates of the target object's head before rotation corresponding to the third rotation angle information.

[0135] Step 702: Determine the third target rotation axis coordinates after the head of the target object is rotated according to the third rotation angle information and the third initial rotation axis coordinates.

[0136] Step 703: input the third target rotation axis coordinate, the initial coordinate, and the third rotation angle information into the rotation coordinate calculation model to obtain the second intermediate coordinate after the target object's head is rotated.

[0137] Step 704: Determine the fourth initial rotation axis coordinates of the target object's head before rotation, corresponding to the fourth rotation angle information.

[0138] Step 705: Determine the fourth target rotation axis coordinates after the head of the target object is rotated according to the fourth rotation angle information and the fourth initial rotation axis coordinates.

[0139] Step 706: input the fourth target rotation axis coordinate, the second intermediate coordinate, and the fourth rotation angle information into the rotation coordinate calculation model to obtain the third intermediate coordinate after the target object's head is rotated.

[0140] Step 707: Determine the fifth initial rotation axis coordinates of the target object's head before rotation, corresponding to the fifth rotation angle information.

[0141] Step 708: Determine the fifth target rotation axis coordinates after the head of the target object is rotated according to the fifth rotation angle information and the fifth initial rotation axis coordinates.

[0142] Step 709: input the fifth target rotation axis coordinate, the third intermediate coordinate, and the fifth rotation angle information into the rotation coordinate calculation model to obtain the target coordinate after the head of the target object is rotated.

[0143] The following is a unified description of steps 701 to 709:

[0144] The target coordinates are coordinates output by performing calculations using a rotation coordinate calculation model when the rotation information includes three rotation angle information.

[0145] The rotation information may include third rotation angle information, fourth rotation angle information and fifth rotation angle information. Here, the third rotation angle information, the fourth rotation angle information and the fifth rotation angle information may be any rotation angle information of horizontal rotation angle information, up and down rotation angle information and left and right deflection angle information, but the third rotation angle information, the fourth rotation angle information and the fifth rotation angle information are not the same.

[0146] The third initial rotation axis coordinate is the rotation axis coordinate corresponding to the target object before the head is rotated based on the third rotation information, which can be any coordinate on the X-axis, any coordinate on the Y-axis, or any coordinate on the Z-axis, and the embodiment of the present invention does not limit this.

[0147] In an embodiment of the present invention, after the third target rotation axis coordinates, initial coordinates, and third rotation angle information are input into the rotation coordinate calculation model to obtain the second intermediate coordinates after the target object's head is rotated, the second intermediate coordinates can be used as the initial coordinates of the third intermediate coordinates after the target object's head is rotated based on the fourth rotation angle information, and the third intermediate coordinates can be determined as the initial coordinates of the target coordinates after the target object's head is rotated based on the fifth rotation angle information. In this way, the rotation axis coordinates, rotation information, and initial coordinates can be input into the rotation coordinate calculation model to obtain the target coordinates output by the rotation coordinate calculation model.

[0148] For example, suppose Figure 4 Taking the left-handed coordinate system shown in the figure as an example, the initial coordinates of the target object's head before rotation are (0, 0, a), the third rotation angle information is a horizontal rotation of 90 degrees to the left, the fourth rotation angle information is a downward rotation of 90 degrees, and the fifth rotation angle information is a leftward rotation of 90 degrees. It can be determined that before the target object's head rotates, the rotation axis is the Y axis, and the initial rotation axis coordinates are (0, n y, 0). After the target object's head rotates the third rotation angle information, the rotation axis is still the Y axis, so the third target rotation axis coordinate after the target object's head rotates can be determined to be (0, n y , 0). Afterwards, the third target rotation axis coordinate, the initial coordinate, and the third rotation angle information can be input into the rotation coordinate calculation model exemplified by the above formula (I), and the second intermediate coordinate after the rotation of the target object's head is obtained as (0, 0, -a).

[0149] Furthermore, before the target object's head rotates the fourth rotation angle information, the initial rotation axis coordinates are determined to be: (0, n y , 0), after the target object's head rotates the fourth angle information, the rotation axis is the Z axis. Then, the coordinates of the fourth target rotation axis can be determined as: (0, 0, n z ). Then, the fourth target rotation axis coordinate, the second intermediate coordinate, and the fourth rotation angle information are input into the rotation coordinate calculation model exemplified by the above formula (I), and the third intermediate coordinate after the rotation of the target object's head is obtained as (0, 0, -a).

[0150] Finally, it is determined that the initial rotation axis corresponding to the fifth rotation angle information of the target object's head is the Z axis, and the coordinates of the fifth initial rotation axis are (0, 0, n z ), the rotation axis corresponding to the fifth rotation angle information of the target object's head is the X axis, and the coordinate of the fifth target rotation axis is (n x , 0, 0). Then, the fifth target rotation axis coordinate, the third intermediate coordinate, and the fifth rotation angle information are input into the rotation coordinate calculation model illustrated in the above formula (I), and the target coordinate after the rotation of the target object's head is obtained as (0, -a, 0).

[0151] It should be noted that the above is merely an exemplary description of the sequence of calculating the target coordinates according to the third rotation angle information, the fourth rotation angle information and the fifth rotation angle information, and the embodiment of the present invention does not limit this.

[0152] So far, completed Figure 7 A description of the process shown.

[0153] Step 304: input the target coordinates into a preset angle calculation formula to obtain the angle information of the target object's head rotating in the left-handed coordinate system.

[0154] Step 305: Determine that the angle information is the angle information of the corresponding speaker.

[0155] The following is a unified description of step 304 and step 305:

[0156] In one embodiment, to determine the coordinates of the corresponding speaker after the target object's head is rotated, after obtaining the target coordinates after the target object's head is rotated, the angle information corresponding to the target coordinates can be determined. Here, the angle information is the angle information corresponding to the target coordinates after the target object's head is rotated. In this way, the angle information can be determined to be the angle information of the corresponding speaker.

[0157] Optionally, the spatial coordinate system can be converted into a spherical coordinate system, and the angle information of the corresponding speaker in the spherical coordinate value can be calculated according to the target coordinates. In this way, it can be avoided that after the target object is rotated by the head, the audio experience is not good due to the different distances between the target object and the speaker.

[0158] Specifically, the target coordinates may be input into a preset angle calculation formula as exemplified in the following formula (II) to obtain the angle information of the target object's head rotating in the left-handed coordinate system.

[0159]

[0160] For example, suppose the target coordinates after the target object's head is rotated are ( 1, 2), then, according to the above description, the target coordinates are input into the angle calculation formula shown in formula (ii), and the angle information of the rotation of the target object's head in the left-hand coordinate system is obtained as θ=45 degrees and φ=30 degrees.

[0161] Step 306: For each preset HRIR data in the preset HRIR database, determine the preset angle information of the preset HRIR data.

[0162] Step 307: Determine the difference between the preset angle information and the angle information.

[0163] Step 308: When the difference is less than or equal to the preset difference threshold, determine that the HRIR data corresponding to the difference is the HRIR data matching the angle information.

[0164] The following is a unified description of steps 306 to 308:

[0165] In the embodiment of the present invention, in order to facilitate the determination of HRIR data according to angle information, HRIR data of preset angle intervals may be preset and the HRIR data may be saved in a preset HRIR database.

[0166] For example, assuming that the preset angle interval is 10 degrees and the HRIR data of the corresponding angle is set, HRIR data of 36 different angles can be set, and the HRIR data can be saved in the preset HRIR database.

[0167] In one embodiment, according to the angle information of the corresponding speaker, HRIR data matching the speaker may be determined from the HRIR database so as to perform convolution processing on the HRIR data and the decompressed audio data to generate spatial audio.

[0168] Among them, the above-mentioned HRIR data matching the angle information may be that the preset angle information corresponding to the HRIR data is consistent with the above-mentioned angle information or the difference between the two is the smallest, and this embodiment does not impose any limitation on this.

[0169] Optionally, the preset angle information corresponding to each HRIR data in the preset HRIR database may be determined, the difference between each preset angle information and the preset angle may be calculated, and the difference may be compared with a preset difference threshold (e.g., 4). If the difference is less than or equal to the difference threshold, it may be determined that the HRIR data corresponding to the difference is the HRIR data that matches the angle information; conversely, if the difference is greater than the difference threshold, it may be determined that the HRIR data corresponding to the difference is not the HRIR data that matches the angle information. In this way, the HRIR data that matches the angle information may be determined.

[0170] Step 309: Perform convolution processing based on the HRIR data and the decompressed audio data to generate spatial audio.

[0171] For a detailed description of step 309 , please refer to the relevant description of step 104 , which will not be repeated here.

[0172] In the technical solution of the present invention, the initial coordinates of the target object's head before rotation are obtained by the target sensor, and the rotation information of the target object's head is collected by the target sensor, and the rotation information includes at least one of the following rotation angle information: horizontal rotation angle information, up and down rotation angle information, left and right deflection angle information. According to the rotation information and the initial coordinates, the target coordinates after the target object's head is rotated are determined. In this way, the target coordinates of the target object's head after one or more rotations can be obtained. The target coordinates are input into a preset angle calculation formula to obtain the angle information of the target object's head rotating in the left-hand coordinate system, and the angle information is determined to be the angle information of the corresponding speaker. For each preset HRIR data in the preset HRIR database, the preset angle information of the preset HRIR data is determined, and the difference between the preset angle information and the angle information is determined. When the difference is less than or equal to the preset difference threshold, the HRIR data corresponding to the difference is determined to be the HRIR data matching the angle information. Then, convolution processing is performed based on the HRIR data and the decompressed audio data to generate spatial audio. In this way, the HRIR data that matches the angle information can be determined, and convolution processing can be performed on the HRIR data and the decompressed audio data to generate spatial audio. This can provide audio experiences in different directions after the target object's head rotates, thereby improving the experience.

[0173] Corresponding to the above-mentioned embodiment of the spatial audio generation method, the present invention also provides a block diagram of an embodiment of a device.

[0174] See also Figure 8 , is a block diagram of an embodiment of a spatial audio generation device provided by an embodiment of the present invention. Figure 8 As shown, the device comprises:

[0175] A coordinate acquisition module 801 is used to acquire the target coordinates of the target object after the head is rotated through a target sensor;

[0176] An angle information determination module 802, used to determine angle information of a corresponding speaker according to the target coordinates;

[0177] An HRIR data determination module 803 is used to determine head-related impulse response HRIR data matching the angle information;

[0178] The audio generation module 804 is used to perform convolution processing based on the HRIR data and the decompressed audio data to generate spatial audio.

[0179] In a possible implementation, the coordinate acquisition module 801 includes (not shown in the figure):

[0180] An initial coordinate acquisition unit, used to acquire the initial coordinates of the target object's head before rotation through a target sensor;

[0181] A rotation information acquisition unit, used to acquire rotation information of the target object's head through a target sensor;

[0182] A coordinate determination unit is used to determine the target coordinates of the target object's head after rotation according to the rotation information and the initial coordinates.

[0183] In a possible implementation manner, the rotation information includes at least one of the following rotation angle information: horizontal rotation angle information, up and down rotation angle information, left and right deflection angle information;

[0184] In the case where the rotation information includes a rotation angle information, the coordinate determination unit is specifically configured to:

[0185] Determine the initial rotation axis coordinates of the target object's head before rotation corresponding to the rotation angle information;

[0186] Determining the target rotation axis coordinates after the head of the target object is rotated according to the rotation angle information and the initial rotation axis coordinates;

[0187] The target rotation axis coordinates, initial coordinates, and rotation angle information are input into a preset rotation coordinate calculation model to obtain the target coordinates after the head of the target object is rotated.

[0188] In a possible implementation manner, when the rotation information includes two rotation angle information, the two rotation angle information include first rotation angle information and second rotation angle information;

[0189] The coordinate determination unit is further used for:

[0190] Determine a first initial rotation axis coordinate of the target object's head before rotation corresponding to the first rotation angle information;

[0191] Determine, according to the first rotation angle information and the first initial rotation axis coordinate, a first target rotation axis coordinate after the head of the target object is rotated;

[0192] Inputting the first target rotation axis coordinate, the initial coordinate, and the first rotation angle information into the rotation coordinate calculation model to obtain the first intermediate coordinate after the head of the target object is rotated;

[0193] Determine a second initial rotation axis coordinate of the target object's head before rotation corresponding to the second rotation angle information;

[0194] Determine the second target rotation axis coordinates after the head of the target object is rotated according to the second rotation angle information and the second initial rotation axis coordinates;

[0195] The second target rotation axis coordinates, the first intermediate coordinates, and the second rotation angle information are input into the rotation coordinate calculation model to obtain the target coordinates after the head of the target object is rotated.

[0196] In a possible implementation manner, when the rotation information includes three rotation angle information, the three rotation angle information include third rotation angle information, fourth rotation angle information and fifth rotation angle information;

[0197] The coordinate determination unit is further used for:

[0198] Determine a third initial rotation axis coordinate of the target object's head before rotation corresponding to the third rotation angle information;

[0199] Determine the third target rotation axis coordinate after the head of the target object is rotated according to the third rotation angle information and the third initial rotation axis coordinate;

[0200] Inputting the third target rotation axis coordinate, the initial coordinate, and the third rotation angle information into the rotation coordinate calculation model to obtain the second intermediate coordinate after the head of the target object is rotated;

[0201] Determine a fourth initial rotation axis coordinate of the target object's head before rotation corresponding to the fourth rotation angle information;

[0202] Determine, according to the fourth rotation angle information and the fourth initial rotation axis coordinate, a fourth target rotation axis coordinate after the head of the target object is rotated;

[0203] Inputting the fourth target rotation axis coordinate, the second intermediate coordinate, and the fourth rotation angle information into the rotation coordinate calculation model to obtain the third intermediate coordinate after the head of the target object is rotated;

[0204] Determine the fifth initial rotation axis coordinates of the target object's head before rotation corresponding to the fifth rotation angle information;

[0205] Determine, according to the fifth rotation angle information and the fifth initial rotation axis coordinate, the fifth target rotation axis coordinate after the head of the target object is rotated;

[0206] The fifth target rotation axis coordinate, the third intermediate coordinate, and the fifth rotation angle information are input into the rotation coordinate calculation model to obtain the target coordinate after the head of the target object is rotated.

[0207] In a possible implementation manner, the angle information determination module 802 is specifically configured to:

[0208] The target coordinates are input into a preset angle calculation formula to obtain the angle information of the target object's head rotating in the left-handed coordinate system. The angle calculation formula is as follows:

[0209]

[0210] Among them, θ and is the angle information of the target object's head rotation in the left-handed coordinate system, and (x′, y′, z′) is the target coordinates of the target object's head after rotation;

[0211] The angle information is determined to be the angle information of the corresponding speaker.

[0212] In a possible implementation manner, the HRIR data determination module is specifically used to:

[0213] For each preset HRIR data in a preset HRIR database, determining preset angle information of the preset HRIR data;

[0214] Determining a difference between the preset angle information and the angle information;

[0215] When the difference is less than or equal to a preset difference threshold, it is determined that the HRIR data corresponding to the difference is the HRIR data matching the angle information.

[0216] Fig. 9A schematic diagram of the structure of an audio device provided by an embodiment of the present invention, Fig. 9 The audio device 900 shown includes: a target sensor 901, at least one processor 902, a memory 903, at least one network interface 904, and a user interface 905. The various components in the audio device 900 are coupled together via a bus system 906. It is understood that the bus system 906 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 906 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 906 is not described in detail. Fig. 9 Various buses are labeled as bus system 906.

[0217] The target sensor is used to collect the target coordinates after the head of the target object is rotated.

[0218] The user interface 905 may include a display, a keyboard or a pointing device (eg, a mouse, a trackball), a touch pad or a touch screen, etc.

[0219] It can be understood that the memory 903 in the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM). The memory 903 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0220] In some implementations, the memory 903 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 9031 and an application program 9032 .

[0221] The operating system 9031 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 9032 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application 9032.

[0222] In the embodiment of the present invention, by calling the program or instruction stored in the memory 903, specifically, the program or instruction stored in the application 9032, the processor 902 is used to execute the method steps provided by each method embodiment, for example, including:

[0223] Acquiring the target coordinates of the target object after the head is rotated by the target sensor;

[0224] Determining angle information of a corresponding speaker according to the target coordinates;

[0225] Determining head-related impulse response (HRIR) data matching the angle information;

[0226] Convolution processing is performed based on the HRIR data and the decompressed audio data to generate spatial audio.

[0227] The method disclosed in the above embodiment of the present invention can be applied to the processor 902, or implemented by the processor 902. The processor 901 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 902 or the instruction in the form of software. The above processor 902 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 903, and the processor 902 reads the information in the memory 903 and completes the steps of the above method in combination with its hardware.

[0228] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0229] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0230] The audio device provided in this embodiment may be Fig. 9 The audio device shown in , can be executed as Figure 1 and Figure 3 All steps of the spatial audio generation method in Figure 1 and Figure 3 For details, please refer to the technical effect of the spatial audio generation method. Figure 1 and Figure 3 For the sake of brevity, the relevant description is not repeated here.

[0231] The embodiment of the present invention also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.

[0232] When one or more programs in the storage medium can be executed by one or more processors, the spatial audio generation method executed on the audio device side can be implemented.

[0233] The processor is used to execute the spatial audio generation program stored in the memory to implement the following steps of the spatial audio generation method performed on the audio device side:

[0234] Acquiring the target coordinates of the target object after the head is rotated by the target sensor;

[0235] Determining angle information of a corresponding speaker according to the target coordinates;

[0236] Determining head-related impulse response (HRIR) data matching the angle information;

[0237] Convolution processing is performed based on the HRIR data and the decompressed audio data to generate spatial audio.

[0238] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0239] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0240] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A spatial audio generation method, characterized in that: Applied to an audio device, the audio device is provided with a target sensor, and the method comprises: Acquiring the target coordinates after the head of the target object is rotated by the target sensor, including: acquiring the initial coordinates before the head of the target object is rotated by the target sensor; collecting the rotation information of the head of the target object by the target sensor; and determining the target coordinates after the head of the target object is rotated according to the rotation information and the initial coordinates; The rotation information includes at least one of the following rotation angle information: horizontal rotation angle information, up and down rotation angle information, left and right deflection angle information; when the rotation information includes one rotation angle information, determining the target coordinates after the head of the target object is rotated according to the rotation information and the initial coordinates, including: determining the initial rotation axis coordinates before the head of the target object is rotated corresponding to the rotation angle information; determining the target rotation axis coordinates after the head of the target object is rotated according to the rotation angle information and the initial rotation axis coordinates; inputting the target rotation axis coordinates, the initial coordinates, and the rotation angle information into a preset rotation coordinate calculation model to obtain the target coordinates after the head of the target object is rotated; the preset rotation coordinate calculation model includes: Among them, (n x , n y , n z ) is the target rotation axis coordinate, n x is the coordinate of the target rotation axis on the X axis, n y is the coordinate of the target rotation axis on the Y axis, n z is the coordinate of the target rotation axis on the Z axis, α is the rotation angle information, (x, y, z) is the initial coordinate of the target object's head before rotation, and (x′, y′, z′) is the target coordinate of the target object's head after rotation; Determining angle information of a corresponding speaker according to the target coordinates; Determining head-related impulse response (HRIR) data matching the angle information; Convolution processing is performed based on the HRIR data and the decompressed audio data to generate spatial audio.

2. The method according to claim 1, characterized in that In the case where the rotation information includes two rotation angle information, the two rotation angle information include first rotation angle information and second rotation angle information; The step of determining the target coordinates of the target object after the head is rotated according to the rotation information and the initial coordinates further includes: Determine a first initial rotation axis coordinate of the target object's head before rotation corresponding to the first rotation angle information; Determine, according to the first rotation angle information and the first initial rotation axis coordinate, a first target rotation axis coordinate after the head of the target object is rotated; Inputting the first target rotation axis coordinate, the initial coordinate, and the first rotation angle information into the rotation coordinate calculation model to obtain the first intermediate coordinate after the head of the target object is rotated; Determine a second initial rotation axis coordinate of the target object's head before rotation corresponding to the second rotation angle information; Determine the second target rotation axis coordinates after the head of the target object is rotated according to the second rotation angle information and the second initial rotation axis coordinates; The second target rotation axis coordinates, the first intermediate coordinates, and the second rotation angle information are input into the rotation coordinate calculation model to obtain the target coordinates after the head of the target object is rotated.

3. The method according to claim 1, characterized in that In the case where the rotation information includes three rotation angle information, the three rotation angle information include third rotation angle information, fourth rotation angle information and fifth rotation angle information; The step of determining the target coordinates of the target object after the head is rotated according to the rotation information and the initial coordinates further includes: Determine a third initial rotation axis coordinate of the target object's head before rotation corresponding to the third rotation angle information; Determine the third target rotation axis coordinate after the head of the target object is rotated according to the third rotation angle information and the third initial rotation axis coordinate; Inputting the third target rotation axis coordinate, the initial coordinate, and the third rotation angle information into the rotation coordinate calculation model to obtain the second intermediate coordinate after the head of the target object is rotated; Determine a fourth initial rotation axis coordinate of the target object's head before rotation corresponding to the fourth rotation angle information; Determining a fourth target rotation axis coordinate after the head of the target object is rotated according to the fourth rotation angle information and the fourth initial rotation axis coordinate; Inputting the fourth target rotation axis coordinate, the second intermediate coordinate, and the fourth rotation angle information into the rotation coordinate calculation model to obtain the third intermediate coordinate after the head of the target object is rotated; Determine the fifth initial rotation axis coordinates of the target object's head before rotation corresponding to the fifth rotation angle information; Determining a fifth target rotation axis coordinate after the head of the target object is rotated according to the fifth rotation angle information and the fifth initial rotation axis coordinate; The fifth target rotation axis coordinate, the third intermediate coordinate, and the fifth rotation angle information are input into the rotation coordinate calculation model to obtain the target coordinate after the head of the target object is rotated.

4. The method according to claim 1, characterized in that The step of determining angle information of a corresponding speaker according to the target coordinates includes: The target coordinates are input into a preset angle calculation formula to obtain the angle information of the target object's head rotating in the left-handed coordinate system. The angle calculation formula is as follows: Among them, θ and is the angle information of the target object's head rotation in the left-handed coordinate system, and (x′, y′, z′) is the target coordinates of the target object's head after rotation; The angle information is determined to be the angle information of the corresponding speaker.

5. The method according to claim 1, characterized in that The determining of HRIR data matching the angle information comprises: For each preset HRIR data in a preset HRIR database, determining preset angle information of the preset HRIR data; Determining a difference between the preset angle information and the angle information; When the difference is less than or equal to a preset difference threshold, it is determined that the HRIR data corresponding to the difference is the HRIR data matching the angle information.

6. A spatial audio generation device, characterized in that: The device comprises: A coordinate acquisition module, used to acquire the target coordinates of the target object after the head is rotated through the target sensor, including: acquiring the initial coordinates of the target object's head before rotation through the target sensor; collecting the rotation information of the target object's head through the target sensor; and determining the target coordinates of the target object after the head is rotated according to the rotation information and the initial coordinates; The rotation information includes at least one of the following rotation angle information: horizontal rotation angle information, up and down rotation angle information, left and right deflection angle information; when the rotation information includes one rotation angle information, determining the target coordinates after the head of the target object is rotated according to the rotation information and the initial coordinates, including: determining the initial rotation axis coordinates before the head of the target object is rotated corresponding to the rotation angle information; determining the target rotation axis coordinates after the head of the target object is rotated according to the rotation angle information and the initial rotation axis coordinates; inputting the target rotation axis coordinates, the initial coordinates, and the rotation angle information into a preset rotation coordinate calculation model to obtain the target coordinates after the head of the target object is rotated; the preset rotation coordinate calculation model includes: Among them, (n x , n y , n z ) is the target rotation axis coordinate, n x is the coordinate of the target rotation axis on the X axis, n y is the coordinate of the target rotation axis on the Y axis, n z is the coordinate of the target rotation axis on the Z axis, α is the rotation angle information, (x, y, z) is the initial coordinate of the target object's head before rotation, and (x′, y′, z′) is the target coordinate of the target object's head after rotation; An angle information determination module, used to determine the angle information of the corresponding speaker according to the target coordinates; An HRIR data determination module, used to determine head-related impulse response HRIR data matching the angle information; The audio generation module is used to perform convolution processing based on the HRIR data and the decompressed audio data to generate spatial audio.

7. An audio device, characterized in that: include: target sensors, processors, and memory; The target sensor is used to collect the target coordinates after the head of the target object rotates; The processor is configured to execute the spatial audio generation program stored in the memory to implement the spatial audio generation method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the spatial audio generation method according to any one of claims 1 to 5.

Images