Multi-user interaction scene linkage method based on VR equipment

Abstract

The invention relates to the technical field of VR (Virtual Reality) equipment, in particular to a multi-user interaction scene linkage method based on VR equipment, which comprises the following steps: acquiring audio data sent by a user side for controlling a virtual three-dimensional role model in a virtual scene in real time, synchronously acquiring the posture of the user side for controlling the virtual three-dimensional role model, and according to the posture of the virtual three-dimensional role model, carrying out interaction on the virtual three-dimensional role model. Determining the propagation direction of audio data sent by the user side through the virtual three-dimensional role model; according to the invention, intelligent adjustment and distribution processing are brought to the audio data through the orientation information of the audio data sent by the virtual three-dimensional role model in the virtual scene by the user side and the relative orientation information of other user sides; the audio data received by each user side in the virtual scene can be regulated and controlled according to the relative position management of the audio data source user, so that the propagation of the audio data in the virtual scene better conforms to the audio propagation logic of a real scene, and a more real virtual scene simulation effect is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of VR equipment, and in particular to a multi-user interactive scene linkage method based on VR equipment. Background Art

[0002] VR interaction is a key link in virtual reality technology. With the help of handles and somatosensory devices, users can naturally operate and interact in virtual scenes, such as grabbing virtual objects with handles and controlling the movement of characters with body movements, which greatly enhances the sense of immersion and makes people feel as if they are in a realistic virtual world, opening up a new interactive experience.

[0003] The invention patent with application number 201610270381.4 discloses a multi-user voice interaction method based on a virtual reality VR scene, which is characterized in that it includes the following steps: when multiple users are in a virtual reality VR scene, voice collection is performed through the microphone of the current user's mobile terminal to obtain the sound signal of the surrounding environment, wherein the virtual reality VR scene is implemented by the current user's mobile terminal in conjunction with VR glasses and headphones; noise reduction processing is performed on the sound signal to obtain the voice of the speaking user; the VR sound source of the VR scene is obtained; the voice of the speaking user is synchronously synthesized with the W sound source to generate a synthesized voice, and the current user's headphones are controlled to output the synthesized voice.

[0004] The application aims to solve the problem that "the VR experience is currently limited to a single person, and the interaction between users in scenarios such as multiplayer VR games or multiplayer VR conferences is relatively weak."

[0005] On the other hand, during the audio interaction process of VR devices, the audio sent by the user is relatively different from that received by other users. Most of the audio is received through fixed algorithms and running logic processing. Compared with the changes in audio propagation and reception order in real scenes, there is no relatively intelligent setting. As a result, there is still room for improvement in the sense of reality of virtual scenes simulated by VR devices during audio interaction.

[0006] Therefore, a multi-user interactive scene linkage method based on VR devices is proposed. Summary of the invention

[0007] In view of the above-mentioned shortcomings of the prior art, the present invention provides a multi-user interactive scene linkage method based on VR equipment, which solves the technical problems raised in the above-mentioned background technology.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A multi-user interactive scene linkage method based on VR equipment, comprising:

[0010] The audio data emitted by the user end controlling the virtual three-dimensional character model in the virtual scene is collected in real time, the posture of the virtual three-dimensional character model controlled by the user end is collected synchronously, and the direction in which the audio data emitted by the user end is transmitted through the virtual three-dimensional character model is determined according to the posture of the virtual three-dimensional character model; the postures of the virtual three-dimensional character models of other user ends in the virtual scene are collected, and based on the postures of the virtual three-dimensional character models of other user ends and the propagation direction of the audio data emitted by the user end through the virtual three-dimensional character model, an audio data distribution target model is constructed; the constructed audio data distribution target model is obtained, the audio data distribution target model is traversed, and the distribution degree of the virtual three-dimensional character models of other user ends in the audio data distribution target model is analyzed; the distribution degree analysis results of the virtual three-dimensional character models of other user ends are obtained, and the audio data is copied, adjusted, and distributed according to the distribution degree analysis results and the parameters of the audio data emitted by the user end that emits the audio data; in the audio data distribution stage, an audio data distribution queue is set, and the audio data to be distributed is distributed to each other user end based on the audio data distribution queue.

[0011] Furthermore, smooth noise reduction processing is synchronously performed on the audio data collected in the virtual scene. During the collection phase, the posture of the virtual three-dimensional character model uses the center point of the head of the virtual three-dimensional character model as a positioning point, and the line connecting the positioning point and the nose tip of the virtual three-dimensional character model as a direction identification line. The combination of the positioning point and the direction identification line is recorded as the posture of the virtual three-dimensional character model.

[0012] Among them, when determining the propagation direction of audio data, the coordinates of the virtual three-dimensional character model are determined based on the position of the positioning point in the posture of the virtual three-dimensional character model relative to the virtual scene, that is, the starting point of the audio data propagation direction. Based on the positioning point on the direction identification line as the starting point and the other end point as the end point, the audio data propagation angle is determined relative to the three-dimensional coordinate axis system of the virtual scene. The combination of the audio data propagation angle and the starting point of the propagation direction is recorded as the audio data propagation direction.

[0013] Furthermore, the smooth noise reduction processing logic of the audio data is expressed as:

[0014]

[0015] Where: y[n] is the audio data sequence after smoothing and noise reduction; M is the window length of the moving average; n is the total number of elements in the original audio data sequence; x[i] is the value of the i-th element in the original audio data sequence; λ is the judgment value;

[0016] Among them, the window length y[n] of the moving average is a positive odd number, and the judgment value λ is customized by the user. The above formula is used to adjust the original audio data sequence x[n] to the audio data sequence y[n] after smoothing and denoising, and the output of the audio data after smoothing and denoising has been completed.

[0017] Furthermore, the virtual three-dimensional character models of other user terminals in the virtual scene are: all user terminals other than the user terminal that sends audio data in the virtual scene;

[0018] When constructing the three-dimensional model of audio data, the coordinate axis system of the virtual space is used as a carrier, and all directional identification lines are represented in the coordinate axis system of the virtual space. The identification result of all directional identification lines in the coordinate axis system is the audio data scattering target model.

[0019] Furthermore, in the audio data distribution target model, the distribution degree analysis logic for the virtual three-dimensional character models of other user terminals is expressed as follows:

[0020]

[0021] Where: θ(A) is the dispersion degree of other user terminals A based on the analysis of their virtual 3D character models; N (A，B) is the set of obstacles in the virtual scene between the direction identification line to which the other user terminal A belongs and the end close to the direction identification line to which the reference user terminal B belongs; j is the reduction factor of the audio by the jth obstacle; k max (A, B) is the maximum value of the angle formed by extending the orientation identification line of the other user terminal A and the orientation identification line of the reference user terminal B under any viewing angle within 180 degrees; D(A, B) is the straight-line distance between the adjacent ends of the orientation identification line of the other user terminal A and the orientation identification line of the reference user terminal B; γ is the normalization factor;

[0022] Wherein, the normalization factor γ>0, referring to the user terminal B, ie, the user terminal that sends the audio data, based on the above formula, the corresponding θ(A) is obtained for each other user terminal.

[0023] Furthermore, the and exists In the case of , f(·) represents the control function. If the value of the formula in the brackets of f(·) is greater than or equal to zero, then f(·) takes the value of the formula in the brackets itself. If the value of the formula in the brackets of f(·) is less than zero, f(·)=0;

[0024] The normalization factor γ is used to control the dispersion of the calculations of other user terminals to be within the range of 0 to 1;

[0025] Among them, the reduction factor of the audio by obstacles in the virtual scene is customized by the virtual scene management background. When setting the reduction factor, it is subject to: all reduction factor values ​​are within the range of 0.01 to 0.99, and the reference obstacles are set according to the material of the same obstacles in reality.

[0026] Furthermore, the parameter of the audio data is the intensity of the audio data;

[0027] The copying of audio data is subject to: the number of audio data copies is equal to that of other users;

[0028] Adjustment of audio data is subject to: the copied audio data is configured one by one on other user terminals, adjustment logic is set, and each audio data configured on other user terminals is adjusted based on the adjustment logic;

[0029] Distribution of audio data is subject to: distributing the adjusted audio data configured on other user terminals to the corresponding other users;

[0030] The audio data to be copied is audio data that has undergone smooth noise reduction processing.

[0031] Furthermore, the adjustment logic of the audio data is expressed as:

[0032] DB z =DB 0 ×θ(i);

[0033] Where: DB z is the intensity of the adjusted audio data; DB 0 is the original intensity of the audio data; θ(i) is the dispersion degree calculated by the i-th other user end;

[0034] The audio data distributed by each other user terminal is adjusted based on the above formula.

[0035] Furthermore, the audio data distribution queue is set to comply with:

[0036] The distance between each other user end and the user end that sends the audio data in the virtual scene is obtained, and a distribution queue is set according to the distance, so that other user ends that are close to the user end that sends the audio data in the virtual scene receive the audio data first, and other user ends that are far from the user end that sends the audio data receive the audio data later.

[0037] Compared with the known public technology, the technical solution provided by the present invention has the following beneficial effects:

[0038] The present invention provides a multi-user interactive scene linkage method based on VR equipment. During the execution of the method, the direction of the audio data emitted by the virtual three-dimensional role model of the user terminal in the virtual scene and the relative direction information of other user terminals are used to bring intelligent adjustment and distribution processing to the audio data, so that the audio data received by each user terminal in the virtual scene can be regulated according to the relative position management of the audio data source user, so that the propagation of the audio data in the virtual scene is more in line with the audio propagation logic of the real scene, thereby achieving a more realistic virtual scene simulation effect;

[0039] Synchronously, during the audio data generation stage, a specified audio data processing logic is also provided to enable the audio data distributed in the virtual scene to be effectively smoothed and noise-reduced before distribution, thereby enhancing the virtual experience provided by the VR device. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 The figure is a flowchart of a multi-user interactive scene linkage method based on VR devices. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The present invention will be further described below in conjunction with the embodiments.

[0044] Example:

[0045] A multi-user interactive scene linkage method based on VR devices in this embodiment is as follows: Figure 1 As shown, including:

[0046] Collecting audio data sent by a user terminal controlling a virtual three-dimensional character model in a virtual scene in real time, synchronously collecting the posture of the user terminal controlling the virtual three-dimensional character model, and determining the direction in which the audio data sent by the user terminal is transmitted through the virtual three-dimensional character model according to the posture of the virtual three-dimensional character model;

[0047] The audio data collected in the virtual scene is subjected to smooth noise reduction processing synchronously. During the collection phase of the posture of the virtual three-dimensional character model, the center point of the head of the virtual three-dimensional character model is used as the positioning point, and the line connecting the positioning point and the nose tip of the virtual three-dimensional character model is used as the orientation mark line. The combination of the positioning point and the orientation mark line is recorded as the posture of the virtual three-dimensional character model.

[0048] When determining the propagation direction of the audio data, the coordinates of the virtual three-dimensional character model are determined based on the position of the positioning point in the posture of the virtual three-dimensional character model relative to the virtual scene, that is, the starting point of the propagation direction of the audio data. The positioning point on the orientation mark line is used as the starting point and the other end point is used as the end point. The propagation angle of the audio data is determined relative to the three-dimensional coordinate axis system of the virtual scene. The combination of the propagation angle of the audio data and the starting point of the propagation direction is recorded as the propagation direction of the audio data.

[0049] The smooth noise reduction processing logic of audio data is expressed as:

[0050]

[0051] Where: y[n] is the audio data sequence after smoothing and noise reduction; M is the window length of the moving average; n is the total number of elements in the original audio data sequence; x[i] is the value of the i-th element in the original audio data sequence; λ is the judgment value;

[0052] The moving average window length y[n] is a positive odd number, and the judgment value λ is customized by the user. The above formula is used to adjust the original audio data sequence x[n] to the smoothed and denoised audio data sequence y[n], and the output of the smoothed and denoised audio data is completed.

[0053] By setting the above logical formula, a specified smooth noise reduction processing logic is brought to the audio data, which effectively improves the accuracy of the application data in the subsequent steps of the above method and improves the effect of this method on audio data processing.

[0054] The virtual three-dimensional character models of other user terminals in the virtual scene are: all user terminals other than the user terminal that sends audio data in the virtual scene;

[0055] When constructing the three-dimensional model of audio data, the coordinate axis system of the virtual space is used as a carrier, and all directional identification lines are represented in the coordinate axis system of the virtual space. The identification result of all directional identification lines in the coordinate axis system is the audio data scattering target model;

[0056] Collecting the postures of virtual three-dimensional character models of other user terminals in the virtual scene, and constructing an audio data dissemination target model based on the postures of the virtual three-dimensional character models of other user terminals and the propagation direction of the audio data sent by the user terminal through the virtual three-dimensional character model;

[0057] Acquire the constructed audio data dispersion target model, traverse the audio data dispersion target model, and perform dispersion degree analysis on virtual three-dimensional character models of other user terminals in the audio data dispersion target model;

[0058] In the audio data distribution target model, the distribution degree analysis logic for the virtual three-dimensional character models of other user terminals is expressed as follows:

[0059]

[0060] Where: θ(A) is the dispersion degree of other user terminals A based on the analysis of their virtual 3D character models; N (A，B) is the set of obstacles in the virtual scene between the direction identification line to which the other user terminal A belongs and the end close to the direction identification line to which the reference user terminal B belongs; j is the reduction factor of the audio by the jth obstacle; k max (A, B) is the maximum value of the angle formed by extending the orientation identification line of the other user terminal A and the orientation identification line of the reference user terminal B under any viewing angle within 180 degrees; D(A, B) is the straight-line distance between the adjacent ends of the orientation identification line of the other user terminal A and the orientation identification line of the reference user terminal B; γ is the normalization factor;

[0061] Wherein, the normalization factor γ>0, referring to the user end B, i.e., the user end that sends the audio data, based on the above formula, the corresponding θ(A) is obtained for each other user end;

[0062] and exists In the case of , f(·) represents the control function. If the value of the formula in the brackets of f(·) is greater than or equal to zero, then f(·) takes the value of the formula in the brackets itself. If the value of the formula in the brackets of f(·) is less than zero, f(·)=0;

[0063] The normalization factor γ is used to control the dispersion of the calculations of other user terminals to be within the range of 0 to 1;

[0064] Through the above logic formula, the analysis logic of the virtual 3D character model dispersion degree is limited to ensure stable calculation of the virtual 3D character model dispersion degree, and provide adjustment logic for the final adjustment of the audio data in this embodiment.

[0065] Among them, the reduction factor of the audio by obstacles in the virtual scene is customized by the virtual scene management background. When setting the reduction factor, it is subject to: all reduction factor values ​​are within the range of 0.01 to 0.99, and the reference obstacles are set according to the material of the same obstacles in reality;

[0066] Obtaining the dispersion analysis results of the virtual three-dimensional character models of other user terminals, and copying, adjusting, and distributing the audio data according to the dispersion analysis results and the parameters of the audio data sent by the user terminal that sent the audio data;

[0067] In the audio data distribution phase, an audio data distribution queue is set, and the audio data to be distributed is distributed to other user terminals based on the audio data distribution queue;

[0068] The parameter of the audio data is the intensity of the audio data;

[0069] The copying of audio data is subject to: the number of audio data copies is equal to that of other users;

[0070] Adjustment of audio data is subject to: the copied audio data is configured one by one on other user terminals, adjustment logic is set, and each audio data configured on other user terminals is adjusted based on the adjustment logic;

[0071] Distribution of audio data is subject to: distributing the adjusted audio data configured on other user terminals to the corresponding other users;

[0072] The audio data to be copied is the audio data that has undergone smooth noise reduction processing;

[0073] The logic of adjusting audio data is expressed as:

[0074] DB z =DB 0 ×θ(i);

[0075] Where: DB z is the intensity of the adjusted audio data; DB 0 is the original intensity of the audio data; θ(i) is the dispersion degree calculated by the i-th other user end;

[0076] Wherein, the audio data distributed by each other user terminal is adjusted based on the above formula;

[0077] The audio data distribution queue is set to comply with:

[0078] The distance between each other user end and the user end that sends the audio data in the virtual scene is obtained, and a distribution queue is set according to the distance, so that other user ends that are close to the user end that sends the audio data in the virtual scene receive the audio data first, and other user ends that are far from the user end that sends the audio data receive the audio data later.

[0079] In this embodiment, the execution of the method in the above embodiment is provided, which effectively provides improvements for each user terminal in the virtual scene during the stage of receiving audio data, thereby effectively enhancing the virtual scene VR experience.

[0080] In summary, during the execution of the method in the above embodiment, the orientation of the audio data emitted by the user terminal in the virtual three-dimensional character model in the virtual scene and the relative orientation information of other user terminals are used to intelligently adjust and distribute the audio data, so that the audio data received by each user terminal in the virtual scene can be regulated according to the relative position management of the audio data source user, so that the propagation of audio data in the virtual scene is more in line with the audio propagation logic of the real scene, thereby achieving a more realistic virtual scene simulation effect; synchronously, in the audio data generation stage, a specified audio data processing logic is also provided, so that the audio data distributed in the virtual scene can be effectively smoothed and denoised before distribution, thereby enhancing the virtual experience provided by the VR device.

[0081] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-user interactive scene linkage method based on VR equipment, characterized in that: include: Collecting audio data sent by a user terminal controlling a virtual three-dimensional character model in a virtual scene in real time, synchronously collecting the posture of the user terminal controlling the virtual three-dimensional character model, and determining the direction in which the audio data sent by the user terminal is transmitted through the virtual three-dimensional character model according to the posture of the virtual three-dimensional character model; Collecting the postures of virtual three-dimensional character models of other user terminals in the virtual scene, and constructing an audio data dissemination target model based on the postures of the virtual three-dimensional character models of other user terminals and the propagation direction of the audio data sent by the user terminal through the virtual three-dimensional character model; Acquire the constructed audio data dispersion target model, traverse the audio data dispersion target model, and perform dispersion degree analysis on virtual three-dimensional character models of other user terminals in the audio data dispersion target model; Obtaining the dispersion analysis results of the virtual three-dimensional character models of other user terminals, and copying, adjusting, and distributing the audio data according to the dispersion analysis results and the parameters of the audio data sent by the user terminal that sent the audio data; In the audio data distribution phase, an audio data distribution queue is set, and the audio data to be distributed is distributed to other user terminals based on the audio data distribution queue.

2. A multi-user interactive scene linkage method based on VR equipment according to claim 1, characterized in that: The audio data collected in the virtual scene is synchronously subjected to smooth noise reduction processing. During the collection phase, the posture of the virtual three-dimensional character model is obtained by taking the center point of the head of the virtual three-dimensional character model as a positioning point, and the line connecting the positioning point and the nose tip of the virtual three-dimensional character model as a direction identification line. The combination of the positioning point and the direction identification line is recorded as the posture of the virtual three-dimensional character model. Among them, when determining the propagation direction of audio data, the coordinates of the virtual three-dimensional character model are determined based on the position of the positioning point in the posture of the virtual three-dimensional character model relative to the virtual scene, that is, the starting point of the audio data propagation direction. Based on the positioning point on the direction identification line as the starting point and the other end point as the end point, the audio data propagation angle is determined relative to the three-dimensional coordinate axis system of the virtual scene. The combination of the audio data propagation angle and the starting point of the propagation direction is recorded as the audio data propagation direction.

3. A multi-user interactive scene linkage method based on VR equipment according to claim 2, characterized in that: The smooth noise reduction processing logic of the audio data is expressed as: Where: y[n] is the audio data sequence after smoothing and noise reduction; M is the window length of the moving average; n is the total number of elements in the original audio data sequence; x[i] is the value of the i-th element in the original audio data sequence; λ is the judgment value; Among them, the window length y[n] of the moving average is a positive odd number, and the judgment value λ is customized by the user. The above formula is used to adjust the original audio data sequence x[n] to the audio data sequence y[n] after smoothing and denoising, and the output of the audio data after smoothing and denoising has been completed.

4. A multi-user interactive scene linkage method based on VR equipment according to claim 2, characterized in that: The virtual three-dimensional role models of other user terminals in the virtual scene are: all user terminals other than the user terminal that sends audio data in the virtual scene; When constructing the three-dimensional model of audio data, the coordinate axis system of the virtual space is used as a carrier, and all directional identification lines are represented in the coordinate axis system of the virtual space. The identification result of all directional identification lines in the coordinate axis system is the audio data scattering target model.

5. The multi-user interactive scene linkage method based on VR equipment according to claim 1, characterized in that: In the audio data distribution target model, the distribution degree analysis logic for virtual three-dimensional character models of other user terminals is expressed as follows: Where: θ(A) is the dispersion degree of the other user terminal A based on the analysis of its virtual 3D character model; N(A, B) is the set of obstacles in the virtual scene between the direction identification line of the other user terminal A and the near end of the direction identification line of the reference user terminal B; χ j is the reduction factor of the audio by the jth obstacle; k max (A, B) is the maximum value of the angle formed by extending the orientation identification line of the other user terminal A and the orientation identification line of the reference user terminal B under any viewing angle within 180 degrees; D(A, B) is the straight-line distance between the adjacent ends of the orientation identification line of the other user terminal A and the orientation identification line of the reference user terminal B; γ is the normalization factor; Wherein, the normalization factor γ>0, referring to the user terminal B, ie, the user terminal that sends the audio data, based on the above formula, the corresponding θ(A) is obtained for each other user terminal.

6. A multi-user interactive scene linkage method based on VR equipment according to claim 5, characterized in that: Said and exists In the case of , f(·) represents the control function. If the value of the formula in the brackets of f(·) is greater than or equal to zero, then f(·) takes the value of the formula in the brackets itself. If the value of the formula in the brackets of f(·) is less than zero, f(·)=0; The normalization factor γ is used to control the dispersion of the calculations of other user terminals to be within the range of 0 to 1; Among them, the reduction factor of the audio by obstacles in the virtual scene is customized by the virtual scene management background. When setting the reduction factor, it is subject to: all reduction factor values ​​are within the range of 0.01 to 0.99, and the reference obstacles are set according to the material of the same obstacles in reality.

7. The multi-user interactive scene linkage method based on VR equipment according to claim 1, characterized in that: The parameter of the audio data is the intensity of the audio data; The copying of audio data is subject to: the number of audio data copies is equal to that of other users; Adjustment of audio data is subject to: the copied audio data is configured one by one on other user terminals, adjustment logic is set, and each audio data configured on other user terminals is adjusted based on the adjustment logic; Distribution of audio data is subject to: distributing the adjusted audio data configured on other user terminals to the corresponding other users; The audio data to be copied is audio data that has undergone smooth noise reduction processing.

8. A multi-user interactive scene linkage method based on VR equipment according to claim 7, characterized in that: The logic of adjusting audio data is expressed as: DB z =DB0×θ(i); Where: DB z is the intensity of the adjusted audio data; DB0 is the original intensity of the audio data; θ(i) is the dispersion degree of the corresponding calculation of the i-th other user end; The audio data distributed by each other user terminal is adjusted based on the above formula.

9. The multi-user interactive scene linkage method based on VR equipment according to claim 1, characterized in that: The audio data distribution queue is set to comply with: The distance between each other user end and the user end that sends the audio data in the virtual scene is obtained, and a distribution queue is set according to the distance, so that other user ends that are close to the user end that sends the audio data in the virtual scene receive the audio data first, and other user ends that are far from the user end that sends the audio data receive the audio data later.

Images