A multi-channel sound simulation processing method and system applied to virtual reality

Abstract

The multi-channel sound simulation processing method and system for virtual reality of the present invention can locate the first initial positioning report of the sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information; the first initial positioning report is modified by the reference positioning report estimated from the prior positioning information to obtain the positioning information of the Xth group of virtual reality sound effect simulation information, thereby improving the accuracy and reliability of the sound quality interference characteristics positioning. In this way, the first positioning information of the Xth group of virtual reality sound effect simulation information can be used for audio node deployment to minimize the sound effect attenuation problem as much as possible, so as to achieve the sound simulation effect in multiple directions and multiple positions, and thus ensure the high-quality virtual reality sound simulation effect.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual reality, and particularly to a multi-channel sound simulation processing method and system applied to virtual reality. Background Art

[0002] Currently, the rise of virtual reality technology (VR / AR / MR) mostly focuses on the visual simulation effect, but the simulation processing effect on sound is not ideal. As we all know, human senses are diverse, and the sound simulation with obvious shortcomings will inevitably seriously affect the final virtual reality effect. Therefore, how to ensure high-quality virtual reality sound simulation effect is one of the current work focuses. Summary of the Invention

[0003] To improve the technical problems existing in the related art, the present invention provides a multi-channel sound simulation processing method and system applied to virtual reality.

[0004] In a first aspect, an embodiment of the present invention provides a multi-channel sound simulation processing method applied to virtual reality, which is applied to a multi-channel sound simulation processing system. The method includes:

[0005] Performing sound quality interference feature positioning on the Xth group of virtual reality sound effect simulation information of the to-be-located multi-channel sound simulation task to determine a first initial positioning report of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information, where X is an integer greater than 1;

[0006] According to the Xth group of virtual reality sound effect simulation information, the first initial positioning report, and a reference positioning report for the sound quality interference feature in the Xth group of virtual reality sound effect simulation information, determining first positioning information of the Xth group of virtual reality sound effect simulation information. The first positioning information includes a first positioning tag set of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information, where the reference positioning report is estimated based on the positioning information of the previous X - 1 groups of virtual reality sound effect simulation information of the Xth group of virtual reality sound effect simulation information;

[0007] Deploying audio nodes by using the first positioning information of the Xth group of virtual reality sound effect simulation information.

[0008] Preferably, performing sound quality interference feature positioning on the Xth group of virtual reality sound effect simulation information of the to-be-located multi-channel sound simulation task to determine a first initial positioning report of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information includes:

[0009] According to the estimated scoring set of acoustic quality interference characteristics in the Xth group of virtual reality sound effect simulation information, disassemble the Xth group of virtual reality sound effect simulation information into a first sound effect simulation set with acoustic quality interference characteristics, a second sound effect simulation set without acoustic quality interference characteristics, and a third sound effect simulation set whose presence or absence of acoustic quality interference characteristics is undetermined;

[0010] Locate the acoustic quality interference characteristics of the first sound effect simulation set and the third sound effect simulation set of the Xth group of virtual reality sound effect simulation information, and determine the first initial positioning report of the acoustic quality interference characteristics in the Xth group of virtual reality sound effect simulation information.

[0011] Preferably, the method further includes:

[0012] According to the second positioning information of the X - 1 groups of virtual reality sound effect simulation information before the Xth group of virtual reality sound effect simulation information, change the first positioning information of the Xth group of virtual reality sound effect simulation information to determine the second positioning information of the Xth group of virtual reality sound effect simulation information.

[0013] Preferably, the method further includes:

[0014] According to the second positioning information of the X - 1 groups of virtual reality sound effect simulation information before the Xth group of virtual reality sound effect simulation information, estimate the perturbation factor of the acoustic quality interference characteristics in the Xth group of virtual reality sound effect simulation information, and determine the reference positioning report of the acoustic quality interference characteristics in the Xth group of virtual reality sound effect simulation information.

[0015] Preferably, the method further includes:

[0016] According to the reference positioning report of the acoustic quality interference characteristics in the Xth group of virtual reality sound effect simulation information and the X - 1th group of virtual reality sound effect simulation information, correct the estimated scoring set of the acoustic quality interference characteristics in the X - 1th group of virtual reality sound effect simulation information to determine the estimated scoring set of the acoustic quality interference characteristics in the Xth group of virtual reality sound effect simulation information.

[0017] Preferably, the locating the acoustic quality interference characteristics of the first sound effect simulation set and the third sound effect simulation set of the Xth group of virtual reality sound effect simulation information to determine the first initial positioning report of the acoustic quality interference characteristics in the Xth group of virtual reality sound effect simulation information includes:

[0018] Mine the voiceprint description of the virtual reality sound effect simulation information of the first sound effect simulation set and the third sound effect simulation set to obtain the first simulation voiceprint description;

[0019] Locate the acoustic quality interference characteristics of the first simulation voiceprint description to determine the second initial positioning report of the acoustic quality interference characteristics in the Xth group of virtual reality sound effect simulation information;

[0020] Determine a set number of first initial positioning reports from the second initial positioning reports according to the credibility coefficients of each second initial positioning report.

[0021] Preferably, the determining the first positioning information of the Xth group of virtual reality sound effect simulation information according to the Xth group of virtual reality sound effect simulation information, the first initial positioning report, and the reference positioning report for the sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information includes:

[0022] Perform upsampling processing on the reference positioning reports for each sound quality interference characteristic in the Xth group of virtual reality sound effect simulation information to determine the third initial positioning reports for each sound quality interference characteristic;

[0023] Align the third initial positioning reports and the first initial positioning reports respectively to determine the sound quality interference characteristics corresponding to each first initial positioning report;

[0024] According to the first initial positioning report and the virtual reality sound effect simulation information of the first sound effect simulation set corresponding to the sound effect simulation set where the first initial positioning report is located, and the third initial positioning report and the virtual reality sound effect simulation information of the second sound effect simulation set corresponding to the sound effect simulation set where the third initial positioning report is located, perform weighted processing on the initial positioning reports for each sound quality interference characteristic in the Xth group of virtual reality sound effect simulation information to obtain the first positioning label set for each sound quality interference characteristic in the Xth group of virtual reality sound effect simulation information.

[0025] Preferably, the aligning the third initial positioning reports and the first initial positioning reports respectively to determine the sound quality interference characteristics corresponding to each first initial positioning report includes:

[0026] Determine the difference comparison results between each third initial positioning report and each first initial positioning report respectively;

[0027] Regard the third initial positioning reports with difference comparison results greater than or equal to the difference comparison result determination value with the first initial positioning report as the aligned third initial positioning reports;

[0028] Regard the sound quality interference characteristics corresponding to the third initial positioning reports aligned with the first initial positioning report as the sound quality interference characteristics corresponding to the first initial positioning report.

[0029] Preferably, each second positioning information includes a second positioning tag set of the sound quality interference feature. The method of changing the first positioning information of the Xth group of virtual reality sound effect simulation information according to the second positioning information of the X - 1 groups of virtual reality sound effect simulation information before the Xth group of virtual reality sound effect simulation information to determine the second positioning information of the Xth group of virtual reality sound effect simulation information includes:

[0030] Determine the positioning tag set distribution of the first sound quality interference feature, where the first sound quality interference feature is any sound quality interference feature in the Xth group of virtual reality sound effect simulation information. The positioning tag set distribution of the first sound quality interference feature includes the second positioning tag set of the first sound quality interference feature in the second positioning information of the X - 1 groups of virtual reality sound effect simulation information, and the first positioning tag set of the first sound quality interference feature in the first positioning information of the Xth group of virtual reality sound effect simulation information;

[0031] For any positioning tag set in the positioning tag set distribution of the first sound quality interference feature, consider the positioning tag sets in the positioning tag set distribution whose quantization perturbation values with respect to the positioning tag set are less than or equal to the quantization perturbation value determination value as the core tag set of the positioning tag set;

[0032] Determine the third positioning tag set with the largest number of core tag sets from the positioning tag set distribution of the first sound quality interference feature;

[0033] Weight the third positioning tag set and all the core tag sets of the third positioning tag set to determine the second positioning tag set of the first sound quality interference feature in the Xth group of virtual reality sound effect simulation information.

[0034] Preferably, the method further includes: estimating the perturbation factor of the sound quality interference feature in the (X + 1)th group of virtual reality sound effect simulation information according to the second positioning information of the X - 1 groups of virtual reality sound effect simulation information before the Xth group of virtual reality sound effect simulation information and the second positioning information of the Xth group of virtual reality sound effect simulation information, and determining the reference positioning report of the sound quality interference feature in the (X + 1)th group of virtual reality sound effect simulation information.

[0035] Preferably, the method further includes:

[0036] Correct the estimated scoring set of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information according to the reference positioning report of the sound quality interference feature in the (X + 1)th group of virtual reality sound effect simulation information and the Xth group of virtual reality sound effect simulation information, and determine the estimated scoring set of the sound quality interference feature in the (X + 1)th group of virtual reality sound effect simulation information.

[0037] Preferably, performing acoustic quality interference feature localization on the Xth group of virtual reality sound effect simulation information for the to-be-located multi-channel sound simulation task, and determining a first initial localization report of the acoustic quality interference features in the Xth group of virtual reality sound effect simulation information, including:

[0038] Performing voiceprint description mining on the Xth group of virtual reality sound effect simulation information to obtain a second simulated voiceprint description;

[0039] Performing acoustic quality interference feature localization on the second simulated voiceprint description to determine a fourth initial localization report of the acoustic quality interference features in the Xth group of virtual reality sound effect simulation information;

[0040] Determining a set number of first initial localization reports from the fourth initial localization reports according to the credibility coefficients of each fourth initial localization report.

[0041] Preferably, the first localization information further includes the topic keywords of the acoustic quality interference features in the Xth group of virtual reality sound effect simulation information. Determining the first localization information of the Xth group of virtual reality sound effect simulation information according to the Xth group of virtual reality sound effect simulation information, the first initial localization report, and the reference localization report for the acoustic quality interference features in the Xth group of virtual reality sound effect simulation information, including:

[0042] Performing differential analysis on the second acoustic quality interference feature according to the virtual reality sound effect simulation information of the third sound effect simulation set corresponding to the first localization tag set where the second acoustic quality interference feature is located, to determine the topic keywords of the second acoustic quality interference feature, where the second acoustic quality interference feature is any one of the acoustic quality interference features in the Xth group of virtual reality sound effect simulation information.

[0043] In a second aspect, the present invention further provides a multi-channel sound simulation processing system, including a processor and a memory; the processor and the memory are communicatively connected, and the processor is configured to read and execute a computer program from the memory to implement the method described above.

[0044] In a third aspect, the present invention further provides a readable storage medium, on which a program is stored, and when the program is executed by a processor, the method described above is implemented.

[0045] According to an embodiment of the present application, perform sound quality interference feature localization on the X-th group of virtual reality sound effect simulation information for the multi-channel sound simulation task to be located, and determine the first initial localization report of the sound quality interference feature in the X-th group of virtual reality sound effect simulation information; according to the X-th group of virtual reality sound effect simulation information, the first initial localization report, and the reference localization report for the sound quality interference feature in the X-th group of virtual reality sound effect simulation information, determine the first localization information of the X-th group of virtual reality sound effect simulation information; use the determined first localization information of the X-th group of virtual reality sound effect simulation information to deploy audio nodes.

[0046] In this way, the first initial localization report of the sound quality interference feature in the X-th group of virtual reality sound effect simulation information can be located; the reference localization report estimated from the prior localization information is used to modify the first initial localization report to obtain the localization information of the X-th group of virtual reality sound effect simulation information, thereby improving the accuracy and reliability of the sound quality interference feature localization. In this way, the first localization information of the X-th group of virtual reality sound effect simulation information can be used to deploy audio nodes to minimize the sound effect attenuation problem as much as possible, so as to achieve the sound simulation effect in multiple directions and multiple positions, and thus ensure the high-quality virtual reality sound simulation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention.

[0048] Figure 1 It is a schematic hardware structure diagram of a multi-channel sound simulation processing system provided by an embodiment of the present invention.

[0049] Figure 2 It is a schematic flowchart of a multi-channel sound simulation processing method applied to virtual reality provided by an embodiment of the present invention.

[0050] Figure 3 It is a schematic communication architecture diagram of an application environment of a multi-channel sound simulation processing method applied to virtual reality provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are only examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0052] It should be noted that the terms "first", "second", etc. in the description, claims and the above drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0053] The method embodiments provided by the embodiments of the present invention can be executed in a multi-channel sound simulation processing system, a computer device or a similar computing device. Taking the operation on a multi-channel sound simulation processing system as an example, Figure 1 is a hardware structure block diagram of a multi-channel sound simulation processing system implementing a multi-channel sound simulation processing method applied to virtual reality according to an embodiment of the present invention. As Figure 1 shown, the multi-channel sound simulation processing system 10 may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processors 102 may include, but are not limited to, processing devices such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above multi-channel sound simulation processing system may further include a transmission device 106 for communication functions. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above multi-channel sound simulation processing system. For example, the multi-channel sound simulation processing system 10 may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown in the figure.

[0054] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to a multi-channel sound simulation processing method applied to virtual reality in the embodiments of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories may be connected to the multi-channel sound simulation processing system 10 through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0055] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the multi-channel sound simulation processing system 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a RadioFrequency (RF) module, which is used to communicate with the Internet wirelessly.

[0056] Based on this, please refer to Figure 2 , Figure 2 is a schematic flowchart of a multi-channel sound simulation processing method applied to virtual reality provided by an embodiment of the present invention. This method is applied to a multi-channel sound simulation processing system. Further, the method may at least include the technical solutions recorded in the following steps.

[0057] S100. Perform sound quality interference feature localization on the Xth group of virtual reality sound effect simulation information of the multi-channel sound simulation task to be located, and determine the first initial localization report of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information, where X is an integer greater than 1.

[0058] In some possible embodiments, performing sound quality interference feature localization on the Xth group of virtual reality sound effect simulation information of the multi-channel sound simulation task to be located as described in S100 and determining the first initial localization report of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information may include the technical solutions described in S110 and S120.

[0059] S110. According to the estimated score set of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information, disassemble the Xth group of virtual reality sound effect simulation information into a first sound effect simulation set with sound quality interference features, a second sound effect simulation set without sound quality interference features, and a third sound effect simulation set for which it is not determined whether there are sound quality interference features.

[0060] S120. Perform sound quality interference feature localization on the first sound effect simulation set and the third sound effect simulation set of the Xth group of virtual reality sound effect simulation information, and determine the first initial localization report of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information.

[0061] Applying to S110 and S120 can ensure the integrity and credibility of the first initial localization report.

[0062] Based on the above, the method further includes: changing the first positioning information of the Xth group of virtual reality sound effect simulation information according to the second positioning information of the X-1 groups of virtual reality sound effect simulation information before the Xth group of virtual reality sound effect simulation information, and determining the second positioning information of the Xth group of virtual reality sound effect simulation information.

[0063] In some examples, each second positioning information includes a second positioning tag set of the sound quality interference feature. Based on this, the changing the first positioning information of the Xth group of virtual reality sound effect simulation information according to the second positioning information of the X-1 groups of virtual reality sound effect simulation information before the Xth group of virtual reality sound effect simulation information, and determining the second positioning information of the Xth group of virtual reality sound effect simulation information includes: determining the distribution of the positioning tag sets of the first sound quality interference feature, where the first sound quality interference feature is any one of the sound quality interference features in the Xth group of virtual reality sound effect simulation information, and the distribution of the positioning tag sets of the first sound quality interference feature includes the second positioning tag set of the first sound quality interference feature in the second positioning information of the X-1 groups of virtual reality sound effect simulation information, and the first positioning tag set of the first sound quality interference feature in the first positioning information of the Xth group of virtual reality sound effect simulation information; for any one of the positioning tag sets in the distribution of the positioning tag sets of the first sound quality interference feature, regarding the positioning tag sets in the distribution of the positioning tag sets whose quantization perturbation value with the positioning tag set is less than or equal to the quantization perturbation value determination value as the core tag set of the positioning tag set; determining the third positioning tag set with the largest number of core tag sets from the distribution of the positioning tag sets of the first sound quality interference feature; weighting the third positioning tag set and all the core tag sets of the third positioning tag set to determine the second positioning tag set of the first sound quality interference feature in the Xth group of virtual reality sound effect simulation information.

[0064] Based on the above, the method further includes: estimating the perturbation factor of the sound quality interference feature in the (X+1)th group of virtual reality sound effect simulation information according to the second positioning information of the X-1 groups of virtual reality sound effect simulation information before the Xth group of virtual reality sound effect simulation information and the second positioning information of the Xth group of virtual reality sound effect simulation information, and determining the reference positioning report of the sound quality interference feature in the (X+1)th group of virtual reality sound effect simulation information.

[0065] Based on the above, the method further includes: correcting the estimated score set of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information according to the reference positioning report of the sound quality interference feature in the (X+1)th group of virtual reality sound effect simulation information and the Xth group of virtual reality sound effect simulation information, and determining the estimated score set of the sound quality interference feature in the (X+1)th group of virtual reality sound effect simulation information.

[0066] Based on the above, the method further includes: according to the reference positioning report of the sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information, and the (X - 1)th group of virtual reality sound effect simulation information, correcting the estimated scoring set of the sound quality interference characteristics in the (X - 1)th group of virtual reality sound effect simulation information, and determining the estimated scoring set of the sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information.

[0067] For some possible embodiments, the positioning of the sound quality interference characteristics for the first sound effect simulation set and the third sound effect simulation set of the Xth group of virtual reality sound effect simulation information described in S120, to determine the first initial positioning report of the sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information, may further include the following: performing voiceprint description mining on the virtual reality sound effect simulation information of the first sound effect simulation set and the third sound effect simulation set to obtain a first simulation voiceprint description; performing sound quality interference characteristic positioning on the first simulation voiceprint description to determine the second initial positioning report of the sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information; and determining a set number of first initial positioning reports from the second initial positioning reports according to the credibility coefficients of each second initial positioning report.

[0068] In some other embodiments, the positioning of the sound quality interference characteristics for the Xth group of virtual reality sound effect simulation information of the multi-channel sound simulation task to be positioned described in S100, to determine the first initial positioning report of the sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information, includes: performing voiceprint description mining on the Xth group of virtual reality sound effect simulation information to obtain a second simulation voiceprint description; performing sound quality interference characteristic positioning on the second simulation voiceprint description to determine the fourth initial positioning report of the sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information; and determining a set number of first initial positioning reports from the fourth initial positioning reports according to the credibility coefficients of each fourth initial positioning report.

[0069] S200. Determine the first positioning information of the Xth group of virtual reality sound effect simulation information according to the Xth group of virtual reality sound effect simulation information, the first initial positioning report, and the reference positioning report for the sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information, where the first positioning information includes the first positioning tag set of the sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information.

[0070] Wherein, the reference positioning report is estimated according to the positioning information of the (X - 1) groups of virtual reality sound effect simulation information before the Xth group of virtual reality sound effect simulation information.

[0071] In some possible embodiments, determining the first positioning information of the Xth group of virtual reality sound effect simulation information according to the Xth group of virtual reality sound effect simulation information, the first initial positioning report, and the reference positioning report for the sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information described in S200 may include the technical solutions described in S210-S230.

[0072] S210. Perform upsampling processing on the reference positioning reports for the respective sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information to determine the third initial positioning reports for the respective sound quality interference characteristics.

[0073] S220. Align the third initial positioning reports and the first initial positioning reports respectively to determine the sound quality interference characteristics corresponding to the respective first initial positioning reports.

[0074] In some possible embodiments, aligning the third initial positioning reports and the first initial positioning reports respectively to determine the sound quality interference characteristics corresponding to the respective first initial positioning reports described in S220 may include the following: respectively determining the difference comparison results between the respective third initial positioning reports and the respective first initial positioning reports; regarding the third initial positioning reports with the difference comparison results greater than or equal to the difference comparison result determination value from the first initial positioning reports as the aligned third initial positioning reports; regarding the sound quality interference characteristics corresponding to the third initial positioning reports aligned with the first initial positioning reports as the sound quality interference characteristics corresponding to the first initial positioning reports.

[0075] S230. According to the first initial positioning report and the virtual reality sound effect simulation information of the first sound effect simulation set corresponding to the sound effect simulation set where the first initial positioning report is located, and the third initial positioning report and the virtual reality sound effect simulation information of the second sound effect simulation set corresponding to the sound effect simulation set where the third initial positioning report is located, perform initial positioning report weighting on the respective sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information to obtain the first positioning tag set for the respective sound quality interference characteristics in the Xth group of virtual reality sound effect simulation information.

[0076] Applied to S210-S230, it can ensure the accuracy and credibility of the first positioning information.

[0077] In some other embodiments, the first positioning information further includes a subject keyword of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information. Based on this, the determination of the first positioning information of the Xth group of virtual reality sound effect simulation information according to the Xth group of virtual reality sound effect simulation information, the first initial positioning report, and the reference positioning report for the sound quality interference feature in the Xth group of virtual reality sound effect simulation information described in S200 may include the following: Differentiated analysis is performed on the second sound quality interference feature to determine the subject keyword of the second sound quality interference feature according to the virtual reality sound effect simulation information of the third sound effect simulation set corresponding to the first positioning tag set of the second sound quality interference feature, and the second sound quality interference feature is any one of the sound quality interference features in the Xth group of virtual reality sound effect simulation information.

[0078] S300. Deploy audio nodes by using the first positioning information of the Xth group of virtual reality sound effect simulation information.

[0079] For S300, the audio node deployment can be achieved in the following way: In the sound simulation site that needs to be arranged, several audio nodes are arranged to push multi-channel audio streams through existing communication technologies, including but not limited to Ethernet, Bluetooth, 5G, and WIFI, etc., so as to achieve the sound simulation effect of multiple spatial positions and multiple directions.

[0080] In this way, it is possible to achieve multi-point and multi-direction sound simulation, and the control is more refined than that of traditional 5.1 / 7.1 channel speakers. In addition, using traditional audio stream input and API interface audio stream input provides higher efficiency for the development of VR / AR / MR applications. Further, the number of nodes can be flexibly arranged according to the specifications, budget, and fineness requirements of the VR / AR / MR application site.

[0081] Based on the above content, the method further includes: estimating the perturbation factor of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information according to the second positioning information of the X - 1 groups of virtual reality sound effect simulation information before the Xth group of virtual reality sound effect simulation information, and determining the reference positioning report of the sound quality interference feature in the Xth group of virtual reality sound effect simulation information.

[0082] In summary, according to the embodiments of the present application, for the Xth group of virtual reality sound effect simulation information of the multi-channel sound simulation task to be located, the acoustic quality interference feature location is performed to determine the first initial location report of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information; according to the Xth group of virtual reality sound effect simulation information, the first initial location report, and the reference location report for the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information, the first location information of the Xth group of virtual reality sound effect simulation information is determined; the first location information of the Xth group of virtual reality sound effect simulation information is used for audio node deployment.

[0083] In this way, the first initial location report of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information can be located; the first initial location report is modified by the reference location report estimated from the prior location information to obtain the location information of the Xth group of virtual reality sound effect simulation information, thereby improving the accuracy and reliability of the acoustic quality interference feature location. In this way, the first location information of the Xth group of virtual reality sound effect simulation information can be used for audio node deployment to minimize the sound effect attenuation problem as much as possible, so as to achieve the sound simulation effect in multiple directions and multiple positions, and thus ensure the high-quality virtual reality sound simulation effect.

[0084] Based on the same or similar inventive concept as described above, the embodiment of the present invention further provides a schematic diagram of the architecture of an application environment 30 of a multi-channel sound simulation processing method applied to virtual reality, including a multi-channel sound simulation processing system 10 and intelligent audio nodes 20 that communicate with each other. The multi-channel sound simulation processing system 10 and the intelligent audio nodes 20 implement or partially implement the technical solutions described in the above method embodiments when running.

[0085] Furthermore, a readable storage medium is provided, on which a program is stored, and when the program is executed by a processor, the above method is implemented.

[0086] In several embodiments provided by the embodiments of the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the part of the module, program segment, or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0087] In addition, the various functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0088] If the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a media service server 10, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs. It should be noted that in this article, the terms "including", "comprising", or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article, or device. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device including the said element.

[0089] The foregoing are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A multi-channel sound simulation processing method applied to virtual reality, characterized in that Applied to a multi-channel sound simulation processing system, the method includes: Performing acoustic quality interference feature localization on the Xth group of virtual reality sound effect simulation information of the multi-channel sound simulation task to be located, and determining a first initial localization report of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information, where X is an integer greater than 1; According to the Xth group of virtual reality sound effect simulation information, the first initial localization report, and a reference localization report for the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information, determining the first localization information of the Xth group of virtual reality sound effect simulation information, where the first localization information includes a first localization tag set of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information, and wherein the reference localization report is estimated based on the localization information of the previous X - 1 groups of virtual reality sound effect simulation information of the Xth group of virtual reality sound effect simulation information; Deploying audio nodes by using the first localization information of the Xth group of virtual reality sound effect simulation information.

2. The method according to claim 1, wherein The performing acoustic quality interference feature localization on the Xth group of virtual reality sound effect simulation information of the multi-channel sound simulation task to be located and determining a first initial localization report of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information includes: According to the estimated score set of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information, disassembling the Xth group of virtual reality sound effect simulation information into a first sound effect simulation set with acoustic quality interference features, a second sound effect simulation set without acoustic quality interference features, and a third sound effect simulation set whose presence or absence of acoustic quality interference features is undetermined; Performing acoustic quality interference feature localization on the first sound effect simulation set and the third sound effect simulation set of the Xth group of virtual reality sound effect simulation information, and determining a first initial localization report of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information; Wherein, the method further includes: modifying the first localization information of the Xth group of virtual reality sound effect simulation information according to the second localization information of the previous X - 1 groups of virtual reality sound effect simulation information of the Xth group of virtual reality sound effect simulation information, and determining the second localization information of the Xth group of virtual reality sound effect simulation information.

3. The method according to claim 1, characterized in that, The method further includes: Estimating a perturbation factor of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information according to the second localization information of the previous X - 1 groups of virtual reality sound effect simulation information of the Xth group of virtual reality sound effect simulation information, and determining a reference localization report of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information.

4. The method according to claim 2, wherein The method further includes: Correcting the estimated score set of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information according to the reference localization report of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information and the X - 1th group of virtual reality sound effect simulation information, and determining the estimated score set of the acoustic quality interference feature in the Xth group of virtual reality sound effect simulation information. Among them, the acoustic quality interference feature localization of the first acoustic effect simulation set and the third acoustic effect simulation set of the Xth group of virtual reality acoustic effect simulation information, and determining the first initial localization report of the acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information includes: mining acoustic texture descriptions for the virtual reality acoustic effect simulation information of the first acoustic effect simulation set and the third acoustic effect simulation set to obtain a first simulated acoustic texture description; performing acoustic quality interference feature localization on the first simulated acoustic texture description to determine a second initial localization report of the acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information; and determining a set number of first initial localization reports from the second initial localization reports according to the credibility coefficients of each second initial localization report.

5. The method according to claim 1, characterized in that The determining of the first localization information of the Xth group of virtual reality acoustic effect simulation information according to the Xth group of virtual reality acoustic effect simulation information, the first initial localization report, and the reference localization report for the acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information includes: Performing upsampling processing on the reference localization reports of each acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information to determine third initial localization reports of each acoustic quality interference feature; Aligning the third initial localization reports and the first initial localization reports respectively to determine the acoustic quality interference features corresponding to each first initial localization report; Performing weighted initial localization reports on each acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information according to the first initial localization report and the virtual reality acoustic effect simulation information of the first acoustic effect simulation set corresponding to the acoustic effect simulation set where the first initial localization report is located, and the third initial localization report and the virtual reality acoustic effect simulation information of the second acoustic effect simulation set corresponding to the acoustic effect simulation set where the third initial localization report is located, to obtain a first localization tag set of each acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information; Among them, the aligning of the third initial localization reports and the first initial localization reports respectively to determine the acoustic quality interference features corresponding to each first initial localization report includes: respectively determining the difference comparison results between each third initial localization report and each first initial localization report; regarding the third initial localization reports with difference comparison results greater than or equal to the difference comparison result determination value with the first initial localization report as the aligned third initial localization reports; and regarding the acoustic quality interference features corresponding to the third initial localization reports aligned with the first initial localization report as the acoustic quality interference features corresponding to the first initial localization report.

6. The method according to claim 2, wherein Each second localization information includes a second localization tag set of the acoustic quality interference feature. The changing of the first localization information of the Xth group of virtual reality acoustic effect simulation information according to the second localization information of the X - 1 groups of virtual reality acoustic effect simulation information before the Xth group of virtual reality acoustic effect simulation information to determine the second localization information of the Xth group of virtual reality acoustic effect simulation information includes: Determine the distribution of the positioning tag set of the first acoustic quality interference feature, where the first acoustic quality interference feature is any one of the acoustic quality interference features in the Xth group of virtual reality acoustic effect simulation information, and the distribution of the positioning tag set of the first acoustic quality interference feature includes the second positioning tag set in the second positioning information of the X - 1 groups of virtual reality acoustic effect simulation information for the first acoustic quality interference feature, and the first positioning tag set in the first positioning information of the Xth group of virtual reality acoustic effect simulation information for the first acoustic quality interference feature; For any positioning tag set in the distribution of the positioning tag set of the first acoustic quality interference feature, consider the positioning tag sets in the distribution of the positioning tag set whose quantization perturbation value with the positioning tag set is less than or equal to the quantization perturbation value determination value as the core tag set of the positioning tag set; Determine the third positioning tag set with the largest number of core tag sets from the distribution of the positioning tag set of the first acoustic quality interference feature; Weight the third positioning tag set and all the core tag sets of the third positioning tag set to determine the second positioning tag set of the first acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information; Wherein, the method further includes: estimating the perturbation factor of the acoustic quality interference feature in the (X + 1)th group of virtual reality acoustic effect simulation information according to the second positioning information of the X - 1 groups of virtual reality acoustic effect simulation information before the Xth group of virtual reality acoustic effect simulation information and the second positioning information of the Xth group of virtual reality acoustic effect simulation information, and determining the reference positioning report of the acoustic quality interference feature in the (X + 1)th group of virtual reality acoustic effect simulation information; Wherein, the method further includes: correcting the estimated scoring set of the acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information according to the reference positioning report of the acoustic quality interference feature in the (X + 1)th group of virtual reality acoustic effect simulation information and the Xth group of virtual reality acoustic effect simulation information, and determining the estimated scoring set of the acoustic quality interference feature in the (X + 1)th group of virtual reality acoustic effect simulation information.

7. The method according to claim 1, characterized in that The positioning of the acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information for the multi-channel sound simulation task to be positioned, and determining the first initial positioning report of the acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information includes: Performing voiceprint description mining on the Xth group of virtual reality acoustic effect simulation information to obtain a second simulated voiceprint description; Performing positioning of the acoustic quality interference feature on the second simulated voiceprint description to determine the fourth initial positioning report of the acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information; Determine a set number of first initial positioning reports from the fourth initial positioning reports according to the credibility coefficients of the respective fourth initial positioning reports.

8. The method according to claim 1, wherein The first positioning information further includes the theme keywords of the acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information. The determination of the first positioning information of the Xth group of virtual reality acoustic effect simulation information according to the Xth group of virtual reality acoustic effect simulation information, the first initial positioning report, and the reference positioning report for the acoustic quality interference feature in the Xth group of virtual reality acoustic effect simulation information includes: Differentiate and analyze the second acoustic quality interference feature based on the virtual reality acoustic effect simulation information of the third acoustic effect simulation set corresponding to the first positioning tag set of the second acoustic quality interference feature in the acoustic effect simulation set, and determine the theme keywords of the second acoustic quality interference feature, where the second acoustic quality interference feature is any one of the acoustic quality interference features in the Xth group of virtual reality acoustic effect simulation information.

9. A multi-channel sound simulation processing system, characterized in that, It includes a processor and a memory; the processor and the memory are communicatively connected, and the processor is configured to read and execute a computer program from the memory to implement the method according to any one of claims 1-8 above.

10. A readable storage medium, characterized in that, A program is stored thereon, and when the program is executed by the processor, it implements the method according to any one of claims 1-8 above.

Images