Sound effect processing method, device and storage medium

Abstract

The present application provides a sound effect processing method, device and storage medium. The sound effect processing method includes: obtaining information about a real object, the information including the association relationship between the real object and a virtual scene and the acoustic properties of the real object; determining the sound output effect of the virtual scene based on the information about the real object; and outputting the sound of the virtual scene based on the sound output effect.

Description

Technical Field

[0001] The present application relates to the field of sound effect processing technology, and in particular to a sound effect processing method, device and storage medium. Background Art

[0002] When existing augmented reality (AR) products are running games and other applications, users see a scene that is a fusion of the real environment and virtual objects, rather than a purely virtual environment. The sound effects of the played audio are only calculated based on the position and surface properties of the virtual objects in the virtual scene, and cannot be adjusted based on the actual objects in the real scene, thereby reducing the user's immersion in the AR scene and resulting in poor realism of the virtual environment. Summary of the Invention

[0003] In response to the above-mentioned technical problems existing in the prior art, the present application provides a sound effect processing method, device and storage medium.

[0004] The present invention provides a method for processing sound effects, including:

[0005] Obtaining information about a real object, the information including an association relationship between the real object and a virtual scene and acoustic properties of the real object;

[0006] Determining a sound output effect of the virtual scene according to the information of the real object;

[0007] The sound of the virtual scene is output based on the sound output effect.

[0008] In some embodiments, obtaining information about a real object specifically includes:

[0009] Obtain image information of real objects in real scenes;

[0010] determining a target category included in the information of the real object based on the image information;

[0011] The determining, based on the information of the real object, the sound output effect of the virtual scene specifically includes:

[0012] An acoustic attribute corresponding to the target category is determined according to the determined target category.

[0013] In some embodiments, determining the acoustic attribute corresponding to the target category according to the determined target category specifically includes:

[0014] Determining material information of the real object according to the determined target category;

[0015] The acoustic properties of the real object are determined based on the material information; wherein the acoustic properties include at least one or more of the following: a reflection coefficient and an absorption coefficient of the real object to sound.

[0016] In some embodiments, determining the sound output effect of the virtual scene based on the information of the real object specifically includes:

[0017] Constructing a scene model including the real scene and the virtual scene;

[0018] Acquiring a first characteristic parameter of a sound source in the scene model, where the first characteristic parameter includes at least a position parameter;

[0019] A sound output effect of the virtual scene is generated based on the first characteristic parameter of the sound source.

[0020] In some embodiments, determining the sound output effect of the virtual scene based on the information of the real object further specifically includes:

[0021] determining second characteristic parameters of a real object in the real scene based on the scene model, where the second characteristic parameters include at least a size parameter;

[0022] A sound output effect of the virtual scene is generated according to the first characteristic parameter of the sound source and the second characteristic parameter of the real object in the real scene.

[0023] In some embodiments, determining the target category included in the information of the real object based on the image information specifically includes:

[0024] Determining that the real object in the image information corresponds to at least two categories;

[0025] Determine the probability value corresponding to each category;

[0026] The category corresponding to the highest probability value among the plurality of probability values ​​is taken as the target category to which the real object belongs; or

[0027] A category corresponding to at least one probability value among the plurality of probability values ​​is used as a subcategory of the target category.

[0028] In some embodiments, determining the acoustic attribute corresponding to the target category according to the determined target category specifically includes:

[0029] In the case where the target category includes multiple subcategories, determining acoustic properties corresponding to each subcategory;

[0030] The determining, based on the information of the real object, the sound output effect of the virtual scene specifically includes:

[0031] determining, based on the acoustic properties corresponding to the subcategories, sub-sound output effects corresponding to the respective acoustic properties;

[0032] The sub-sound output effects are mixed based on a preset ratio to obtain the sound output effect of the virtual scene.

[0033] In some embodiments, determining the acoustic properties of the real object based on the material information specifically includes:

[0034] In a case where the material information of the real object includes different materials, determining a coverage area of ​​the real object by each material based on the image information of the real object;

[0035] An acoustic property of the real object is determined based on the coverage area.

[0036] The present application also provides a sound effect processing device, including:

[0037] an acquisition module configured to obtain information about a real object, the information including an association relationship between the real object and a virtual scene and acoustic properties of the real object;

[0038] a determination module configured to determine a sound output effect of the virtual scene according to information of the real object;

[0039] An output module is configured to output the sound of the virtual scene based on the sound output effect.

[0040] An embodiment of the present application further provides a storage medium storing a computer program, which implements the steps of the above-mentioned sound effect processing method when executed by a processor.

[0041] Compared with the prior art, the beneficial effect of the embodiments of the present application is that: the present application obtains information about real objects and determines the sound output effect of the virtual scene based on the information of the real objects, wherein the information of the real objects may include the association relationship between the real objects and the virtual scene and the acoustic properties of the real objects. In this way, the sound output effect of the virtual scene can be adjusted in combination with the real objects in the real scene, thereby increasing the user's immersion in the virtual scene, thereby effectively increasing the realism of the sound feedback from the virtual environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar components in different views. The same reference numerals with letter suffixes or different letter suffixes may represent different instances of similar components. The accompanying drawings generally illustrate various embodiments by way of example and not limitation, and together with the description and claims, serve to illustrate the disclosed embodiments. Where appropriate, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive of the embodiments of the present apparatus or method.

[0043] Figure 1 This is a flowchart of a sound effect processing method according to an embodiment of the present application;

[0044] Figure 2 A schematic diagram of a virtual scene corresponding to the sound effect processing method according to an embodiment of the present application;

[0045] Figure 3 This is a flowchart of the sound effect processing method according to the second embodiment of the present application;

[0046] Figure 4 This is a flowchart of the sound effect processing method of embodiment 3 of the present application. DETAILED DESCRIPTION

[0047] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0048] It should be understood that various modifications may be made to the embodiments of the invention herein. Therefore, the above description should not be considered as limiting, but merely as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0049] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0050] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0051] It should also be understood that although the present application has been described with reference to certain specific examples, those skilled in the art will readily be able to implement many other equivalent forms of the present application.

[0052] The above and other aspects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0053] Specific embodiments of the present application will be described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments described are merely examples of the present application and may be implemented in a variety of ways. Well-known and / or repetitive functions and structures are not described in detail to avoid obscuring the present application with unnecessary or redundant details. Therefore, the specific structural and functional details described herein are not intended to be limiting, but rather to serve as a basis and representative basis for the claims to teach those skilled in the art to variously utilize the present application with substantially any suitable detailed structure.

[0054] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," which may all refer to one or more of the same or different embodiments according to the present application.

[0055] The present application provides a method for processing sound effects, which can be applied to electronic devices, including but not limited to one of the following: a laptop computer, a smartphone, and a smart wearable device. For example, the electronic device can be an AR (augmented reality) device, such as AR glasses.

[0056] Furthermore, if Figure 1 As shown, the sound effect processing method includes steps S101 to S103.

[0057] Step S101 : obtaining information of a real object 1 , wherein the information includes an association relationship between the real object 1 and a virtual scene and acoustic properties of the real object 1 .

[0058] Specifically, the real object 1 can be understood as an object that actually exists in the real environment, for example, an object in the real environment that the AR glasses want to integrate when the user wears the AR glasses. Figure 2 , Figure 2 The wall shown in FIG can be understood as a real object.

[0059] Specifically, the virtual scene is constructed by integrating a real object 1 and a virtual object 2 in a real environment. The relationship between the real object 1 and the virtual scene may include at least the coordinates of the real object 1 in the virtual scene, the relative positional relationship between the real object 1 and the virtual object 2, etc. The virtual object 2 may include virtual objects such as cylinders and spheres constructed in the virtual scene.

[0060] For example, Figure 2 As shown, Figure 2Figure 1 shows a schematic diagram of a virtual scene, which includes a real object 1, a virtual object 2, and a sound source. The sound source can be understood as the sound emitted or played by the object, such as stereo audio. This application does not impose any specific restrictions on this, as long as it can produce audio.

[0061] Specifically, the acoustic properties of the real object 1 may include at least one or more of the following: pitch, loudness, timbre, and acoustic parameters that affect at least one of the pitch, loudness, and timbre.

[0062] Step S102: determining the sound output effect of the virtual scene according to the information of the real object 1.

[0063] Step S103: outputting the sound of the virtual scene based on the sound output effect.

[0064] Specifically, the information of the above-mentioned real object 1 may also include at least information that can affect the acoustic properties, and the information may include one or more of the following: the size, material, and category of the real object 1. The acoustic properties of the real object 1 can be determined by obtaining the information of the real object 1.

[0065] Specifically, an electronic device using the above-mentioned sound effect processing method may also include an acoustic property management module, which stores a mapping relationship between a real object 1 and a virtual scene. After obtaining information about the real object 1, the module can determine the sound output effect of the real object 1 that has a mapping relationship with the real object 1 based on the information of the real object 1. Then, the sound output effect corresponding to the real object 1 and the sound output effect corresponding to the virtual object 2 are merged to obtain the sound output effect to be output by the virtual scene, so that the sound output effect not only corresponds to the virtual object 2, but can also generate the sound of the virtual scene in combination with the real object 1. Specifically, after the sound source in the virtual scene emits a sound, the real object can process the sound in the virtual scene (such as reflection, absorption, etc.), thereby obtaining a sound output effect processed by the real object, thereby enhancing the user's immersive experience.

[0066] For example, Figure 2 As shown, Figure 2 After the sound source shown in the figure emits a sound, the prior art generates the sound output effect of the virtual scene only based on the processing of the sound by the virtual objects in the virtual scene, such as Figure 2 In the example above, the virtual objects constructed as cylinders and spheres reflect sound to generate the sound output effect of the virtual scene. However, after obtaining information about the real object 1 constructed as a wall, the present application can combine the reflection or absorption of sound in the virtual scene by the wall to obtain the sound output effect of the virtual scene, effectively increasing the realism of the sound reflected by the virtual environment.

[0067] The present application obtains information about a real object 1 and determines the sound output effect of a virtual scene based on the information about the real object 1. The information about the real object 1 may include the association between the real object 1 and the virtual scene and the acoustic properties of the real object 1. In this way, the sound output effect of the virtual scene can be adjusted in combination with the real objects in the real scene, thereby increasing the user's sense of immersion in the virtual scene and effectively increasing the realism of the sound feedback from the virtual environment.

[0068] The above-mentioned sound effect processing method can be applied to AR game scenarios, enterprise service scenarios, educational and puzzle scenarios, etc. For example, in AR game scenarios, players can experience various immersive sound effects according to different real-life environments; in enterprise service scenarios, AR spatial sound effects can be enabled in digital exhibitions and commercial exhibition halls, allowing users to experience different sound effects in various real-life environments; in educational and puzzle scenarios, combined with real scenes of various cartoon shapes, participants can wear AR glasses to experience the immersive sound effects of the combination of virtual objects 2 and real objects 1.

[0069] In some embodiments, as Figure 3 As shown, the step S101 of obtaining the information of the real object 1 specifically includes steps S201 and S202. Step S201: obtaining image information of the real object 1 in a real scene. Step S202: determining the target category included in the information of the real object 1 based on the image information.

[0070] The step S102 of determining the sound output effect of the virtual scene according to the information of the real object 1 specifically includes step S203. Step S203: determining an acoustic attribute corresponding to the target category according to the determined target category.

[0071] In this way, the acoustic properties corresponding to the real object 1 can be determined in combination with the target category included in the information of the real object 1. The above method for determining the acoustic properties is efficient and accurate.

[0072] Specifically, the above-mentioned image information of the real object 1 in the real scene can be collected by a camera of an electronic device or by a depth sensor. The image information can be a two-dimensional plane image or a three-dimensional depth image, and this application does not make any specific limitations on this.

[0073] Specifically, based on a computer vision and machine learning software library, the image information obtained above can be input and the target category corresponding to the current real object 1 can be output, thereby improving the accuracy of the target category corresponding to the determined real object 1. The computer vision and machine learning software library can be OpenCV (OpenCV is a cross-platform computer vision and machine learning software library released under the Apache2.0 license). This application does not make specific restrictions on this. It is sufficient to be able to output the target category of the object in the image when the image is obtained.

[0074] Specifically, the above-mentioned target categories may at least include categories divided by material, such as metal, wood, etc., and may also include categories divided by the object's own attributes, such as walls, windows, floors, etc., and of course may also include categories divided by size, such as large object class, small object class, etc.

[0075] Specifically, the target category included in the information of the real object 1 determined based on the image information may correspond to not only one category but also multiple categories, so as to accurately determine the attributes of the real object 1 .

[0076] In some embodiments, step S202 of determining the target category included in the information of the real object 1 based on the image information may specifically include: segmenting the real object 1 contained in the image information to obtain segmented images each including each real object 1; and determining the target category of the real object 1 based on the segmented image.

[0077] In some embodiments, determining the acoustic attribute corresponding to the target category according to the determined target category in step S203 specifically includes:

[0078] Determining material information of the real object 1 according to the determined target category;

[0079] The acoustic properties of the real object 1 are determined based on the material information; wherein the acoustic properties include at least one or more of the following: a reflection coefficient and an absorption coefficient of the real object 1 to sound.

[0080] In this way, the acoustic properties can be further determined in combination with the material information of the real object 1, thereby improving the accuracy of the determined acoustic properties.

[0081] Specifically, the material information may include at least one or more materials constituting the real object 1 , and may also include acoustic-related parameters of the corresponding materials. The material information may be used to determine the acoustic properties of the real object 1 .

[0082] In some embodiments, determining the sound output effect of the virtual scene according to the information of the real object 1 in step S102 specifically includes:

[0083] Constructing a scene model including the real scene and the virtual scene;

[0084] Acquiring a first characteristic parameter of a sound source in the scene model, where the first characteristic parameter includes at least a position parameter;

[0085] A sound output effect of the virtual scene is generated based on the first characteristic parameter of the sound source.

[0086] In this way, the sound output effect of the virtual scene can be determined by combining the scene model and the position of the sound source, making the sound output effect more realistic and further enhancing the user's immersive experience.

[0087] In some optional embodiments, the constructing of the scene model including the real scene and the virtual scene may specifically include: obtaining depth information of the real object 1 in the real scene; determining point cloud information of the real object 1 based on the depth information; and generating a virtual scene model including the real object 1 and the virtual object 2 based on the point cloud information.

[0088] Specifically, the first characteristic parameter of the sound source may also include the pitch, loudness, and timbre of the sound source. The position parameter included in the first characteristic parameter of the sound source may be understood as a coordinate parameter of the sound source, or a distance parameter of the sound source relative to the real object 1 and / or the virtual object 2.

[0089] In some embodiments, determining the sound output effect of the virtual scene according to the information of the real object 1 in step S102 further specifically includes:

[0090] determining a second characteristic parameter of the real object 1 in the real scene based on the scene model, wherein the second characteristic parameter includes at least a size parameter;

[0091] The sound output effect of the virtual scene is generated according to the first characteristic parameter of the sound source and the second characteristic parameter of the real object 1 in the real scene.

[0092] Since the size parameters of the real object 1 will affect the reflection or absorption of sound by the real object 1, the above method can combine the position parameters of the sound source and the size parameters of the real object 1 to generate the sound output effect of the virtual scene, thereby improving the accuracy of the sound output effect.

[0093] Specifically, the size parameters of the real object 1 are used to indicate its size in the virtual scene, and may specifically include volume parameters, length, width, and height parameters of the real object 1 .

[0094] In some embodiments, determining the target category included in the information of the real object 1 based on the image information in step S202 specifically includes:

[0095] Determining that the real object 1 in the image information corresponds to at least two categories;

[0096] Determine the probability value corresponding to each category;

[0097] The category corresponding to the highest probability value among the multiple probability values ​​is used as the target category to which the real object 1 belongs; or,

[0098] A category corresponding to at least one probability value among the plurality of probability values ​​is used as a subcategory of the target category.

[0099] In this way, the target category to which the real object 1 belongs can be further accurately determined by the probability value corresponding to the category, thereby further improving the accuracy of the acoustic properties determined based on the target category.

[0100] For example, if real object 1 is a chair, it can be determined that the chair corresponds to at least metal and wood. That is, the chair may be made of wood alone, metal alone, or a combination of metal and wood. The probability that the chair is made of metal is 50%, and the probability that it is made of wood is 70%. The higher probability value can be determined as the target category of real object 1, that is, the chair is determined to be wood. Alternatively, the categories corresponding to multiple probability values ​​can be determined as subcategories of the target category, that is, wood and metal are subcategories of the chair.

[0101] In some embodiments, determining the probability value corresponding to each category may specifically include:

[0102] inputting the image information into a first database;

[0103] Obtain a target category of the real object 1 contained in the image information output by the first database based on the image information and a probability value corresponding to the target category.

[0104] In some embodiments, determining the acoustic attribute corresponding to the target category according to the determined target category in step S203 specifically includes: if the target category includes multiple subcategories, determining the acoustic attribute corresponding to each subcategory respectively.

[0105] The step S102 of determining the sound output effect of the virtual scene based on the information of the real object 1 specifically includes:

[0106] determining, based on the acoustic properties corresponding to the subcategories, sub-sound output effects corresponding to the respective acoustic properties;

[0107] The sub-sound output effects are mixed based on a preset ratio to obtain the sound output effect of the virtual scene.

[0108] In this way, when the target category contains multiple subcategories, the sound output effect of the virtual scene can be obtained by mixing the acoustic properties corresponding to each subcategory and then determining the sub-sound output effects, rather than being limited to determining one acoustic property through one target category, so as to further improve the accuracy of the generated sound output effect.

[0109] Specifically, the preset proportion may be related to the probability value corresponding to each subcategory, that is, a subcategory with a higher probability value has a higher proportion, and a subcategory with a lower probability value has a lower proportion.

[0110] In some embodiments, determining the acoustic properties of the real object 1 based on the material information specifically includes:

[0111] In a case where the material information of the real object 1 includes different materials, determining the coverage area of ​​the real object 1 by each material based on the image information of the real object 1;

[0112] The acoustic properties of the real object 1 are determined according to the coverage area.

[0113] In this way, by measuring the coverage area of ​​each material of the real object 1 , the sound processing effect of the surface of the real object can be determined, thereby further determining the acoustic properties of the real object 1 .

[0114] For example, when it is determined that the material information of the real object 1 includes metal materials and wood materials, it is determined that for the chair, the coverage area of ​​the metal material is 15 square centimeters and the coverage area of ​​the wood material is 5 square centimeters, and the acoustic properties of the real object 1 are determined based on the different coverage areas of the two.

[0115] The embodiment of the present application also provides a sound effect processing device 110, such as Figure 4 As shown, the sound effect processing device 110 includes an acquisition module 101, a determination module 102, and an output module 103. The acquisition module 101 is configured to obtain information about a real object 1, including the association between the real object 1 and the virtual scene and the acoustic properties of the real object 1. The determination module 102 is configured to determine the sound output effect of the virtual scene based on the information about the real object 1. The output module 103 is configured to output the sound of the virtual scene based on the sound output effect.

[0116] The present application obtains information about a real object 1 and determines the sound output effect of a virtual scene based on the information about the real object 1. The information about the real object 1 may include the association between the real object 1 and the virtual scene and the acoustic properties of the real object 1. In this way, the sound output effect of the virtual scene can be adjusted in combination with the real objects in the real scene, thereby increasing the user's sense of immersion in the virtual scene and effectively increasing the realism of the virtual environment.

[0117] In some embodiments, the acquisition module 101 is further configured to: acquire image information of a real object 1 in a real scene; and determine, based on the image information, a target category included in the information of the real object 1. The determination module 102 is further configured to: determine, based on the determined target category, an acoustic attribute corresponding to the target category.

[0118] In some embodiments, the determination module 102 is further configured to: determine material information of the real object 1 according to the determined target category;

[0119] The acoustic properties of the real object 1 are determined based on the material information; wherein the acoustic properties include at least one or more of the following: a reflection coefficient and an absorption coefficient of the real object 1 to sound.

[0120] In some embodiments, the determining module 102 is further configured to:

[0121] Constructing a scene model including the real scene and the virtual scene;

[0122] Acquiring a first characteristic parameter of a sound source in the scene model, where the first characteristic parameter includes at least a position parameter;

[0123] A sound output effect of the virtual scene is generated based on the first characteristic parameter of the sound source.

[0124] In some embodiments, the determining module 102 is further configured to:

[0125] determining a second characteristic parameter of the real object 1 in the real scene based on the scene model, wherein the second characteristic parameter includes at least a size parameter;

[0126] The sound output effect of the virtual scene is generated according to the first characteristic parameter of the sound source and the second characteristic parameter of the real object 1 in the real scene.

[0127] In some embodiments, the acquisition module 101 is further configured to:

[0128] Determining that the real object 1 in the image information corresponds to at least two categories;

[0129] Determine the probability value corresponding to each category;

[0130] The category corresponding to the highest probability value among the multiple probability values ​​is used as the target category to which the real object 1 belongs; or,

[0131] A category corresponding to at least one probability value among the plurality of probability values ​​is used as a subcategory of the target category.

[0132] In some embodiments, the determining module 102 is further configured to:

[0133] In the case where the target category includes multiple subcategories, determining acoustic properties corresponding to each subcategory;

[0134] The determining of the sound output effect of the virtual scene according to the information of the real object 1 specifically includes:

[0135] determining, based on the acoustic properties corresponding to the subcategories, sub-sound output effects corresponding to the respective acoustic properties;

[0136] The sub-sound output effects are mixed based on a preset ratio to obtain the sound output effect of the virtual scene.

[0137] In some embodiments, the determining module 102 is further configured to:

[0138] In a case where the material information of the real object 1 includes different materials, determining the coverage area of ​​the real object 1 by each material based on the image information of the real object 1;

[0139] The acoustic properties of the real object 1 are determined according to the coverage area.

[0140] An embodiment of the present application further provides a storage medium storing a computer program, which implements the steps of the above-mentioned sound effect processing method when executed by a processor.

[0141] Note that the various units in the various embodiments of the present application can be implemented as computer-executable instructions stored on a memory, which can implement the corresponding steps when executed by a processor; they can also be implemented as hardware with corresponding logical computing capabilities; or they can be implemented as a combination of software and hardware (firmware). In some embodiments, the processor can be implemented as any one of an FPGA, an ASIC, a DSP chip, a SOC (system on a chip), an MPU (for example, but not limited to Cortex), etc. The processor can be communicatively coupled to the memory and configured to execute the computer-executable instructions stored therein. The memory can include read-only memory (ROM), flash memory, random access memory (RAM), dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM, static memory (for example, flash memory, static random access memory), etc., on which computer-executable instructions are stored in any format. The computer-executable instructions can be accessed by the processor, read from the ROM or any other suitable storage location, and loaded into the RAM for execution by the processor to implement the wireless communication method according to the various embodiments of the present application.

[0142] It should be noted that, among the various components of the system of the present application, the components are logically divided according to the functions to be implemented, but the present application is not limited to this, and the various components can be re-divided or combined as needed. For example, some components can be combined into a single component, or some components can be further decomposed into more sub-components.

[0143] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that in practice, a microprocessor or digital signal processor (DSP) can be used to implement some or all of the functions of some or all of the components in the system according to the embodiments of the present application. The present application can also be implemented as an apparatus or device program (e.g., a computer program and a computer program product) for performing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium or can be in the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form. In addition, the present application can be implemented with the aid of hardware comprising several different elements and with the aid of a suitably programmed computer. In a unit claim that lists several means, several of these means can be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

[0144] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., solutions that intersect various embodiments), adaptations, or changes. The elements in the claims are to be interpreted broadly based on the language employed in the claims and are not limited to the examples described in this specification or during the prosecution of this application, which examples are to be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, with the true scope and spirit being indicated by the following claims and the full scope of their equivalents.

[0145] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of their solutions) can be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the application. This should not be interpreted as an intention that a disclosed feature that is not required to be protected is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a specific disclosed embodiment. Thus, the following claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of this application should be determined with reference to the appended claims and the full scope of equivalents to which these claims are entitled.

[0146] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.

Claims

1. A sound effect processing method, comprising: Obtaining information about a real object, the information including an association relationship between the real object and a virtual scene and acoustic properties of the real object; Determining a sound output effect of the virtual scene according to the information of the real object; Outputting the sound of the virtual scene based on the sound output effect; wherein determining the sound output effect of the virtual scene based on the information of the real object specifically includes: Construct scene models including real scenes and virtual scenes; Acquiring first characteristic parameters of a sound source in the scene model, where the first characteristic parameters include at least a position parameter, where the position parameter is a coordinate parameter of the sound source, or a distance parameter of the sound source relative to a real object and / or a virtual object; determining, based on the scene model, second characteristic parameters of the real object in the real scene, where the second characteristic parameters include at least a size parameter, and the size parameter is the size of the real object in the virtual scene; A sound output effect of the virtual scene is generated according to the first characteristic parameter of the sound source and the second characteristic parameter of the real object in the real scene.

2. The sound effect processing method according to claim 1, wherein obtaining information about the real object comprises: Obtain image information of real objects in real scenes; determining a target category included in the information of the real object based on the image information; The determining, based on the information of the real object, the sound output effect of the virtual scene specifically includes: An acoustic attribute corresponding to the target category is determined according to the determined target category.

3. The sound effect processing method according to claim 2, wherein determining the acoustic attribute corresponding to the target category according to the determined target category specifically comprises: Determining material information of the real object according to the determined target category; The acoustic properties of the real object are determined based on the material information; wherein the acoustic properties include at least one or more of the following: a reflection coefficient and an absorption coefficient of the real object to sound.

4. The sound effect processing method according to claim 2, wherein determining the target category included in the information of the real object based on the image information specifically comprises: Determining that the real object in the image information corresponds to at least two categories; Determine the probability value corresponding to each category; The category corresponding to the highest probability value among the multiple probability values ​​is used as the target category to which the real object belongs; or, A category corresponding to at least one probability value among the plurality of probability values ​​is used as a subcategory of the target category.

5. The sound effect processing method according to claim 2 or 4, wherein determining the acoustic attribute corresponding to the target category according to the determined target category specifically comprises: In the case where the target category includes multiple subcategories, determining acoustic properties corresponding to each subcategory; The determining, based on the information of the real object, the sound output effect of the virtual scene specifically includes: determining, based on the acoustic properties corresponding to the subcategories, sub-sound output effects corresponding to the respective acoustic properties; The sub-sound output effects are mixed based on a preset ratio to obtain the sound output effect of the virtual scene.

6. The sound effect processing method according to claim 3, wherein determining the acoustic properties of the real object based on the material information specifically comprises: In a case where the material information of the real object includes different materials, determining a coverage area of ​​the real object by each material based on the image information of the real object; An acoustic property of the real object is determined based on the coverage area.

7. A sound effect processing device, comprising: an acquisition module configured to obtain information about a real object, the information including an association relationship between the real object and a virtual scene and acoustic properties of the real object; a determination module configured to determine a sound output effect of the virtual scene according to information of the real object; an output module configured to output the sound of the virtual scene based on the sound output effect; wherein the determination module is further configured to: Construct scene models including real scenes and virtual scenes; Acquiring first characteristic parameters of a sound source in the scene model, where the first characteristic parameters include at least a position parameter, where the position parameter is a coordinate parameter of the sound source, or a distance parameter of the sound source relative to a real object and / or a virtual object; determining, based on the scene model, second characteristic parameters of the real object in the real scene, where the second characteristic parameters include at least a size parameter, and the size parameter is the size of the real object in the virtual scene; A sound output effect of the virtual scene is generated according to the first characteristic parameter of the sound source and the second characteristic parameter of the real object in the real scene.

8. A storage medium storing a computer program, wherein when the computer program is executed by a processor, the steps of the sound effect processing method according to any one of claims 1 to 6 are implemented.

Images