Ambient sound generation method and ambient sound generation system
Through the ambient sound generation method based on user needs and physical space-time environments, the problem of environmental sounds in the prior art cannot be flexibly adjusted and coordinated, the generation of multi-channel ambient sounds is realized, and the user's auditory experience and spatial three-dimensional sense are improved.
Patent Information
- Application Number
- CN202311696242.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-11
- Publication Date
- 2025-06-13
AI Technical Summary
The existing ambient sound generation technology cannot effectively adjust flexibly according to user needs and physical space-time environment, and cannot coordinate with the music being played by the user, resulting in poor auditory experience.
The target ambient sound information is determined based on the ambient sound design reference information, including the desired ambient sound information input by the user, the current music information and the physical information in time and space, and the corresponding ambient sound generation information is generated, and the multi-channel target ambient sound is generated through rendering, synthesis and mixing processing.
It realizes flexible adjustment and coordination of ambient sounds, improves the user's auditory experience and the spatial three-dimensional sense of ambient sounds, and provides a more immersive auditory experience.
Smart Images
Figure CN120151765A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of audio processing, and more particularly to an ambient sound generation method and an ambient sound generation system. Background Art
[0002] With the wide application of ambient sound or background sound in civilian and commercial fields, the generation of ambient sound also faces higher requirements.
[0003] Currently, in the generation of ambient sound, on the one hand, the generated ambient sound is usually played to the user by a mono or closely mounted stereo speaker, and the sound can be regarded as a point source, which makes the user not feel as immersive as in a real environment, and the spatial stereo effect and immersive experience of the ambient sound are poor; on the other hand, the current ambient sound generation usually only includes a single white noise or a single background sound, and cannot flexibly adapt to the different needs of users. For example, it cannot be coordinated with the music that the user is playing, nor can it flexibly change according to the user's current geographical location or weather environment information, which results in a poor auditory experience for the user.
[0004] Therefore, there is a need for a method and system that can flexibly adjust and change the generated ambient sound according to the user's needs and the current physical space-time environment on the premise of realizing good and convenient generation of the ambient sound, and can be coordinated with other music that the user is playing, and the ambient sound has, for example, multiple channels and is played by multiple sound sources, so as to improve the user's auditory experience and the spatial stereo sense of the ambient sound. Summary of the Invention
[0005] In view of the above problems, the present invention provides an ambient sound generation method and an ambient sound generation system. By using the system and method provided by the present invention, on the basis of realizing good generation of the ambient sound, the generated ambient sound can be flexibly adjusted and changed according to the user's needs and the current physical space-time environment, and can be coordinated with other music that the user is playing, and the ambient sound has, for example, multiple channels and is played by multiple sound sources, so as to improve the user's auditory experience and the spatial stereo sense of the ambient sound.
[0006] According to one aspect of the present disclosure, an ambient sound generation method is proposed, including: determining target ambient sound information based on ambient sound design reference information; determining ambient sound generation information corresponding to the target ambient sound information based on the target ambient sound information, where the ambient sound generation information includes information of at least one sub-ambient sound and characteristic parameter information corresponding to the sub-ambient sound; and generating a target ambient sound based on the ambient sound generation information.
[0007] In some embodiments, the ambient sound design reference information includes at least one of: desired ambient sound information input by the user, current music information, and spatio-temporal physical information.
[0008] In some embodiments, when the ambient sound design reference information includes the expected ambient sound information input by the user, determining the target ambient sound information includes: determining the target ambient sound information based on content analysis of the expected ambient sound information.
[0009] In some embodiments, when the ambient sound design reference information includes the current music information, determining the target ambient sound information includes: determining the music characteristic information of the current music information, and determining the target ambient sound information based on the music characteristic information.
[0010] In some embodiments, the music characteristic information includes at least one of the following: lyric content information, music type information, music rhythm information, and music style information.
[0011] In some embodiments, the spatio-temporal physical information includes at least one of the following: current location information, current time information, and current weather information.
[0012] In some embodiments, the characteristic parameter information corresponding to the sub-ambient sound includes at least one of the following: volume information, trigger density information, azimuth information, movement speed information, and location information.
[0013] In some embodiments, generating the target ambient sound based on the ambient sound generation information includes: obtaining corresponding ambient sound materials from a preset ambient sound material library based on the information of each sub-ambient sound; performing rendering synthesis and mixing processing on the ambient sound materials based on the characteristic parameter information of each sub-ambient sound.
[0014] In some embodiments, the target ambient sound is an audio signal with at least two channels, and the target ambient sound is played through multiple playback devices in a channel-separated manner.
[0015] In some embodiments, the target ambient sound is an audio signal with greater than or equal to three channels, and the number of playback devices corresponds to the number of channels of the target ambient sound.
[0016] According to another aspect of the present disclosure, an ambient sound generation system is proposed, including: a controller configured to execute the ambient sound generation method as described above; a playback device configured to receive the target ambient sound from the controller and play the target ambient sound.
[0017] In some embodiments, the target ambient sound is an audio signal with at least two channels, and among them, the playback device is a plurality of playback devices configured to play the target ambient sound in a channel-separated manner. Description of the Drawings
[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. The following drawings are not deliberately drawn to scale in actual size, and the focus is on showing the gist of the present invention.
[0019] Figure 1 Fig. shows an exemplary flowchart of an ambient sound generation method 100 according to an embodiment of the present disclosure;
[0020] Figure 2 Fig. shows an exemplary flowchart of a process S103 for generating a target ambient sound according to an embodiment of the present disclosure;
[0021] Figure 3 Fig. shows a schematic diagram of an ambient sound generation system 200 according to an embodiment of the present disclosure. Detailed Embodiments
[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts also fall within the scope of protection of the present invention.
[0023] As shown in this application and the claims, unless the context clearly indicates an exception, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.
[0024] Although this application makes various references to certain modules in the system according to the embodiments of this application, however, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.
[0025] Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the operations before or below do not necessarily need to be executed precisely in order. On the contrary, as needed, various steps can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several steps can be removed from these processes.
[0026] According to one aspect of the present disclosure, an ambient sound generation method 100 is provided.
[0027] It should be understood that the "ambient sound" described in this application, that is, background sound, is intended to represent the sound in a specific scene or environment. According to different scenes and environments, the ambient sound may include natural sounds such as wind sound, rain sound, and ocean wave sound, may also include animal sounds such as bird chirping and frog croaking, may further include sounds from items such as horn sound and fan humming sound, and also may include sounds from people in the environment such as noisy crowd sound and cheering sound.
[0028] Figure 1 An exemplary flowchart of the ambient sound generation method 100 according to an embodiment of the present disclosure is shown. Referring to Figure 1 , the ambient sound generation method 100 includes: First, in step S101, based on the ambient sound design reference information, the target ambient sound information is determined.
[0029] The ambient sound design reference information refers to the reference information used to determine the target ambient sound, which may be, for example, a single piece of information or may also be multiple pieces of information. It should be understood that the embodiments of the present disclosure are not limited by the specific number of the ambient sound design reference information.
[0030] The ambient sound design reference information may include, for example, one of the expected ambient sound information input by the user, the current music information, and the spatio-temporal physical information, or may also include the expected ambient sound information input by the user, the current music information, and the spatio-temporal physical information at the same time. The embodiments of the present disclosure are not limited by the type and source of the ambient sound design reference information.
[0031] The target ambient sound information refers to the information used to characterize the ambient content of the ambient sound to be generated, which may be, for example, string information such as "a quiet river with cricket sound" or "a street-side coffee shop in the urban area", or may also be other types of information. It should be understood that the embodiments of the present disclosure are not limited by the type and composition method of the target ambient sound information.
[0032] For example, the ambient sound design reference information may be input into a preset function or algorithm, and the target ambient sound information is determined based on the solution of the function or algorithm, or the ambient sound design reference information may also be processed by a neural network to generate the corresponding target ambient sound information.
[0033] After obtaining the target ambient sound information, further, in step S102, based on the target ambient sound information, the ambient sound generation information corresponding to the target ambient sound information is determined.
[0034] The ambient sound generation information refers to the information used to characterize the relevant content required to generate the target ambient sound.
[0035] For example, the environmental sound generation information may include information about at least one sub-environmental sound and the characteristic parameter information corresponding to the sub-environmental sound.
[0036] The information about the sub-environmental sound refers to the information of the sub-components used to form the target environmental sound. For example, it may be the name or content of the sub-environmental sound, such as "cicada chirping", "thunder", etc. Or it may also be a preset digital code or binary code corresponding to the sub-environmental sound, etc. The embodiments of the present disclosure are not limited by the specific composition and type of the information about the sub-environmental sound.
[0037] The characteristic parameter information corresponding to the sub-environmental sound refers to the information that represents the specific performance characteristics that the sub-environmental sound needs to present in the target environmental sound. For example, the characteristic parameter information corresponding to the sub-environmental sound includes one or more of: volume information, trigger density information, azimuth information, movement speed information, position information. It should be understood that the embodiments of the present disclosure are not limited by the specific composition of the characteristic parameters of the sub-environmental sound.
[0038] The process of determining the environmental sound generation information corresponding to the target environmental sound information based on the target environmental sound information may be, for example: inputting the target environmental sound information into a pre-trained neural network, processing and analyzing the target environmental sound information based on the neural network, and obtaining the corresponding sub-environmental sound information and the characteristic parameters of the sub-environmental sound at the output end of the neural network; or, alternatively, the environmental sound generation information corresponding to the target environmental sound information may be obtained based on a preset comparison rule or mapping rule.
[0039] It should be understood that the above only gives an exemplary process for generating the environmental sound generation information corresponding to the target environmental sound information, and the embodiments of the present disclosure are not limited thereto.
[0040] After obtaining the environmental sound generation information, in step S103, based on the target environmental sound information, the environmental sound generation information corresponding to the target environmental sound information is determined.
[0041] The target environmental sound refers to the audio information corresponding to the target environmental sound information. For example, the process of generating the target environmental sound based on the environmental sound generation information may be: based on the information of each sub-environmental sound, obtaining the corresponding environmental sound materials in a preset environmental sound material library; then, based on the characteristic parameter information of each sub-environmental sound, processing the environmental sound materials, such as rendering synthesis and mixing processing (the present application is not limited by the specific rendering, synthesis, and mixing methods), to obtain the target environmental sound.
[0042] Based on the above, in the present application, when generating the target ambient sound, the target ambient sound information is determined based on the ambient sound design reference information, and based on the target ambient sound information, the ambient sound generation information corresponding to the target ambient sound information, including the information of at least one sub-ambient sound and the characteristic parameter information corresponding to the sub-ambient sound, is determined. Subsequently, the target ambient sound is generated based on the ambient sound generation information, enabling the generation process of the target ambient sound to be simply and conveniently implemented via the method of the present application, and the generated target ambient sound has high flexibility and reliability.
[0043] In some embodiments, the ambient sound design reference information includes at least one of: the desired ambient sound information input by the user, the current music information, and the spatio-temporal physical information.
[0044] The desired ambient sound information input by the user refers to the content information of the ambient sound that the user expects to hear. For example, it can be the direct ambient sound content information, such as "thunder", "rain", or it can also be the scene information represented by the expected ambient sound, such as "train station", "pond side", etc. It can also be the information including the volume level or the distance of the ambient sound, such as "slight cicada chirping", "center position of the storm", etc. According to the actual situation, the desired ambient sound can also include other content information. The embodiments of the present disclosure are not limited by the specific composition content and type of the desired ambient sound information.
[0045] The desired ambient sound information input by the user can be, for example, directly input by the user from the keyboard, or can also be selected by the user from the preset ambient sound types and keyword tags given by the system, or can also be the desired ambient sound information obtained after the system performs preliminary processing (such as preprocessing or analysis) on the content directly input by the user. It should be understood that the embodiments of the present disclosure are not limited by the input method of the desired ambient sound information.
[0046] The current music information refers to the information used to characterize the music played by the user at the current moment. For example, it can be the audio signal of the music, or it can also be the characteristic information of the music, etc. The embodiments of the present disclosure are not limited by the specific composition of the music information.
[0047] The current music information can be, for example, the ambient music information collected by an external sensor (such as collecting the audio signal currently played around the user), or it can also be the music input detected in the internal interface or functional module of the system. It should be understood that the embodiments of the present disclosure are not limited by the specific acquisition method of the current music information.
[0048] The spatio-temporal physical information refers to information associated with the spatio-temporal position and physical environment where the user is currently located, which can be, for example, one or more of the user's current location information, current time information, and current weather information. It should be understood that the above only gives some examples of spatio-temporal physical information, and the embodiments of the present disclosure are not limited by its specific composition.
[0049] In this application, by setting that the environmental sound design reference information includes at least one of the expected environmental sound information input by the user, the currently played music stream information, and the spatio-temporal physical information, when determining the target environmental sound information, it is possible to comprehensively consider one or more aspects such as the user's needs, the current physical spatio-temporal environment, and other music played by the user, so that the generated target environmental sound can better adapt to the user's needs and / or the external physical environment state where the user is currently located, and when the user has already played music, the generated target environmental sound can be well coordinated with the current music, thereby improving the flexibility and reliability of the environmental sound generation process, and after mixing in the environmental sound, compared with the single music playing form before adding the environmental sound, the user obtains a better auditory experience.
[0050] In some embodiments, when the environmental sound design reference information includes the expected environmental sound information input by the user, determining the target environmental sound information includes: determining the target environmental sound information based on content analysis of the expected environmental sound information.
[0051] The content analysis refers to the process of analyzing and processing the content of the expected environmental sound information according to preset rules. According to actual needs, it can be, for example, a complex semantic analysis process or a relatively direct keyword splicing and conversion process. It should be understood that the embodiments of the present disclosure are not limited thereto.
[0052] For example, the expected environmental information can be processed and analyzed according to a preset neural network or algorithm, and the target environmental sound information is output at the output end of the neural network or algorithm.
[0053] For example, when the user inputs "walking on the streets of London on Christmas day in heavy snow", it can first, for example, divide the sentence of the expected environmental sound information through a content analysis algorithm or function, extract keywords "Christmas", "London", "snow" (intensity "strong"), "street", and further integrate and sort them based on a preset neural network to finally obtain the target environmental sound information of "heavy snow on the streets of London at Christmas".
[0054] For example, when the user selects "pond", "summer", "night", and "rain" from the ambient sound keywords pre-provided by the system via a mouse, and further sets the intensity of the "rain" keyword to "weak", then, for example, through the calculation of a preset algorithm, the information content can be sorted and combined, and for example, the target ambient sound information can be obtained as: "light rain in the pond on a summer night".
[0055] However, it should be understood that the above only gives some examples of determining the target ambient sound information based on the expected ambient sound information, and the embodiments of the present disclosure are not limited thereto.
[0056] Based on the above, in the present application, when the ambient sound design reference information includes the expected ambient sound information input by the user, determining the target ambient sound information includes: determining the target ambient sound information based on the content analysis of the expected ambient sound information, so that the target ambient sound information can be determined based on the expected ambient sound information input by the user in a simple and convenient manner, and the accuracy and precision of the determined target ambient sound information are improved, which is conducive to making the subsequent generated target ambient sound well adapted to the user's expectations and improving the user's auditory experience.
[0057] In some embodiments, when the ambient sound design reference information includes the current music information, determining the target ambient sound information includes: determining the music characteristic information of the current music information, and determining the target ambient sound information based on the music characteristic information.
[0058] The music characteristic information of the current music information refers to the information used to characterize the characteristics of the current music information, which may include, for example, the lyric content information, music type information, music rhythm information, music style information, etc. of the current music information. However, it should be understood that the embodiments of the present disclosure are not limited thereto.
[0059] It should be understood that, for example, the music characteristic information corresponding to the current music information can be directly obtained via the current music information. For example, when the current music information is the title of a song, for example, the lyrics, type (such as pop music), rhythm, style (such as "soothing"), etc. corresponding to the song can be directly obtained in the song library. Or, the music characteristic information of the current music information can also be determined by performing feature extraction and analysis processing on the current music information. For example, when the current music information is an audio signal, for example, the music characteristic information corresponding thereto can be determined by performing time-frequency domain analysis on the audio signal.
[0060] It should be understood that the above only gives some examples of determining the music characteristic information of the current music information, and the embodiments of the present disclosure are not limited thereto.
[0061] Moreover, the process of determining the target ambient sound information based on the music characteristic information can, for example, process the music characteristic information through a preset algorithm or function to determine the ambient sound keywords associated with the music characteristic information, and generate the target ambient sound based on the ambient sound keywords. For example, if the music characteristic information of the current music information is music style information, and if the music style information is "country music", then, for example, the corresponding ambient sound keyword can be determined as "country", and the target ambient sound information can be generated based on the ambient sound keyword subsequently. Alternatively, the ambient sound information corresponding to the music characteristic information can also be obtained based on a preset rule or mapping relationship and used as the target ambient sound information.
[0062] Based on the above, in the present application, when the ambient sound design reference information includes the current music information, determining the target ambient sound information includes: determining the music characteristic information of the current music information, and determining the target ambient sound information based on the music characteristic information, so that the target ambient sound information can be generated according to the current music information in a simple and convenient manner, and thus when the user is already playing music, the generated target ambient sound can be coordinated and adapted to the music being played, facilitating the provision of a better auditory experience for the user.
[0063] In some embodiments, the music characteristic information includes at least one of: lyric content information, music type information, music rhythm information, and music style information.
[0064] The lyric content information refers to the lyric content of the currently played music. The lyric content can, for example, be the lyrics within a preset time period subsequent to the current moment, or can also be all the lyrics of the music. It should be understood that the embodiments of the present disclosure are not limited by the specific composition of the lyric content information.
[0065] The music type information refers to the information characterizing the type of the currently played music, which can, for example, be classical, pop music, rock music, jazz music, country music, electronic music, ethnic music, etc. It should be understood that the embodiments of the present disclosure are not limited by the specific composition of the music type information.
[0066] The music rhythm information refers to the information characterizing the rhythm state of the music, which can, for example, be the rhythm pattern of the music, such as triplets, syncopation rhythm pattern, front dotted rhythm pattern, back dotted rhythm pattern, etc., or can also be a roughly divided rhythm state, such as fast rhythm, slow rhythm, etc. The embodiments of the present disclosure are not limited by the specific composition of the music rhythm information.
[0067] The music style information can, for example, characterize the characteristics and differences of music in aspects such as performance form, melody, rhythm, harmony, etc. For example, it represents the sub-style type to which the music belongs under a certain specific type. Specifically, if the music is of the classical music type, its style can, for example, be symphony, chamber music, concerto, opera, etc. It should be understood that the embodiments of the present disclosure are not limited thereto.
[0068] According to actual needs, the music characteristic information can, for example, further include other contents such as instrument characteristic information (characterizing the characteristics of the instruments it contains) and music emotion characteristic information.
[0069] Based on the above, in this application, by setting the music characteristic information to include at least one of lyric content information, music type information, music rhythm information, and music style information, when determining the target ambient sound information based on the music characteristic information of the music, the characteristics of the music at multiple levels and dimensions can be comprehensively considered, so that the determined target ambient sound information can better adapt to the currently played music.
[0070] In some embodiments, the spatio-temporal physical information includes at least one of current location information, current time information, and current weather information.
[0071] The current location information, for example, characterizes the geographical location where the user is currently located, such as information about the street, building, city, province, country, etc. where he is located.
[0072] The current time information, for example, characterizes the current time-related information, such as calendar information, clock time information, etc.
[0073] The current weather information, for example, characterizes the weather conditions currently existing in the area where the user is located.
[0074] For example, when the ambient sound design reference information includes spatio-temporal physical information, the process of determining the target ambient sound information based on the ambient sound design reference information can, for example, include: based on the user's current location information, current time information, and current weather information, according to a preset rule, respectively determine the ambient sound keywords corresponding thereto, and generate the target ambient sound information based on the ambient sound keywords.
[0075] For example, if the user is currently in Beijing, the calendar is in December, and the time is at night, the weather information is heavy snow, and the preset rule is, for example, to obtain the state opposite to the current one to provide the user with a different scene experience (it should be understood that this preset rule is only an example and can be modified according to actual needs), then, for example, the ambient sound keywords "Hainan", "summer", "daytime", and "sunny" can be determined accordingly, and the target ambient sound information "Hainan summer daytime sunny" can be determined therefrom.
[0076] Based on the above, by setting the spatio-temporal physical information to include at least one of the current location information, the current time information, and the current weather information, it is possible to take into account the spatio-temporal state and the physical environment state of the user when generating the target ambient sound information, so as to better generate the target ambient sound that meets the user's current needs and improve the user's auditory experience.
[0077] For example, in some embodiments, when determining the target ambient sound information based on the ambient sound design reference information, the ambient sound design reference information will, for example, simultaneously include two or three of the desired ambient sound information input by the user, the current music information, and the spatio-temporal physical information. Some examples are given below to illustrate it more specifically.
[0078] For example, when determining the target ambient sound information based on both the desired ambient sound information input by the user and the current music information, if the desired ambient sound information input by the user is "night", and the current music information is soft-style country music, then, for example, the target ambient sound keyword "night view" can be determined based on "night" according to a preset algorithm, and the target ambient sound keyword "country" can be determined based on the music characteristic information of the current music information, and the target ambient sound information "country night view" can be generated.
[0079] For example, when determining the target ambient sound information based on both the desired ambient sound information input by the user and the spatio-temporal physical information, if the desired ambient sound information input by the user is "café", the detected current geographical location of the user is "street side", and the calendar is at "Christmas", then, for example, the target ambient sound keyword "adjacent to the street" can be determined based on the geographical location "street side" according to a preset algorithm, and the target ambient keyword "festival" can be determined based on the calendar "Christmas", and the target ambient sound information "street-side café festival" can be generated.
[0080] For example, when determining the target ambient sound information based on both the current music information and the spatio-temporal physical information, if the music type of the current music information is folk song, and the weather in the city where the user is located is cloudy, then, for example, the keyword "sunny" can be determined based on "cloudy" according to a preset algorithm, and the keyword "mountains and rivers" can be determined based on "folk song", and the target ambient sound information "sunny mountains and rivers" can be generated.
[0081] For example, when determining the target ambient sound information based on the desired ambient sound information input by the user, the current music information, and the spatio-temporal physical information at the same time, if the desired ambient sound information input by the user is "bonfire barbecue", the user's location is "by the sea", and the currently played song is of the light music type, then, for example, the target ambient sound keyword "sea wave" can be determined based on the location "by the sea", and the target ambient sound keyword "gentle breeze" can be determined based on the "light music" type, and the target ambient sound information "sea wave gentle breeze bonfire barbecue" can be generated.
[0082] It should be understood that the above are only some examples, and the embodiments of the present disclosure are not limited thereto.
[0083] In some embodiments, the characteristic parameter information corresponding to the sub-environment sound includes at least one of volume information, trigger density information, azimuth information, movement speed information, and position information.
[0084] The volume information refers to the volume ratio or volume proportion of the sub-environment sound in the target environment sound.
[0085] The trigger density information refers to the frequency of occurrence of the sub-environment sound or the set of sub-environment sounds in the target environment sound.
[0086] The azimuth information refers to the azimuth of the sub-environment sound relative to the expected position (for example, it is desired that the cicada sound appears directly in front of the user, the frog sound appears at the 11 o'clock horizontal azimuth and -15-degree pitch angle to the left front of the user).
[0087] The movement speed information refers to the movement mode and movement speed that the sub-environment sound is expected to have. For example, if it is desired to create an environment where "a butterfly flies in front of the user", the sub-environment sound such as the sound of ocean waves is expected to be in a uniformly decelerated movement from far to near.
[0088] The position information refers to the position of the sub-environment sound relative to the user. For example, if it is desired to present the environmental sound of fireworks exploding 500 meters away from the user, the sub-environment sound of "fireworks explosion" has, for example, the position information "500m".
[0089] Based on the above, in the present application, by setting the characteristic parameter information corresponding to the sub-environment sound to include at least one of volume information, trigger density information, azimuth information, movement speed information, and position information, it is possible to flexibly select and set various characteristics of each sub-environment sound in the target environment sound, thereby facilitating the realistic and accurate generation of the target environment sound and improving the user's auditory experience.
[0090] In some embodiments, based on the environmental sound generation information, the process S103 of generating the target environment sound can be described more specifically.
[0091] Figure 2 An exemplary flowchart of the process S103 of generating the target environment sound according to an embodiment of the present disclosure is shown. Referring to Figure 2 , first, in step S1031, based on the information of each sub-environment sound, the corresponding environmental sound material is obtained from the preset environmental sound material library.
[0092] It should be understood that the preset ambient sound material library refers to a resource library storing multiple individual ambient sound materials. The individual ambient sound materials may include, for example, natural sounds such as wind sounds, rain sounds, and ocean wave sounds, may also include animal sounds such as bird chirping sounds and frog croaking sounds, may further include sounds from items such as whistle sounds and fan humming sounds, and may also include sounds from people in the environment such as a noisy crowd and cheering sounds.
[0093] For example, according to the information based on each sub-ambient sound, such as the name of each sub-ambient sound, the corresponding ambient sound material can be obtained from the preset ambient sound material library. For example, the rain ambient sound material corresponding to the sub-ambient sound name of "rain sound" can be obtained.
[0094] Thereafter, in step S1032, based on the characteristic parameter information of each sub-ambient sound, rendering synthesis and mixing processing are performed on the ambient sound material.
[0095] For example, after obtaining the corresponding ambient sound materials corresponding to the information of the multiple sub-ambient sounds, the relevant attributes (such as volume, density, etc.) of the ambient sound material can be adjusted accordingly according to the characteristic parameter information of the sub-ambient sound. And for example, in the process of overall rendering and fusion to form the target ambient sound, the fusion parameters are set based on the characteristic parameter information, so that each sub-ambient sound in the fused target ambient sound can present characteristics corresponding to the characteristic parameter information (such as presenting corresponding azimuth, motion state, motion speed, position, etc.).
[0096] Rendering synthesis and mixing processing refers to an audio processing method for generating a target ambient sound based on sub-ambient sounds. The embodiments of the present disclosure are not limited by the specific composition and content of the rendering synthesis and mixing processing.
[0097] Based on the above, in the present application, when generating a target ambient sound, first, based on the information of each sub-ambient sound, the corresponding ambient sound material is obtained from the preset ambient sound material library; thereafter, based on the characteristic parameter information of each sub-ambient sound, rendering synthesis and mixing processing are performed on the ambient sound material, so that in the process of generating the target ambient sound, the corresponding ambient sound material can be accurately and reliably selected, and according to the actual situation, the ambient sound material is adjusted accordingly based on the characteristic parameter information, so that the generated target ambient sound can well and precisely meet the needs of users and improve the user's auditory experience.
[0098] In some embodiments, the target ambient sound is an audio signal having at least two channels, and the target ambient sound is played in separate channels by a plurality of playback devices flexibly distributed in a physical space.
[0099] For example, the target ambient sound can be, for example, four-channel, or it can also be six-channel. Embodiments of the present disclosure are not limited by the number of channels of the audio signal of the target ambient sound.
[0100] For example, an audio signal of the target ambient sound with multiple channels can be directly generated, or a single audio signal can be first generated, and then the single audio signal is subjected to multi-channel rendering, such as the Vector Based Amplitude Panning (VBAP) algorithm, to obtain an audio signal with multiple channels. Embodiments of the present disclosure are not limited by the specific manner of generating the multi-channel audio signal.
[0101] The multiple playback devices can be, for example, speaker devices, or they can also be other devices for implementing audio playback. Embodiments of the present disclosure are not limited by the specific type of the audio playback device.
[0102] The spatial positions of the multiple playback devices can be described more specifically. For example, the multiple devices can be arranged in a preset device array, such as a circular array or a rectangular array, or they can be randomly and flexibly distributed in a physical space, such as an irregular arrangement. However, it should be understood that embodiments of the present disclosure are not limited thereto.
[0103] It should be understood that the multiple playback devices can have a preset interval distance relative to each other, that is, it indicates that the playback devices are separately arranged from each other and located in different physical positions, thereby forming multiple playback sound sources, rather than just a point sound source of a mono speaker or a closely installed stereo speaker.
[0104] It should be understood that embodiments of the present disclosure are not limited by the specific interval distance between the multiple playback devices.
[0105] In this application, considering that the position information of the playback device will also affect the generation effect of the final target ambient sound, by setting the target ambient sound as an audio signal with at least two channels, and the target ambient sound is played in separate channels through multiple playback devices arranged at intervals, compared with the current output of the target ambient sound only through a mono speaker or a closely installed stereo speaker, the output ambient sound can have multiple channels and be played by multiple sound sources, improving the user's auditory experience and the spatial stereo sense of the ambient sound, and being more conducive to the user obtaining an immersive feeling.
[0106] In some embodiments, the target ambient sound is an audio signal with greater than or equal to three channels, and the number of the playback devices corresponds to the number of channels of the target ambient sound.
[0107] By setting the target ambient sound as an audio signal with greater than or equal to three channels, the number of channels of the target ambient sound can be further expanded, thereby further enhancing the stereoscopic effect of the target ambient sound; in addition, by setting the number of the playback devices corresponding to the number of channels of the target ambient sound, the multi-channel audio can be well broadcast based on multiple playback devices, improving the stereoscopic effect and immersion of the target ambient sound, and further enhancing the user experience.
[0108] Figure 3 FIG. 4 shows a schematic diagram of an ambient sound generation system 200 according to an embodiment of the present disclosure.
[0109] Referring to Figure 3 , according to another aspect of the present disclosure, an ambient sound generation system 200 is proposed, including: a controller 210 and a playback device 220.
[0110] The controller is configured to execute the ambient sound generation method 100 as described above.
[0111] The playback device 220 is configured to receive the target ambient sound from the controller and play the target ambient sound.
[0112] The playback device may be, for example, a speaker device or other devices for realizing audio playback. It should be understood that the embodiments of the present disclosure are not limited by the specific type and composition of the playback device.
[0113] It should be understood that, according to actual needs, the controller may be integrated with one of the playback devices, or the controller may be separately provided from the playback devices. The embodiments of the present disclosure are not limited thereto.
[0114] The ambient sound generation system 200 may, for example, implement the method as described above and have the functions as described above, which will not be elaborated here.
[0115] In some embodiments, the target ambient sound is an audio signal with at least two channels, and among them, the playback devices are multiple playback devices flexibly distributed in a physical space and are configured to play the target ambient sound by channels.
[0116] For example, the target ambient sound may be, for example, four channels or six channels. The embodiments of the present disclosure are not limited by the number of channels of the audio signal of the target ambient sound.
[0117] The spatial positions of the multiple playback devices may be described more specifically. For example, the multiple devices may be arranged in a preset device array, such as a circular array or a rectangular array, or may be randomly and flexibly distributed in a physical space, such as an irregular arrangement. However, it should be understood that the embodiments of the present disclosure are not limited thereto.
[0118] It should be understood that the multiple playback devices may, for example, have a preset interval distance relative to each other, that is, it is characterized that the playback devices are separately arranged from each other and located in different physical positions, thereby forming multiple playback sound sources, rather than just a point sound source of a monophonic speaker or a closely installed stereo speaker.
[0119] It should be understood that the embodiments of the present disclosure are not limited by the specific interval distance between the multiple playback devices.
[0120] In this application, considering that the position information of the playback devices will also affect the generation effect of the final target ambient sound, by setting the target ambient sound as an audio signal having at least two channels, and the target ambient sound is played in a channel-separated manner by multiple playback devices flexibly distributed in a physical space, compared with the current output of the target ambient sound only through a monophonic or closely installed stereo speaker, the output ambient sound can have multiple channels and be played by multiple sound sources, improving the user's auditory experience and the spatial stereoscopic effect of the ambient sound, and being more conducive to the user obtaining an immersive feeling.
[0121] The program part in the technology can be considered as a "product" or "article" in the form of executable code and / or related data, which is participated in or implemented through a computer-readable medium. Tangible and permanent storage media can include any memory or storage used by a computer, a processor, or similar devices or related modules. For example, various semiconductor memories, tape drives, disk drives, or any similar devices capable of providing storage functions for software.
[0122] All software or a part of it may sometimes communicate through a network, such as the Internet or other communication networks. Such communication can load the software from one computer device or processor to another. For example: from a server or host computer of a device to a hardware platform of a computer environment, or other computer environments implementing a system, or a system with a similar function related to providing information required for target tracking. Therefore, another medium capable of transmitting software elements can also be used as a physical connection between local devices, such as light waves, radio waves, electromagnetic waves, etc., which are propagated through cables, optical fibers, or air. Physical media used to carry the carrier, such as cables, wireless connections, or optical fibers and other similar devices, can also be considered as media carrying software. Here, unless the tangible "storage" medium is restricted, other terms representing computer or machine "readable media" all represent the media participating in the process of the processor executing any instruction.
[0123] This application uses specific terms to describe the embodiments of this application. For example, "the first / second embodiment", "an embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that the "an embodiment" or "one embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application can be appropriately combined.
[0124] In addition, those skilled in the art can understand that various aspects of this application can be illustrated and described by several patentable types or situations, including any new and useful process, machine, product, or composition of matter, or any new and useful improvement thereof. Accordingly, various aspects of this application can be executed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software can be referred to as "data block", "module", "engine", "unit", "component", or "system". In addition, various aspects of this application may be embodied as a computer product located in one or more computer-readable media, which includes computer-readable program code.
[0125] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those of ordinary skill in the art to which this invention belongs. It should also be understood that terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense, unless expressly defined as such herein.
[0126] The above is the description of the present invention and should not be considered as a limitation thereof. Although several exemplary embodiments of the present invention have been described, those skilled in the art will readily understand that many modifications can be made to the exemplary embodiments without departing from the novel teachings and advantages of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention as defined by the claims. It should be understood that the above is the description of the present invention and should not be considered as limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present invention is defined by the claims and their equivalents.
Claims
1. An environmental sound generation method, comprising: determining target environmental sound information based on environmental sound design reference information; determining environmental sound generation information corresponding to the target environmental sound information based on the target environmental sound information, where the environmental sound generation information includes information of at least one sub - environmental sound and characteristic parameter information corresponding to the sub - environmental sound; generating a target environmental sound based on the environmental sound generation information.
2. The environmental sound generation method according to claim 1, wherein the environmental sound design reference information includes at least one of: expected environmental sound information input by a user, current music information, and spatio - physical information.
3. The environmental sound generation method according to claim 2, in the case where the environmental sound design reference information includes expected environmental sound information input by a user, determining the target environmental sound information comprising: determining the target environmental sound information based on content analysis of the expected environmental sound information.
4. The environmental sound generation method according to claim 2, wherein in the case where the environmental sound design reference information includes current music information, determining the target environmental sound information includes: determining music characteristic information of the current music information, and determining the target environmental sound information based on the music characteristic information.
5. The environmental sound generation method according to claim 4, wherein the music characteristic information includes at least one of: lyric content information, music type information, music rhythm information, and music style information.
6. The environmental sound generation method according to claim 2, wherein the spatio - physical information includes at least one of: current location information, current time information, and current weather information.
7. The environmental sound generation method according to claim 1, wherein the characteristic parameter information corresponding to the sub - environmental sound includes at least one of: volume information, trigger density information, azimuth information, movement speed information, and position information.
8. The environmental sound generation method according to claim 1, wherein generating a target environmental sound based on the environmental sound generation information includes: acquiring corresponding environmental sound materials in a preset environmental sound material library based on the information of each sub - environmental sound; performing rendering synthesis and mixing processing on the environmental sound materials based on the characteristic parameter information of each sub - environmental sound.
9. The environmental sound generation method according to claim 1, wherein the target environmental sound is an audio signal with at least two channels, and the target environmental sound is played by multiple playback devices in a channel - separated manner.
10. The environmental sound generation method according to claim 9, wherein the target environmental sound is an audio signal with greater than or equal to three channels, and the number of the playback devices corresponds to the number of channels of the target environmental sound.
11. An environmental sound generation system, comprising: a controller configured to execute the environmental sound generation method according to any one of claims 1 - 9; a playback device configured to receive the target environmental sound from the controller and play the target environmental sound.
12. The environmental sound generation system according to claim 11, wherein the target environmental sound is an audio signal with at least two channels, And among them, the playback devices are multiple playback devices, and the playback devices are configured to play the target ambient sound in channels.