Sound source scene discrimination method, apparatus and device, and medium

Abstract

The invention provides a sound source scene discrimination method. The method comprises the following steps: starting a sensor module; obtaining attitude change information and real-time image information of the sound effect device, sending the attitude change information and the real-time image information to a sensor fusion module, and confirming real-time position information of the sound effect device according to the attitude change information and the real-time image information; a preset first model judges a use scene of the sound effect device according to the real-time position information so as to confirm a moving track of the sound effect device; the sound source information of the sound effect device is confirmed according to the moving track, and the audio processing module carries out 3D audio processing on the sound effect device according to the real-time position information and the sound source information. According to the technical scheme provided by the embodiment of the invention, accurate use scene information of the sound effect device is obtained by combining sensors such as the camera unit and the gyroscope unit and visual identification, and the position of a sound source is judged under the condition that the earphone and a person move at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of TWS earphones, and in particular to a method, device, equipment and medium for distinguishing a sound effect scene. Background Art

[0002] At present, when distinguishing the sound effect scene of headphones, the user's posture is evaluated by setting a gyroscope in the headphones. Since the gyroscope is set inside the headphones, the headphones and the head have the same movement trajectory. Therefore, when the user's head moves, the direction of the sound source will also change. However, the existing technology cannot handle the situation where the headphones and the person move at the same time, and determine the location of the sound source. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides a sound source scene discrimination method, device, equipment and medium, which can determine the location of the sound source when the earphone and the person move at the same time.

[0004] In a first aspect, an embodiment of the present invention provides a sound source scene identification method, characterized in that it is applied to a multi-sensor fusion sound effect device, the sound effect device includes a sensor module, a sensor fusion module and an audio processing module, the sensor fusion module is respectively connected to the sensor module and the audio processing module, wherein the sensor module is provided with a camera unit and a gyroscope unit, and the sound source scene identification method includes:

[0005] Starting the sensor module;

[0006] The gyroscope unit obtains posture change information of the sound effect device, the camera unit obtains real-time image information of the sound effect device, and sends the posture change information and the real-time image information to the sensor fusion module;

[0007] The sensor fusion module confirms the real-time position information of the sound effect device according to the posture change information and the real-time image information, and sends the real-time position information to the audio processing module;

[0008] The preset first model determines the usage scenario of the sound effect device according to the real-time position information to confirm the movement trajectory of the sound effect device;

[0009] The sound source information of the sound effect device is confirmed according to the movement trajectory, and the audio processing module performs 3D audio processing on the sound effect device according to the real-time position information and the sound source information.

[0010] In some embodiments of the present invention, the preset first model determines the usage scenario of the sound effect device according to the real-time location information, including:

[0011] Inputting the posture change information and the real-time image information into the first model;

[0012] The first model constructs a first convolutional layer according to the real-time image information, and fills the first convolutional layer to obtain a first average pooling layer;

[0013] Performing stride padding on the first average pooling layer to obtain a second convolutional layer, and confirming a second average pooling layer according to the second convolutional layer;

[0014] A first fully connected layer, a second fully connected layer, and a third fully connected layer of the second average pooling layer are constructed to confirm the usage scenario of the sound effect device.

[0015] In some embodiments of the present invention, the sound effect device is further provided with a reverberation module, the reverberation module is provided with an early reflection unit and a late reverberation unit, and the audio processing module performs 3D audio processing on the sound effect device according to the real-time position information and the sound source information, including:

[0016] Acquire audio data information, input the audio data information into the early reflection unit and the first model, the early reflection unit obtains first reflection audio according to the audio data information, and inputs the first reflection audio into the first model and the late reverberation unit respectively, wherein the first model is preset with an HRTF database;

[0017] The HRTF database sends the first access direction data and the early reflection direction data to the first model, and the first model outputs audio rendering information according to the audio data information, the first reflected audio, the first access direction data, and the early reflection direction data;

[0018] The late reverberation unit outputs a first reverberation audio according to the first reflected audio;

[0019] The first reverberation audio and the audio rendering data are balanced so that the sound effect device outputs balanced audio.

[0020] In some embodiments of the present invention, the audio processing module performs 3D audio processing on the sound effect device according to the real-time position and the sound source information, including:

[0021] confirming whether the sound effect device is in a sports environment according to the movement trajectory information;

[0022] When the sound effect device is in the sports environment, the audio processing module stops performing 3D processing on the sound effect device;

[0023] When the sound effect device is not in the sports environment, the audio processing module performs 3D processing on the sound effect device according to the posture change information obtained by the gyroscope unit.

[0024] In some embodiments of the present invention, the audio processing module performs 3D processing on the sound effect device according to the posture change information obtained by the gyroscope unit, including:

[0025] Acquiring time difference information, sound level difference information and spectrum difference information of the sound effect device, and obtaining sound source transmission characteristic information of the sound effect device according to the time difference information, sound level difference information and spectrum difference information;

[0026] Constructing a sound source transmission network according to the sound source transmission characteristic information, wherein the sound source transmission network is provided with a plurality of sound source transmission planes and a plurality of sound source transmission vertical planes;

[0027] The sound source of the sound effect device is simulated according to the sound source transmission network so that the sound effect device outputs spatial audio.

[0028] In some embodiments of the present invention, simulating the sound source of the sound effect device according to the sound source transmission network includes:

[0029] When a user uses the sound effect device, obtaining shape information and size information of the user's head, ears, and torso;

[0030] The time difference information, the sound level difference information, and the spectrum difference information are simulated according to the shape information and the size information, so that the user can perceive the direction and distance of the sound source.

[0031] In some embodiments of the present invention, the confirming the usage scenario of the sound effect device includes:

[0032] Converting objects in the real-time image information into image signals;

[0033] Obtaining morphological information of the object according to the image signal, and converting the morphological information into a digital signal according to preset pixel distribution information, brightness information and color information;

[0034] The target feature in the real-time image information is extracted according to the digital signal, the identification result of the target feature is confirmed, and the sound effect device is controlled to output a sound source according to the identification result.

[0035] In a second aspect, an embodiment of the present invention provides a sound source scene identification device, characterized in that it includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the sound source scene identification method as described in any one of claims 1 to 7.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, comprising the device for distinguishing a sound source scene as described in the second aspect.

[0037] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the sound source scene identification method as described in the first aspect above.

[0038] The sound source scene identification method according to the embodiment of the present invention has at least the following beneficial effects:

[0039] Start the sensor module; the gyroscope unit obtains the posture change information of the sound effect device, the camera unit obtains the real-time image information of the sound effect device, and sends the posture change information and the real-time image information to the sensor fusion module; the sensor fusion module confirms the real-time position information of the sound effect device according to the posture change information and the real-time image information, and sends the real-time position information to the audio processing module; the preset first model determines the use scenario of the sound effect device according to the real-time position information to confirm the moving trajectory of the sound effect device; the sound source information of the sound effect device is confirmed according to the moving trajectory, and the audio processing module performs 3D audio processing on the sound effect device according to the real-time position information and the sound source information. According to the technical solution of this embodiment, accurate use scenario information of the sound effect device is obtained by combining sensors such as the camera unit and the gyroscope unit and visual recognition, and the position of the sound source is determined when the earphones and the person are moving at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of a method for distinguishing a sound source scene provided by an embodiment of the present invention;

[0041] Figure 2 is a flow chart of a preset first model provided in an embodiment of the present invention for determining a usage scenario of a sound effect device according to real-time location information;

[0042] Figure 3 It is a flow chart of an audio processing module provided by an embodiment of the present invention performing 3D audio processing on a sound effect device according to real-time position information and sound source information;

[0043] Figure 4is a flow chart after 3D audio processing provided by an embodiment of the present invention;

[0044] Figure 5 is a flow chart of an audio processing module according to an embodiment of the present invention performing 3D processing on a sound effect device according to posture change information;

[0045] Figure 6 It is a flow chart of simulating the sound source of the sound effect device according to the sound source transmission network provided by an embodiment of the present invention;

[0046] Figure 7 is a flow chart of a usage scenario of a sound effect device for confirming sound effects provided by an embodiment of the present invention;

[0047] Figure 8 It is a structural diagram of a sound source scene identification device provided by another embodiment of the present invention. DETAILED DESCRIPTION

[0048] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0049] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0050] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0051] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0052] The embodiment of the present invention provides a method for distinguishing a sound source scene, characterized in that it is applied to a sound effect device of multi-sensor fusion, the sound effect device includes a sensor module, a sensor fusion module and an audio processing module, the sensor fusion module is respectively connected to the sensor module and the audio processing module, wherein the sensor module is provided with a camera unit and a gyroscope unit. Specifically, by combining sensors such as a camera unit and a gyroscope unit, more accurate scene information is obtained in combination with visual recognition, and by assisting 3D sound effect processing, the sound effect device obtains better sound effects. Furthermore, by judging that the wearer of the sound effect device is in a closed environment such as a car, the sound effect processing of the 3D audio is turned off. When the wearer is not in a closed environment such as a car, the data of the gyroscope is used to perform 3D sound effect processing on the audio.

[0053] The control method of the embodiment of the present invention is further described below based on the accompanying drawings.

[0054] Reference Figure 1 , Figure 1 A flow chart of a sound source scene identification method provided by an embodiment of the present invention, the sound source scene identification method includes but is not limited to the following steps:

[0055] Step S11, starting the sensor module;

[0056] Step S12, the gyroscope unit obtains posture change information of the sound effect device, the camera unit obtains real-time image information of the sound effect device, and sends the posture change information and real-time image information to the sensor fusion module;

[0057] It should be noted that the gyroscope obtains the attitude change information of the sound effect device by measuring the change of the rotor torque. The camera unit is used to capture the real-time image information of the sound effect device. The camera unit usually includes one or more image sensors, which can convert the captured light into electrical signals and generate digital images after processing, so as to perform image processing and analysis on the environment where the sound effect device is located. The information obtained by the gyroscope unit and the camera unit is then integrated and processed by the sensor fusion module to further improve the accuracy and reliability of the information. By fusing the information of the gyroscope unit and the camera unit, the sensor fusion module can provide more comprehensive and accurate status information of the sound effect device. It helps to reduce the errors and uncertainties that may be caused by a single sensor.

[0058] Step S13, the sensor fusion module confirms the real-time position information of the sound effect device according to the posture change information and the real-time image information, and sends the real-time position information to the audio processing module;

[0059] It should be noted that the sensor fusion module can accurately determine the real-time position of the sound effect device through posture change information and implementation image information. The information provided by the sensor fusion module is used to control and optimize the sound effect device. For example, the playback direction or volume of the sound effect device can be adjusted according to the posture change information to provide a better auditory experience. At the same time, real-time image information can also be used to monitor environmental changes around the sound effect device so that corresponding measures can be taken in time to complete the correction and adjustment of the sound source of the sound effect device.

[0060] Step S14, the preset first model determines the usage scenario of the sound effect device according to the real-time position information to confirm the movement trajectory of the sound effect device;

[0061] It should be noted that the first model is an AI model. The first model determines whether the wearer of the sound effect device is in a closed environment such as a car through real-time location information. When the wearer of the sound effect device is in a closed mobile environment such as a car, it is considered that the sound source also moves, thereby determining the movement trajectory of the sound effect device. In this embodiment, through scene judgment and confirmation of the movement trajectory, the sound effect processing can more accurately match the use environment and movement state of the sound effect device, so that the sound effect device can handle the situation where the sound source and the user move at the same time, such as when a person uses a mobile phone in a car.

[0062] Step S15, confirming the sound source information of the sound effect device according to the moving trajectory, and the audio processing module performs 3D audio processing on the sound effect device according to the real-time position information and the sound source information.

[0063] It should be noted that the 3D audio processing combined with the motion trajectory and sound source information can generate sound effects that match the actual position of the sound effect device and the sound source information, so that the headphone wearer can accurately perceive the direction and distance of the sound. In addition, the audio processing module can accurately determine the environment in which the sound effect device is located, and accurately determine the needs and effects of the sound effect processing.

[0064] It should be noted that by combining sensors such as the camera unit and the gyroscope unit, combined with visual recognition to obtain more accurate scene information, and by assisting 3D sound effect processing, the sound effect device can obtain better sound effects. Furthermore, by judging that the wearer of the sound effect device is in a closed environment such as a car, the sound effect processing of the 3D audio is turned off. When the wearer is not in a closed environment such as a car, the gyroscope data is used to perform 3D sound effect processing on the audio.

[0065] In addition, in one embodiment, referring to Figure 2 ,exist Figure 1 Step S14 of the illustrated embodiment also includes but is not limited to the following steps:

[0066] Step S21, inputting posture change information and real-time image information into the first model;

[0067] Step S22, the first model constructs a first convolutional layer according to the real-time image information, fills the first convolutional layer, and obtains a first average pooling layer;

[0068] Step S23, performing stride padding on the first average pooling layer to obtain a second convolutional layer, and confirming a second average pooling layer according to the second convolutional layer;

[0069] Step S24, constructing the first fully connected layer, the second fully connected layer and the third fully connected layer of the second average pooling layer to confirm the usage scenario of the sound effect device.

[0070] It should be noted that by inputting posture change information and real-time image information into the model at the same time, the complementary information provided by these two data sources can be fully utilized. The posture change information can reflect the dynamic characteristics of the sound effect device, while the real-time image information provides rich spatial and environmental information. The first convolutional layer extracts local features in the image, such as edges and textures, through convolution operations. These features are crucial for identifying the type of environment in which the sound effect device is located (such as indoor, outdoor, natural landscape, urban architecture, etc.). With the deepening of the convolution layer, the model can capture higher-level features, such as the shape and color distribution of objects, further enhancing the ability of scene recognition. The average pooling layer downsamples the feature map output by the convolution layer, reduces the size of the feature map, reduces the computational complexity, and retains important feature information. The fully connected layer after the second average pooling layer maps the extracted features to the classification space, and learns the feature representation of different scenes through training to achieve accurate classification of the use scene of the sound effect device. The layer-by-layer processing of the first, second and third fully connected layers enables the model to gradually approach the classification results of the real scene. Since the model can process multiple types of image and posture change information, it can maintain good recognition performance under different environmental conditions (such as illumination changes, occlusion, perspective changes, etc.). By using a combination of average pooling layers and fully connected layers, the model can learn more robust feature representations during training and reduce dependence on specific training data.

[0071] In addition, in one embodiment, referring to Figure 3 ,exist Figure 1 Step S11 of the illustrated embodiment also includes but is not limited to the following steps:

[0072] Step S31, obtaining audio data information, inputting the audio data information into an early reflection unit and a first model, wherein the early reflection unit obtains a first reflection audio according to the audio data information, and inputs the first reflection audio into the first model and a late reverberation unit respectively, wherein the first model is preset with an HRTF database;

[0073] Step S32: The HRTF database sends the first access direction data and the early reflection direction data to the first model, and the first model outputs audio rendering information according to the audio data information, the first reflected audio, the first access direction data, and the early reflection direction data;

[0074] Step S33, the late reverberation unit outputs the first reverberation audio according to the first reflected audio;

[0075] Step S34: equalize the first reverberation audio and the audio rendering data so that the sound effect device outputs balanced audio.

[0076] It should be noted that audio data information is obtained from the sound source of the sound effect device, and the audio is preprocessed, such as denoising or format conversion, to ensure the quality and consistency of the audio data information, and early reflection audio is obtained based on the audio information, and HRTF information matching the first access direction data and the early reflection direction data is extracted from the HRTF database and sent to the first model, and the HRTF data is used to perform spatial processing on the audio and reflected audio to simulate the sound perception in different directions, and the late reverberation audio is calculated based on the first reflected audio and environmental parameters using a reverberation algorithm or a predefined reverberation model, and finally the audio rendering information and the late reverberation audio are obtained, and the audio rendering information and the first reverberation audio are equalized, and the equalized sound effect is finally output through the sound effect device.

[0077] In addition, in one embodiment, referring to Figure 4 ,exist Figure 1 Step S15 of the illustrated embodiment also includes but is not limited to the following steps:

[0078] Step S41, confirming whether the sound effect device is in a sports environment according to the movement trajectory information;

[0079] Step S42, when the sound effect device is in the sports environment, the audio processing module stops performing 3D processing on the sound effect device;

[0080] Step S43: when the sound effect device is not in a sports environment, the audio processing module performs 3D processing on the sound effect device according to the posture change information obtained by the gyroscope unit.

[0081] It should be noted that 3D audio technology can restore real sound effects in three-dimensional space, and by simulating the human ear receiving sound, as well as the difference between the left and right ears, the positioning and reverberation of sound, the user can feel the direction and distance of the sound source. The implementation of 3D audio technology mainly relies on two key concepts: head-related transfer function (HRTF) and spatial audio rendering. The key technology is spatial positioning technology. Spatial positioning technology is the key to realizing 3D audio. By accurately measuring the time difference and sound intensity difference of the sound reaching the ears of different listeners, the source of the sound can be determined.

[0082] In addition, in one embodiment, referring to Figure 5 ,exist Figure 4 Step S43 of the illustrated embodiment also includes but is not limited to the following steps:

[0083] Step S51, obtaining time difference information, sound level difference information and spectrum difference information of the sound effect device, and obtaining sound source transmission characteristic information of the sound effect device according to the time difference information, sound level difference information and spectrum difference information;

[0084] Step S52, constructing a sound source transmission network according to the sound source transmission characteristic information, wherein the sound source transmission network is provided with a plurality of sound source transmission planes and a plurality of sound source transmission vertical planes;

[0085] Step S53 , simulating the sound source of the sound effect device according to the sound source transmission network, so that the sound effect device outputs spatial audio.

[0086] It should be noted that the position and direction of the sound source are determined by recording the time difference information of the sound of the sound effect device from the sound source to the human ear; the sound level difference information when the sound is output to the human ear is measured to determine the attenuation of the sound source in different directions; the difference in the spectrum characteristics when the sound source is played is analyzed to identify the spectrum changes of the sound source in different directions. According to the sound source transmission characteristic information, a sound source transmission network including multiple sound source transmission planes and multiple sound source transmission vertical planes is constructed. These planes and vertical planes represent the propagation path and reflection of the sound in three-dimensional space. In the network, each node represents a possible sound source position, and each path represents the propagation mode of the sound from one node to another. Using the sound source transmission network, we can simulate the propagation of the sound source in the sound effect device at different positions and directions. According to the simulation results, the sound effect device generates an audio output with a sense of space. Including the directionality, distance and environment of the sound. In this embodiment, by simulating the propagation of sound in three-dimensional space, the sound effect device can output spatial audio with directionality and distance, so that the audience can perceive the sound source and the surrounding environment more realistically. Using a sound source transmission network for simulation can more accurately reflect the propagation characteristics of sound in different environments and conditions, thereby improving the realism and immersion of the sound effects.

[0087] In addition, in one embodiment, referring to Figure 6 ,exist Figure 5 Step S53 in the illustrated embodiment also includes but is not limited to the following steps:

[0088] Step S61, when the user uses the sound effect device, obtaining shape information and size information of the user's head, ears and torso;

[0089] Step S62: simulating the time difference information, the sound level difference information and the spectrum difference information according to the shape information and the size information, so as to enable the user to perceive the direction and distance of the sound source.

[0090] It should be noted that HRTF, or head-related transfer function, is a sound localization algorithm that describes the transmission characteristics of sound from the sound source to the listener's ears when it propagates in three-dimensional space. HRTF includes information such as time difference, sound level difference, and spectrum difference, which are determined by the shape and size of the listener's head, ears, and torso. By simulating these characteristics, HRTF can help create a sound with a sense of space, allowing the listener to perceive the direction and distance of the sound.

[0091] In addition, in one embodiment, referring to Figure 7 ,exist Figure 2 Step S24 of the illustrated embodiment also includes but is not limited to the following steps:

[0092] Step S71, converting objects in the real-time image information into image signals;

[0093] Step S72, obtaining morphological information of the object according to the image signal, and converting the morphological information into a digital signal according to preset pixel distribution information, brightness information and color information;

[0094] Step S73, extracting target features in the real-time image information according to the digital signal, confirming the identification result of the target features, and controlling the sound effect device to output the sound source according to the identification result.

[0095] It should be noted that the real-time image information of the environment in which the sound effect device is located is obtained through the camera unit, the objects in the image are identified and converted into image signals, the morphological information of the objects in the image information is obtained according to the image signals, and the morphological information is converted into a digital signal according to the preset pixel distribution information, brightness information and color information, and the target features in the real-time image information are extracted from the digital signal. According to the discrimination result, the sound effect device is controlled to output the corresponding sound source. Since the whole process is based on real-time image information, the sound effect device can respond quickly to changes in the image. By extracting the target features and controlling the sound effect device according to the discrimination result, personalized sound effect output can be achieved. Combining image information with sound effects can create a more realistic and immersive experience.

[0096] like Figure 8 As shown, Figure 8 : is a structural diagram of a sound source scene identification device provided by an embodiment of the present invention. The present invention also provides a sound source scene identification device, including:

[0097] The processor 801 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0098] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 802, and the processor 801 calls and executes the sound source scene identification method of the embodiment of this application;

[0099] Input / output interface 803, used to implement information input and output;

[0100] The communication interface 804 is used to realize the communication interaction between the present apparatus and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WI FI, Bluetooth, etc.);

[0101] A bus 805 that transmits information between the various components of the device (e.g., the processor 801, the memory 802, the input / output interface 803, and the communication interface 804);

[0102] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0103] An embodiment of the present application further provides an electronic device, comprising the sound source scene identification device as described above.

[0104] The embodiment of the present application further provides a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned sound source scene identification method is implemented.

[0105] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are implemented to be located in one place, or may also be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0106] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0107] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions under the shared conditions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A method for distinguishing a sound source scene, characterized in that: In a sound effect device applied to multi-sensor fusion, the sound effect device includes a sensor module, a sensor fusion module and an audio processing module, the sensor fusion module is connected to the sensor module and the audio processing module respectively, wherein the sensor module is provided with a camera unit and a gyroscope unit, and the sound source scene discrimination method includes: Starting the sensor module; The gyroscope unit obtains posture change information of the sound effect device, the camera unit obtains real-time image information of the sound effect device, and sends the posture change information and the real-time image information to the sensor fusion module; The sensor fusion module confirms the real-time position information of the sound effect device according to the posture change information and the real-time image information, and sends the real-time position information to the audio processing module; The preset first model determines the usage scenario of the sound effect device according to the real-time position information to confirm the movement trajectory of the sound effect device; The sound source information of the sound effect device is confirmed according to the movement trajectory, and the audio processing module performs 3D audio processing on the sound effect device according to the real-time position information and the sound source information.

2. The method for distinguishing a sound source scene according to claim 1, characterized in that: The preset first model determines the usage scenario of the sound effect device according to the real-time location information, including: Inputting the posture change information and the real-time image information into the first model; The first model constructs a first convolutional layer according to the real-time image information, and fills the first convolutional layer to obtain a first average pooling layer; Performing stride padding on the first average pooling layer to obtain a second convolutional layer, and confirming a second average pooling layer according to the second convolutional layer; A first fully connected layer, a second fully connected layer, and a third fully connected layer of the second average pooling layer are constructed to confirm the usage scenario of the sound effect device.

3. The method for distinguishing a sound source scene according to claim 1, characterized in that: The sound effect device is further provided with a reverberation module, the reverberation module is provided with an early reflection unit and a late reverberation unit, and the audio processing module performs 3D audio processing on the sound effect device according to the real-time position information and the sound source information, including: Acquire audio data information, input the audio data information into the early reflection unit and the first model, the early reflection unit obtains first reflection audio according to the audio data information, and inputs the first reflection audio into the first model and the late reverberation unit respectively, wherein the first model is preset with an HRTF database; The HRTF database sends the first access direction data and the early reflection direction data to the first model, and the first model outputs audio rendering information according to the audio data information, the first reflected audio, the first access direction data, and the early reflection direction data; The late reverberation unit outputs a first reverberation audio according to the first reflected audio; The first reverberation audio and the audio rendering data are balanced so that the sound effect device outputs balanced audio.

4. The method for distinguishing a sound source scene according to claim 1, characterized in that: The audio processing module performs 3D audio processing on the sound effect device according to the real-time position and the sound source information, including: confirming whether the sound effect device is in a sports environment according to the movement trajectory information; When the sound effect device is in the sports environment, the audio processing module stops performing 3D processing on the sound effect device; When the sound effect device is not in the sports environment, the audio processing module performs 3D processing on the sound effect device according to the posture change information obtained by the gyroscope unit.

5. The method for distinguishing a sound source scene according to claim 4, characterized in that: The audio processing module performs 3D processing on the sound effect device according to the posture change information obtained by the gyroscope unit, including: Acquiring time difference information, sound level difference information and spectrum difference information of the sound effect device, and obtaining sound source transmission characteristic information of the sound effect device according to the time difference information, sound level difference information and spectrum difference information; Constructing a sound source transmission network according to the sound source transmission characteristic information, wherein the sound source transmission network is provided with a plurality of sound source transmission planes and a plurality of sound source transmission vertical planes; The sound source of the sound effect device is simulated according to the sound source transmission network so that the sound effect device outputs spatial audio.

6. The method for distinguishing a sound source scene according to claim 5, characterized in that: The simulating the sound source of the sound effect device according to the sound source transmission network includes: When the user uses the sound effect device, shape information and size information of the user's head, ears and torso; The time difference information, the sound level difference information, and the spectrum difference information are simulated according to the shape information and the size information, so that the user can perceive the direction and distance of the sound source.

7. The method for distinguishing a sound source scene according to claim 2, characterized in that: The confirming the usage scenario of the sound effect device includes: Converting objects in the real-time image information into image signals; Obtaining morphological information of the object according to the image signal, and converting the morphological information into a digital signal according to preset pixel distribution information, brightness information and color information; The target features in the real-time image information are extracted according to the digital signal, the identification result of the target features is confirmed, and the sound effect device is controlled to output a sound source according to the identification result.

8. A sound source scene identification device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the sound source scene discrimination method as described in any one of claims 1 to 7.

9. An electronic device, characterized in that: It includes the sound source scene identification device as described in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the sound source scene identification method according to any one of claims 1 to 7.

Images