Audio processing method, related device and communication system

Abstract

The invention provides an audio processing method, a related device and a communication system. When the virtual conference is accessed, the first equipment can judge whether equipment in the same physical space as the first equipment exists in other equipment accessed to the virtual conference and judge the physical distance between the equipment in the same physical space as the first equipment and the first equipment; and further determining whether transmission sound from the devices in the same physical space as the first device can cause repeated sound interference or not. If the transmission sound can cause repeated sound interference, the first device can not play the transmission sound or play the transmission sound after mixing the transmission sound with the transparent transmission sound. Therefore, the problem of repeated sound interference when a plurality of participants perform the virtual conference in the same physical space can be solved, and the experience of the user participating in the virtual conference is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an audio processing method, related devices and a communication system. Background Art

[0002] The current virtual conference system based on augmented reality (AR) or virtual reality (VR) can simulate a virtual conference room. Participants can use virtual images to sit around the table in the virtual conference room to hold meetings and interact. Each participant can wear an AR / VR device to watch the scene in the virtual conference room. In addition, the AR / VR device can also perform spatial audio modeling based on the sound field environment information of the virtual conference room and the positional relationship of each participant in the virtual conference room, so that when the user hears the voice of a certain participant, the position of the participant in the hearing sense is consistent with the position of the participant seen in the virtual conference room. That is, the above-mentioned spatial audio modeling allows the sound heard by the user to match the virtual conference room scene seen.

[0003] However, when multiple participants are in the same physical space, especially when the participants are physically close to each other, when a participant near the user speaks, the user will hear the voice of this participant not only from the transparent sound generated by the AR / VR device collecting the ambient sound, but also from the transmission sound sent by the AR / VR device of this participant. That is, the user may hear the voice of a participant speaking twice. This will cause repeated sound interference to the user, affecting the user's meeting experience. Summary of the invention

[0004] The present application provides an audio processing method, a related device and a communication system. The present application selects and processes the audio generated in a virtual meeting to solve the problem of repeated sound interference in a virtual meeting and improve the user's meeting experience.

[0005] In the first aspect, the present application provides an audio processing method. In which a first device receives a first audio from a second device, and generates a second audio according to the collected sound signal; when the first device and the second device are in the same physical space and the physical distance between the first device and the second device meets a first condition, the first device plays the second audio and does not play the first audio, or the first device detects a first delay between the first audio and the second audio, and when the first delay is greater than a delay threshold, filters out the human voice part in the first audio to obtain a third audio, and plays the second audio and the third audio.

[0006] The first audio may be obtained by collecting ambient sound by the second device. The first audio may be referred to as transmission sound. The second audio may be generated by the first device using transparent playback technology to collect sound signals. The second audio may be referred to as transparent transmission sound. The third audio may be generated by the first device after performing crosstalk filtering to eliminate the first audio.

[0007] The physical distance between the first device and the second device satisfying the first condition may indicate that the first device and the second device are located close to each other.

[0008] It can be understood that when the first device and the second device are in the same physical space and close to each other, the first device and the second device can both collect the sound of the user wearing the second device. That is to say, if the user wearing the second device speaks, the first audio and the second audio can both include the sound of the user wearing the second device. In addition, the first device and the second device can also collect the sound of the user wearing the first device. That is to say, if the user wearing the first device speaks, the first audio and the second audio can both include the sound of the user wearing the first device. Therefore, when the first device and the second device are in the same physical space and close to each other, no matter whether the user wearing the first device speaks or the user wearing the second device speaks, the first audio and the second audio will produce repeated sound interference.

[0009] It can be seen that when the first device and the second device are in the same physical space and are in close positions, the first device plays the second audio instead of the first audio, which can ensure that the user wearing the first device can hear both his own voice and the voice of the user wearing the second device, while avoiding interference from repeated sounds. Alternatively, when the time delay between the first audio and the second audio is large, the first device avoids interference from repeated sounds by filtering out the human voice part in the first audio. In other words, the above method can first determine whether the transmitted sound and the transmitted sound in the first device will cause repeated sound interference, and select and process the audio when repeated sound interference will occur, so that the audio played last will not cause repeated sound interference to the user, thereby improving the user's experience in participating in virtual meetings.

[0010] In combination with the first aspect, in some embodiments, the first device and the second device are in the same physical space, specifically including: the IP address of the first device and the IP address of the second device indicate that the first device and the second device are connected to the same network; the physical distance between the first device and the second device satisfies the first condition, specifically including: the physical distance between the first device and the second device is less than the first distance, and / or the loudness of the sound signal related to the second audio in the first audio is greater than the first loudness.

[0011] The first distance may also be referred to as a distance threshold. The first distance may be preset. For example, the first distance may be 3 meters, 4 meters, 5 meters, etc.

[0012] In combination with the first aspect, in some embodiments, the first device and the second device are devices connected to the same virtual conference, wherein the first device and the second device are worn by different users, the device for playing audio in the first device adopts an ear-blocking design, and the second audio is transparent sound.

[0013] In combination with the first aspect, in some embodiments, when the first device plays the second audio and the third audio, it can specifically use the second audio as the main sound source and the third audio as the non-main sound source to mix and play the second audio and the third audio.

[0014] It is understandable that when the first device and the second device are in the same physical space and close to each other, the sound quality of the voice of the user wearing the first device or the user wearing the second device in the second audio is better. Moreover, compared with the first audio from the first device, the user wearing the first device will first listen to the content of his or her speech or the user wearing the second device through the second audio. Therefore, when the first device mixes the second audio and the third audio, the second audio is used as the main sound source and the third audio is used as the non-main sound source. Among them, the proportion of information of the main sound source in the mixed audio is higher than the proportion of information of the non-main sound source. That is, when the user listens to the mixed audio, the content he hears is more from the main sound source.

[0015] In combination with the first aspect, in some embodiments, when the first device and the second device are in the same physical space, the physical distance between the first device and the second device satisfies the first condition, and the first delay is less than or equal to the delay threshold, the first device uses the second audio as the main sound source and the first audio as the non-main sound source, mixes and plays the first audio and the second audio.

[0016] It can be seen that when the time delay between the first audio and the second audio is small, the first audio and the second audio will not cause repeated sound interference to the user. Therefore, the first device can directly mix and play the first audio and the second audio.

[0017] In combination with the first aspect, in some embodiments, when the first device and the second device are in the same physical space and the physical distance between the first device and the second device does not meet the first condition, the first device plays the first audio and does not play the second audio, or, when the first delay is greater than the delay threshold, the first electronic device filters out the vocal part in the second audio to obtain the fourth audio, and plays the first audio and the fourth audio.

[0018] The physical distance between the first device and the second device does not satisfy the first condition, specifically including: the physical distance between the first device and the second device is greater than the first distance, or the loudness of the sound signal related to the second audio in the first audio is less than the first loudness. The physical distance between the first device and the second device does not satisfy the first condition, which may indicate that the first device and the second device are far apart.

[0019] When the first device and the second device are in the same physical space but far apart, the first device has a poor sound pickup effect for the speech of the user wearing the second device. In other words, the sound quality of the voice of the user wearing the second device in the second audio is poor. The first device plays the first audio instead of the second audio to ensure that the user wearing the first device can clearly hear the speech of the user wearing the second device while avoiding interference from repeated sounds. Alternatively, when the time delay between the first audio and the second audio is large, the first device avoids interference from repeated sounds by filtering out the human voice part in the second audio.

[0020] In combination with the first aspect, in some embodiments, when the first device plays the first audio and the fourth audio, the first audio is used as the main sound source and the fourth audio is used as the non-main sound source, and the first audio and the fourth audio are mixed and played.

[0021] In combination with the first aspect, in some embodiments, when the first device and the second device are in the same physical space, the physical distance between the first device and the second device does not meet the first condition, and the first delay is less than or equal to the delay threshold, the first device uses the first audio as the main sound source and the second audio as the non-main sound source, mixes the first audio and the second audio, and plays them.

[0022] In combination with the first aspect, in some embodiments, when the first device and the second device are not in the same physical space, the first device plays the first audio and the second audio.

[0023] In a second aspect, the present application provides an audio processing method, wherein a first device receives a first audio from a second device; when the first device and the second device are in the same physical space and the physical distance between the first device and the second device satisfies a first condition, the first device does not play the first audio.

[0024] In combination with the second aspect, in some embodiments, the first device and the second device are in the same physical space, specifically including: the IP address of the first device and the IP address of the second device indicate that the first device and the second device are connected to the same network; the physical distance between the first device and the second device satisfies the first condition, specifically including: the physical distance between the first device and the second device is less than the first distance, and / or the loudness of the sound signal related to the second audio in the first audio is greater than the first loudness.

[0025] In combination with the second aspect, in some embodiments, the first device and the second device are devices connected to the same virtual conference, wherein the first device and the second device are worn by different users, and the device for playing audio in the first device adopts a non-ear-blocking design.

[0026] It can be seen that when the first device adopts a non-ear-blocking design, the first device can detect whether there are other devices in the same physical space as the first device, as well as devices that are closer to the first device among the devices in the same physical space as the first device. The user wearing the first device can quickly and relatively clearly hear the voice of the participant who is closer through the ambient sound. Therefore, when the second device is closer to the first device, the first device can no longer play the first audio sent by the second device, so as to prevent the user from repeatedly listening to the same speech of the participant who is closer to both sides. This can solve the problem of repeated sound interference.

[0027] In combination with the second aspect, in some embodiments, when the first device and the second device are in the same physical space and the physical distance between the first device and the second device does not meet the first condition, the first device plays the first audio; when the first device and the second device are not in the same physical space, the first device plays the first audio.

[0028] In a third aspect, the present application provides an electronic device, which may include an audio input module, an audio output module, a memory, and a processor. The audio input module may be used to collect sound signals. The audio output module may be used to play audio. The memory may be used to store a computer program. The processor may be used to call the computer program so that the electronic device executes any possible implementation method in the first aspect or the second aspect.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, enables the electronic device to execute any possible implementation method in the first aspect or the second aspect.

[0030] In a fifth aspect, the present application provides a computer program product, which may include computer instructions. When the computer instructions are run on an electronic device, the electronic device executes any possible implementation method in the first aspect or the second aspect.

[0031] In a sixth aspect, the present application provides a chip, which is applied to an electronic device, and the chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute any possible implementation method as in the first aspect or the second aspect.

[0032] It is understandable that the electronic device provided in the third aspect, the computer-readable storage medium provided in the fourth aspect, the computer program product provided in the fifth aspect, and the chip provided in the sixth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a schematic diagram of a virtual conference scenario provided by an embodiment of the present application;

[0034] Figure 2 is a schematic diagram of the architecture of a communication system 10 provided in an embodiment of the present application;

[0035] Figure 3 is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application;

[0036] Figure 4A is a flow chart of an audio processing method provided by an embodiment of the present application;

[0037] Figure 4B is a schematic diagram of the time delay between acoustic signals provided in an embodiment of the present application;

[0038] Figure 5 is a flowchart of another audio processing method provided by an embodiment of the present application;

[0039] Figure 6 It is a structural diagram of an electronic device 100 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in a "or" relationship.

[0041] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0042] In the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0043] Here we first introduce a scenario of using AR / VR technology for virtual meetings.

[0044] Figure 1 A schematic diagram of a virtual meeting scenario is shown as an example.

[0045] like Figure 1 As shown, the user can participate in the virtual meeting by wearing the electronic device 100. The electronic device 100 can be worn on the user's head. For example, the electronic device 100 can be a device that supports AR / VR technology, such as AR / VR glasses, AR / VR head-mounted display (head-mounted display, HMD), AR / VR all-in-one machine, etc. The specific type of the electronic device 100 is not limited in the embodiment of the present application.

[0046] Among them, all participants in the virtual meeting can wear devices of the same type as the electronic device 100 to access the virtual meeting. Some participants in the virtual meeting can be located in the same physical space in the real environment, and some participants can also be located in different physical spaces in the real environment. For example, in the real environment, some participants may be in the company's conference room, and some participants may be at home. This application does not limit the location of the participants in the virtual meeting in the real environment.

[0047] The electronic device 100 can display on the display screen Figure 1 User interface 210 is shown. As can be seen from user interface 210, electronic device 100 can use AR / VR technology to provide a virtual conference room and display one or more virtual images of participants in the virtual conference room. For example, virtual image 211, virtual image 212, virtual image 213, virtual image 214, etc. A virtual image can represent a participant in a virtual meeting.

[0048] In some embodiments, the character features of the avatar (such as the gender, hairstyle, clothing, etc. of the avatar) can be selected and set by the participant corresponding to the avatar. The seat position of the avatar in the virtual conference room can also be selected by the user. For example, the seat positions of avatar 211 and avatar 212 in the virtual conference room are adjacent. The seat position of avatar 213 is opposite to avatar 211.

[0049] In some embodiments, the electronic device 100 can present the scene of the virtual meeting according to the perspective of the virtual image of the user wearing the electronic device 100 in the virtual meeting room. For example, the seat selected by the user wearing the electronic device 100 in the virtual meeting room is located at Figure 1 The user interface 210 presents the virtual conference room scene that can be seen from the perspective opposite to the virtual image 214. This can give participants an immersive experience when participating in a virtual meeting.

[0050] In some embodiments, the electronic device 100 can perform spatial audio modeling based on the sound field environment information of the virtual conference room and the positional relationship of each participant in the virtual conference room, so that when the user hears the voice of a certain participant, the position of the participant in the hearing sense is consistent with the position of the participant seen in the virtual conference room. Among them, the sound field environment of the virtual conference room can be selected and set by the participants. For example, before conducting a virtual meeting, the participants can set the room size, room reflective material, reverberation time, number of seats and other information of the virtual conference room. The electronic device 100 can determine the sound field environment information of the virtual conference room based on the parameters set by the above users. The electronic device 100 performs spatial audio modeling and can determine the audio playback parameters used to play the voices of each participant in the virtual conference room, so as to achieve the effect of matching the sound heard by the user with the virtual conference room scene seen.

[0051] Exemplarily, the electronic device 100 receives audio 1 sent from the electronic device of the virtual image 211. The audio 1 may be the audio of the speech of the participant corresponding to the virtual image 211. The electronic device 100 may play the audio 1 using the audio playback parameters of the virtual image 211 determined based on the spatial audio modeling, so that the user can distinguish that the voice of the participant corresponding to the virtual image 211 is emitted from the left front. The electronic device 100 receives audio 2 sent from the electronic device of the virtual image 213. The audio 2 may be the audio of the speech of the participant corresponding to the virtual image 213. The electronic device 100 may play the audio 2 using the audio playback parameters of the virtual image 213 determined based on the spatial audio modeling, so that the user can distinguish that the voice of the participant corresponding to the virtual image 213 is emitted from the right front.

[0052] In some embodiments, the electronic device 100 adopts an ear-blocking design. The ear-blocking design may mean that the device for playing audio in the electronic device 100 will block the user's ears and isolate part of the ambient sound. For example, the electronic device 100 can play audio through in-ear headphones. In the case of adopting an ear-blocking design, the electronic device 100 can collect ambient sound and process the ambient sound through a transparent playback technology to generate a transparent sound. The electronic device 100 can play the transparent sound. The transparent sound can also be called transparent audio and other names. It can be understood that when the user's ears are blocked, the user's listening to the sound of his own speech may be affected. The above-mentioned ambient sound may include the sound of the user's own speech. The electronic device 100 can facilitate the user to listen to the sound of his own speech normally by playing the above-mentioned transparent sound. In addition, the electronic device 100 can send the collected ambient sound to the electronic devices worn by other participants in the virtual meeting so that other participants can hear the user's speech. In addition, in the virtual meeting scenario, when other participants speak, the electronic device worn by the speaker can send the collected sound signal to the electronic device 100. The electronic device 100 can receive and play the transmission sound of other participants' speeches. The transmission sound can also be called transmission audio.

[0053] It can be seen that in the case of the above-mentioned ear blocking design, the electronic device 100 can play both the transmitted sound and the transmitted sound.

[0054] When there are other participants near the electronic device 100, the ambient sound collected by the electronic device 100 may include the voice of the nearby participants. That is to say, when a participant near the electronic device 100 speaks, the user wearing the electronic device 100 will hear the voice of the participant from the transparent sound played by the electronic device 100, and will also hear the voice of the participant from the transmission sound played by the electronic device 100. There may be a time delay between the above-mentioned transparent sound and transmission sound. Therefore, when other participants near the electronic device 100 speak, the user wearing the electronic device 100 will hear the voice of the participant twice in succession. This will cause repeated sound interference to the user, affecting the user's conference experience.

[0055] In some embodiments, the electronic device 100 adopts a non-blocking ear design. The non-blocking ear design may mean that the device for playing audio in the electronic device 100 will not block the user's ears and will not isolate the ambient sound. For example, the electronic device 100 can play audio through bone conduction headphones. In the case of a non-blocking ear design, since the ambient sound is not isolated, the user can directly hear the ambient sound, so the electronic device 100 can play the transparent sound without using transparent playback technology. Among them, the electronic device 100 can send the collected ambient sound to the electronic devices worn by other participants in the virtual meeting so that other participants can hear the user's speech. In a virtual meeting scenario, when other participants speak, the electronic device worn by the speaker can send the collected sound signal to the electronic device 100. The electronic device 100 can receive and play the transmission sound of other participants' speeches.

[0056] It can be seen that in the case of the above-mentioned non-ear-blocking design, the electronic device 100 can only play the transmission sound.

[0057] When other participants speak near the electronic device 100, the voices of other participants can be directly transmitted to the ears of the user wearing the electronic device 100 through a medium such as air, and heard by the user wearing the electronic device 100. In addition, the user wearing the electronic device 100 can also hear the voice of the participant speaking from the transmission sound played by the electronic device 100. Among them, the time when the user hears the speaker's voice through the transmission sound is usually later than the time when the user directly hears the speaker's voice through the ambient sound. Therefore, in the case of a non-ear-blocking design and when other participants speak near the electronic device 100, the user wearing the electronic device 100 can also hear the voice of the participant speaking twice in succession. This will cause the problem of repeated sound interference to the user, affecting the user's conference experience.

[0058] For easier understanding, AR / VR technology is introduced here.

[0059] The electronic device 100 can use AR / VR technology to provide a virtual environment. Among them, the electronic device 100 can use AR / VR technology to render and display one or more virtual objects. The virtual object can be generated by the electronic device 100 itself using computer graphics technology, computer simulation technology, etc., or it can be generated by other electronic devices using computer graphics technology, computer simulation technology, etc. and sent to the electronic device 100. Other electronic devices can be, for example, servers, or mobile phones, computers, and other devices connected or paired with the electronic device 100. For example, the above-mentioned virtual environment can include a virtual conference room, a virtual cinema, a virtual supermarket, and the like. The above-mentioned virtual objects can also be referred to as virtual images, virtual elements, or virtual images. Virtual objects can be two-dimensional or three-dimensional. Virtual objects are fake objects that do not really exist in the physical world. Virtual objects can be virtual objects that imitate objects that exist in the real physical world, thereby bringing an immersive experience to users. The types of virtual objects can include virtual animals, virtual characters, virtual trees, virtual buildings, virtual labels, icons, pictures, or videos, and the like.

[0060] Different virtual objects may be located at different positions in the virtual environment. For example, virtual objects of participants may be located at different positions in the virtual conference room. The electronic device 100 may determine the positional relationship of the virtual objects of participants in the virtual conference room.

[0061] The virtual environment is relative to the real environment. The real environment may refer to the real physical environment or physical space where the user and the electronic device 100 are currently located. Objects existing in the real environment are real objects. For example, real objects may include participants in a virtual meeting.

[0062] Based on AR / VR technology, the electronic device 100 can display virtual objects on a display screen or present a virtual environment to the user by projecting optical signals to the user's retina. The user can experience a three-dimensional virtual environment by wearing the electronic device 100. In this three-dimensional virtual environment, the virtual objects are equivalent to being mapped to the actual real environment, and the user feels or thinks that the virtual objects seem to exist in the actual real environment. For example, in the above Figure 1 In the scene shown, the user wearing the electronic device 100 can view the user interface 210. The user interface 210 presents a three-dimensional virtual conference room and virtual objects participating in the meeting in the virtual conference room to the user. Each virtual object participating in the meeting can correspond to a real object, that is, a participant in a real environment. In some embodiments, the electronic device 100 can map the virtual object to the real environment. In this way, the user can see both the real object in the real environment and the virtual object.

[0063] Figure 2The schematic diagram of the architecture of the communication system 10 provided in the present application is exemplified.

[0064] like Figure 2 As shown, the communication system 10 may include multiple electronic devices supporting AR / VR technology, such as electronic device 100, electronic device 101, and electronic device 102. Among them, electronic device 100, electronic device 101, and electronic device 102 can all be wearable devices that use AR / VR technology to provide a virtual environment to the user. For example, in the above Figure 1 In the virtual conference scenario shown, multiple participants can wear electronic device 100, electronic device 101, and electronic device 102 respectively to participate in the virtual conference.

[0065] A communication connection can be established between each electronic device in the communication system 10. For example, the electronic device 100, the electronic device 101, and the electronic device 102 can communicate with each other. The embodiment of the present application does not limit the method for establishing a communication connection between electronic devices in the communication system 10. For example, the communication connection may include but is not limited to a wireless fidelity (Wi-Fi) connection, a Bluetooth connection, and the like.

[0066] In a scenario where electronic devices such as electronic device 100, electronic device 101, and electronic device 102 are connected to the same virtual conference, these electronic devices can provide the same virtual conference room. Among them, electronic device 100 can send data such as virtual image information set by the user wearing electronic device 100 and selected location information to other electronic devices connected to the virtual conference, such as electronic device 101 and electronic device 102. The above virtual image information can be used to display the virtual image. For example, the virtual image information may include but is not limited to the gender, hairstyle, clothing and other information of the virtual image. After receiving the information sent by electronic device 100, other electronic devices can render the virtual image of the user wearing electronic device 100 at the corresponding seat in the virtual conference room. Electronic device 100 can also receive virtual image information and selected location information sent by other electronic devices connected to the virtual conference. For example, electronic device 100 receives virtual image information and selected location information sent by electronic device 101. Electronic device 100 can render the virtual image of the user wearing electronic device 101 at the corresponding seat in the virtual conference room. In this way, the user wearing electronic device 100 can watch the scene of the virtual image of each participant sitting at the corresponding position in the virtual conference room for a virtual meeting.

[0067] The electronic device 100 may also send the collected ambient sound to other electronic devices connected to the virtual conference so that other participants can listen to the speech of the user wearing the electronic device 100. The electronic device 100 may receive transmission sounds sent by other electronic devices connected to the virtual conference. For example, the electronic device 100 receives transmission sounds sent by the electronic device 101. The transmission sounds may include the speech of the participant wearing the electronic device 101. In this way, the user wearing the electronic device 100 can listen to the speeches of other participants in the virtual conference in real time.

[0068] In some embodiments, the user can control the movement of the virtual image through somatosensory interactions such as hand / arm movements, head movements, and / or eye movements. For example, the electronic device 100 can detect the posture of the user wearing the electronic device 100, and adjust the virtual image of the user according to the changes in the user's posture. For example, when the user wearing the electronic device 100 turns his head to the left, the electronic device 100 can adjust the head of the user's virtual image to turn left. Among them, the electronic device 100 can send the information that the virtual image's head is turning left to other electronic devices connected to the virtual meeting, so that other electronic devices can adjust the virtual image of the user wearing the electronic device 100.

[0069] Optionally, each electronic device in the communication system 10 can be used in conjunction with a handheld device. For example, the handheld device can include a controller, a gyro mouse, a stylus, or other handheld computing devices. The handheld device can be configured with a variety of sensors, such as an acceleration sensor, a gyro sensor, a magnetic sensor, etc. Electronic devices such as the electronic device 100 can detect and track information such as user posture changes through the handheld device.

[0070] The structure of the electronic device 100 is described below. The structures of other electronic devices in the communication system 10 (such as the electronic device 101, the electronic device 102, etc.) can refer to the structure of the electronic device 100.

[0071] Figure 3 The schematic diagram of the hardware structure of the electronic device 100 is exemplarily shown.

[0072] like Figure 3 As shown, the electronic device 100 may include a processor 110, a memory 120, a communication module 130, a sensor module 140, a button 150, an input / output interface 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a display device 180, a camera 190, a battery 1100, and the like.

[0073] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than those shown in the figure. For example, it may also include an infrared transceiver, an ultrasonic transceiver, a motor, and a flashlight, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0074] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a video processing unit (VPU) controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0075] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0076] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0077] In the present application, a computer program is stored in the memory, which is used to enable the controller or processor to implement the audio processing method of the present application through an interface or protocol.

[0078] The electronic device can implement wireless communication functions through the communication module 130. The communication module 130 may include an antenna, a wireless communication module, a mobile communication module, a modem processor, a baseband processor, etc. The communication module 130 may also include more or fewer devices.

[0079] Antennas are used to transmit and receive electromagnetic wave signals. An electronic device may include multiple antennas, each of which may be used to cover a single or multiple communication frequency bands. Different antennas may also be reused to improve antenna utilization. For example, an antenna may be reused as a diversity antenna for a wireless local area network. In other embodiments, an antenna may be used in conjunction with a tuning switch.

[0080] The mobile communication module can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to electronic devices. The mobile communication module may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module may receive electromagnetic waves through an antenna, filter, amplify, etc. the received electromagnetic waves, and transmit them to a modulation and demodulation processor for demodulation. The mobile communication module may also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna.

[0081] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker, etc.), or displays an image or video through a display device 180.

[0082] The wireless communication module can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module can be one or more devices integrating at least one communication processing module. The wireless communication module receives electromagnetic waves via an antenna, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module can also receive the signal to be sent from the processor 110, modulate the frequency of the signal, amplify it, and convert it into electromagnetic waves for radiation through the antenna.

[0083] The electronic device implements the display function through a GPU, a display device 180, and an application processor. The GPU is a microprocessor for image processing, connecting the display device 180 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0084] In the embodiment of the present application, the display device 180 may be used to present one or more virtual objects, so that the electronic device 100 provides a scene of a virtual environment for the user. The display device 180 may present virtual objects in one or more of the following ways:

[0085] 1. In some embodiments, the display device 180 is a display screen, and the display screen may include a display panel. The display panel of the display device 180 can be used to display virtual objects, thereby presenting a three-dimensional virtual environment to the user. The user can see the virtual object from the display panel and experience the virtual reality scene. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc.

[0086] 2. In some embodiments, the display device 180 may include an optical device for projecting an optical signal (e.g., a light beam) directly onto the user's retina. The user can directly see the virtual object through the optical signal projected by the optical device and feel the three-dimensional virtual environment. The optical device may be a micro projector, etc.

[0087] The number of display devices 180 in the electronic device may be two, corresponding to the two eyeballs of the user respectively. The contents displayed on the two display devices may be displayed independently. Different images may be displayed on the two display devices to enhance the stereoscopic effect of the image. In some possible embodiments, the number of display devices 180 in the electronic device may also be one, corresponding to both eyeballs of the user.

[0088] The electronic device can realize the shooting function through the ISP, the camera 190, the video codec, the GPU, the display device 180 and the application processor.

[0089] The ISP is used to process data fed back by the camera 190. For example, when taking a photo, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, and the light signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye.

[0090] The camera 190 is used to capture still images or videos. In some embodiments, the electronic device may include 1 or N cameras 190, where N is a positive integer greater than 1.

[0091] In some embodiments, the camera 190 may capture hand images or body images of the user, and the processor 110 may be used to analyze the images captured by the camera 190 to identify hand movements or body movements input by the user.

[0092] In some embodiments, the camera 190 can be used in conjunction with an infrared device (such as an infrared transmitter) to detect the user's eye movements, such as eye gaze direction, blinking operations, gazing operations, etc., thereby achieving eye tracking.

[0093] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0094] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and continuously self-learn. NPU can realize applications such as intelligent cognition of electronic devices, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0095] The memory 120 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the memory 120. The memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application program (such as VR / AR / MR application) required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area may store data (such as audio data, virtual image data, etc.) created during the use of the electronic device. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0096] The electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the microphone 170C, the headphone interface 170D, and the application processor, etc. For example, collecting ambient sound, playing audio, etc.

[0097] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module can also be used to encode and decode audio signals. Speaker 170A, also known as "speaker", is used to convert audio electrical signals into sound signals. In some embodiments, the electronic device 100 may include a speaker array consisting of multiple speakers. Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. The electronic device can be provided with at least one microphone 140. In some embodiments, the electronic device can be provided with multiple microphones 170C to realize functions such as sound source positioning. The headphone jack 170D is used to connect wired headphones.

[0098] In some embodiments, the electronic device 100 may use a device designed to block ears to play audio. For example, the electronic device 100 includes headphones. The headphones block the user's ears when worn. In other embodiments, the electronic device 100 may use a device designed not to block ears to play audio. For example, the electronic device 100 includes headphones. The headphones do not block the user's ears when worn.

[0099] In some embodiments, the electronic device may include one or more buttons 150 that can control the electronic device and provide the user with access to functions on the electronic device. In some embodiments, the electronic device may include an input / output interface 160 that can connect other devices to the electronic device through suitable components. Components may include, for example, audio / video jacks, data connectors, etc.

[0100] The sensor module 140 may include various sensors, for example, a proximity light sensor, a distance sensor, a gyro sensor, an ambient light sensor, an acceleration sensor, a temperature sensor, a magnetic sensor, a bone conduction sensor, a fingerprint sensor, and the like.

[0101] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture, etc., and this application does not limit this. OS, Or other operating systems.

[0102] Based on the above-mentioned virtual conference scenario and electronic device 100, the present application provides an audio processing method to solve the problem of repeated sound interference in virtual conferences and improve the user's conference experience.

[0103] When accessing a virtual conference, the electronic device 100 can determine whether the electronic device 100 is in the same physical space as other electronic devices accessing the virtual conference. If the electronic device 100 is not in the same physical space as other electronic devices accessing the virtual conference, the electronic device 100 can play the transmission sound after receiving the transmission sound sent by other electronic devices. If the electronic device 100 is in the same physical space as other electronic devices accessing the virtual conference, the electronic device 100 can not play the transmission sound, or mix the transmission sound and the transparent sound before playing the mixed audio.

[0104] Among them, when the electronic device 100 adopts an ear-blocking design, the electronic device 100 can generate transparent sound based on the collected ambient sound. The electronic device 100 can detect the distance between the electronic device 100 and the electronic device 100 that is located in the same physical space as the electronic device 100 among other electronic devices connected to the virtual conference. If it is far away from the electronic device 100, the electronic device 100 can use the transmission sound sent from the electronic device at a longer distance as the main sound source and the transparent sound as the non-main sound source for mixing. If it is close to the electronic device 100, the electronic device 100 can use the transparent sound as the main sound source and the transmission sound sent from the electronic device at a closer distance as the non-main sound source for mixing. In addition, before mixing the main sound source and the non-main sound source, the electronic device 100 can detect the time delay between the main sound source and the non-main sound source. If the time delay is large, the electronic device 100 can perform crosstalk filtering on the non-main sound source to filter out the effective components in the non-main sound source. The effective component may include the voice part in the non-main sound source. Then, the electronic device 100 can mix the main sound source and the filtered non-main sound source and play the mixed audio. If the delay is small, the electronic device 100 directly mixes the main sound source and the non-main sound source.

[0105] It is understandable that if other electronic devices are in the same physical space as the electronic device 100, when a participant wearing other electronic devices speaks, the transmitted sound played by the electronic device 100 may include the speeches of other participants. In the present application, the electronic device 100 can mix the transmitted sound with the transmission sound sent from an electronic device in the same physical space as the electronic device 100 to ensure that when a participant around the user speaks, the user only listens to the voice of the participant once. In addition, when the time delay between the transmitted sound and the transmission sound is large, the electronic device 100 performs crosstalk filtering and elimination, which can better avoid the sound of the same participant's speech appearing twice in the mixed audio. In this way, in a virtual conference scenario, the problem of repeated sound interference when multiple participants are in the same physical space can be solved.

[0106] When the electronic device 100 adopts a non-ear-blocking design, the electronic device 100 can detect the distance between the electronic device 100 and the electronic device 100 that is located in the same physical space among other electronic devices connected to the virtual conference. If the distance is far from the electronic device 100, the electronic device 100 can play the transmission sound sent from the electronic device at the far distance. If the distance is close to the electronic device 100, the electronic device 100 may not play the transmission sound sent from the electronic device at the close distance.

[0107] It is understandable that if other electronic devices are in the same physical space as the electronic device 100 and are far away, it may be difficult for the user wearing the electronic device 100 to clearly hear the speech of the participant who is far away through the ambient sound. Therefore, the electronic device 100 can play the transmission sound sent by the electronic device that is far away, so that the user can accurately hear the content of the speech of the participant who is far away in the same space. If other electronic devices are in the same physical space as the electronic device 100 and are close to each other, the user wearing the electronic device 100 can quickly and relatively clearly hear the sound of the speech of the participant who is close to the distance through the ambient sound. Therefore, the electronic device 100 may no longer play the transmission sound sent by the electronic device that is close to the distance, so as to avoid the user from repeatedly listening to the same speech of the participant who is close to both sides. This can solve the problem of repeated sound interference.

[0108] Figure 4A The flowchart of an audio processing method provided by the present application is exemplified.

[0109] Figure 4A The method shown may be a method for performing audio processing in an electronic device 100 using an ear-blocking design. Figure 4A As shown, the method may include steps S411 to S421. Among them:

[0110] S411 : The electronic device 100 determines the transmitted sound according to the collected sound signal, and receives the transmitted sound from the electronic device 101 .

[0111] When the electronic device 100 adopts an ear-blocking design, the electronic device 100 can collect ambient sound (or ambient sound signal). The ambient sound may include a background sound part. When there is a person speaking in the environment where the electronic device 100 is located, the ambient sound may also include a human voice part. The human voice part may also be called speech. The electronic device 100 may use a transparent playback technology to process the collected ambient sound. The audio obtained through the above transparent playback process may be called transparent sound.

[0112] In some embodiments, the electronic device 100 can filter out part or all of the background sound in the ambient sound to generate transparent sound.

[0113] The above-mentioned sound transmission can help users hear the sounds of nearby people or objects even when their ears are blocked by the electronic device 100.

[0114] The transmission sound from the electronic device 101 may be audio collected by the electronic device 101 and transmitted through the communication connection between the electronic device 100 and the electronic device 101. In the case where the user wearing the electronic device 101 speaks, the transmission sound from the electronic device 101 may include the voice of the user wearing the electronic device 101 speaking.

[0115] Here, an example of electronic device 100 and electronic device 101 accessing the same virtual conference is used for explanation. Among them, more electronic devices can be connected to the virtual conference. That is, it is not limited to the participants wearing electronic device 100 and electronic device 101, and more participants can participate in the virtual conference. In addition to receiving the transmission sound from electronic device 101, electronic device 100 can also receive transmission sounds sent by more other electronic devices connected to the virtual conference. The situation in which electronic device 100 receives transmission sounds sent by other electronic devices can refer to the situation in which electronic device 100 receives transmission sounds sent by electronic device 101.

[0116] S412: The electronic device 100 determines whether the electronic device 100 and the electronic device 101 are in the same physical space.

[0117] Since both the electronic device 100 and the electronic device 101 are worn by the user, the position of the electronic device 100 can represent the position of the user wearing the electronic device 100 in the real environment, and the position of the electronic device 101 can represent the position of the user wearing the electronic device 101 in the real environment. In other words, the electronic device 100 and the electronic device 101 being in the same physical space can represent that the user wearing the electronic device 100 and the user wearing the electronic device 101 are in the same physical space. The electronic device 100 and the electronic device 101 not being in the same physical space can represent that the user wearing the electronic device 100 and the user of the electronic device 101 are not in the same physical space.

[0118] When two users are in the same physical space, especially when they are physically close to each other, the two users can hear each other's speech without transmitting sound.

[0119] In a possible implementation, the electronic device 100 and the electronic device 101 can determine whether the electronic device 100 and the electronic device 101 are in the same physical space by means of an internet protocol (IP) address. The IP address of the electronic device 100 and the IP address of the electronic device 101 can reflect whether the electronic device 100 and the electronic device 101 are connected to the same network. If the electronic device 100 and the electronic device 101 are connected to the same network, the electronic device 100 can determine that the electronic device 100 and the electronic device 101 are in the same physical space. If the electronic device 100 and the electronic device 101 are connected to different networks, the electronic device 100 can determine that the electronic device 100 and the electronic device 101 are not in the same physical space.

[0120] Exemplarily, when both the electronic device 100 and the electronic device 101 access the network through the same router, the electronic device 100 and the electronic device 101 access the same network. Among them, even if the user wearing the electronic device 100 and the user wearing the electronic device 101 are in different rooms, the electronic device 100 determines the electronic device 100 and the electronic device 101 as being in the same physical space. It can be understood that the signal coverage of a router is limited. Even if the user wearing the electronic device 100 and the user wearing the electronic device 101 are in different rooms, the actual physical distance between the two users is usually not too far apart. For example, the user wearing the electronic device 100 may be in the bedroom. The user wearing the electronic device 101 may be in the living room. Both the electronic device 100 and the electronic device 101 access the network through the router located in the living room.

[0121] When the electronic device 100 and the electronic device 101 access the network through different routers, the electronic device 100 and the electronic device 101 access different networks. At this time, the electronic device 100 can determine that it is not in the same physical space as the electronic device 101.

[0122] The above determination of whether electronic devices are in the same physical space by IP address is only an exemplary description of the present application and shall not constitute a limitation on the present application. The electronic device 100 may also determine whether the electronic device 100 is in the same physical space as the electronic device 101 by other methods.

[0123] If the electronic device 100 and the electronic device 101 are not in the same physical space, the electronic device 100 may execute the following step S413.

[0124] If the electronic device 100 and the electronic device 101 are in the same physical space, the electronic device 100 may execute the following step S414.

[0125] S413: The electronic device 100 plays the transparent sound and the transmitted sound.

[0126] When the electronic device 100 and the electronic device 101 are not in the same physical space, the user wearing the electronic device 100 cannot hear the voice of the user wearing the electronic device 101 through the ambient sound in the environment in which he is located. That is, the voice of the user wearing the electronic device 101 will not be included in the transparent sound determined by the electronic device 100. The speech of the user wearing the electronic device 101 will not cause repeated sound interference to the user wearing the electronic device 100. Therefore, the electronic device 100 can directly play the transmission sound from the electronic device 101. In addition, since the transparent sound and the transmission sound will not cause repeated sound interference, the electronic device 100 can also play the transparent sound.

[0127] Among them, the electronic device 100 can superimpose the transparent sound and the transmitted sound and then play them. For example, the transparent sound is a1(t). The transmitted sound from the electronic device 101 is a2(t). The electronic device 100 superimposes a1(t) and a2(t) to obtain a3(t). a3(t)=a1(t)+a2(t). The electronic device 100 can play the audio corresponding to a3(t). This application does not limit the above method of superimposing the transparent sound and the transmitted sound.

[0128] S414: The electronic device 100 determines whether the physical distance between the electronic device 100 and the electronic device 101 is greater than a distance threshold.

[0129] When the electronic device 100 and the electronic device 101 are in the same physical space, the user wearing the electronic device 100 may hear the voice of the user wearing the electronic device 101 speaking through the ambient sound in the environment in which he is located. That is, the voice of the user wearing the electronic device 101 speaking will be included in the transparent sound determined by the electronic device 100. The speech of the user wearing the electronic device 101 will cause repeated sound interference to the user wearing the electronic device 100. Therefore, the electronic device 100 can detect the physical distance between the electronic device 100 and the electronic device 101 to determine the sound quality of the speech of the nearby participant heard by the user wearing the electronic device 100 through the transparent sound. The better the sound quality of the speech of the nearby participant heard through the transparent sound, the more likely the transparent sound and the transmission sound will cause repeated sound interference to the user.

[0130] Among them, the greater the physical distance between the electronic device 100 and the electronic device 101, the greater the physical distance between the user wearing the electronic device 100 and the user wearing the electronic device 101, and the worse the sound quality of the user wearing the electronic device 100 listening to the speech of the user wearing the electronic device 101 through the transparent sound transmission. It can be understood that the sound in the environment is transmitted through a medium such as air, and will gradually attenuate during the transmission process. The farther the distance between two people, the more the sound of one person's speech is attenuated when it is transmitted to the other person in the environment, and the more difficult it is for the other person to hear clearly what the other person is saying. On the contrary, the closer the distance between two people, the less the sound of one person's speech is attenuated when it is transmitted to the other person in the environment and the shorter the propagation distance, the faster it can reach the other person, and the faster and more clearly the other person can hear what the other person is saying.

[0131] When the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold, the electronic device 100 may execute the following step S415.

[0132] When the physical distance between the electronic device 100 and the electronic device 101 is less than or equal to the distance threshold, the electronic device 100 may perform the following step S416.

[0133] The above distance threshold may be preset. The present application does not limit the value of the distance threshold. For example, the distance threshold may be 3 meters, 4 meters, 5 meters, etc.

[0134] In a possible implementation, the electronic device 100 can determine the physical distance between the electronic device 100 and the electronic device 101 according to the signal (such as a Wi-Fi signal or a Bluetooth signal, etc.) communicated between the electronic device 100 and the electronic device 101. Among them, the electronic device 100 and the electronic device 101 can send signals to each other. The electronic device 100 determines the physical distance between the electronic device 100 and the electronic device 101 according to the time when the signal is transmitted between the electronic device 100 and the electronic device 101.

[0135] In a possible implementation, the electronic device 100 may use sound source localization technology to determine the physical distance between the electronic device 100 and the electronic device 101. The electronic device 100 may collect ambient sound based on an audio collection device such as a microphone array, and identify the physical distance between one or more sound sources in the ambient sound and the electronic device 100. For example, the ambient sound collected by the electronic device 100 includes human voices, and the electronic device 100 may use sound source localization technology to determine the physical distance between the user who emits the human voice and the electronic device 100. In the case where the ambient sound collected by the electronic device 100 includes human voices, if the electronic device 100 receives a transmission sound from the electronic device 101, the electronic device 100 may perform a correlation judgment between the human voice in the ambient sound and the transmission sound from the electronic device 101 to determine whether the human voice in the ambient sound is related to the transmission sound of the electronic device 101. The human voice in the ambient sound is related to the transmission sound of the electronic device 101, which may indicate that the human voice in the ambient sound is emitted by the user wearing the electronic device 101. Therefore, the physical distance between the human voice source determined by the electronic device 100 using the sound source localization technology and the electronic device 100 is the physical distance between the electronic device 100 and the electronic device 101 .

[0136] In a possible implementation, the electronic device 100 can also determine the physical distance between the electronic device 100 and the electronic device 101 based on the image captured by the camera. The electronic device 100 may store facial feature data of the user wearing the electronic device 101. When the user wearing the electronic device 100 and the user wearing the electronic device 101 are in the same physical space, the electronic device 100 can capture an image containing the user wearing the electronic device 101. The electronic device 100 can determine the position of the user wearing the electronic device 101 in the image captured by the electronic device 100 based on the facial feature data of the user, and then calculate the physical distance between the user (that is, the electronic device 101) and the electronic device 100.

[0137] The above method is only an exemplary description of the present application and shall not be construed as limiting the present application. The electronic device 100 may also use other methods to determine the physical distance between the electronic device 100 and the electronic device 101 .

[0138] In some embodiments, the quality of the sound effect of the speech of the nearby participant heard by the user wearing the electronic device 100 through the transparent sound transmission is not limited to judging by the physical distance between the electronic devices. The electronic device 100 can also judge the quality of the sound effect of the speech of the nearby participant heard by the user wearing the electronic device 100 through the transparent sound transmission according to whether the parameters such as the loudness of the human voice in the ambient sound (or the transparent sound transmission) or the sound pickup quality meet the preset conditions. The above-mentioned sound pickup quality can include the balancing effect of different frequency signals in the collected ambient sound, which can be used to reflect the subjective hearing of the ambient sound. The better the sound pickup quality, the clearer the user can hear the sound in the ambient sound.

[0139] Exemplarily, when the ambient sound is collected, the electronic device 100 can separate the human voice from the ambient sound and detect the loudness of the human voice. If the loudness of the human voice is greater than the preset loudness, it can be indicated that the user who emits the human voice is close to the user wearing the electronic device 100. If the loudness of the human voice is less than or equal to the preset loudness, it can be indicated that the user who emits the human voice is far away from the user wearing the electronic device 100. The electronic device 100 can make a correlation judgment between the human voice in the ambient sound and the transmission sound from the electronic device 101 to determine whether the human voice in the ambient sound is related to the transmission sound of the electronic device 101. If the human voice in the ambient sound is related to the transmission sound of the electronic device 101 and the loudness of the human voice is greater than the preset loudness, the electronic device 100 can perform the following step S416. If the human voice in the ambient sound is related to the transmission sound of the electronic device 101 and the loudness of the human voice is less than or equal to the preset loudness, the electronic device 100 can perform the following step S415.

[0140] S415. The electronic device 100 determines the transmitted sound as the main sound source and determines the transmitted sound as the non-main sound source.

[0141] When the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold, the sound quality of the voice of the user wearing the electronic device 101 in the transmitted sound determined by the electronic device 100 is poor. Therefore, when the electronic device 100 mixes the transmitted sound and the transmitted sound, the transmitted sound is used as the main sound source and the transmitted sound is used as the non-main sound source. In this way, the user wearing the electronic device 100 can better listen to the content of the speech of the user wearing the electronic device 101.

[0142] Among them, the proportion of information from the main sound source in the mixed audio is higher than that from the non-main sound source. That is, when users listen to the mixed audio, the content they hear is more from the main sound source.

[0143] S416: The electronic device 100 determines the transmitted sound as the main sound source and the transmitted sound as the non-main sound source.

[0144] When the physical distance between the electronic device 100 and the electronic device 101 is less than or equal to the distance threshold, the sound quality of the voice of the user wearing the electronic device 101 in the transparent sound determined by the electronic device 100 is better. In addition, compared with the transmission sound from the electronic device 101, the user wearing the electronic device 100 usually first listens to the content of the speech of the user wearing the electronic device 101 through the transparent sound. Therefore, when the electronic device 100 mixes the transmission sound and the transparent sound, the transparent sound is used as the main sound source and the transmission sound is used as the non-main sound source.

[0145] S417: The electronic device 100 determines the time delay between the transmitted sound and the transmitted sound.

[0146] The transparent sound and the transmission sound are obtained by the electronic device 100 through different channels. There is usually a time delay between the transparent sound and the transmission sound. The time delay between the transparent sound and the transmission sound can indicate the time difference between the user hearing the speech of the user wearing the electronic device 101 through the transparent sound and the transmission sound. The above-mentioned transparent sound is determined and played to the user after the electronic device 100 collects the ambient sound. The above-mentioned transmission sound is collected by the electronic device 101, transmitted to the electronic device 100 through the communication network, and then played to the user by the electronic device 100. It takes a long time for the above-mentioned transmission sound to be transmitted in the communication network. Therefore, under normal circumstances, the user wearing the electronic device 100 will hear the speech of the user wearing the electronic device 101 through the transparent sound earlier than hearing the speech of the user wearing the electronic device 101 through the transmission sound.

[0147] In order to avoid repeated sounds in the mixed audio obtained by mixing the transparent sound and the transmitted sound, the electronic device 100 can detect the time delay between the transparent sound and the transmitted sound.

[0148] Figure 4B A schematic diagram showing the time delay between the transmitted sound and the transmitted sound is shown as an example.

[0149] like Figure 4B As shown, the acoustic signal 431 may be an acoustic signal of a transparent sound. The acoustic signal 432 may be an acoustic signal of a transmitted sound. The electronic device 100 may perform a correlation judgment on the acoustic signal 431 and the acoustic signal 432 to determine the acoustic signal corresponding to the speech of the user wearing the electronic device 101 in the transparent sound and the transmitted sound. Among them, the electronic device 100 may compare the correlation between the acoustic signal 431 and the acoustic signal 432 starting at different times. For example, the electronic device 100 detects that the part of the acoustic signal 431 starting at time t1 has the highest correlation with the part of the acoustic signal 432 starting at time t1+Δt1. Then, the electronic device 100 may determine that the time delay between the transparent sound and the transmitted sound is Δt1.

[0150] The embodiment of the present application does not limit the delay estimation method of the electronic device 100. For example, the electronic device 100 may use a delay estimation method combining fractional low-order covariance (FLOC) and linear prediction (LPC).

[0151] S418: The electronic device 100 determines whether the delay is greater than a delay threshold.

[0152] It can be seen from the Haas effect that if the time difference between the sound waves of two identical sound sources reaching the listener is within 35 milliseconds (ms), the listener cannot distinguish the two sound sources, and the two identical sound sources will not cause repeated sound interference to the listener. Therefore, the electronic device 100 can determine whether the delay between the main sound source and the non-main sound source is greater than the delay threshold. The above delay threshold can be 35ms, 34ms, 36ms, etc. The embodiment of the present application does not limit the specific value of the delay threshold.

[0153] If the delay between the main sound source and the non-main sound source is less than or equal to the delay threshold, the electronic device 100 may perform the following step S419. If the delay between the main sound source and the non-main sound source is greater than the delay threshold, the electronic device 100 may perform the following step S420.

[0154] S419: The electronic device 100 mixes the main sound source and the non-main sound source to obtain and play the mixed audio.

[0155] In some embodiments, the electronic device 100 may use a mixing method such as an adaptive mixing weighting algorithm or an automatic alignment algorithm to mix the main sound source and the non-main sound source. The embodiment of the present application does not limit the mixing method used by the electronic device 100.

[0156] S420: The electronic device 100 performs crosstalk filtering on the non-primary sound source to filter out effective components in the non-primary sound source.

[0157] S421: The electronic device 100 mixes the main sound source and the filtered non-main sound source to obtain and play the mixed audio.

[0158] When the delay between the main sound source and the non-main sound source is greater than the delay threshold, the electronic device 100 may filter out the effective components in the non-main sound source before mixing. The effective components in the non-main sound source may include the human voice part in the non-main sound source. This can avoid the voice of the same participant speaking twice in the mixed audio, causing repeated sound interference to the user.

[0159] In some embodiments, the electronic device 100 stores audio playback parameters for playing the audio of the speech of the user wearing the electronic device 101. The audio playback parameters can be determined by the electronic device 100 through spatial audio modeling based on the positional relationship between the user wearing the electronic device 101 and the user wearing the electronic device 100 in the virtual conference room. The electronic device 100 uses the audio parameters to play the mixed audio determined in the above step S419 or step S421, which can bring an immersive experience to the users participating in the virtual meeting. Among them, when the user hears a participant speaking, the source of the sound can be determined from the auditory sense to match the position of the virtual image of this participant seen in the virtual conference room.

[0160] In some embodiments, when the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold, the electronic device 100 may not need to distinguish between the main sound source and the non-main sound source. Among them, the electronic device 100 can play the transparent sound and the transmission sound after superimposing them. For details, please refer to the aforementioned step S413. It can be understood that if the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold, the electronic device 100 can only collect a small amount of sound of the user wearing the electronic device 101, and it may not even collect the sound of the user wearing the electronic device 101. Then, the transparent sound almost does not contain the sound of the user wearing the electronic device 101. Therefore, the transparent sound and the transmission sound from the electronic device 101 will not cause repeated sound interference to the user wearing the electronic device 100. The electronic device 100 can play the audio formed by the direct superposition of the transparent sound and the transmission sound.

[0161] In some embodiments, the above steps S415 to S421 are optional. Among them, when the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold, the electronic device 100 may only play the transmission sound when receiving the transmission sound from the electronic device 101, but not the transparent sound. When the physical distance between the electronic device 100 and the electronic device 101 is less than or equal to the distance threshold, the electronic device 100 may play the transparent sound, but not the transmission sound from the electronic device 101. This can also avoid the repeated sound interference to the user caused by playing the transparent sound and the transmission sound at the same time.

[0162] In some embodiments, when the electronic device 100 and the electronic device 101 are located in the same physical space, especially when the physical distance between the electronic device 100 and the electronic device 101 is less than a distance threshold, if the user wearing the electronic device 100 speaks, the transmitted sound generated by the electronic device 100 includes the sound of the user wearing the electronic device 100 speaking, and the electronic device 101 can collect the sound of the user wearing the electronic device 100 speaking. That is, the transmitted sound from the electronic device 101 also includes the sound of the user wearing the electronic device 100 speaking. The time delay between the transmitted sound and the transmitted sound in the above step S417 can also represent the time difference for the user wearing the electronic device 100 to hear his own voice through the transmitted sound and the transmitted sound. The electronic device 100 generates a signal based on the transmitted sound. Figure 4A The method for playing audio can not only reduce the situation where users hear other users' speeches repeatedly, but also reduce the situation where users hear their own speeches repeatedly.

[0163] In some embodiments, if the electronic device 100 and the electronic device 101 are in the same physical space and the physical distance between the two is greater than the distance threshold, the electronic device 100 can determine whether the loudness of the human voice part in the transparent sound is louder or the loudness of the human voice part in the transmitted sound is louder, or the electronic device 100 can determine which of the loudness of the relevant parts of the transparent sound and the transmitted sound is louder. For example, if the loudness of the human voice part in the transmitted sound is louder, the electronic device 100 can execute the above step S415, determine the transmitted sound as the main sound source, and determine the transparent sound as the non-main sound source. If the loudness of the human voice part in the transparent sound is louder, the electronic device 100 can execute the above step S416, determine the transparent sound as the main sound source, and determine the transmitted sound as the non-main sound source. It can be understood that the repeated sound interference in the virtual meeting may be that the user will repeatedly hear the sound of his own voice, or the user may hear the sound of other users speaking. When the electronic device 100 and the electronic device 101 are in the same physical space and far apart, if the user wearing the electronic device 101 speaks, the transmission sound and the transparent sound may both contain the sound of the user wearing the electronic device 101 speaking. However, the sound quality of the user wearing the electronic device 101 speaking in the transmission sound is better. Therefore, the electronic device 100 uses the transmission sound as the main sound source and the transparent sound as the non-main sound source, which can not only allow the user wearing the electronic device 100 to clearly hear the sound of the user wearing the electronic device 101 speaking, but also reduce the interference of repeated sounds. If the user wearing the electronic device 100 speaks, the transmission sound and the transparent sound may both contain the sound of the user wearing the electronic device 100 speaking. However, the sound quality of the user wearing the electronic device 100 speaking in the transparent sound is better. Therefore, the electronic device 100 uses the transparent sound as the main sound source and the transmission sound as the non-main sound source, which can allow the user wearing the electronic device 100 to clearly hear the sound of his own speech and reduce the interference of repeated sounds. It can be seen that in a virtual meeting, each participant usually speaks alternately. When electronic device 100 and electronic device 101 are in the same physical space and are far apart, electronic device 100 can adjust the main sound source and non-main sound source in real time according to changes in the speaker, while ensuring that the user wearing electronic device 100 can clearly hear the speaker's speech, reducing repeated sound interference and improving the user's conference experience.

[0164] By the above Figure 4AIt can be seen from the method shown that when the audio to be played in the electronic device 100 includes transparent sound and transmission sound, the electronic device 100 can determine whether the transparent sound and transmission sound will cause repeated sound interference to the user wearing the electronic device 100, and the degree of interference, to determine the processing and playing method of the transparent sound and transmission sound. In the case where the transparent sound and transmission sound will cause repeated sound interference to the user (that is, the electronic device 100 and the electronic device 101 are in the same physical space), the electronic device 100 can determine one of the transparent sound and the transmission sound as the main sound source according to the degree of interference, and the other as the non-main sound source, and then mix the sound. For example, if the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold, the degree of interference is small. If the physical distance between the electronic device 100 and the electronic device 101 is less than or equal to the distance threshold, the degree of interference is large. In addition, the electronic device 100 can also perform crosstalk filtering and elimination on the non-main sound source during mixing. This can better reduce repeated sound interference and enhance the user's experience of participating in virtual meetings.

[0165] Figure 5 The flowchart of another audio processing method provided by the present application is exemplified.

[0166] Figure 5 The method shown may be a method for performing audio processing in an electronic device 100 that adopts a non-ear-blocking design. Figure 5 As shown, the method may include steps S511 to S516. Among them:

[0167] S511 : The electronic device 100 receives a transmission sound from the electronic device 101 .

[0168] When the electronic device 100 adopts a non-blocking ear design, the ears of the user wearing the electronic device 100 are not blocked, and the user can directly hear the sounds in the environment (such as his own voice, the voice of other people talking nearby, etc.). Therefore, the electronic device 100 does not need to generate and play the transmitted sound. The transmitted sound from the electronic device 101 can refer to the aforementioned Figure 4A The introduction of step S411 is shown.

[0169] S512: The electronic device 100 determines whether the electronic device 100 and the electronic device 101 are in the same physical space.

[0170] Step S512 may refer to the aforementioned Figure 4A The introduction of step S412 is shown.

[0171] S513: The electronic device 100 plays a transmission sound.

[0172] When the electronic device 100 and the electronic device 101 are not in the same physical space, the user wearing the electronic device 100 cannot hear the voice of the user wearing the electronic device 101 through the ambient sound in the environment in which he is located. That is, the user wearing the electronic device 100 can only hear the voice of the user wearing the electronic device 101 through the transmission sound. Therefore, the electronic device 100 can directly play the transmission sound from the electronic device 101.

[0173] S514: The electronic device 100 determines whether the physical distance between the electronic device 100 and the electronic device 101 is greater than a distance threshold.

[0174] Step S514 can refer to the above Figure 4A The introduction of step S414 is shown.

[0175] S515: The electronic device 100 plays a transmission sound.

[0176] When the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold, the user wearing the electronic device 100 is far away from the user wearing the electronic device 101. When the sound of the user wearing the electronic device 101 is transmitted through a medium such as air to the user wearing the electronic device 100, it is greatly attenuated. The user wearing the electronic device 100 may find it difficult to hear the content of the user wearing the electronic device 101 speaking from the ambient sound. Therefore, the electronic device 100 can play the transmission sound.

[0177] S516: The electronic device 100 does not play the transmission sound.

[0178] When the physical distance between the electronic device 100 and the electronic device 101 is less than or equal to the distance threshold, the user wearing the electronic device 100 is relatively close to the user wearing the electronic device 101. When the sound of the speech of the user wearing the electronic device 101 is transmitted through a medium such as air to the user wearing the electronic device 100, the attenuation is small. The user wearing the electronic device 100 can clearly hear the content of the speech of the user wearing the electronic device 101 from the ambient sound. At this time, if the electronic device 100 plays the transmission sound, the user may hear the speech of the user wearing the electronic device 101 again from the transmission sound in addition to hearing the speech of the user wearing the electronic device 101 from the ambient sound. That is, it will cause the problem of repeated sound interference to the user wearing the electronic device 100. Therefore, after the electronic device 100 receives the transmission sound from the electronic device 101, it is not necessary to play the transmission sound.

[0179] By the above Figure 5It can be seen from the method shown that, when a user can hear other nearby users speaking through ambient sound, the electronic device 100 can determine whether the received transmission sound will cause repeated sound interference to the user wearing the electronic device 100, to determine whether to play the transmission sound. For example, when the electronic device 100 and the electronic device 101 are in the same physical space and the physical distance between the two is close, the transmission sound from the electronic device 101 will cause greater repeated sound interference to the user. The electronic device 100 may not play the transmission sound from the electronic device 101. This can avoid the problem of repeated sound interference and enhance the user's experience of participating in virtual meetings.

[0180] Figure 6 The structure of an electronic device 100 provided in an embodiment of the present application is exemplarily shown.

[0181] like Figure 6 As shown, the electronic device 100 may include an audio input module 611 , a communication module 612 , an audio output module 613 , an audio processing module 614 , an acoustic signal selection module 615 , and a delay estimation module 616 .

[0182] The audio input module 611 may be used to collect ambient sound. For example, the audio input module 611 may include a microphone.

[0183] The communication module 612 can be used for the electronic device 100 to communicate with other electronic devices (such as the electronic device 101). The communication module 612 can refer to the aforementioned Figure 3 For example, the electronic device 100 can access the virtual conference through the communication module 612 and receive the transmission sound and other data sent by other electronic devices (such as the electronic device 101) that access the virtual conference. The electronic device 100 can also send the ambient sound and other data collected by the audio input module 611 to other electronic devices that access the virtual conference through the communication module 612.

[0184] The audio output module 613 may be used to play audio. For example, the audio output module 613 may include a speaker.

[0185] The audio processing module 614 may be used to perform crosstalk filtering, audio mixing, etc. on the audio. In some embodiments, the audio processing module 614 may also be used to process the ambient sound collected by the audio input module 611 to generate transparent sound.

[0186] The sound signal selection module 615 can be used to select audio to determine the audio that the electronic device 100 needs to play. When the electronic device 100 accesses a virtual meeting, the sound signal selection module 615 can determine whether one or more audios in the electronic device 100 will cause repeated sound interference to the user when playing, and then select appropriate audio to play to reduce repeated sound interference.

[0187] The time delay estimation module 616 may be used to estimate the time delay between two acoustic signals.

[0188] In some embodiments, the electronic device 100 adopts an ear-blocking design. The electronic device 100 contains both transmitted sound and transmitted sound. Here, the transmitted sound from the electronic device 101 is used as an example for explanation. The sound signal selection module 615 can determine whether the electronic device 100 and the electronic device 101 are in the same physical space. For a specific method introduction, please refer to the aforementioned Figure 4A Step S412 shown. If the electronic device 100 and the electronic device 101 are not in the same physical space, the transparent sound and the transmission sound will not cause repeated sound interference to the user when they are played. Therefore, the sound signal selection module 615 can instruct the audio output module 613 to play the transparent sound and the transmission sound. In this way, the user wearing the electronic device 100 can listen to the content of the speech of the user wearing the electronic device 101 through the transmission sound, and can listen to the sound of his own speech and other nearby sounds through the transparent sound. Since the transparent sound does not contain the sound of the user speaking of the electronic device 101, the audio output module 613 plays the transparent sound and the transmission sound from the electronic device 101 without causing repeated sound interference to the user.

[0189] If the electronic device 100 and the electronic device 101 are in the same physical space, the transmitted sound and the transmitted sound may cause repeated sound interference to the user during playback, and the degree of interference depends on the physical distance between the electronic device 100 and the electronic device 101. The greater the physical distance between the electronic device 100 and the electronic device 101, the smaller the degree of repeated sound interference. Conversely, the greater the degree of repeated sound interference. The acoustic signal selection module 615 can determine whether the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold. For a detailed description of the method, please refer to the aforementioned Figure 4A Step S414 is shown.

[0190] If the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold, the sound signal selection module 615 can determine the transmission sound as the main sound source and determine the transparent sound as the non-main sound source. If the physical distance between the electronic device 100 and the electronic device 101 is less than or equal to the distance threshold, the sound signal selection module 615 can determine the transparent sound as the main sound source and determine the transmission sound as the non-main sound source. Then, the sound signal selection module 615 can instruct the audio processing module 614 to mix the main sound source and the non-main sound source.

[0191] The audio processing module 614 may first instruct the delay estimation module 616 to determine the delay between the main sound source and the non-main sound source (i.e., the transmitted sound and the transmitted sound). If the delay between the main sound source and the non-main sound source is greater than the delay threshold, the audio processing module 614 may perform crosstalk filtering on the non-main sound source to filter out the effective components in the non-main sound source, and then mix the main sound source and the filtered non-main sound source to obtain a mixed audio (refer to the aforementioned Figure 4A If the time delay between the main sound source and the non-main sound source is less than the time delay threshold, the audio processing module 614 can mix the main sound source and the non-main sound source to obtain mixed audio.

[0192] The audio processing module 614 may instruct the audio output module 613 to play the mixed audio.

[0193] In some embodiments, if the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold, the acoustic signal selection module 615 can select the transmission sound for playback. Among them, the acoustic signal selection module 615 can instruct the audio output module 613 to play the transmission sound instead of the transparent sound. If the physical distance between the electronic device 100 and the electronic device 101 is less than the distance threshold, the acoustic signal selection module 615 can select the transparent sound for playback. Among them, the acoustic signal selection module 615 can instruct the audio output module 613 to play the transparent sound instead of the transmission sound.

[0194] In some embodiments, the electronic device 100 adopts a non-blocking ear design. The electronic device 100 contains transmission sound but does not contain penetrating sound. Here, the transmission sound from the electronic device 101 is used as an example for explanation. The sound signal selection module 615 can determine whether the electronic device 100 and the electronic device 101 are in the same physical space. For a specific method introduction, please refer to the aforementioned Figure 4A If the electronic device 100 and the electronic device 101 are not in the same physical space, the user wearing the electronic device 100 can only hear the content of the speech of the user wearing the electronic device 101 through the transmission sound. Therefore, the sound signal selection module 615 can instruct the audio output module 613 to play the transmission sound.

[0195] If the electronic device 100 and the electronic device 101 are in the same physical space, the transmitted sound and the ambient sound transmitted through a medium such as air and heard by the user may cause repeated sound interference to the user, and the degree of interference depends on the physical distance between the electronic device 100 and the electronic device 101. The greater the physical distance between the electronic device 100 and the electronic device 101, the smaller the degree of repeated sound interference. Conversely, the greater the degree of repeated sound interference. The acoustic signal selection module 615 can determine whether the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold. For a detailed description of the method, please refer to the aforementioned Figure 4A Step S414 is shown.

[0196] If the physical distance between the electronic device 100 and the electronic device 101 is greater than the distance threshold, the sound signal selection module 615 can select the transmission sound to play. Among them, the sound signal selection module 615 can instruct the audio output module 613 to play the transmission sound. If the physical distance between the electronic device 100 and the electronic device 101 is less than or equal to the distance threshold, the transmission sound from the electronic device 101 does not need to be played. That is, the audio output module 613 does not need to play the transmission sound from the electronic device 101.

[0197] It should be noted that Figure 6 The structure shown is only an exemplary description of the present application and should not be construed as limiting the present application. The electronic device 100 may also include more or fewer modules, or combine some modules, or separate some modules.

[0198] Depend on Figure 6 As can be seen from the structure shown, the electronic device 100 can select appropriate audio for playback based on the sound signal selection module, which can solve the problem of multiple participants hearing the same content twice when holding a virtual meeting in the same physical space, thereby improving the user experience of participating in virtual meetings.

[0199] It should be noted that, without causing any contradiction or conflict, any feature in any embodiment of the present application, or any part of any feature, can be combined, and the combined technical solution is also within the scope of the embodiments of the present application.

[0200] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An audio processing method, characterized in that: The method comprises: The first device receives the first audio from the second device, and generates the second audio according to the collected sound signal; When the first device and the second device are in the same physical space and the physical distance between the first device and the second device meets a first condition, the first device plays the second audio and does not play the first audio, or the first device detects a first delay between the first audio and the second audio, and when the first delay is greater than a delay threshold, filters out the human voice part in the first audio to obtain a third audio, and plays the second audio and the third audio.

2. The method according to claim 1, characterized in that: The first device and the second device are in the same physical space, specifically including: the IP address of the first device and the IP address of the second device indicate that the first device and the second device are connected to the same network; The physical distance between the first device and the second device satisfies a first condition, which specifically includes: the physical distance between the first device and the second device is less than a first distance, and / or the loudness of an acoustic signal related to the second audio in the first audio is greater than the first loudness.

3. The method according to claim 1 or 2, characterized in that: The first device and the second device are devices connected to the same virtual conference, wherein the first device and the second device are worn by different users, the device for playing audio in the first device adopts an ear-blocking design, and the second audio is transparent sound.

4. The method according to any one of claims 1 to 3, characterized in that The playing of the second audio and the third audio specifically includes: The first device uses the second audio as a main sound source and the third audio as a non-main sound source, and mixes and plays the second audio and the third audio.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: When the first device and the second device are in the same physical space, the physical distance between the first device and the second device satisfies the first condition, and the first delay is less than or equal to the delay threshold, the first device uses the second audio as a main sound source and the first audio as a non-main sound source, and mixes and plays the first audio and the second audio.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: When the first device and the second device are in the same physical space and the physical distance between the first device and the second device does not meet the first condition, the first device plays the first audio and does not play the second audio; or, when the first delay is greater than the delay threshold, the first electronic device filters out the human voice part in the second audio to obtain a fourth audio, and plays the first audio and the fourth audio.

7. The method according to claim 6, characterized in that The playing of the first audio and the fourth audio specifically includes: The first device uses the first audio as a main sound source and the fourth audio as a non-main sound source, and mixes and plays the first audio and the fourth audio.

8. The method according to claim 6 or 7, characterized in that: The method further comprises: When the first device and the second device are in the same physical space, the physical distance between the first device and the second device does not meet the first condition, and the first delay is less than or equal to the delay threshold, the first device uses the first audio as a main sound source and the second audio as a non-main sound source, and mixes and plays the first audio and the second audio.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: When the first device and the second device are not in the same physical space, the first device plays the first audio and the second audio.

10. An audio processing method, characterized in that: The method comprises: The first device receives the first audio from the second device; When the first device and the second device are in the same physical space and the physical distance between the first device and the second device meets a first condition, the first device does not play the first audio.

11. The method according to claim 10, characterized in that The first device and the second device are in the same physical space, specifically including: the IP address of the first device and the IP address of the second device indicate that the first device and the second device are connected to the same network; The physical distance between the first device and the second device satisfies a first condition, which specifically includes: the physical distance between the first device and the second device is less than a first distance, and / or the loudness of an acoustic signal related to the second audio in the first audio is greater than the first loudness.

12. The method according to claim 10 or 11, characterized in that: The first device and the second device are devices connected to the same virtual conference, wherein the first device and the second device are worn by different users, and the device for playing audio in the first device adopts a non-ear-blocking design.

13. The method according to any one of claims 10 to 12, characterized in that: The method further comprises: When the first device and the second device are in the same physical space and the physical distance between the first device and the second device does not satisfy the first condition, the first device plays the first audio; When the first device and the second device are not in the same physical space, the first device plays the first audio.

14. An electronic device, characterized in that: The electronic device comprises an audio input module, an audio output module, a memory and a processor, wherein the audio input module is used to collect sound signals; the audio output module is used to play audio; the memory is used to store computer programs; and the processor is used to call the computer program so that the electronic device executes the method described in any one of claims 1 to 9 or claims 10 to 13.

15. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 9 or claims 10 to 13.

16. A computer program product, characterized in that The computer program product includes computer instructions. When the computer instructions are run on an electronic device, the electronic device executes the method according to any one of claims 1 to 9 or claims 10 to 13.

Images