Video conference method and device, equipment and storage medium

Abstract

The invention relates to the technical field of video conferences, and provides a video conference method, device and equipment and a storage medium, and the method comprises the steps: displaying a video picture of a video conference, the video picture comprising at least one speaker; acquiring a sight focus of a user wearing the near-to-eye display equipment on the video picture; and according to the sight focus, determining target sound data from preset sound data of at least one speaker corresponding to the video conference, and outputting the target sound data so as to improve the broadcasting effect of the target sound data of the video conference.

Description

Technical Field

[0001] The present application relates to the field of video conferencing technology, and in particular to a video conferencing method, apparatus, device and storage medium. Background Art

[0002] Currently, when a user participates in a video conference, if there are many speakers speaking at the same time in the video conference, it is easy for the user to not be able to hear the voice data of the speaker he or she is paying attention to, resulting in poor broadcasting effect of the voice data of the video conference. Summary of the invention

[0003] The main purpose of the present application is to provide a video conferencing method, apparatus, equipment and storage medium, aiming to solve the technical problem of poor broadcasting effect of video conference sound data caused by the user being unable to hear the speech data of the speaker he is concerned about when there are many speakers speaking at the same time in the video conference.

[0004] In a first aspect, the present application provides a video conferencing method, applied to a near-eye display device, the video conferencing method comprising:

[0005] Displaying a video screen of a video conference, wherein the video screen includes at least one speaker;

[0006] Obtaining a visual focus of a user wearing a near-eye display device on the video screen;

[0007] According to the sight focus, target sound data is determined from preset sound data of at least one speaker corresponding to the video conference, and the target sound data is output.

[0008] In a second aspect, the present application provides a video conferencing device, the video conferencing device comprising:

[0009] A screen display module, used to display a video screen of a video conference, wherein the video screen includes at least one speaker;

[0010] A focus acquisition module, used to acquire the sight focus of a user wearing a near-eye display device on the video screen;

[0011] The target determination module is used to determine target sound data from preset sound data of at least one speaker corresponding to the video conference according to the sight focus, and output the target sound data.

[0012] In a third aspect, the present application provides a near-eye display device, the near-eye display device comprising a memory and a processor;

[0013] The memory is used to store computer programs;

[0014] The processor is used to execute the computer program and implement the steps of the video conferencing method as described above when executing the computer program.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the video conferencing method as described above are implemented.

[0016] The present application provides a video conferencing method, apparatus, device and storage medium. The video conferencing method includes: displaying a video screen of a video conference, where the video screen includes at least one speaker; obtaining the visual focus of a user wearing a near-eye display device on the video screen; and determining target sound data from preset sound data of at least one speaker corresponding to the video conference based on the visual focus, and outputting the target sound data.

[0017] In the process of displaying the video screen of a video conference, the near-eye display device can determine the speaker that the user is paying attention to from at least one speaker included in the video screen according to the user's line of sight focus on the video screen. For example, the speaker on whom the user's line of sight focuses in the video screen is the speaker that the user is paying attention to. Based on this, the near-eye display device can obtain the target sound data of the speaker that the user is paying attention to from the preset sound data of at least one speaker corresponding to the video conference. Accordingly, the near-eye display device can output the target sound data. If the target sound data is broadcast, the user can more conveniently hear the target sound data of the speaker that he or she is paying attention to, which is conducive to improving the broadcast effect of the near-eye display device on the target sound data of the video conference. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a flowchart of a video conferencing method provided in an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a video screen of a video conference involved in an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a video screen of a video conference involved in yet another embodiment of the present application;

[0022] Figure 4 is a schematic block diagram of a video conferencing device provided in an embodiment of the present application;

[0023] Figure 5 It is a schematic block diagram of a near-eye display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0026] Embodiments of the present application provide an interaction method, apparatus, device, and storage medium for a near-eye display device. The interaction method of the near-eye display device can be applied to a near-eye display device. A near-eye display device may include an extended reality (XR) device. XR devices may include augmented reality (AR) glasses, virtual reality (VR) glasses, mixed reality (MR) glasses, AR helmets, VR helmets, MR helmets, and the like, without limitation.

[0027] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0028] See also Figure 1 , Figure 1 It is a flow chart of a video conferencing method provided in an embodiment of the present application. It should be noted that the video conferencing method provided in an embodiment of the present application can be used for near-eye display devices.

[0029] like Figure 1 As shown, the video conferencing method includes steps S101 to S104.

[0030] S101. Display a video screen of a video conference, where the video screen includes at least one speaker.

[0031] For example, when the near-eye display device detects an access instruction to a video conference, it accesses the video conference and displays the video screen of the video conference. For example, a user wearing a near-eye display device can send an access instruction to the near-eye display device through gesture information, voice information, device posture information, touch information, etc., which is not limited here.

[0032] Exemplarily, the speaker may be determined based on at least one of a speaker identification in an offline conference scenario and a speaker identification of an online participant.

[0033] In some embodiments, the video conference that the user attends may have a corresponding offline meeting, and the video screen displayed by the near-eye display device may include screens related to the offline meeting scene. Figure 2 As shown, the video screen may include screens corresponding to participants in the offline conference scene. Since participants in the offline conference scene can choose whether to speak according to their own needs, the video screen may include at least one speaker.

[0034] In some embodiments, the video conference that the user attends is in the form of an online conference. In the case where the video conference is in the form of an online conference, the video screen displayed by the near-eye display device may include the speaker identification of the online participant. Figure 3 As shown, the speaker identification may include a preset icon corresponding to the speaker, a virtual image corresponding to the speaker, etc., which are not limited here. The preset icon may include an avatar, name, etc. of the speaker, which are not limited here. Since all participants in the online participant list can choose whether to speak according to their own needs, the video screen may include at least one speaker.

[0035] Of course, it is not limited to this. For example, the video conference can also be a conference format in which online and offline conferences are held in parallel. Accordingly, the video screen of the video conference can include at least one of the screen corresponding to the participants in the offline conference scene and the speaker identification of the online participants, which is not limited here.

[0036] When the near-eye display device displays the video screen of the video conference, the user can determine the speaker he or she is interested in from at least one speaker included in the video screen, so that the near-eye display device can subsequently determine the target sound data.

[0037] In this way, the near-eye display device displays the video screen of the video conference, and the video screen includes at least one speaker, so that the user can intuitively view the speaker of the video conference, which is conducive to improving the visualization effect of the speaker in the video conference. Correspondingly, the video screen of the video conference displayed by the near-eye display device can be used for the near-eye display device to subsequently determine the target sound data, which is conducive to improving the convenience of the near-eye display device in determining the target sound data.

[0038] S102: Obtain the visual focus of a user wearing a near-eye display device on a video screen.

[0039] For example, when participating in a video conference, if the user has a speaker that he or she is interested in, the user can focus his or her gaze on the speaker that he or she is interested in. Based on this, the near-eye display device can obtain the user's gaze focus on the video screen, so as to subsequently determine the sound data of the speaker that the user is interested in based on the user's gaze focus.

[0040] In some embodiments, an eye tracking sensor may be provided on the near-eye display device. The eye tracking sensor may be used to identify the gaze position of the user (also referred to as the gaze focus position). When the gaze position of the user is identified, the near-eye display device may determine the gaze focus of the user on the video screen in combination with the video screen displayed by the near-eye display device.

[0041] The line of sight focus can be used by subsequent near-eye display devices to determine target sound data.

[0042] S103: Determine target sound data from preset sound data of at least one speaker corresponding to the video conference according to the sight focus, and output the target sound data.

[0043] For example, during a video conference, the near-eye display device can obtain preset sound data of at least one speaker corresponding to the video conference. For example, when a speaker speaks, the near-eye display device can obtain corresponding sound data, and the near-eye display device can determine the corresponding sound data as the preset sound data of the speaker. In addition, the near-eye display device can combine the preset sound data of all speakers as the preset sound data of at least one speaker corresponding to the video conference.

[0044] When the near-eye display device obtains the user's visual focus on the video screen, the near-eye display device can determine that the user's visual focus is on at least one speaker based on the visual focus. Accordingly, the near-eye display device can determine that the speaker on which the user's visual focus is focused is the target speaker that the user is paying attention to. Accordingly, the near-eye display device can determine the sound data of the target speaker included in the preset sound data as the target sound data.

[0045] In this way, the near-eye display device can determine the target speaker based on the user's line of sight, which is helpful to improve the accuracy of determining the target speaker.

[0046] In some embodiments, a target speaker is determined from speakers included in the video screen based on line of sight focus; preset sound data whose sound direction is within a target sound range corresponding to the target speaker and / or whose voiceprint information conforms to the target voiceprint information of the target speaker is determined as target sound data.

[0047] For example, when the visual focus is within the picture range of the speaker in the video screen, the near-eye display device can determine that the speaker is the target speaker. The target speaker is used to indicate the speaker that the user is paying attention to. Of course, it is not limited to this. When the visual focus is within the picture range of each of the multiple speakers in the video screen, the near-eye display device can also determine that the multiple speakers are target speakers. For example, when the seats of multiple speakers are relatively compact, when the near-eye display device displays the video screen under a preset shooting angle, the picture ranges of each of the multiple speakers in the video screen can have overlapping picture ranges. Accordingly, when the visual focus is in the overlapping picture range, the near-eye display device can determine that the multiple speakers corresponding to the overlapping picture range are all target speakers, which is not limited here.

[0048] When the target speaker is determined, at least one of a target vocal range corresponding to the target speaker and target voiceprint information of the target speaker may be obtained. The target vocal range is used to indicate the position range of at least one sound source position corresponding to the sound data collected by the near-eye display device when the target speaker speaks. The target voiceprint information is used to indicate the voiceprint information of the target speaker. The target vocal range and the target voiceprint information may be predetermined or updated in real time, and are not limited here.

[0049] For example, when a speaker is speaking, he or she can change his or her head posture, such as changing at least one of the head position and head direction. Taking the speaker in an offline meeting scenario as an example, the speaker can lower his or her head to check the document or look up at the participants while speaking, and the speaker's head posture will change. In response to the change in the speaker's head posture, the speaker's mouth posture will also change, which is equivalent to the sound source position corresponding to the sound data will change. Based on this, the near-eye display device can determine the target sound range corresponding to the target speaker according to the change range of the head posture.

[0050] At least one of the target sound range and the target voiceprint information can be used by the near-eye display device to filter out the target sound data from the preset sound data corresponding to the video conference. The target sound data is used to indicate the sound data of the target speaker.

[0051] For example, the near-eye display device can determine whether there is sound data in the preset sound data whose sound direction is within the target sound range corresponding to the target speaker. If there is sound data in the preset sound data whose sound direction is within the target sound range, and the possibility that the sound data is the sound data of the target speaker is greater than or equal to the preset probability threshold, the near-eye display device can determine that the sound data is the target sound data, so as to filter the target sound data from the preset sound data according to the target sound range.

[0052] For example, the near-eye display device can determine whether there is sound data whose voiceprint information matches the target voiceprint information of the target speaker in the preset sound data. If there is sound data whose voiceprint information matches the target voiceprint information in the preset sound data, and the possibility that the sound data is the sound data of the target speaker is greater than or equal to the preset probability threshold, the near-eye display device can determine that the sound data is the target sound data, so as to filter the target sound data from the preset sound data according to the target voiceprint information.

[0053] Of course, it is not limited to this. The near-eye display device can also integrate the target vocal range and target voiceprint information corresponding to the target speaker to filter out the target sound data from the preset sound data.

[0054] In this way, when determining the target sound data, the near-eye display device can determine the target speaker based on the user's line of sight, which is conducive to improving the convenience of determining the target speaker. Accordingly, when determining the target speaker, the near-eye display device can flexibly filter the preset sound data based on at least one of the target sound range and target voiceprint information corresponding to the target speaker to obtain the target sound data, which is conducive to improving the flexibility of determining the target sound data.

[0055] In some embodiments, when the target sound data is determined, the near-eye display device may output the target sound data, such as broadcasting the target sound data.

[0056] For example, the near-eye display device can set at least one of the broadcast order and broadcast volume of the target sound data so that the user can better hear the target sound data of the target speaker he is paying attention to, thereby improving the broadcast effect of the near-eye display device on the target sound data.

[0057] For example, when there are multiple speakers speaking at the same time, including a target speaker and speakers other than the target speaker, the near-eye display device can determine that the preset broadcast priority of the target sound data of the target speaker is the highest. The near-eye display device can then set the broadcast order of the target sound data to be earlier than the broadcast order of the sound data of speakers other than the target speaker, so as to give priority to broadcasting the target sound data.

[0058] Of course, it is not limited to this. For example, when there are multiple speakers speaking at the same time, and the multiple speakers include a target speaker and speakers other than the target speaker, the near-eye display device can determine that the target sound data of the target speaker needs to be highlighted. Based on this, the near-eye display device can broadcast the target sound data according to the preset broadcast volume. Correspondingly, the near-eye display device can reduce the broadcast volume of the sound data of speakers other than the target speaker, and can also not broadcast the sound data of speakers other than the target speaker before the target sound data is broadcast, etc., which are not limited here.

[0059] In some implementations, when there are multiple target speakers, target sound data of at least one target speaker is broadcasted according to a preset broadcasting priority among the multiple target speakers.

[0060] For example, during a video conference, a user can look at multiple speakers, and the number of target speakers determined by the near-eye display device according to the user's line of sight focus can also be multiple. In the case where multiple target speakers speak in the same time period, if the near-eye display device simultaneously broadcasts the target sound data of each of the multiple target speakers, it is easy to cause the user to be unable to hear the multiple target sound data clearly. Based on this, the near-eye display device can broadcast the target sound data of at least one target speaker according to the preset broadcast priority between the multiple target speakers. The preset broadcast priority can be pre-set or set by the user, and is not limited here. For example, the near-eye display device can broadcast the target sound data of the target speaker with a higher priority according to the preset broadcast priority between the multiple target speakers, and then broadcast the target sound data of the target speaker with a lower priority. Of course, it is not limited to this. When the near-eye display device broadcasts multiple target sound data according to the preset broadcast priority, the near-eye display device can stop broadcasting the remaining target sound data in response to the received end broadcast instruction. The near-eye display device can also determine the current broadcast duration of the target sound data, so that when the current broadcast duration reaches a preset duration threshold, the remaining target sound data will be stopped from being broadcast. There is no limitation here.

[0061] In this way, when there are multiple target speakers, the target sound data of at least one target speaker is broadcast according to the preset broadcast priority among the multiple target speakers. The near-eye display device can determine the target sound data that the user is most concerned about according to the preset broadcast priority, and give priority to broadcasting the target sound data that the user is most concerned about, which is conducive to improving the near-eye display device's broadcast accuracy and broadcast flexibility for the target sound data, and enhancing the user's auditory experience of the target sound data.

[0062] In some implementations, when the sound volume corresponding to the target sound data is less than or equal to a preset volume threshold, the target sound data is broadcast according to a preset broadcast volume, and the preset broadcast volume is greater than the preset volume threshold.

[0063] For example, when the target speaker's speaking voice is relatively low, the sound volume corresponding to the target sound data is also relatively low. When the sound volume corresponding to the target sound data is relatively low, the user may not be able to hear the target sound data clearly. Based on this, the near-eye display device can determine whether the sound volume corresponding to the target sound data is less than or equal to the preset volume threshold. If the sound volume corresponding to the target sound data is less than or equal to the preset volume threshold, it can be determined that there is a high possibility that the user cannot hear the target sound data clearly. Based on this, the near-eye display device can broadcast the target sound data according to the preset broadcast volume, and the preset broadcast volume is greater than the preset volume threshold, so that the user can hear the target sound data clearly. Among them, the preset broadcast volume can be pre-set or set by the user, which is not limited here.

[0064] In this way, when the sound volume corresponding to the target sound data is less than or equal to the preset volume threshold, the target sound data is broadcast according to the preset broadcast volume. The preset broadcast volume is greater than the preset volume threshold, so that the user can clearly hear the target sound data broadcast by the near-eye display device, which is conducive to improving the broadcast flexibility of the near-eye display device for the target sound data, and improving the user's auditory experience of the target sound data.

[0065] In some implementations, when the target sound data is output, the target text information corresponding to the target sound data is displayed.

[0066] For example, when the near-eye display device outputs the target sound data, if the user's speech speed corresponding to the target sound data is fast, it is easy for the user to not be able to hear the target sound data currently being broadcast. Based on this, in order to enhance the user's auditory experience of the target sound data, the near-eye display device can assist the user in obtaining the target sound data by displaying the target text information corresponding to the target sound data. Among them, the text type corresponding to the target text information, such as font type, font size, language type, etc., can be pre-set or set by the user, and is not limited here.

[0067] In this way, the near-eye display device displays the target text information corresponding to the target sound data when broadcasting the target sound data, so that the user can view the target text information corresponding to the target sound data when listening to the target sound data broadcast by the near-eye display device, which is beneficial to improving the user's accuracy in knowing the target sound data and improving the flexibility of the near-eye display device in conveying the target sound data.

[0068] Exemplarily, the near-eye display device can obtain preset sound data corresponding to the video conference. For example, the preset sound data corresponding to the video conference can be stored in the cloud corresponding to the near-eye display device. Accordingly, the cloud can send the preset sound data corresponding to the video conference to the near-eye display device by data broadcasting, which is not limited here.

[0069] Before acquiring the preset sound data, the near-eye display device may predetermine the preset sound data.

[0070] In some embodiments, initial sound data is obtained; when the sound direction of the initial sound data is within a preset sound range corresponding to one of the speakers included in the video screen, and / or the voiceprint information of the initial sound data conforms to the preset voiceprint information of the speaker, the initial sound data is processed according to the preset identity identifier of the speaker to obtain initial sound data carrying the preset identity identifier; based on the initial sound data carrying the preset identity identifier, the preset sound data of at least one speaker corresponding to the video conference is determined.

[0071] For example, during a video conference, the near-eye display device may collect initial sound data. The near-eye display device may also determine the preset sound range corresponding to each speaker in the video screen and the preset voiceprint information of each speaker. For example, a positioning sensor may be provided on the near-eye display device. The near-eye display device may locate the position of the speaker through the positioning sensor. The near-eye display device may determine the preset sound range corresponding to the speaker according to the position of the speaker. For example, when the speaker speaks, the head posture may change. In response to the change in the speaker's head posture, the speaker's mouth position will change, and the sound source position corresponding to the sound data will change. Based on this, the near-eye display device may determine the preset sound range corresponding to the preset speaker according to the change range of the head posture. Accordingly, the near-eye display device may determine whether the initial sound data is the sound data of the preset speaker based on at least one of the preset sound range and the preset voiceprint information corresponding to the preset speaker. For example, when the sound direction of the initial sound data is within the preset sound range corresponding to one of the speakers included in the video screen, and / or the voiceprint information of the initial sound data conforms to the preset voiceprint information of the speaker, the near-eye display device can determine that the possibility that the initial sound data is the sound data of the speaker is greater than or equal to the preset probability threshold, and the near-eye display device can process the initial sound data according to the preset identity identifier of the speaker to obtain the initial sound data carrying the preset identity identifier. The preset identity identifier may include the speaker's device identification number, the speaker's account information, the speaker's name, etc., which are not limited here. The device identification number may include a Media Access Control Address (MAC address), a device identification number (Identity Document, ID), etc., which are not limited here.

[0072] Accordingly, the near-eye display device can determine the preset sound data of at least one speaker corresponding to the video conference based on all the initial sound data carrying the preset identity identifier.

[0073] For example, when the preset sound data corresponding to the video conference is determined based on all the initial sound data carrying the preset identity identifier, the near-eye display device can perform subsequent processing on the initial sound data based on the preset identity identifier. For example, when storing the preset sound data, it can be stored in different storage locations according to the different preset identity identifiers. For another example, when outputting the preset sound data, the preset sound data can be translated according to the language information corresponding to the preset identity identifier and the corresponding translation language, and the translated preset sound data can be broadcast, etc., which is not limited here.

[0074] In some implementations, preset sound data is converted into preset text according to a preset identity identifier to obtain conference record information of the video conference.

[0075] For example, the near-eye display device can distinguish the speakers corresponding to each preset sound data based on the preset identity identifier corresponding to the preset sound data. Based on this, the preset sound data is converted into preset text according to the preset identity identifier, and the obtained video conference record information can determine the preset text corresponding to the preset sound data of each speaker.

[0076] When the user needs to record or review relevant meeting information of a video conference, the user can obtain the meeting record information of the video conference through the near-eye display device, so as to clarify the specific speech of the speaker based on the meeting record information. The user does not need to write the meeting record information by himself, which is conducive to improving the convenience of determining the meeting record information.

[0077] The video conferencing method provided in the above embodiment includes: displaying a video screen of the video conference, the video screen including at least one speaker; obtaining the line of sight focus of a user wearing a near-eye display device on the video screen; based on the line of sight focus, determining target sound data from preset sound data of at least one speaker corresponding to the video conference, and outputting the target sound data.

[0078] In the process of displaying the video screen of the video conference, the near-eye display device can determine the speaker that the user is paying attention to from at least one speaker included in the video screen according to the user's line of sight focus on the video screen. For example, the speaker on whom the user's line of sight focuses in the video screen is the speaker that the user is paying attention to. Based on this, the near-eye display device can obtain the target sound data of the speaker that the user is paying attention to from the preset sound data corresponding to the video conference. Accordingly, the near-eye display device can output the target sound data. For example, if the target sound data is broadcast, the user can more conveniently hear the target sound data of the speaker that he or she is paying attention to, which is conducive to improving the broadcast effect of the near-eye display device on the target sound data of the video conference.

[0079] See also Figure 4 , Figure 4 : is a schematic block diagram of a video conferencing device provided in an embodiment of the present application. The video conferencing device can be configured in a near-eye display device to perform the aforementioned video conferencing method. The near-eye display device may include an XR device. The XR device may include AR glasses, VR glasses, MR glasses, AR helmets, VR helmets, MR helmets, etc., which are not limited here.

[0080] like Figure 4 As shown, the video conferencing device includes a picture display module 110 , a focus acquisition module 120 and a target determination module 130 .

[0081] The screen display module 110 is used to display the video screen of the video conference, where the video screen includes at least one speaker.

[0082] The focus acquisition module 120 is used to acquire the sight focus of the user wearing the near-eye display device on the video screen.

[0083] The target determination module 130 is used to determine target sound data from preset sound data of at least one speaker corresponding to the video conference according to the sight focus, and output the target sound data.

[0084] Exemplarily, the target determination module 130 includes a speaker determination submodule and a sound data determination submodule.

[0085] The speaker determination submodule is used to determine a target speaker from speakers included in the video screen according to the sight focus.

[0086] The sound data determination submodule is used to determine that the preset sound data whose sound azimuth is within the target sound range corresponding to the target speaker and / or whose voiceprint information conforms to the target voiceprint information of the target speaker is the target sound data.

[0087] Exemplarily, the target determination module 130 includes a first broadcast submodule.

[0088] The first broadcasting submodule is used to broadcast the target sound data of at least one of the target speakers according to a preset broadcasting priority among the target speakers when there are multiple target speakers.

[0089] Exemplarily, the target determination module 130 includes a second broadcast submodule.

[0090] The second broadcast submodule is used to broadcast the target sound data according to a preset broadcast volume when the sound volume corresponding to the target sound data is less than or equal to a preset volume threshold, and the preset broadcast volume is greater than the preset volume threshold.

[0091] Exemplarily, the video conferencing device further includes a data acquisition submodule, a first data processing submodule and a second data processing submodule.

[0092] The data acquisition submodule is used to acquire initial sound data.

[0093] The first data processing submodule is used to process the initial sound data according to the preset identity identifier of the speaker to obtain the initial sound data carrying the preset identity identifier when the sound direction of the initial sound data is within the preset sound range corresponding to one of the speakers included in the video screen and / or the voiceprint information of the initial sound data conforms to the preset voiceprint information of the speaker.

[0094] The second data processing submodule is used to determine the preset sound data of at least one speaker corresponding to the video conference according to the initial sound data carrying the preset identity identifier.

[0095] Exemplarily, the video conferencing device also includes a conference recording submodule.

[0096] The conference record submodule is used to convert the preset sound data into preset text according to the preset identity identifier to obtain the conference record information of the video conference.

[0097] Exemplarily, the video conferencing device also includes a text display submodule.

[0098] The text display submodule is used to display the target text information corresponding to the target sound data when the target sound data is broadcast.

[0099] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0100] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0101] Exemplarily, the above method and apparatus may be implemented in the form of a computer program, which may be run on a near-eye display device to participate in a video conference through the near-eye display device. Exemplarily, the near-eye display device may include an XR device, such as VR glasses, AR glasses, MR glasses, VR helmets, AR helmets, MR helmets, etc., which are not limited here.

[0102] See also Figure 5 , Figure 5 It is a schematic block diagram of the structure of a near-eye display device provided in an embodiment of the present application.

[0103] like Figure 5 As shown, the near-eye display device includes a memory and a processor. The memory and the processor may be connected via a system bus, and the memory may include a storage medium and an internal memory.

[0104] The storage medium can store an operating system and a computer program. When the computer program is executed, the processor can execute any video conferencing method.

[0105] The processor is used to provide computing and control capabilities to support the operation of the entire near-eye display device.

[0106] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any video conferencing method.

[0107] Those skilled in the art will understand that Figure 5 The structure shown in the figure is merely a block diagram of a portion of the structure related to the scheme of the present application, and does not constitute a limitation on the near-eye display device to which the scheme of the present application is applied. The specific near-eye display device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0108] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0109] In one embodiment, the processor is used to execute a computer program and implement the following steps when executing the computer program:

[0110] Displaying a video screen of a video conference, wherein the video screen includes at least one speaker;

[0111] Obtaining a visual focus of a user wearing a near-eye display device on the video screen;

[0112] Determining target sound data from preset sound data of at least one of the speakers corresponding to the video conference according to the sight focus;

[0113] The target sound data is broadcasted.

[0114] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the video conference described above can refer to the corresponding process in the aforementioned video conferencing method embodiment, and will not be repeated here.

[0115] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the video conferencing method of the present application.

[0116] The computer-readable storage medium may be an internal storage unit of the near-eye display device described in the aforementioned embodiment, such as a hard disk or memory of the near-eye display device. The computer-readable storage medium may also be an external storage device of the near-eye display device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the near-eye display device.

[0117] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0118] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0119] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A video conferencing method, characterized in that: Applied to a near-eye display device, the video conferencing method comprises: Displaying a video screen of a video conference, wherein the video screen includes at least one speaker; Obtaining a visual focus of a user wearing a near-eye display device on the video screen; According to the sight focus, target sound data is determined from preset sound data of at least one speaker corresponding to the video conference, and the target sound data is output.

2. The video conferencing method according to claim 1, characterized in that: The step of determining target sound data from preset sound data of at least one speaker corresponding to the video conference according to the sight focus includes: Determining a target speaker from speakers included in the video screen according to the sight focus; It is determined that preset sound data whose sound azimuth is within the target sound azimuth corresponding to the target speaker and / or whose voiceprint information conforms to the target voiceprint information of the target speaker is the target sound data.

3. The video conferencing method according to claim 2, characterized in that: The outputting the target sound data comprises: When there are multiple target speakers, the target sound data of at least one target speaker is broadcasted according to a preset broadcasting priority among the multiple target speakers.

4. The video conferencing method according to claim 1, characterized in that: The outputting the target sound data comprises: When the sound volume corresponding to the target sound data is less than or equal to a preset volume threshold, the target sound data is broadcast according to a preset broadcast volume, and the preset broadcast volume is greater than the preset volume threshold.

5. The video conferencing method according to any one of claims 1 to 4, characterized in that: Before determining the target sound data from the preset sound data of at least one speaker corresponding to the video conference according to the sight focus, the video conference method further includes: Get initial sound data; When the sound direction of the initial sound data is within the preset sound range corresponding to one of the speakers included in the video screen, and / or the voiceprint information of the initial sound data conforms to the preset voiceprint information of the speaker, the initial sound data is processed according to the preset identity identifier of the speaker to obtain the initial sound data carrying the preset identity identifier; According to the initial sound data carrying the preset identity identifier, preset sound data of at least one speaker corresponding to the video conference is determined.

6. The video conferencing method according to claim 5, characterized in that: After determining the preset sound data of at least one speaker corresponding to the video conference according to the initial sound data carrying the preset identity identifier, the video conference method further includes: According to the preset identity identifier, the preset sound data is converted into preset text to obtain the conference record information of the video conference.

7. The video conferencing method according to any one of claims 1 to 4, characterized in that: The video conferencing method further includes: When the target sound data is output, target text information corresponding to the target sound data is displayed.

8. A video conferencing device, characterized in that: The video conferencing device comprises: A screen display module, used to display a video screen of a video conference, wherein the video screen includes at least one speaker; A focus acquisition module, used to acquire the sight focus of a user wearing a near-eye display device on the video screen; The target determination module is used to determine target sound data from preset sound data of at least one speaker corresponding to the video conference according to the sight focus, and output the target sound data.

9. A near-eye display device, characterized in that: The near-eye display device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the steps of the video conferencing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the video conferencing method according to any one of claims 1 to 7 are implemented.

Images