Audio optimization method for video conference and video conference terminal

Abstract

The invention provides an audio optimization method for a video conference and a video conference terminal, and relates to the technical field of video conferences, and the method comprises the steps: carrying out the conventional parameter configuration after responding to a startup instruction; after the conventional parameter configuration is completed, adjusting the angle of the camera according to an adjusting instruction of the user on the angle of the camera; after the conference is detected, capturing a sound source through a microphone to determine the direction of a speaking person; controlling the camera to focus the direction to obtain video data; and uploading the video data. By implementing the method, the interference of standing positions of different persons on audio data acquisition can be reduced, and the overall audio quality of the video conference is improved.

Description

Technical Field

[0001] The present application relates to the technical field of video conferencing, and in particular to an audio optimization method for video conferencing and a video conferencing terminal. Background Art

[0002] With the increase of remote work and international cooperation, video conferencing has become an important tool for cross-regional communication. Fast and accurate information exchange plays a key role in business decision-making and educational dissemination. Therefore, efficient video conferencing systems are indispensable to modern society.

[0003] In video conferencing, the system usually needs to process complex audio and video signals to ensure the clarity and continuity of communication. Existing video conferencing systems use various algorithms to optimize audio and video inputs, such as automatic gain control, echo cancellation, etc., to adapt to different meeting environments and the needs of participants. By providing clear audio and video streams, all participants can clearly see and hear the meeting content.

[0004] However, existing audio optimization methods are less effective in conference rooms with multiple participants. This is mainly reflected in the interference of sound and camera angles between different people, resulting in poor overall audio quality, which affects the communication effect of the meeting and the experience of the participants. Summary of the invention

[0005] The present application provides a video conference audio optimization method and a video conference terminal, which are used to improve the overall audio quality of a video conference.

[0006] In a first aspect, the present application provides a method for optimizing audio in a video conference, which is applied to a video conference terminal, and the method includes: After responding to the power-on instruction, conventional parameter configuration is performed, and the conventional parameters include network connection settings, network status detection, and audio parameter configuration; After completing the general parameter configuration, adjust the camera angle according to the user's adjustment instructions for the camera angle; After entering the meeting, the microphone is used to capture the sound source and determine the direction of the speaker; Control the camera to focus on the direction to obtain video data; Upload the video data.

[0007] Through the above embodiments, the video conferencing terminal reduces the interference of different personnel positions on audio data collection and improves the overall audio quality of the video conference by optimizing a series of steps such as routine parameter configuration, camera angle adjustment, speaker positioning, camera focus control and video data uploading during the meeting after the video conferencing terminal is turned on.

[0008] In some embodiments, before the step of adjusting the camera angle according to the user's instruction to adjust the camera angle after completing the conventional parameter configuration, the method further includes: Constructing a three-dimensional scene graph of the conference room based on the conference room layout data input by the user; Determine the number of cameras to be installed and the preset installation position corresponding to each camera in the three-dimensional scene graph according to the effective shooting distance of the camera; The preset installation location is displayed in the three-dimensional scene graph.

[0009] Through the above embodiment, the video conferencing terminal generates a virtual three-dimensional space model based on the conference room layout data provided by the user, and calculates the number of cameras required to cover the entire scene and the optimized layout positions according to camera performance parameters such as shooting distance, and generates a visual camera installation guide to guide staff to install the camera.

[0010] In some embodiments, after entering the meeting, the step of capturing the sound source through the microphone to determine the direction of the speaker specifically includes: After detecting the voice signal, the microphone is used to capture the sound source and determine the direction of the speaker; Control the camera to obtain the visual image in the direction; Facial recognition technology is used to analyze the direction and expression of the face in the visual image to determine the location information of the speaker.

[0011] Through the above embodiment, the video conferencing terminal uses the microphone array to determine the direction of the sound source, and instructs the camera in the corresponding direction to capture the image, and then detects and analyzes the face in the image to obtain a more accurate speaker position, which facilitates the subsequent camera to accurately focus on the speaker.

[0012] In some embodiments, the step of controlling the camera to focus on the direction to obtain video data specifically includes: Determine multiple nearby cameras based on the location information of the speaker; According to the videos captured by the multiple cameras, a camera with the smallest and most complete facial deviation of the speaker is selected and set as the target camera; The target camera is controlled to focus on the speaker and obtain video data.

[0013] Through the above embodiment, the video conferencing terminal first selects multiple candidate cameras nearby according to the coordinates of the speaker, and then optimizes the target camera by analyzing factors such as the completeness of the face in each shot, the offset, etc. This ensures that even if the speaker is not in the ideal front position, it can automatically switch to the appropriate viewing angle and continue tracking and shooting, thereby improving the stability of the video content and enabling the audio and video information of the speech process to be fully recorded.

[0014] In some embodiments, before the step of controlling the target camera to focus on the speaker and obtain video data, the step further includes: Detect lighting parameters of the video captured by the camera; The fill light device is controlled to perform fill light according to the difference between the illumination parameter and the preset standard parameter.

[0015] Through the above embodiment, before focusing and shooting, the video conferencing terminal detects the brightness of the picture and evaluates whether the lighting conditions required for imaging are met. If the light is insufficient, the external fill light device is linked to control the fill light intensity according to the difference between the actual brightness and the preset standard. While dynamically optimizing the video quality, it also prevents problems such as overexposure, so that the facial features of the speaker can be presented more clearly.

[0016] In some embodiments, after the step of controlling the camera to focus on the direction to acquire video data, the method further includes: The video data is input into a machine learning model to obtain a first translation result of the lip movement of the speaker; The first translation result is integrated with the second translation result obtained by speech recognition to obtain translated subtitles for the video conference.

[0017] Through the above embodiment, the video conferencing terminal inputs the speaker's facial video frame into the pre-trained lip reading recognition model to obtain a text translation independent of the voice, which is cross-checked with the traditional voice recognition results to generate a more accurate and complete conference record subtitle. This makes up for the shortcomings of voice recognition in noisy environments, dialects and other difficult recognition scenarios.

[0018] In some embodiments, the step of integrating the first translation result with the second translation result obtained by speech recognition to obtain the translated subtitles of the video conference specifically includes: Detecting whether the first translation result is consistent with the second translation result; If yes, setting the second translation result obtained by the speech recognition as the translated subtitle; If not, the content at the inconsistent position in the first translation result is embedded into the second translation result to obtain an integrated translation subtitle.

[0019] Through the above embodiment, the video conferencing terminal cross-checks the translation results of the lip reading recognition model with the traditional speech recognition results. If there is any inconsistency, the contents of the two are integrated, and complementary error correction is performed to generate a more accurate and complete conference record subtitle for the participants to understand.

[0020] In a second aspect, the present application provides a video conferencing terminal, the video conferencing terminal comprising: one or more processors and a memory; The memory is coupled to the one or more processors, and is used to store computer program code, which includes computer instructions. The one or more processors call the computer instructions so that the video conferencing terminal can implement a video conferencing audio optimization method provided in the above embodiment, which will not be repeated here.

[0021] In a third aspect, the present application provides a computer-readable storage medium, including instructions. When the instructions are executed on a video conferencing terminal, the video conferencing terminal can implement a video conferencing audio optimization method provided in the above embodiment, which will not be repeated here.

[0022] In a fourth aspect, the present application provides a computer program product. When the computer program product runs on a video conferencing terminal, the video conferencing terminal can implement a video conferencing audio optimization method provided in the above embodiment, which will not be described in detail here.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By analyzing and calculating factors such as camera parameters and lighting conditions, the image quality is evaluated in real time during the meeting, and the camera focus and fill light intensity are dynamically adjusted to ensure that the video content remains clear and stable. By actively adapting to changing external conditions, the audio and video acquisition quality and user experience are improved.

[0024] 2. Integrate multiple artificial intelligence technologies such as speech recognition, facial expression analysis, and lip reading recognition into video conferencing scenarios, and use speech positioning and face detection to achieve accurate and efficient speaker tracking, making it easier for the camera to focus on the speaker and improve image quality. Through cross-validation of speech and lip reading recognition, more complete and accurate conference subtitles are generated to facilitate understanding by participants.

[0025] 3. Based on the conference room layout data provided by the user, a virtual three-dimensional space model is generated. According to the camera performance parameters such as shooting distance, the number of cameras required to cover the entire scene and the optimized layout positions are calculated, and a visual camera installation guide is generated to guide the staff to install the camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flowchart of an audio optimization method for a video conference in an embodiment of the present application; Figure 2 is another flowchart of an audio optimization method for a video conference in an embodiment of the present application; Figure 3 It is a schematic diagram of the physical device structure of the video conferencing terminal in the embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more listed items.

[0028] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0029] For ease of understanding, the following is a flow chart of the method provided in this implementation. Figure 1 The figure is a flow chart of an audio optimization method for a video conference in an embodiment of the present application.

[0030] S101, after responding to the power-on instruction, perform conventional parameter configuration.

[0031] After the video conferencing terminal is started, the user can configure the general parameters. In the embodiment of the present application, the user can configure the basic parameters, such as the terminal name, terminal number, server address, and whether the QR code for joining the meeting is displayed. The network connection can also be set, such as the selection of wired network and wireless network, and further, the currently connected network can be tested to view the current network test results. In addition, the video parameters (such as video input interface, picture quality, etc.) and audio parameters (such as audio input interface, audio output interface, etc.) can also be configured, which is not limited here.

[0032] In addition, in addition to the general parameters, users can also set call parameters on the video conferencing terminal, such as allowing call permissions, whether to answer automatically, whether to turn on Do Not Disturb, and video call layout, etc. The specific adjustable parameters can be set according to actual needs and are not limited here.

[0033] S102: After completing the configuration of conventional parameters, adjust the camera angle according to the user's instruction to adjust the camera angle.

[0034] After the general parameter configuration is completed, the video conferencing terminal enters the standby state, and the corresponding display interface will display the real-time image captured by its own camera.

[0035] Furthermore, since the sitting postures and heights of the participants are different, if the camera is fixed, there may be problems with distorted images and poor composition, affecting the video effect and conference experience. Therefore, the terminal supports users to manually adjust the camera angle to optimize the picture. Specifically, users can issue pitch and rotation instructions for the camera through input devices such as directional keys and joysticks on the video conferencing terminal, or through the virtual control bar on the touch screen connected to the video conferencing terminal. After receiving the user's instructions, the video conferencing terminal sends instructions to control the camera to move to the specified position. When the user is satisfied with the preview picture, he can press the confirmation key again to save the current camera angle parameters.

[0036] S103: After entering the meeting is detected, the direction of the speaker is determined by capturing the sound source through the microphone.

[0037] When the meeting officially begins, participants usually take turns to speak and discuss. In order to allow the camera to focus on the speaker at any time, the video conferencing terminal has multiple built-in microphones distributed in different directions of the device. When a voice is heard from a certain direction, the direction of the speaker is determined based on the differences in time and intensity of the sound waves from each microphone.

[0038] Of course, in another embodiment, the microphone can be wirelessly connected to the video conferencing terminal, and the microphone sends its own location information to the video conferencing terminal in real time, so that the video conferencing terminal determines the location of the speaker based on the position of the microphone that receives the maximum sound intensity.

[0039] S104: Control the camera to focus on the direction to obtain video data.

[0040] After locating the speaking area or direction, the video conferencing terminal controls the camera to quickly turn to that direction, automatically adjusts the focus, and detects faces from the acquired video images. Through the facial recognition algorithm, the facial features of the face are analyzed, especially the movement amplitude and frequency of the lips, to determine which person in the picture is speaking.

[0041] After finding the target face, the control unit of the terminal will calculate its position offset in the picture in real time, and control the electric mechanism through instructions to fine-tune the camera angle so that the target face is always in the center of the picture. Of course, in other embodiments, if the camera angle cannot be adjusted after being determined, the camera can be controlled to automatically focus on the target face, so that the image of the speaker in the video is clearer.

[0042] S105: Upload the video data.

[0043] When the video conferencing terminal obtains high-quality video images of the speaker, it will be encoded in real time and uploaded to the video conferencing server or other participating terminals via the network for other users to watch.

[0044] Specifically, the video conferencing terminal determines the frame rate, bit rate and resolution of the video data upload based on the current network status and the video parameters set by the user in step S101. At the same time, the video conferencing terminal will also perform adaptive network optimization on the uploaded video stream. The transmission strategy is dynamically adjusted by real-time monitoring of the bandwidth, packet loss rate and other parameters of the uplink network. For example, when the network is good, the UDP protocol is used to reduce retransmission delays; when the network is congested, switch to reliable transmission solutions such as QUIC and FEC to ensure the integrity of the picture arrival. At the same time, SVC layered coding, multi-path transmission and other technologies can be used to allow receivers with different bandwidths to obtain video quality that suits them, which is not limited here.

[0045] In the above embodiment, the video conferencing terminal reduces the interference of different people's positions on audio data collection and improves the overall audio quality of the video conference by optimizing a series of steps such as general parameter configuration, camera angle adjustment, speaker positioning, camera focus control and video data uploading during the meeting.

[0046] The following is a more detailed description of the process of the method provided by this embodiment. Figure 2 As shown, it is another flowchart of an audio optimization method for a video conference in an embodiment of the present application.

[0047] S201: Construct a three-dimensional scene graph of the conference room based on conference room layout data input by a user.

[0048] The user can input the layout data of the conference room in the video conferencing terminal, such as the length, width, and height of the conference room, the location and size of the conference table, and the placement of chairs.

[0049] The video conferencing terminal uses 3D modeling technology to construct a digital 3D model of a conference room in a virtual space in real proportion based on the layout data. It should be noted that the construction of the 3D model can be achieved by installing relevant modeling software, which is not limited here.

[0050] S202: Determine the number of cameras to be installed and the preset installation position corresponding to each camera in the three-dimensional scene graph according to the effective shooting distance of the camera.

[0051] Since a single camera has a limited field of view and resolution, it is difficult to cover the entire conference room. Therefore, it is necessary to calculate the best camera combination solution based on the spatial size of the conference room and the performance parameters of the camera.

[0052] Specifically, the video conferencing terminal first needs to obtain the effective shooting distance parameter of each camera model, that is, the maximum allowable distance between the camera and the subject under the premise of ensuring a certain imaging quality. This parameter can be obtained from the camera specification or measured experimentally. Then the video conferencing terminal uses the effective shooting distance as the radius to generate a set of circular areas in the three-dimensional scene graph, covering the main space of the conference room (such as the area corresponding to the conference table and chairs). By calculating the number of circular areas, the number of cameras that need to be deployed in the conference room can be obtained. Among them, the center position corresponds to the preset installation position coordinates of each camera.

[0053] It should be noted that when calculating the number of circular areas, on the basis of ensuring that the main space of the conference room is covered, the overlap of circular areas corresponding to different cameras is minimized. Of course, the position and number of cameras can also be adjusted according to actual needs and are not limited here.

[0054] In the above embodiment, the video conferencing terminal generates a virtual three-dimensional space model based on the conference room layout data provided by the user, and calculates the number of cameras required to cover the entire scene and the optimized layout positions according to camera performance parameters such as shooting distance, and generates a visual camera installation guide to guide staff to install the camera.

[0055] S203: Determine the location information of the speaker through sound source localization and facial recognition technology.

[0056] During the meeting, in order for the camera to accurately capture the image of the current speaker, the video conferencing terminal needs to determine the speaker's location in real time.

[0057] Specifically, after the video conferencing terminal captures the voice signal through the microphone array arranged in the conference room, it calculates the time difference and volume difference of each microphone receiving the sound, and then determines the direction of the sound source. It is understandable that since the speaker's voice is the clearest and loudest, the direction with the highest sound intensity is generally toward the area where the speaker is located.

[0058] To further improve positioning accuracy, the video conferencing terminal takes this area as the area of ​​interest, obtains high-definition images from the camera in this direction first, and uses facial recognition algorithms to detect faces in the picture. Then, by analyzing information such as the direction of the face and the amplitude of the mouth movement, the speaker who is speaking is identified from the multiple people in the area of ​​interest. Then, based on the relative position of the speaker's face in the picture, the spatial position of the speaker in the conference room coordinate system is calculated.

[0059] For example, when someone starts speaking in a meeting, the speaker is determined to be in the southwest of the conference room by calculating the difference in audio signals from the four microphones. The video conferencing terminal automatically wakes up cameras 1 and 2 located in the southwest corner of the wall, and obtains close-up images of the speaking area from the video streams they capture. After face detection, a face is identified that is speaking. The face is to the left in the image of camera 1 and to the right in the image of camera 2. The final triangulation calculation shows that the person is 6 meters away from the southwest corner and 3 meters away from the south wall, which is point (6, 3) in the conference room coordinate system.

[0060] In the above embodiment, the video conferencing terminal uses a microphone array to determine the direction of the sound source, and instructs the camera in the corresponding direction to capture the image, and then detects and analyzes the face in the image to obtain a more accurate speaker position, so that the subsequent camera can focus on the speaker accurately.

[0061] S204: Determine multiple nearby cameras based on the location information of the speaker.

[0062] Since there are multiple cameras installed in the conference room, each camera is located at a different position and angle, the images of the speakers captured will also be different.

[0063] In order to select the camera with the best visual effect, the video conferencing system determines which cameras are closer to the speaker based on the speaker's coordinates. Generally speaking, the closer the camera is to the subject, the clearer the picture details will be, but it should not be too close to cause an incomplete view.

[0064] Therefore, the user can set a reasonable distance threshold range on the video conferencing terminal, such as 1 to 3 meters, and then calculate the spatial distance between each camera and the speaker in the three-dimensional virtual coordinate system. The cameras whose distances fall within the threshold range are marked as candidates, and usually multiple cameras meet the conditions at the same time.

[0065] S205: Select a camera with the smallest facial deviation and the most complete face of the speaker from the videos captured by multiple cameras to obtain video data for uploading.

[0066] After screening in step S204, the video conference terminal obtains multiple candidate cameras for shooting the speaker. This step is used to select the one with the best visual effect from the multiple candidate cameras as the final shooting position, and obtain its video image for network uploading and conference live broadcast.

[0067] Specifically, the video conferencing terminal obtains the real-time video stream collected by each candidate camera, and then intercepts the key frame image containing the speaker. Based on the face recognition algorithm, it is detected whether the speaker's face area appears completely in the picture instead of being cut off by the edge; at the same time, the offset degree of the speaker's face in the whole picture is detected. The corresponding angle of the camera with the most complete face and the largest proportion is determined as the best shooting angle.

[0068] In the above embodiment, the video conferencing terminal first selects multiple candidate cameras nearby according to the coordinates of the speaker, and then optimizes the target camera by analyzing factors such as the completeness of the face in each shot, the offset, etc. This ensures that even if the speaker is not in the ideal front position, it can automatically switch to the appropriate viewing angle and continue tracking and shooting, thereby improving the stability of the video content and enabling the audio and video information of the speech process to be fully recorded.

[0069] S206: Controlling a fill light device to perform fill light according to a difference between a lighting parameter corresponding to the video data and a preset standard parameter.

[0070] During a video conference, good lighting conditions are one of the key factors to ensure image quality. However, due to the layout of the conference room, people blocking the view, changes in external light, etc., the brightness of the video screen may be too dark or overexposed, affecting the visual effect and information communication.

[0071] To address this problem, the video conferencing terminal analyzes the brightness histogram, average brightness and other parameters of the video frame to determine whether the current light intensity meets the imaging requirements. At the same time, the video conferencing terminal uses the built-in light sensor to detect the ambient light intensity in real time. The video analysis results are compared with the sensor data, and the preset optimal light standard is referenced to obtain the light difference.

[0072] The video conferencing terminal can intelligently adjust the external fill light equipment, such as LED lights and reflectors, according to the light difference. For example, when the picture is dark, the terminal controls the fill light to increase the output power, and when the ambient light is too strong, the reflector is used to diffuse the light to avoid overexposure.

[0073] In the above embodiment, before focusing and shooting, the video conferencing terminal detects the brightness of the screen to evaluate whether the lighting conditions required for imaging are met. If the light is insufficient, the external fill light device is linked to control the fill light intensity according to the difference between the actual brightness and the preset standard. While dynamically optimizing the video quality, it also prevents problems such as overexposure, so that the facial features of the speaker can be presented more clearly.

[0074] S207: Input the video data into a machine learning model to obtain a first translation result of the speaker's lip movement.

[0075] The video conferencing terminal inputs the collected facial video frames of the speaker into the pre-trained lip reading recognition model. The model adopts an end-to-end deep neural network structure, integrating the characteristics of CNN (convolutional neural network) and RNN (recurrent neural network), and can effectively extract the spatiotemporal characteristics of lip movement. After multi-layer network feature learning and sequence mapping, the lip reading recognition model outputs a text result synchronized with the video, that is, the lip reading translation of the speaker.

[0076] Among them, lip reading recognition is an artificial intelligence technology that analyzes human lip movements and converts them into text. It uses visual information as input, establishes a mapping relationship between lip shape and speech through a deep learning algorithm, and realizes a visual understanding of speech content.

[0077] S208: Whether the first translation result is consistent with the second translation result.

[0078] Specifically, the video conferencing terminal triggers the lip reading recognition and speech recognition engines simultaneously, and independently completes the speech transcription task for the video and audio inputs respectively. The lip reading model processes the facial video stream and outputs the lip reading translation, which is recorded as the first translation result. The speech recognition system analyzes the sound recorded by the microphone and outputs the auditory translation, which is recorded as the second translation result.

[0079] The video conference terminal aligns the two translation results word by word in real time according to the time axis to determine whether they are consistent. If they are consistent, step S209 is executed to set the second translation result obtained by speech recognition as the translated subtitle. On the contrary, if they are inconsistent, step S210 is executed to embed the content of the inconsistent position in the first translation result into the second translation result to obtain an integrated translated subtitle.

[0080] S209: Setting the second translation result obtained by the speech recognition as a translated subtitle.

[0081] Specifically, in the cross-validation phase, if the results of lip reading recognition and speech recognition are highly consistent, the speech transcription can be directly adopted to generate subtitles for simultaneous interpretation.

[0082] S210: embed the content of the inconsistent position in the first translation result into the second translation result to obtain an integrated translation subtitle.

[0083] When the quality of the audio and video signals is poor, such as when there is a lot of ambient noise or unclear lip shape, the translation results given by lip reading recognition and speech recognition may differ to a certain extent. In order to correct the recognition error, the video conferencing terminal embeds the corresponding first translation result at the position of the inconsistent content in the second translation result to obtain an integrated translation subtitle. For example, in one embodiment, the first translation result is "the revenue in the third quarter is about 1.5 billion yuan", and the second translation result is "the revenue in the third quarter is about 5 billion yuan", then the integrated translation subtitle is "the revenue in the third quarter is about 5 billion (1.5) billion yuan".

[0084] In the above embodiment, the video conferencing terminal inputs the speaker's facial video frame into the pre-trained lip reading recognition model to obtain a text translation independent of the speech, which is cross-checked with the traditional speech recognition results to generate a more accurate and complete conference record subtitle, making up for the shortcomings of speech recognition in noisy environments, dialects and other difficult recognition scenarios.

[0085] The video conferencing terminal in the embodiment of the present invention is an electronic device. Figure 3 A schematic diagram of the architecture of an electronic device suitable for implementing an embodiment of the present invention is shown.

[0086] It should be noted that Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0087] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructions (computer programs), or by controlling related hardware through instructions (computer programs), and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. The electronic device of this embodiment includes a storage medium and a processor, wherein a plurality of instructions are stored in the storage medium, and the instructions can be loaded by the processor to execute any step of the method provided in the embodiment of the present invention.

[0088] Specifically, the storage medium and the processor are electrically connected directly or indirectly to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more signal lines. The storage medium stores computer execution instructions for implementing the data access control method, including at least one software function module that can be stored in the storage medium in the form of software or firmware. The processor executes various functional applications and data processing by running the software program and module stored in the storage medium. The storage medium can be, but is not limited to, random access storage medium (Random Access Memory, referred to as: RAM), read-only storage medium (Read Only Memory, referred to as: ROM), programmable read-only storage medium (Programmable Read-Only Memory, referred to as: PROM), erasable read-only storage medium (Erasable Programmable Read-Only Memory, referred to as: EPROM), electrically erasable read-only storage medium (Electric Erasable Programmable Read-Only Memory, referred to as: EEPROM), etc. Among them, the storage medium is used to store programs, and the processor executes the program after receiving the execution instruction.

[0089] Furthermore, the software programs and modules in the above-mentioned storage medium may also include an operating system, which may include various software components and / or drivers for managing system tasks (such as memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components. The processor may be an integrated circuit chip having signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., which may implement or execute the various methods, steps, and logic flow diagrams disclosed in this embodiment. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0090] Since the instructions stored in the storage medium can execute the steps in any method provided in the embodiments of the present invention, the beneficial effects of any method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0091] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for optimizing audio in a video conference, applied to a video conference terminal, characterized in that: The method comprises: After responding to the power-on instruction, performing conventional parameter configuration, the conventional parameters include network connection settings, network status detection and audio parameter configuration; After completing the general parameter configuration, adjust the camera angle according to the user's adjustment instructions for the camera angle; After entering the meeting, the microphone is used to capture the sound source and determine the direction of the speaker; Control the camera to focus on the direction to obtain video data; The video data is uploaded.

2. The method according to claim 1, characterized in that Before the step of adjusting the camera angle according to the user's instruction to adjust the camera angle after completing the conventional parameter configuration, the method further includes: Constructing a three-dimensional scene graph of the conference room based on the conference room layout data input by the user; Determining the number of cameras to be installed and the preset installation position corresponding to each camera in the three-dimensional scene graph according to the effective shooting distance of the camera; The preset installation location is displayed in the three-dimensional scene graph.

3. The method according to claim 1, characterized in that The step of detecting the entry into the meeting and capturing the sound source by a microphone to determine the direction of the speaker specifically includes: After detecting the voice signal, the microphone is used to capture the sound source and determine the direction of the speaker; Controlling the camera to obtain a visual image in the direction; Facial recognition technology is used to analyze the direction and expression of the face in the visual image to determine the location information of the speaker.

4. The method according to claim 1, characterized in that: The step of controlling the camera to focus on the direction to obtain video data specifically includes: Determine multiple nearby cameras based on the location information of the speaker; Selecting a camera with the smallest and most complete facial deviation of the speaker based on the videos captured by the multiple cameras, and setting the camera as the target camera; The target camera is controlled to focus on the speaker and obtain video data.

5. The method according to claim 4, characterized in that Before the step of controlling the target camera to focus on the speaker and obtain video data, the method further includes: Detect lighting parameters of the video captured by the camera; A fill light device is controlled to perform fill light according to the difference between the illumination parameter and the preset standard parameter.

6. The method according to claim 1, characterized in that After the step of controlling the camera to focus on the direction to acquire video data, the method further includes: Inputting the video data into a machine learning model to obtain a first translation result of the lip movement of the speaker; The first translation result is integrated with the second translation result obtained by speech recognition to obtain translated subtitles for the video conference.

7. The method according to claim 6, characterized in that The step of integrating the first translation result with the second translation result obtained by speech recognition to obtain the translated subtitles of the video conference specifically includes: detecting whether the first translation result is consistent with the second translation result; If yes, setting the second translation result obtained by the speech recognition as the translated subtitle; If not, the content at the inconsistent position in the first translation result is embedded into the second translation result to obtain an integrated translation subtitle.

8. A video conferencing terminal, characterized in that: The video conferencing terminal includes: one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the video conferencing terminal to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on the video conferencing terminal, the video conferencing terminal executes the method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is executed on a video conferencing terminal, the video conferencing terminal is enabled to execute the method according to any one of claims 1 to 7.

Images