Method and device for adjusting video picture

Abstract

The invention relates to the field of video conferences, and provides a method and device for adjusting a video picture, and the method comprises the steps: determining a target exposure value corresponding to a target image and / or a target white balance value corresponding to the target image, the target image being an image input by an image input device or a part of an image input by the image input device, and the target white balance value corresponding to the target image; the target image is a to-be-output image of the kth frame after the current frame of the image output device; and outputting the target image according to the target exposure value and / or the target white balance value in the kth frame after the current frame. According to the method, the exposure value and / or the white balance value and other parameter values of the switched target image are determined in advance before the video picture is switched, so that corresponding adjustment can be applied to the video picture according to the pre-determined parameter values at the same time when the video picture is switched; the switching of the video pictures and the adjustment of the corresponding parameters are carried out at the same time, so that the visual effect of the pictures is kept uniform when the pictures of the video conference are switched.

Description

Technical Field

[0001] The present application relates to the field of video conferencing, and more specifically, to a method and device for adjusting a video screen. Background Art

[0002] With the development of video conferencing technology, the functions of video conferencing terminal devices are becoming more and more abundant. Current video conferencing terminal devices can provide multiple viewing modes, such as the director's close-up mode, which can automatically display the close-up of the current speaker and switch as the speaker changes; the director's tracking mode, which can lock the lens on a specific object and track it as the object moves; and the director's equalization mode, which can splice the close-up images of multiple participants in the conference room into one picture and present it.

[0003] However, in any of the above view modes, there is a problem that the brightness or color of the picture changes due to differences in environmental factors such as light before and after the switching. If the parameters such as exposure and white balance of the picture before and after the switching are not adjusted or adjusted in time, the visual experience of the picture before and after the switching will be inconsistent, making the picture look uncoordinated. Therefore, how to maintain the uniformity of the visual effect of the picture when switching the picture in a video conference has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a method and device for adjusting a video screen. By determining in advance parameter values ​​such as the exposure value and / or white balance value of a target image after switching before the video screen is switched, corresponding adjustments can be applied to the video screen according to the pre-determined parameter values ​​at the same time when the video screen is switched, so that the switching of the video screen and the adjustment of the corresponding parameters occur simultaneously, thereby maintaining the uniformity of the visual effect of the screen when the screen of the video conference is switched.

[0005] In a first aspect, a method for adjusting a video screen is provided, the method being applied to a video conferencing terminal device, the video conferencing terminal device comprising an image input device and an image output device, the method comprising: determining a target exposure value corresponding to a target image and / or a target white balance value corresponding to the target image, wherein the target image is an image input by the image input device or a partial image of an image input by the image input device, and the target image is an image to be output by the image output device in the kth frame after the current frame, k being a positive integer greater than or equal to 1; at the kth frame after the current frame, outputting the target image according to the target exposure value and / or the target white balance value.

[0006] Optionally, the image input device may be an image acquisition device such as a video camera, a still camera, a smart phone, or may be a processor such as an image signal processing (ISP) for processing an image signal input by the image acquisition device.

[0007] Optionally, the image output device can be an image display device such as a monitor, a television, a smart phone, etc., or it can be a processor such as a graphics processing unit (GPU) or a digital signal processing (DSP) for processing image signals output to the image display device.

[0008] According to the technical solution provided in the present application, by determining in advance the exposure value and / or white balance value and other parameter values ​​of the target image after switching before the video screen is switched, the corresponding adjustment can be applied to the video screen according to the pre-determined parameter values ​​at the same time when the video screen is switched, so that the switching of the video screen and the adjustment of the corresponding parameters occur simultaneously, thereby maintaining the unity of the visual effect of the screen when the screen is switched in the video conference.

[0009] In combination with the first aspect, in certain implementations of the first aspect, at the kth frame after the current frame, a target image is output according to a target exposure value and / or a target white balance value, including: at the kth frame after the current frame, applying the target exposure value and / or the target white balance value to an image input device.

[0010] According to the above technical solution, by directly applying the determined exposure value and / or white balance value to the image input device, since image acquisition parameters such as the exposure value and the white balance value are directly determined according to the environmental changes of the captured image, the brightness and color of the image can be conveniently and accurately adjusted at the input end, reducing the calculation complexity, thereby improving the accuracy and efficiency of video image adjustment.

[0011] In combination with the first aspect, in certain implementations of the first aspect, when k is greater than 1, at the kth frame after the current frame, the target image is output according to the target exposure value and / or the target white balance value, and the method also includes: fitting the exposure value and / or white balance value of the image input device at the current frame and the target exposure value and / or the target white balance value to determine the exposure value and / or white balance value corresponding to the i-th frame after the current frame, where i is a positive integer greater than or equal to 1 and less than k; at the i-th frame after the current frame, applying the exposure value and / or white balance value corresponding to the i-th frame after the current frame to the image input device.

[0012] According to the above technical solution, image acquisition parameters such as the exposure value and / or white balance value of at least one transition frame between the two frames before and after the switching are calculated by interpolation, so that on the basis of maintaining the uniformity of the visual effects of the front and back pictures, the brightness and color of the intermediate picture in the switching process also change evenly, thereby improving the consistency of the visual effect of the video picture when the lens switches smoothly.

[0013] In combination with the first aspect, in certain implementations of the first aspect, at the kth frame after the current frame, a target image is output according to a target exposure value and / or a target white balance value, including: determining brightness compensation of the target image and / or color compensation of the image according to the target exposure value and / or the target white balance value, wherein the brightness compensation corresponds to the target exposure value and the color compensation corresponds to the target white balance value; at the kth frame after the current frame, applying the brightness compensation and / or color compensation to the target image through an image output device.

[0014] According to the above technical solution, without changing the image acquisition related parameters such as the exposure value or white balance value of the image input device or when the related parameters cannot be changed, by converting the image acquisition related parameters into brightness compensation and color compensation when the image output device post-processes the image, the brightness and color of the image can be adjusted at the output end to be similar to the effect at the input end, thereby improving the flexibility and scope of application of the video picture adjustment method.

[0015] In combination with the first aspect, in certain implementations of the first aspect, brightness compensation of a target image and / or color compensation of the image are determined according to a target exposure value and / or a target white balance value, including: determining a change in a Y channel in a YUV color space of the target image according to the target exposure value.

[0016] According to the above technical solution, by converting the exposure value during image acquisition into the brightness represented by the Y channel during image processing, the required change in the exposure value can be compensated by changing the value of the Y channel, thereby realizing the adjustment of the brightness of the image in the post-processing process, achieving an effect similar to changing the exposure value when acquiring the image.

[0017] In combination with the first aspect, in certain implementations of the first aspect, brightness compensation of a target image and / or color compensation of an image is determined based on a target exposure value and / or a target white balance value, including: determining a change in at least one of an R channel, a G channel, and a B channel in an RGB color space of the target image based on the target white balance value.

[0018] According to the above technical solution, by converting the white balance value during image acquisition into red, green and blue color values ​​represented by the R channel, G channel and B channel during image processing, the required change in the white balance value can be compensated by changing the value of at least one of the R channel, G channel and B channel, thereby realizing image color adjustment in the post-processing process, achieving an effect similar to that of changing the white balance value when acquiring the image.

[0019] In combination with the first aspect, in some implementations of the first aspect, the target image includes a first part and a second part, the first part and the second part are images of two different parts in an image input by an image input device, and determining a target exposure value corresponding to the target image and / or a target white balance value corresponding to the target image includes: determining a first target exposure value and a second target exposure value, and / or a first target white balance value and a second target white balance value, wherein the first target exposure value is a target exposure value corresponding to the first part, the second target exposure value is a target exposure value corresponding to the second part, the first target white balance value is a target white balance value corresponding to the first part, and the second target white balance value is a target white balance value corresponding to the second part; at a k-th frame after a current frame, applying brightness compensation and / or color compensation to the target image through an image output device, includes: at a k-th frame after a current frame, applying brightness compensation corresponding to the first target exposure value to the first part, and applying brightness compensation corresponding to the second target exposure value to the second part through the image output device; and / or at a k-th frame after the current frame, applying color compensation corresponding to the first target white balance value to the first part, and applying color compensation corresponding to the second target white balance value to the second part through the image output device.

[0020] According to the above technical solution, when the target image after switching is composed of multiple parts of an image input by the image input device, since each part can only use the same image acquisition related parameters such as exposure value and white balance value, by independently determining the image acquisition parameters suitable for each part and converting them into brightness compensation and / or color compensation related to image post-processing, it is possible to achieve different degrees of brightness and color adjustment for different parts of the same input image, so that the different parts have a unified visual effect after being spliced ​​together, thereby improving the consistency of the visual effect of the video screen under the splicing view.

[0021] In combination with the first aspect, in certain implementations of the first aspect, before determining the target exposure value corresponding to the target image and / or the target white balance value corresponding to the target image, the method also includes: determining the target image from the image input by the image input device based on at least one method of target detection, sound source localization, and sound and image matching.

[0022] According to the above technical solution, through image recognition technology, voice recognition technology or a combination of the two, the image content to be output can be accurately and quickly located in advance from the input image, thereby providing a prerequisite for accurately and quickly determining parameter values ​​such as the exposure value and / or white balance value of the target image, thereby improving the accuracy and efficiency of video picture adjustment.

[0023] In a second aspect, a device for adjusting a video screen is provided, which is applied to a video conferencing terminal device. The video conferencing terminal device also includes an image input device and an image output device. The device includes: a pre-calculation module, used to determine a target exposure value corresponding to a target image and / or a target white balance value corresponding to the target image, wherein the target image is an image input by the image input device or a partial image of an image input by the image input device, and the target image is an image to be output by the image output device in the kth frame after the current frame, and k is a positive integer greater than or equal to 1; a processing module, used to output the target image according to the target exposure value and / or the target white balance value at the kth frame after the current frame.

[0024] In combination with the second aspect, in some implementations of the second aspect, the processing module is used to: apply the target exposure value and / or the target white balance value to the image input device at the kth frame after the current frame.

[0025] In combination with the second aspect, in certain implementations of the second aspect, when k is greater than 1, the processing module is used to: fit the exposure value and / or white balance value of the image input device in the current frame and the target exposure value and / or target white balance value to determine the exposure value and / or white balance value corresponding to the i-th frame after the current frame, where i is a positive integer greater than or equal to 1 and less than k; and in the i-th frame after the current frame, apply the exposure value and / or white balance value corresponding to the i-th frame after the current frame to the image input device.

[0026] In combination with the second aspect, in certain implementations of the second aspect, the processing module is used to: determine brightness compensation of the target image and / or color compensation of the image based on the target exposure value and / or the target white balance value, wherein the brightness compensation corresponds to the target exposure value and the color compensation corresponds to the target white balance value; and at the kth frame after the current frame, apply the brightness compensation and / or color compensation to the target image through the image output device.

[0027] In combination with the second aspect, in some implementations of the second aspect, the processing module is used to: determine a change amount of a Y channel in a YUV color space of a target image according to a target exposure value.

[0028] In combination with the second aspect, in certain implementations of the second aspect, the processing module is used to: determine a change amount of at least one of the R channel, the G channel, and the B channel in the RGB color space of the target image according to the target white balance value.

[0029] In combination with the second aspect, in some implementations of the second aspect, the target image includes a first part and a second part, the first part and the second part are images of two different parts in an image input by an image input device, and the pre-calculation module is used to: determine a first target exposure value and a second target exposure value, and / or a first target white balance value and a second target white balance value, wherein the first target exposure value is a target exposure value corresponding to the first part, the second target exposure value is a target exposure value corresponding to the second part, the first target white balance value is a target white balance value corresponding to the first part, and the second target white balance value is a target white balance value corresponding to the second part; the processing module is used to: at the kth frame after the current frame, apply brightness compensation corresponding to the first target exposure value to the first part and apply brightness compensation corresponding to the second target exposure value to the second part through the image output device; and / or at the kth frame after the current frame, apply color compensation corresponding to the first target white balance value to the first part and apply color compensation corresponding to the second target white balance value to the second part through the image output device.

[0030] In combination with the second aspect, in certain implementations of the second aspect, the device also includes a determination module, which is used to: determine the target image from the image input by the image input device based on at least one method of target detection, sound source localization, and sound and image matching, before determining the target exposure value corresponding to the target image and / or the target white balance value corresponding to the target image.

[0031] In a third aspect, a computing device is provided, comprising a processor and a memory, wherein the memory is used to store instructions, and the processor is used to call and execute the instructions from the memory, so that the computing device executes the method in the first aspect or any possible implementation of the first aspect.

[0032] Optionally, the processor may be a general-purpose processor, which may be implemented by hardware or software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor may be a general-purpose processor, which is implemented by reading software codes stored in a memory, which may be integrated in the processor or may be located outside the processor and exist independently.

[0033] In a fourth aspect, a chip is provided, which obtains instructions and executes the instructions to implement the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0034] Optionally, as an implementation, the chip includes a processor and a data interface, and the processor reads instructions stored in the memory through the data interface to execute the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0035] Optionally, as an implementation method, the chip may also include a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the method in the above-mentioned first aspect or any possible implementation method of the first aspect.

[0036] In a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device, the computing device executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0037] In a sixth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0038] By way of example, these computer-readable storage media include, but are not limited to, one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), Flash memory, electrically EPROM (EEPROM), and hard drive.

[0039] Optionally, as an implementation manner, the above-mentioned storage medium may specifically be a non-volatile storage medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of a video conferencing terminal device to which the method for adjusting the video screen provided in an embodiment of the present application is applied.

[0041] Figure 2 It is a schematic flowchart of a method for adjusting a video screen provided in an embodiment of the present application.

[0042] Figure 3 It is a schematic flowchart of the method for adjusting the video picture provided in an embodiment of the present application in the director close-up and director smoothing scenarios.

[0043] Figure 4 It is a schematic flowchart of the method for adjusting the video screen provided in an embodiment of the present application in a director dialogue scenario.

[0044] Figure 5 It is a schematic structural block diagram of a device for adjusting a video picture provided in an embodiment of the present application.

[0045] Figure 6 It is a schematic structural block diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0047] The present application will present various aspects, embodiments or features around a system including multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all devices, components, modules, etc. discussed in conjunction with the figures. In addition, combinations of these schemes may also be used.

[0048] In addition, in the embodiments of the present application, words such as "exemplary" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present concepts in a concrete way.

[0049] In the embodiments of the present application, "corresponding (corresponding, relevant)" and "corresponding (corresponding)" can sometimes be used interchangeably. It should be pointed out that when the distinction between them is not emphasized, the meanings they intend to express are consistent.

[0050] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0051] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0052] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0053] To facilitate understanding, relevant terms and concepts that may be involved in the embodiments of the present application are first introduced below.

[0054] 1. 3A algorithm: 3A algorithm is the general name of auto white balance (AWB), auto exposure (AE) and auto focus (AF) algorithms. That is, according to the position of the subject in the picture taken by the image acquisition device and environmental factors such as the intensity and color of the light, the image acquisition related parameter values ​​such as white balance value, exposure value, focal length and focus position are automatically determined.

[0055] 2. Object detection: Find all objects of interest in the image, determine their categories and locations, and thus find specific objects in the image.

[0056] 3. Sound source localization (SSL): Use multiple microphones to measure sound signals at different locations in the environment. Since the time it takes for the sound signal to reach each microphone is delayed to varying degrees, an algorithm is used to process the measured sound signals to obtain the arrival direction (including azimuth, elevation) and distance of the sound source point relative to the microphone, thereby determining the position of the sound source in space.

[0057] 4. Audio and video matching: Taking the frames of all target people in the current picture and the spatial coordinates of the sound source as input, and mapping the spatial coordinates of the sound source to the corresponding pixel position in the image, the person making the sound in the current picture can be found. This process is called audio and video matching.

[0058] 5. Director Close-up: One of the view modes that the video conferencing terminal device can provide. When someone in the panoramic picture input by the image input device starts to speak, the picture output by the image output device is adjusted to a close-up picture of the current speaker. When the speaker changes, the output picture is adjusted to a close-up picture of the new speaker. When no one is speaking, the panoramic picture input by the image input device is output. This mode may also be called speaker positioning in other technical solutions.

[0059] 6. Director smoothing: One of the view modes that video conferencing terminal devices can provide. Based on the director close-up, when the picture switches from panoramic view to close-up, or from one close-up to another close-up, or from close-up to panoramic view, the picture does not switch directly between the two frames, but a smooth transition process. That is, there is a transition frame between the two frames before and after the switch, and the transition frame outputs the intermediate picture between the picture before and after the switch.

[0060] 7. Director Tracking: One of the view modes that video conferencing terminal devices can provide. Based on the director close-up, the close-up image will track the movement of the current speaker in real time.

[0061] 8. Director's average sharing: One of the view modes that the video conferencing terminal device can provide. When there are multiple portraits in the panoramic picture input by the image input device, the close-up images of the multiple portraits can be stitched together as the picture output by the image output device. Similarly, the close-ups of the current multiple speakers can also be stitched together. In this case, this mode can also be called director's dialogue.

[0062] With the development of video conferencing technology, the functions of video conferencing terminal devices are becoming more and more abundant. Current video conferencing terminal devices can provide multiple viewing modes, such as the director's close-up mode, which can automatically display the close-up of the current speaker and switch as the speaker changes; the director's tracking mode, which can lock the lens on a specific object and track it as the object moves; and the director's equalization mode, which can splice the close-up images of multiple participants in the conference room into one picture and present it.

[0063] In any view mode, the picture finally output by the image output device is determined according to the panoramic picture input by the image input device. The image of a panoramic picture has uniform image acquisition parameters, such as a uniform exposure value and a uniform white balance value. However, due to differences in environmental factors such as light, different parts of the panoramic picture may have differences in brightness and / or color when viewed separately, resulting in inconsistent visual experience before and after the picture switches in the view mode. For example, there are two people in the panoramic picture, and the person facing the light source will have a higher facial brightness than the person facing away from the light source. If the close-up of the person facing the light source is switched to the close-up of the person facing away from the light source without adjustment, the picture after switching will be darker than the picture before switching, making the picture look uncoordinated. Even if the white balance, exposure and focus are readjusted after the picture is switched according to the 3A algorithm, the brightness and color of the picture during the adjustment will change repeatedly, and the consistency of the visual effect of the picture cannot be continuously maintained. In addition, the 3A algorithm has worse support for more complex viewing modes. For example, the parameter values ​​of each intermediate picture are uncontrollable during director smoothing, and it cannot solve the problem of inconsistent brightness or color between two parts from the same input image in the director's averaging mode.

[0064] In view of this, an embodiment of the present application provides a method for adjusting a video screen. By determining in advance parameter values ​​such as the exposure value and / or white balance value of the target image after switching before the video screen is switched, the corresponding adjustment can be applied to the video screen according to the pre-determined parameter values ​​at the same time when the video screen is switched, so that the switching of the video screen and the adjustment of the corresponding parameters occur simultaneously, thereby maintaining the unity of the visual effect of the screen when the screen is switched in the video conference.

[0065] Figure 1 A schematic diagram of a video conferencing terminal device to which the method for adjusting a video screen provided in an embodiment of the present application is applied is shown.

[0066] like Figure 1 As shown, the video conferencing terminal device in the embodiment of the present application includes at least one image input device and at least one image output device. The image input device is used to input the captured image signal. The image input device can be an image acquisition device such as a video camera, a camera, a smart phone, or a processor such as an image signal processing (ISP) for processing the image signal input by the image acquisition device. Among them, the image acquisition device such as a video camera, a camera, a smart phone, etc. can be built-in to the video conferencing terminal device, or can be externally connected through the input interface provided by the video conferencing terminal device, and this application does not make specific limitations on this.

[0067] The cameras that can be used as image acquisition devices in the embodiments of the present application include, but are not limited to, pan / tilt / zoom (PTZ) cameras and digital pan / tilt / zoom (EPTZ) cameras. PTZ cameras adjust the shooting angle and zoom of the camera by physical means such as motors or pan / tilt heads, thereby adjusting the field of view actually captured by the camera. EPTZ cameras do not move or zoom physically, so the field of view actually captured by the camera remains unchanged. The software selects a local area in the panoramic image of the actual field of view of the camera and enlarges it to achieve the translation and zoom of the visible area of ​​the output image. In the case where the image input device includes an EPTZ camera, the images input by the image input device mentioned in the embodiments of the present application below refer to the panoramic images of the actual field of view of the EPTZ camera, rather than the images translated and zoomed by digital means.

[0068] The image output device is used to output the image signal to be displayed. The image output device can be, for example, an image display device such as a display, a television, a smart phone, or a processor such as a graphics processing unit (GPU) or a digital signal processing (DSP) for processing the image signal output to the image display device. Among them, the image display device such as a display, a television, a smart phone, etc. can be built-in to the video conferencing terminal device, or can be externally connected through the output interface provided by the video conferencing terminal device, and this application does not make specific restrictions on this.

[0069] The video conferencing terminal device in the embodiment of the present application further includes a control module for controlling the image input device and / or the image output device. The functions of the control module include but are not limited to controlling the translation, tilt and zoom of the image acquisition device, adjusting the image acquisition related parameters such as exposure value and white balance value of the image input device, and adjusting the image display related parameters such as brightness and color of the image output device.

[0070] The video conferencing terminal device in the embodiment of the present application further includes a video module for processing the image signal in the video conferencing terminal device. As an example, the video module may include a codec for encoding and decoding the image signal; it may also include a target detection submodule and / or an audio-visual matching submodule for identifying the target object from the image; and a director's average sharing / dialogue mode submodule, a director's close-up submodule, an autoframing submodule, and other submodules for implementing various viewing modes.

[0071] In order to implement the method of adjusting the video screen in the following embodiment of the present application, the video module may also include exposure pre-calculation, white balance pre-calculation, exposure interpolation, white balance interpolation, exposure post-processing, white balance post-processing and other sub-modules, which are used to determine the adjustment method of the visual effect of the video image in various intelligent scenarios. Among them, exposure pre-calculation and white balance pre-calculation are used to calculate in advance the parameters such as exposure value and white balance value suitable for the target image to be output; exposure interpolation and white balance difference are used to calculate in advance the exposure value and white balance and other parameters of the transition frame in the smoothing process of the director; exposure post-processing and white balance post-processing are used to calculate the corresponding brightness compensation and color compensation and other post-processing related parameters according to the parameters such as exposure value and white balance value.

[0072] The video conferencing terminal device in the embodiment of the present application may also include at least one audio input device and / or at least one audio output device. Similarly, the audio input device may be a device such as a microphone, or a chip for controlling the above device; the audio output device may be a device such as a speaker, earphone, or a chip for controlling the above device. The above-mentioned speakers, earphones, microphones and other devices may also be built-in to the video conferencing terminal device, or externally connected through the input / output interface provided by the video conferencing terminal device.

[0073] The audio input / output device also includes an audio module. The audio module may include a codec for encoding and decoding audio signals. The audio module may also include an audio positioning submodule for performing functions such as voice detection and sound source positioning according to audio signals input by multiple audio input devices.

[0074] The video conferencing terminal device in the embodiment of the present application may also include a network module for remotely connecting to other terminal devices via the network. For example, the image signal and audio signal processed by the video module and the audio module are sent to other video conferencing terminal devices via the Internet, so that the image and audio are output on other video conferencing terminal devices, or the image signal and audio signal collected by other terminal devices are received and output through the local image output device and audio output device. For another example, the control module can remotely control the camera of other terminal devices or receive remote control instructions from other terminal devices by sending and receiving corresponding signals through the Internet.

[0075] Combine the following Figures 2 to 5 , describes in detail the method for adjusting the video screen of the present application.

[0076] Figure 2 A schematic flowchart of a method for adjusting a video picture provided in an embodiment of the present application is shown. Figure 2 The method shown is applied to a video conference terminal device. Optionally, the video conference terminal device may be as follows: Figure 1The video conferencing terminal equipment shown in FIG. Figure 2 As shown, the method includes the following steps.

[0077] S210: Determine a target exposure value corresponding to a target image and / or a target white balance value corresponding to the target image.

[0078] For example, in step S210, the video conferencing terminal device can determine the appropriate exposure value and white balance value of the target image to which the output screen is about to switch. Specifically, the target image is the image to be output by the image output device of the video conferencing terminal device in the kth frame after the current frame, and k is a positive integer greater than or equal to 1. That is, the target image is a video screen that is ready to be output but has not yet been output, so the exposure value and / or white balance value of the target image is determined in advance before the output video screen switches to the target image. At the same time, the target image is also an image input by the image input device of the video conferencing terminal device or a part of the input image. The image input by the image input device mentioned here specifically refers to the panoramic image of the actual field of view of the image input device. That is, the scenario in which the method for adjusting the video screen provided in the present application is applied is that no matter what the actual output screen is, the video conferencing terminal device can always obtain the panoramic image, and any actual output screen is the panoramic image itself or a local focus of the panoramic image.

[0079] For example, the scenario in which the embodiment of the present application is applied can be that the EPTZ camera independently captures images. Due to the characteristics of the EPTZ camera mentioned above that the field of view of the actual captured image will not be changed, the panoramic image of the actual field of view of the EPTZ camera can be used as the image input by the image input device mentioned in the embodiment of the present application, and the image output by the image output device is determined by local cutout and scaling on the panoramic image. For the scenario in which the EPTZ camera and other types of cameras jointly capture images, the panoramic image of the actual field of view of the EPTZ camera can also be used as the image input by the image input device mentioned in the embodiment of the present application. In this case, the output image can be a partial cutout of the panoramic image of the actual field of view of the EPTZ camera, or it can be a partial image actually captured by other cameras through physical movement, zooming, and other methods, but it does not exceed the actual field of view of the EPTZ camera. The scenario in which the embodiment of the present application is applied can also be that multiple arbitrary cameras jointly capture images, such as multiple PTZ cameras. In this case, the sum of the current field of view of multiple PTZ cameras can be called a panoramic image, that is, the image input by the image input device. Taking the case of two PTZ cameras as an example, the currently output video image is a close-up image of speaker 1 captured by PTZ camera 1, and camera 2 captures a close-up image of speaker 2 and speaker 3 that will be output. In this case, the input image refers to the sum of the images captured by PTZ camera 1 and PTZ camera 2, that is, a panoramic image that includes speaker 1, speaker 2 and speaker 3. The target image can be a panoramic image, an image currently captured by PTZ camera 2, or a close-up image of speaker 2 in the image currently captured by PTZ camera 2.

[0080] It should be understood that since the embodiments of the present application are applied to a video conferencing system, the input and output of images are both real-time. In other words, the image input by the image input device is also changing in real time, and the image output by the image output device is also the whole or part of the image input by the image input device at the same time. For example, the image output by the image output device at the kth frame after the current frame is the image or part of the image input by the image input device at the kth frame after the current frame, rather than the image or part of the image input by the image input device at the current frame.

[0081] The appropriate target exposure value and target white balance value of the target image refer to the target image being able to achieve a certain visual effect after setting the image acquisition parameters to the target exposure value and / or target white balance value. For example, the target exposure value and target white balance value in the embodiment of the present application can make the target image give people a visual experience that is consistent with the current output image in terms of brightness and color. On the basis of being able to obtain the target image, the calculation of the target exposure value and the target white balance value can be achieved through the existing 3A algorithm, including but not limited to the gray world method, the perfect reflection wave method, the dynamic threshold method, etc., and the present application does not make specific limitations on this.

[0082] Optionally, before determining the target exposure value corresponding to the target image and / or the target white balance value corresponding to the target image, the video conferencing terminal device may also first determine the target image from the image input by the image input device. Depending on the view mode and the switching scene, the method of determining the target image may be different. For example, in the director close-up or director smooth mode, when the video screen needs to be switched from a close-up to a panoramic view, the video conferencing terminal device can monitor the sound in the environment through the audio input device. When there is no voice input for a certain period of time, the panoramic image input by the image input device can be directly determined as the target image. When the video screen needs to switch from a panoramic view to a close-up or from one close-up to another close-up, the video conferencing terminal device can determine the target image from the image input by the image input device according to at least one method of target detection, sound source localization, and sound and image matching. Specifically, image recognition can be performed by target detection methods, such as identifying the outline of characters in a panoramic image and the facial movements of each character to determine the spokesperson; speech recognition can also be performed by sound source localization methods to determine the spatial coordinates of the sound source and determine the speaker's position based on the correspondence between the coordinate system in the panoramic image and the spatial coordinate system; or sound and image matching can be performed by combining image recognition technology and speech recognition technology, combining the outline of the character identified in the image and the spatial coordinates of the located spokesperson to determine the spokesperson's position, thereby determining the local image of the inventor's position as the target image. Similarly, for the director's equal sharing mode, multiple spokesperson positions can be determined first according to the above method, and the local image of each spokesperson's position can be cut out, and the image formed by cutting out and splicing can be determined as the target image.

[0083] Through the above scheme, the image content to be output can be accurately and quickly located in advance from the input image, thereby providing a prerequisite for accurately and quickly determining the exposure value and / or white balance value and other parameter values ​​of the target image, thereby improving the accuracy and efficiency of video picture adjustment.

[0084] S220: At the kth frame after the current frame, output a target image according to the target exposure value and / or the target white balance value.

[0085] For example, in step S220, the video conference terminal device can apply the corresponding effect to the target image according to the appropriate target exposure value and / or target white balance value of the target image determined in advance in S210, when the output video picture switches to the target image at the kth frame, so that the target image presents the expected visual effect. It should be understood that, as mentioned above, the output target image here refers to the real-time picture input by the image input device at the kth frame of the target image area.

[0086] Through the technical solution of the embodiments of the present application, by determining in advance the exposure value and / or white balance value and other parameter values ​​of the target image after switching before the video screen is switched, the corresponding adjustment can be applied to the video screen according to the pre-determined parameter values ​​at the same time when the video screen is switched, so that the switching of the video screen and the adjustment of the corresponding parameters occur simultaneously, thereby maintaining the unity of the visual effect of the screen when the screen is switched in the video conference.

[0087] For the above step S220, the visual effect corresponding to the target exposure value and / or the target white balance value is presented on the target image. This can be achieved at the input end of the image or at the output end of the image. The two situations are described in detail below.

[0088] In some possible implementations, the video conferencing terminal device may apply the target exposure value and / or target white balance value to the image input device at the kth frame after the current frame. Since the exposure value and the white balance value are both parameters related to image acquisition, the exposure value and / or the white balance value of the image input device can be directly set to the target exposure value and / or the target white balance value. Specifically, for an image acquisition device whose parameters related to image acquisition can be adjusted physically, such as a SLR camera, the target exposure value and / or the target white balance value can be directly set to the image acquisition device, and the image acquisition device achieves the target exposure value and / or the target white balance value by adjusting the lens group; for an image acquisition device whose parameters related to image acquisition can be adjusted digitally, such as an EPTZ camera, the target exposure value and / or the target white balance value can be configured to the ISP connected to the image acquisition device, and the adjustment of the exposure value and / or the white balance value of the image acquisition device is controlled by the processor.

[0089] Through the technical solution of the embodiments of the present application, by directly applying the determined exposure value and / or white balance value to the image input device, since image acquisition parameters such as the exposure value and the white balance value are directly determined according to the environmental changes of the captured image, the brightness and color of the image can be conveniently and accurately adjusted at the input end without the need for other intermediate quantity conversion processes, thereby reducing the calculation complexity and improving the accuracy and efficiency of video image adjustment.

[0090] In some other possible implementations, the video conferencing terminal device can determine the brightness compensation of the target image and / or the color compensation of the image according to the target exposure value and / or the target white balance value, and apply the brightness compensation and / or the color compensation to the target image through the image output device at the kth frame after the current frame. In some cases, the exposure value and the white balance value may not be directly set to the image input device, for example, the target exposure value and / or the target white balance value exceed the settable range of the image input device, and for example, two different exposure value and white balance value combinations need to be applied to an input image in the director averaging mode. Therefore, it is also possible to convert image acquisition-related parameters such as exposure value and white balance value into brightness compensation and color compensation in the image post-processing process through an algorithm, so as to achieve an effect similar to adjusting the exposure value and white balance value through image post-processing at the output end. Specifically, the above-mentioned brightness compensation corresponds to the target exposure value, and the color compensation corresponds to the target white balance value.

[0091] The exposure time during image acquisition affects the total amount of light passing through the image acquisition device, so the change in exposure value can be reflected as a change in brightness-related parameters during image post-processing. For example, the change in the Y channel in the YUV color space of the target image can be determined based on the exposure value. The Y flux in the YUV color space is a flux that reflects the brightness of the image independently of the other two chromaticity fluxes, so the change in the Y flux can adjust the brightness of the image without affecting the color of the image. For example, if the target exposure value is larger than the exposure value of the current input image, the video conferencing terminal device can increase the Y channel of each pixel in the target image by a certain value without actually changing the exposure value (the exposure value of the target image still uses the exposure value of the current input image), thereby achieving an effect similar to increasing the exposure value. Through the above scheme, the exposure value during image acquisition is converted into the brightness represented by the Y channel during image processing, and the required change in the exposure value can be compensated by changing the value of the Y channel, thereby achieving the adjustment of the brightness of the image in the post-processing process, achieving an effect similar to changing the exposure value when acquiring the image.

[0092] During image acquisition, the white balance value affects the overall color of the image by affecting the color temperature of the acquired image. Therefore, changes in the white balance value can be reflected as changes in color-related parameters during image post-processing. For example, the amount of change in at least one of the R channel, G channel, and B channel in the RGB color space of the target image can be determined based on the target white balance value. For example, the corresponding color temperature can be first looked up in a table or calculated based on the white balance value. Since the higher the color temperature, the bluer the overall color of the picture, and the lower the color temperature, the redder the overall color of the picture, the G channel can be normalized to convert the color temperature into the color gain coefficients of the R / G and B / G channels. Multiply the value of the R channel by the color gain coefficient of the R / G channel, and multiply the value of the B channel by the color gain coefficient of the B / G channel, and recalculate the RGB color value of each pixel in the target image:

[0093] R'=R×δ R / G ,

[0094] B'=B×δ R / G ,

[0095] G'=G

[0096] Among them, R', B' and G' respectively represent the color values ​​of the pixels in the target image in the R channel, B channel and G channel, R, B and G respectively represent the color values ​​of the pixels in the corresponding position in the input image in the R channel, B channel and G channel, δ R / G Indicates the color gain coefficient of the R / G channel, δ R / G Represents the color gain coefficient of the B / G channel. The above formula can achieve an effect similar to that of adjusting the white balance value without actually changing the white balance value. Through the above scheme, the white balance value during image acquisition is converted into the red, green and blue color values ​​represented by the R channel, G channel and B channel during image processing. The required change in the white balance value can be compensated by changing the value of at least one of the R channel, G channel and B channel, thereby adjusting the image color in the post-processing process, achieving an effect similar to that of changing the white balance value when acquiring the image.

[0097] Through the technical solution of the embodiments of the present application, without changing the image acquisition related parameters such as the exposure value or white balance value of the image input device or the related parameters cannot be changed, by converting the image acquisition related parameters into brightness compensation and color compensation when the image output device post-processes the image, the brightness and color of the image can be adjusted at the output end to be similar to the effect at the input end, thereby improving the flexibility and scope of application of the video picture adjustment method.

[0098] Combine the following Figure 3 and Figure 4 , the application of the method for adjusting the video screen provided in the embodiment of the present application in a specific view mode is explained.

[0099] Figure 3 The schematic flow chart of the method for adjusting the video screen provided by the embodiment of the present application in the application scenarios of close-up and smooth directing is shown. Figure 3 As shown, after starting the director close-up view mode in the panoramic mode, the video conferencing terminal device first determines the local image of the speaker's location, that is, the target image mentioned above. The method for determining the target image is as described in the above step S210, including but not limited to one or more methods of target detection, sound source localization, and sound and image matching. After pre-calculating the appropriate target exposure value and / or target white balance value of the target image, the target exposure value and / or target white balance value are applied to the target image in different ways depending on whether the director smoothing mode is enabled.

[0100] When the director smoothing mode is not enabled, the video conferencing terminal device directly switches to the target image in the next frame of the current frame after determining the target exposure value and / or the target white balance value. That is, the case of step S220 when k=1. In this case, since the switching of the video screen is in the direct cut mode, the pre-calculated target exposure value and / or target white balance value is directly applied to the target image in the first frame after the switching frame, that is, the current frame.

[0101] It should be understood that the application of the pre-calculated target exposure value and / or target white balance value to the target image mentioned here is the same as the output of the target image according to the target exposure value and / or target white balance value mentioned in S220 above. That is, the target exposure value and / or target white balance value can be applied to the target image by directly adjusting the image acquisition related parameters at the image input end, such as applying the pre-calculated value to the ISP, or can be applied to the target image at the image output end by converting the target exposure value and / or target white balance value into corresponding post-processing related parameters, such as using a GPU to post-process the target image.

[0102] When the director smoothing mode is not enabled, that is, when k>1 in step S220, the video conference terminal device first calculates the exposure value and / or white balance value corresponding to each of the k-1 transition frames between the current frame and the kth frame after the current frame by interpolation, and then applies the corresponding exposure value and / or white balance value to the position of each transition frame during the video screen switching process. Similarly, the method of applying the corresponding exposure value and / or white balance value to the transition frame can be achieved by directly adjusting the image acquisition related parameters at the image input end, or by adjusting the corresponding post-processing related parameters at the image output end, which will not be repeated here.

[0103] Specifically, the method for achieving smooth broadcasting by interpolation is to fit the exposure value and / or white balance value of the image input device at the current frame and the target exposure value and / or target white balance value to determine the exposure value and / or white balance value corresponding to the i-th frame after the current frame, where i is a positive integer greater than or equal to 1 and less than k. Taking the interpolation calculation process of the exposure value as an example, assuming that the exposure value of the current frame (the original speaker) is EV s , the pre-calculated exposure value of the target frame (target speaker) is EV e , the total number of smoothed frames / steps is k, then the exposure value EV of the i-th frame after the current frame is i The result of a linear fit is:

[0104] EV i =EV e +i×(EV s -EV e ) / k

[0105] Among them, EV i Indicates the exposure value corresponding to the i-th frame after the current frame, EV e Indicates the target exposure value of the target image, EV s represents the exposure value of the image input device at the current frame, k represents the frame number of the target image after the current frame, and i represents the frame number of the transition frame after the current frame. It should be understood that the above example only uses a linear fit of the exposure value as an example to illustrate the interpolation method, and other fitting methods such as a quadratic curve can also be used to fit the exposure value and / or white balance value, and this application does not make specific limitations on this.

[0106] Through the technical solution of the embodiment of the present application, image acquisition parameters such as the exposure value and / or white balance value of at least one transition frame between the two frames before and after the switching are calculated by interpolation, so that while maintaining the uniformity of the visual effects of the front and back pictures, the brightness and color of the intermediate picture during the switching process also change evenly, thereby improving the consistency of the visual effect of the video picture when the lens switches smoothly.

[0107] Figure 4 FIG. 1 is a schematic flow chart showing a method for adjusting a video screen provided by an embodiment of the present application in a director dialogue application scenario. Figure 4As shown, after starting the director dialogue view mode in the panoramic mode, the video conference terminal device first determines the local image of each target person in the multiple target persons, that is, the components used to stitch together the target image. After pre-calculating the appropriate exposure value and / or white balance value for each local image, the pre-calculated exposure value and / or white balance value is converted into corresponding brightness compensation and color compensation, and each component of the target image is post-processed through an image output device such as a GPU or DSP, and finally the processed components are stitched together to display the target image.

[0108] For example, take the target image as being composed of two parts, which are respectively called the first part and the second part. The video conference terminal device first determines the first target exposure value and the second target exposure value, and / or the first target white balance value and the second target white balance value, wherein the first target exposure value is the target exposure value corresponding to the first part, the second target exposure value is the target exposure value corresponding to the second part, the first target white balance value is the target white balance value corresponding to the first part, and the second target white balance value is the target white balance value corresponding to the second part. Then, at the kth frame after the current frame, the brightness compensation corresponding to the first target exposure value is applied to the first part and the brightness compensation corresponding to the second target exposure value is applied to the second part through the image output device, and / or, at the kth frame after the current frame, the color compensation corresponding to the first target white balance value is applied to the first part and the color compensation corresponding to the second target white balance value is applied to the second part through the image output device.

[0109] Through the technical solution of the embodiment of the present application, when the target image after switching is composed of multiple parts of an image input by the image input device, since each part can only use the same image acquisition related parameters such as exposure value and white balance value, by independently determining the image acquisition parameters suitable for each part and converting them into brightness compensation and / or color compensation related to image post-processing, it is possible to achieve different degrees of brightness and color adjustment for different parts of the same input image, so that the different parts have a unified visual effect after being spliced ​​together, thereby improving the consistency of the visual effect of the video picture under the splicing view.

[0110] Combination of the above Figures 2 to 4 The method for adjusting the video screen provided by the present application is described below. Figures 5 and 6 , an embodiment of the device for adjusting the video picture provided in this application is described.

[0111] Figure 5 A schematic structural block diagram of a device 500 for adjusting a video picture provided in an embodiment of the present application is shown.

[0112] like Figure 5As shown, the device 500 includes: a pre-calculation module 520 and a processing module 530.

[0113] Specifically, the pre-calculation module 520 is used to determine a target exposure value corresponding to a target image and / or a target white balance value corresponding to a target image, wherein the target image is an image input by an image input device or a partial image of an image input by the image input device, and the target image is an image to be output by an image output device in the kth frame after a current frame, and k is a positive integer greater than or equal to 1.

[0114] Specifically, the processing module 530 is used to output a target image according to a target exposure value and / or a target white balance value at the kth frame after the current frame.

[0115] Optionally, the processing module 530 is specifically configured to apply the target exposure value and / or the target white balance value to the image input device at the kth frame after the current frame.

[0116] Optionally, the processing module 530 is specifically used to fit the exposure value and / or white balance value of the image input device at the current frame and the target exposure value and / or target white balance value at the kth frame after the current frame, when k is greater than 1, to determine the exposure value and / or white balance value corresponding to the i-th frame after the current frame, where i is a positive integer greater than or equal to 1 and less than k; at the i-th frame after the current frame, apply the exposure value and / or white balance value corresponding to the i-th frame after the current frame to the image input device.

[0117] Optionally, the processing module 530 is specifically used to determine the brightness compensation of the target image and / or the color compensation of the image according to the target exposure value and / or the target white balance value, wherein the brightness compensation corresponds to the target exposure value and the color compensation corresponds to the target white balance value; at the kth frame after the current frame, the brightness compensation and / or the color compensation is applied to the target image through the image output device.

[0118] Optionally, the processing module 530 is specifically configured to determine a change amount of a Y channel in a YUV color space of a target image according to a target exposure value.

[0119] Optionally, the processing module 530 is specifically configured to determine a change amount of at least one of an R channel, a G channel, and a B channel in an RGB color space of the target image according to the target white balance value.

[0120] Optionally, the target image includes a first part and a second part, the first part and the second part are images of two different parts in an image input by an image input device, and the pre-calculation module is specifically used to determine a first target exposure value and a second target exposure value, and / or a first target white balance value and a second target white balance value, wherein the first target exposure value is a target exposure value corresponding to the first part, the second target exposure value is a target exposure value corresponding to the second part, the first target white balance value is a target white balance value corresponding to the first part, and the second target white balance value is a target white balance value corresponding to the second part; the processing module is specifically used to apply, through the image output device, at a k-th frame after the current frame, brightness compensation corresponding to the first target exposure value to the first part, and brightness compensation corresponding to the second target exposure value to the second part, and / or, through the image output device, at a k-th frame after the current frame, color compensation corresponding to the first target white balance value to the first part, and color compensation corresponding to the second target white balance value to the second part.

[0121] Optionally, the device 500 may also include a determination module 510, which is used to determine the target image from the image input by the image input device according to at least one method of target detection, sound source localization, and sound and image matching before determining the target exposure value corresponding to the target image and / or the target white balance value corresponding to the target image.

[0122] The above modules can be implemented by software or hardware. For example, the implementation of the processing module 530 is described below by taking the processing module 530 as an example. Similarly, the implementation of the determination module 510 and the pre-calculation module 520 can refer to the implementation of the processing module 530.

[0123] As an example of a software functional unit, the processing module 530 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the processing module 530 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0124] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0125] As an example of a hardware functional unit, the processing module 530 may include at least one computing device, such as a server, etc. Alternatively, the processing module 530 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0126] The multiple computing devices included in the processing module 530 can be distributed in the same region or in different regions. The multiple computing devices included in the processing module 530 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the processing module 530 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0127] It should be noted that, in other embodiments, the determination module 510, the pre-calculation module 520 and the processing module 530 can be used to execute any step in the above-mentioned method for adjusting the video screen, and the steps that the determination module 510, the pre-calculation module 520 and the processing module 530 are responsible for implementing can be specified as needed. By respectively implementing different steps in the above-mentioned method for adjusting the video screen, the full functions of the operation and maintenance channel management cluster can be realized.

[0128] The present application also provides a computing device 100. Figure 6As shown, the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other through the bus 102. The computing device 100 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 100.

[0129] The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus 102 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 102 may include a path for transmitting information between various components of the computing device 100 (eg, the memory 106, the processor 104, and the communication interface 108).

[0130] The processor 104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0131] The memory 106 may include a volatile memory, such as a random access memory (RAM). The processor 104 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0132] The memory 106 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the functions of the aforementioned determination module, pre-calculation module, and processing module, thereby implementing the aforementioned method for adjusting the video picture. That is, the memory 106 stores instructions for executing the aforementioned method for adjusting the video picture.

[0133] The communication interface 108 uses a command distribution module such as, but not limited to, a network interface card, a transceiver, etc. to implement communication between the computing device 100 and other devices or a communication network.

[0134] An embodiment of the present application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to execute the above-mentioned method for adjusting the video screen.

[0135] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the above-mentioned method for adjusting the video screen.

[0136] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state hard disk), etc. The computer-readable storage medium includes instructions, and the instructions instruct the computing device to perform the above-mentioned management. The various technical features can be combined arbitrarily. In order to make the description concise, all possible combinations of the various technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for adjusting a video screen, characterized in that: The method is applied to a video conference terminal device, the video conference terminal device includes an image input device and an image output device, and the method includes: Determine a target exposure value corresponding to a target image and / or a target white balance value corresponding to the target image, wherein the target image is an image input by the image input device or a partial image of the image input by the image input device, and the target image is an image to be output by the image output device in the kth frame after the current frame, where k is a positive integer greater than or equal to 1; At the kth frame after the current frame, the target image is output according to the target exposure value and / or the target white balance value.

2. The method according to claim 1, characterized in that At the kth frame after the current frame, outputting the target image according to the target exposure value and / or the target white balance value comprises: At a k-th frame after the current frame, the target exposure value and / or the target white balance value is applied to the image input device.

3. The method according to claim 2, characterized in that When k is greater than 1, at the kth frame after the current frame, outputting the target image according to the target exposure value and / or the target white balance value, further comprising: Fitting the exposure value and / or white balance value of the image input device at the current frame and the target exposure value and / or the target white balance value to determine the exposure value and / or white balance value corresponding to the i-th frame after the current frame, where i is a positive integer greater than or equal to 1 and less than k; At the i-th frame after the current frame, the exposure value and / or the white balance value corresponding to the i-th frame after the current frame is applied to the image input device.

4. The method according to claim 1, characterized in that: At the kth frame after the current frame, outputting the target image according to the target exposure value and / or the target white balance value comprises: Determining brightness compensation of the target image and / or color compensation of the image according to the target exposure value and / or the target white balance value, wherein the brightness compensation corresponds to the target exposure value, and the color compensation corresponds to the target white balance value; At the kth frame after the current frame, the brightness compensation and / or the color compensation is applied to the target image through the image output device.

5. The method according to claim 4, characterized in that The step of determining the brightness compensation of the target image and / or the color compensation of the image according to the target exposure value and / or the target white balance value comprises: According to the target exposure value, a change amount of a Y channel in a YUV color space of the target image is determined.

6. The method according to claim 4 or 5, characterized in that: The step of determining the brightness compensation of the target image and / or the color compensation of the image according to the target exposure value and / or the target white balance value comprises: According to the target white balance value, a change amount of at least one of an R channel, a G channel, and a B channel in an RGB color space of the target image is determined.

7. The method according to any one of claims 4 to 6, characterized in that The target image includes a first part and a second part, wherein the first part and the second part are images of two different parts of the image input by the image input device. The determining of a target exposure value corresponding to the target image and / or a target white balance value corresponding to the target image includes: Determining a first target exposure value and a second target exposure value, and / or a first target white balance value and a second target white balance value, wherein the first target exposure value is a target exposure value corresponding to the first part, the second target exposure value is a target exposure value corresponding to the second part, the first target white balance value is a target white balance value corresponding to the first part, and the second target white balance value is a target white balance value corresponding to the second part; The step of applying the brightness compensation and / or the color compensation to the target image by the image output device at the kth frame after the current frame includes: At the kth frame after the current frame, applying the brightness compensation corresponding to the first target exposure value to the first part and applying the brightness compensation corresponding to the second target exposure value to the second part through the image output device; and / or At the kth frame after the current frame, the image output device applies the color compensation corresponding to the first target white balance value to the first part, and applies the color compensation corresponding to the second target white balance value to the second part.

8. The method according to any one of claims 1 to 7, characterized in that Before determining a target exposure value corresponding to the target image and / or a target white balance value corresponding to the target image, the method further includes: The target image is determined from the image input by the image input device according to at least one method of target detection, sound source localization, and sound-image matching.

9. A device for adjusting a video image, characterized in that: The device is arranged in a video conference terminal device, and the video conference terminal device further comprises an image input device and an image output device, and the device comprises: a pre-calculation module, used to determine a target exposure value corresponding to a target image and / or a target white balance value corresponding to the target image, wherein the target image is an image input by the image input device or a partial image of the image input by the image input device, and the target image is an image to be output by the image output device in the kth frame after the current frame, where k is a positive integer greater than or equal to 1; A processing module is used to output the target image according to the target exposure value and / or the target white balance value at the kth frame after the current frame.

10. The device according to claim 9, characterized in that The processing module is used for: At a k-th frame after the current frame, the target exposure value and / or the target white balance value is applied to the image input device.

11. The device according to claim 10, characterized in that When k is greater than 1, the processing module is used to: Fitting the exposure value and / or white balance value of the image input device at the current frame and the target exposure value and / or the target white balance value to determine the exposure value and / or white balance value corresponding to the i-th frame after the current frame, where i is a positive integer greater than or equal to 1 and less than k; At the i-th frame after the current frame, the exposure value and / or the white balance value corresponding to the i-th frame after the current frame is applied to the image input device.

12. The device according to claim 9, characterized in that The processing module is used for: Determining brightness compensation of the target image and / or color compensation of the image according to the target exposure value and / or the target white balance value, wherein the brightness compensation corresponds to the target exposure value, and the color compensation corresponds to the target white balance value; At the kth frame after the current frame, the brightness compensation and / or the color compensation is applied to the target image through the image output device.

13. The device according to claim 12, characterized in that The processing module is used for: According to the target exposure value, a change amount of a Y channel in a YUV color space of the target image is determined.

14. The device according to claim 12 or 13, characterized in that The processing module is used for: According to the target white balance value, a change amount of at least one of an R channel, a G channel, and a B channel in an RGB color space of the target image is determined.

15. The device according to any one of claims 12 to 14, characterized in that: The target image includes a first part and a second part, wherein the first part and the second part are images of two different parts of the image input by the image input device. The pre-calculation module is used for: Determining a first target exposure value and a second target exposure value, and / or a first target white balance value and a second target white balance value, wherein the first target exposure value is a target exposure value corresponding to the first part, the second target exposure value is a target exposure value corresponding to the second part, the first target white balance value is a target white balance value corresponding to the first part, and the second target white balance value is a target white balance value corresponding to the second part; The processing module is used for: At the kth frame after the current frame, applying the brightness compensation corresponding to the first target exposure value to the first part and applying the brightness compensation corresponding to the second target exposure value to the second part through the image output device; and / or At the kth frame after the current frame, the image output device applies the color compensation corresponding to the first target white balance value to the first part, and applies the color compensation corresponding to the second target white balance value to the second part.

16. The device according to any one of claims 9 to 15, characterized in that The device further includes a determination module, which is used, before determining a target exposure value corresponding to a target image and / or a target white balance value corresponding to the target image, to: The target image is determined from the image input by the image input device according to at least one method of target detection, sound source localization, and sound-image matching.

17. A computing device, characterized in that The computer comprises a processor and a memory, wherein the processor is configured to execute instructions stored in the memory so that the computing device executes the method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that: The method comprises computer program instructions which, when executed by a computing device, cause the computing device to perform the method according to any one of claims 1 to 8.

Images