Video white balance adjustment method, device, equipment, storage medium and program product

Abstract

This invention relates to the field of medical device technology, and discloses a video white balance adjustment method, apparatus, device, storage medium, and program product. The method includes: downsampling image frames in the video to be processed based on a preset sampling range to obtain sampled images; analyzing the sampled images using a target network model to obtain white balance parameters, wherein the target network model is used to analyze the gain of each color channel in the sampled image to obtain the white balance parameters of the image frames; and calibrating the image frames in the video to be processed sequentially based on the white balance parameters to obtain the target video. This invention eliminates the need to rely on color temperature-gain curves to determine the gain value of image frames to determine white balance parameters, thus adapting to the automatic white balance requirements in special application scenarios such as endoscopy.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, specifically to video white balance adjustment methods, devices, equipment, storage media, and program products. Background Technology

[0002] An endoscope is a medical device. Due to the limitations of the usage environment, multiple light sources are usually fused to acquire images when using an endoscope, such as the fusion of white light and fluorescence. This can lead to distortion and misalignment in the acquired images, increasing the difficulty of achieving image white balance.

[0003] In relevant white balance adjustment schemes, to overcome the reliance on manual operation in manual white balance, automatic white balance is typically used to determine the image's gain value, which is then used to adjust the image's white balance. Specifically, statistical methods can be used to collect data from light sources of different color temperatures, and a color temperature-gain curve can be generated based on the collected results. The image's gain value can then be determined by comparing the image's color temperature data with this color temperature-gain curve. However, statistical methods often fail in special scenarios due to the inability to collect data. Summary of the Invention

[0004] In view of this, the present invention provides a balance adjustment method, apparatus, device, storage medium and program product to solve the problem that white balance algorithms such as statistical methods often fail in special application scenarios such as endoscopy due to the inability to collect data.

[0005] In a first aspect, the present invention provides a video white balance adjustment method, the method comprising: The image frames in the video to be processed are downsampled based on a preset sampling range to obtain a sampled image; The white balance parameters are obtained by analyzing the sampled image through the target network model. The target network model is used to analyze the gain of each color channel in the sampled image to obtain the white balance parameters of the image frame. The target video is obtained by sequentially calibrating the image frames in the video to be processed based on the white balance parameters.

[0006] In this embodiment of the invention, image frames in the video to be processed are first downsampled based on a preset sampling range to obtain sampled images. These sampled images are then analyzed using a target network model to obtain white balance parameters. The target network model analyzes the gain of each color channel in the sampled image to obtain the white balance parameters of the image frames. Then, the image frames in the video to be processed are sequentially calibrated based on the white balance parameters to obtain the target video. This eliminates the need to rely on a color temperature-gain curve to determine the gain value of the image frames and thus the white balance parameters, adapting to the automatic white balance requirements of special application scenarios such as endoscopy.

[0007] In one optional implementation, the sampled image is analyzed using a target network model to obtain white balance parameters, including: Analyze the color cast data of the sampled images using a target network model; Based on color bias data and preset weights, the gain ratio of each color channel in the sampled image is determined. The preset weights are used to indicate the weights assigned to each color channel in the target network model. The white balance parameters are determined based on the gain ratio.

[0008] In this embodiment of the invention, considering that different endoscope models may correspond to different color spaces in the application scenarios of endoscopes, or that users' focus on observing image frames may not be on the reproduction of scene colors, the weights of each color channel can be set during automatic and manual white balance adjustments to adjust the color cast of each color channel, thereby increasing the applicability of the invention.

[0009] In one optional implementation, analyzing the color cast data of the sampled image using a target network model includes: The brightness values ​​of the sampled image are extracted using the target network model; The color cast data is obtained by analyzing the color cast values ​​of each color channel in the sampled image at the current brightness value.

[0010] In this embodiment of the invention, color deviation data can be determined based on the color deviation values ​​of each color channel in the sampled image at the current brightness value, thereby providing a technical basis for determining the gain ratio of each color channel in the sampled image based on the color deviation data and preset weights.

[0011] In one alternative implementation, the method further includes: Obtain color preference settings and determine preset weights based on color preference settings; The model parameters of the network model to be trained are adjusted based on preset weights, and the adjusted network model is trained to obtain the target network model.

[0012] In this embodiment of the invention, considering that different users have different usage habits when using endoscopes, and that the requirements for color tone also differ in different application environments, a color tone preference parameter can be added when training the above-mentioned network model to be trained, so as to better meet the user's application needs and improve the user experience.

[0013] In one optional implementation, the number of sampled images is multiple, and the white balance parameters are obtained by analyzing the sampled images through a target network model, further including: The white balance sub-parameters for each sampled image are obtained by analyzing multiple sampled images using the target network model. The white balance parameters are fitted based on the white balance sub-parameters, and the white balance parameters are determined based on the fitting results.

[0014] In this embodiment of the invention, when there are multiple sampled images, the white balance parameters can be determined based on the white balance sub-parameters of each sampled image object to obtain the optimal white balance parameters for the entire image frame, thereby improving the adjustment accuracy of the white balance mechanism and enhancing the user experience.

[0015] In one optional implementation, image frames in the video to be processed are downsampled based on a preset sampling range to obtain a sampled image, including: The preset sampling range in the image frame is determined based on the sampling configuration; The sampling location is determined based on the number of preset sampling ranges; The image is downsampled by a preset sampling range based on the sampling location to obtain a sampled image.

[0016] In this embodiment of the invention, when setting the sampling configuration, the user can set the sampling range according to the usage scenario to obtain a preset sampling range. For example, if the user wants to focus on observing a specific target area, considering that the main part the user is interested in is usually in the center of the field of view, the white balance effect in the central area is the best parameter for the entire image frame. Therefore, the preset sampling range can be set to the central area of ​​the image frame. Alternatively, if the user's observation has no focus and the observation requirement is the entire field of view, multiple preset negative sampling ranges can be set and distributed as evenly as possible in the image frame. This allows for the determination of the best white balance parameter for the entire image frame based on these multiple preset sampling ranges, thereby improving the adjustment accuracy of the white balance mechanism and enhancing the user experience.

[0017] In one optional implementation, determining the sampling location based on the number of preset sampling ranges includes: When the number of preset sampling ranges is one, the sampling position is determined based on the center position of the image frame; or, When there are multiple preset sampling ranges, the spacing value between the preset sampling ranges is calculated based on the size of the preset sampling range and the image frame, and the sampling position is determined according to the spacing value. The spacing values ​​between the preset sampling ranges may be the same or different.

[0018] In this embodiment of the invention, when setting the sampling configuration, the user can set the sampling range according to the usage scenario to obtain a preset sampling range. For example, if the user wants to focus on observing a specific target area, considering that the main part the user is interested in is usually in the center of the field of view, the white balance effect in the central area is the best parameter for the entire image frame. Therefore, the preset sampling range can be set to the central area of ​​the image frame. Alternatively, if the user's observation has no focus and the observation requirement is the entire field of view, multiple preset negative sampling ranges can be set and distributed as evenly as possible in the image frame. This allows for the determination of the best white balance parameter for the entire image frame based on these multiple preset sampling ranges, thereby improving the adjustment accuracy of the white balance mechanism and enhancing the user experience.

[0019] In a second aspect, the present invention provides a video white balance adjustment device, the device comprising: The sampling module is used to downsample image frames in the video to be processed based on a preset sampling range to obtain a sampled image; The analysis module is used to analyze the sampled image through the target network model to obtain white balance parameters; The calibration module is used to calibrate the image frames in the video to be processed sequentially based on the white balance parameters to obtain the target video.

[0020] Thirdly, the present invention provides a computer device, including: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the video white balance adjustment method described in the first aspect or any corresponding embodiment thereof.

[0021] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the video white balance adjustment method described in the first aspect or any corresponding embodiment thereof.

[0022] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the video white balance adjustment method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1This is a flowchart of a video white balance adjustment method according to an embodiment of the present invention; Figure 2 This is a flowchart of another video white balance adjustment method according to an embodiment of the present invention; Figure 3 This is a schematic diagram showing the sampling position determined based on the center position of the image frame; Figure 4 This is a schematic diagram illustrating the determination of multiple sampling positions within an image frame; Figure 5 This is a structural block diagram of a video white balance adjustment device according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] The application scenarios on which the video white balance adjustment method depends are described here.

[0027] An endoscope is a medical device that can be inserted into the human body through natural orifices or small surgical incisions to observe changes in relevant areas or to perform corresponding procedures on the observed areas through the working channel. Therefore, due to limitations in the operating environment, endoscopes typically employ a method of fusing multiple light sources for image acquisition, such as the fusion of white light and fluorescence. This results in distortion and misalignment of the acquired images, increasing the difficulty of achieving proper white balance.

[0028] In existing white balance adjustment schemes, to avoid reliance on manual white balance, automatic white balance is typically used to determine the image's gain value, which is then used to adjust the image's white balance. Specifically, statistical methods can be used to collect data from light sources of different color temperatures, and a color temperature-gain curve can be generated based on the collected data. The image's gain value is then determined by comparing the image's color temperature data with this color temperature-gain curve. However, statistical methods often fail in special scenarios due to the inability to collect data. Therefore, there is an urgent need for an automatic white balance scheme applicable to special scenarios such as endoscopy use.

[0029] This invention provides a video white balance adjustment method. First, image frames in the video to be processed are downsampled based on a preset sampling range to obtain sampled images. Then, a target network model is used to analyze the sampled images to obtain white balance parameters. The target network model analyzes the gain of each color channel in the sampled images to obtain the white balance parameters of the image frames. Next, the image frames in the video to be processed are sequentially calibrated based on the white balance parameters to obtain the target video. This eliminates the need to rely on a color temperature-gain curve to determine the gain value of the image frames to determine the white balance parameters, thus adapting to the automatic white balance requirements of special application scenarios such as endoscopy.

[0030] According to an embodiment of the present invention, a video white balance adjustment method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] This embodiment provides a video white balance adjustment method, which can be used in the aforementioned medical devices such as endoscopes. Figure 1 This is a flowchart of a video white balance adjustment method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Downsample the image frames in the video to be processed based on a preset sampling range to obtain a sampled image.

[0032] In this embodiment of the invention, the video to be processed can be a video captured by a camera in an endoscope. The camera can send the captured video to be processed back to the host for processing, so that the host can perform white balance calibration on the image frames in the video to be processed.

[0033] Considering that the image frames in the video to be processed are usually 4K images, using all pixels in the full-size image frame to calculate the white balance parameters may result in excessive computation. Therefore, the image frames in the video to be processed can be downsampled to obtain sampled images.

[0034] In practice, a white balance parameter estimation window can be preset, and the image within this window can be downsampled to obtain a sampled image. The position and size of the white balance parameter estimation window can be determined according to actual usage requirements.

[0035] For example, the position of the white balance parameter estimation window can be determined based on the target being examined by the endoscope. If the target is easily observable, and considering that the main subject being observed is usually in the center of the field of view, the white balance parameter estimation window can be set to the center of the image frame. Alternatively, if the field of view is limited during examination, the position of the white balance parameter estimation window can be adjusted based on the position of the observed target.

[0036] When determining the size of the white balance parameter estimation window, the window size can be adjusted to suit the computing resources of the host computer in the endoscope. For example, if the remaining computing resources of the host computer are insufficient to support the processing of the sampled image corresponding to the current window size during actual use, the window size can be reduced to avoid limiting the processing of subsequent image frames.

[0037] Step S102: The sampled image is analyzed by the target network model to obtain the white balance parameters. The target network model is used to analyze the gain of each color channel in the sampled image to obtain the white balance parameters of the image frame.

[0038] In this embodiment of the invention, the target neural network can be a lightweight deep learning network, so that the target neural network can be deployed on the GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), or FPGA (Field Programmable Gate Array) chip of the host.

[0039] Specifically, deep learning networks can be pruned and retrained. The deep neural network can be a convolutional neural network (CNN) or similar network used for image data processing. The resulting lightweight deep learning network, after pruning, learns color deviations under different lighting conditions to obtain white balance parameters, and automatically adjusts image frames based on these parameters to restore natural colors. The specific deep neural network, corresponding pruning strategies, and retraining methods are described in detail below, and will not be elaborated further in this disclosure.

[0040] Step S103: Based on the white balance parameters, the image frames in the video to be processed are calibrated sequentially to obtain the target video.

[0041] In this embodiment of the invention, image frames in the video to be processed, which are captured in real time by the camera, can be streamed and the color deviation of each image frame in the video to be processed can be compensated by the white balance parameters.

[0042] Specifically, considering that during the use of an endoscope, light emitted from a light source is transmitted to the inside of the human body through a beam (optical fiber) to illuminate the target area to be examined, and the video of that target area is captured by the aforementioned camera, the system can detect and adapt to changes in ambient light in real time during streaming processing. The white balance parameters are adjusted using the aforementioned target network model to improve the calibration results of image frames.

[0043] As described above, in this embodiment of the invention, image frames in the video to be processed are first downsampled based on a preset sampling range to obtain sampled images. These sampled images are then analyzed using a target network model to obtain white balance parameters. The target network model analyzes the gain of each color channel in the sampled image to obtain the white balance parameters of the image frames. Then, the image frames in the video to be processed are sequentially calibrated based on the white balance parameters to obtain the target video. This eliminates the need to rely on a color temperature-gain curve to determine the gain value of the image frames and thus the white balance parameters, adapting to the automatic white balance requirements of special application scenarios such as endoscopy.

[0044] This embodiment provides another video white balance adjustment method, which can be used for the aforementioned medical devices such as endoscopes. Figure 2 This is a flowchart of another video white balance adjustment method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Downsample the image frames in the video to be processed based on a preset sampling range to obtain a sampled image.

[0045] Specifically, step S201 includes: Step S2011: Determine the preset sampling range in the image frame according to the sampling configuration.

[0046] Step S2012: Determine the sampling location based on the number of preset sampling ranges.

[0047] Step S2013: Downsample the preset sampling range in the image according to the sampling position to obtain the sampled image.

[0048] In this embodiment of the invention, the above sampling configuration is used to configure the sampling mode during the use of the endoscope. Specifically, when setting the sampling configuration, the user can set the sampling range according to the usage scenario to obtain a preset sampling range.

[0049] For example, if the user wants to focus on observing a specific target area, considering that the main part of the user's attention is usually in the center of the field of view, the white balance effect in the central area is the best parameter for the entire image frame. Therefore, the preset sampling range can be set to the central area of ​​the image frame. Alternatively, if the user's observation has no specific focus and the observation requirement is the entire field of view, multiple preset negative sampling ranges can be set and distributed as evenly as possible within the image frame. This allows for the determination of the optimal white balance parameter for the entire image frame based on these multiple preset sampling ranges.

[0050] Therefore, the sampling position in the image frame can be determined based on the number of preset sampling ranges. The following are several specific implementation methods for determining the sampling position: Method 1: When the number of preset sampling ranges is one, the sampling position is determined based on the center position of the image frame.

[0051] In this embodiment of the invention, when the preset sampling range is one, it indicates that the user's observation requirement is to focus on observing a specific target area. Therefore, the preset sampling range can be set at the center of the image frame, such as... Figure 3 The diagram shows the sampling position determined based on the center position of the image frame, where the center point of the rectangular region corresponding to the preset sampling range coincides with the center point of the image frame.

[0052] Method 2: When there are multiple preset sampling ranges, calculate the spacing value between the preset sampling ranges based on the size of the preset sampling range and the image frame, and determine the sampling position according to the spacing value. The spacing values ​​between the preset sampling ranges may be the same or different.

[0053] In this embodiment of the invention, the user can set the number of preset sampling ranges. Within a certain limit, the more preset sampling ranges set, the higher the accuracy of the white balance parameters determined by the target network model based on the sampled images. It should be understood that, to accommodate the host's computing resources, the total area of ​​the preset sampling ranges can be fixed. When there are multiple preset sampling ranges, the sum of the areas of the preset sampling ranges is less than or equal to this total area.

[0054] Considering that users' observation needs cover the entire field of view, white balance parameter estimation windows corresponding to multiple preset sampling ranges can be evenly distributed across the image frame, for example, as follows: Figure 4 The diagram illustrates the determination of multiple sampling locations within an image frame, where the spacing between the center points of each white balance parameter estimation window can be equal.

[0055] Step S202 involves analyzing the sampled image using a target network model to obtain white balance parameters. The target network model is used to analyze the gain of each color channel in the sampled image to obtain the white balance parameters of the image frame. For details, please refer to [link to relevant documentation]. Figure 1 Step S102 of the illustrated embodiment will not be described again here.

[0056] Step S203: Based on the white balance parameters, the image frames in the video to be processed are calibrated sequentially to obtain the target video. For details, please refer to [link to relevant documentation]. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0057] In this embodiment of the invention, when setting the sampling configuration, the user can set the sampling range according to the usage scenario to obtain a preset sampling range. For example, if the user wants to focus on observing a specific target area, considering that the main part of the user's attention is usually in the center of the field of view, the white balance effect in the central area is the best, which is the optimal parameter for the entire image frame. Therefore, the preset sampling range can be set to the central area of ​​the image frame. Alternatively, if the user's observation has no specific focus and the observation requirement is the entire field of view, multiple preset negative sampling ranges can be set and distributed as evenly as possible within the image frame. This allows for the determination of the optimal white balance parameter for the entire image frame based on these multiple preset sampling ranges, thereby improving the adjustment accuracy of the white balance mechanism and enhancing the user experience.

[0058] In some optional implementations, step S202 above includes: Step S2021: Analyze the color shift data of the sampled image using the target network model.

[0059] Step S2022: Based on color shift data and preset weights, determine the gain ratio corresponding to each color channel in the sampled image. The preset weights are used to indicate the weights assigned to each color channel in the target network model.

[0060] Step S2023: Determine the white balance parameters based on the gain ratio.

[0061] In this embodiment of the invention, when the above-mentioned sampled image is an RGB (ren, green, blue, referring to the three color channels of red, green, and blue) image, the above-mentioned color deviation data can be used to indicate the color deviation value corresponding to each color channel in the RGB image.

[0062] Generally, when performing white balance on an image, it is necessary to analyze the color cast of each color channel in order to adjust each color channel based on the analysis results. For example, if the red channel is oversaturated, its gain is reduced; if the blue channel is undersaturated, its gain is increased to achieve color balance, thereby eliminating unnatural color deviations caused by different light sources and ensuring that the image's colors reflect the true colors of the real scene.

[0063] Based on this, this invention considers that different endoscope models may correspond to different color spaces in the application scenarios of endoscopes, or that users' focus on observing image frames may not be on the reproduction of scene colors. Therefore, in the relevant automatic and manual white balance adjustments, when adjusting the color cast of each color channel, the weight of each color channel is 1. In this invention, the weight of each color channel can be set to be different to increase the gain ratio when compensating for a certain color channel.

[0064] For example, for models with a single-color mode color space, during video capture, the image color can be biased towards a certain color channel, such as the red channel. In this case, when adjusting the white balance of the image frame, the preset weight of the red channel can be increased. The preset weights for the red, green, and blue color channels can be 1.2:0.9:0.9, respectively.

[0065] In this embodiment of the invention, considering that different endoscope models may correspond to different color spaces in the application scenarios of endoscopes, or that users' focus on observing image frames may not be on the reproduction of scene colors, the weights of each color channel can be set during automatic and manual white balance adjustments to adjust the color cast of each color channel, thereby increasing the applicability of the invention.

[0066] In some optional implementations, step S2021 above includes: Step a1: Extract the brightness value of the sampled image using the target network model.

[0067] Step a2: Analyze the color shift values ​​of each color channel in the sampled image at the current brightness value to obtain color shift data.

[0068] In this embodiment of the invention, the sampled image can first be converted into a grayscale image to calculate the color cast value of each color channel of the sampled image at the current brightness value. Specifically, the color cast can be identified by comparing the brightness values ​​of areas that should be white or neutral gray in the same scene.

[0069] Here, if the brightness value of a certain color channel in the sampled image is abnormally high or low, it indicates that there is a color cast problem in that color channel. In this case, the color cast value can be determined by comparing the brightness value of that color channel with that of a white or neutral gray area.

[0070] In this embodiment of the invention, color deviation data can be determined based on the color deviation values ​​of each color channel in the sampled image at the current brightness value, thereby providing a technical basis for determining the gain ratio of each color channel in the sampled image based on the color deviation data and preset weights.

[0071] In some optional implementations, step S202 above further includes: Step b1: Obtain the color preference settings and determine the preset weights based on the color preference settings.

[0072] Step b2: Adjust the model parameters of the network model to be trained based on the preset weights, and train the adjusted network model to obtain the target network model.

[0073] In this embodiment of the invention, considering that different users have different usage habits when using endoscopes, and that the requirements for color tone are also different in different application environments, a color tone preference parameter can be added when training the above-mentioned network model to be trained, so as to adjust the preset weights corresponding to each color channel in the network model to be trained according to the color tone preference parameter.

[0074] For example, the color preference settings mentioned above can include reddish, bluish, greenish, bright, dark, etc. The model parameters corresponding to different color preferences can be different. For example, when the color preference is set to reddish, the preset weight ratio of the red channel can be higher than that of the blue and green channels.

[0075] In this embodiment of the invention, considering that different users have different usage habits when using endoscopes, and that the requirements for color tone also differ in different application environments, a color tone preference parameter can be added when training the above-mentioned network model to be trained, so as to better meet the user's application needs and improve the user experience.

[0076] In some optional implementations, the number of upsampled images is multiple, and step S202 above further includes: Step c1 involves analyzing multiple sampled images using the target network model to obtain the white balance sub-parameters for each sampled image.

[0077] Step c2: Fit the white balance sub-parameters and determine the white balance parameters based on the fitting results.

[0078] In this embodiment of the invention, as can be seen from the above, the number of preset sampling ranges can be one or more. When the preset sampling range is one, the number of sampled images obtained is one. At this time, the white balance parameter of the sampled image output by the target network model can be used as the adjustment parameter corresponding to the video to be processed.

[0079] Furthermore, when there are multiple preset sampling ranges, the number of sampled images obtained is also multiple. The white balance sub-parameters corresponding to each sampled image can be the same or different. In this case, multiple white balance sub-parameters can be fitted to determine the white balance parameters based on the fitting results. Here, when performing data fitting, linear fitting, nonlinear fitting, and interpolation methods can be used to estimate the white balance parameters corresponding to the entire image frame based on the fitting results. The specific fitting method is subject to feasibility and is not limited in this application.

[0080] In this embodiment of the invention, when there are multiple sampled images, the white balance parameters can be determined based on the white balance sub-parameters of each sampled image object to obtain the optimal white balance parameters for the entire image frame, thereby improving the adjustment accuracy of the white balance mechanism and enhancing the user experience.

[0081] In summary, in this embodiment of the invention, image frames in the video to be processed are first downsampled based on a preset sampling range to obtain sampled images. These sampled images are then analyzed using a target network model to obtain white balance parameters. The target network model analyzes the gain of each color channel in the sampled image to obtain the white balance parameters of the image frames. Then, the image frames in the video to be processed are sequentially calibrated based on the white balance parameters to obtain the target video. This eliminates the need to rely on a color temperature-gain curve to determine the gain value of the image frames and thus the white balance parameters, adapting to the automatic white balance requirements of special application scenarios such as endoscopy.

[0082] This embodiment also provides a video white balance adjustment device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0083] This embodiment provides a video white balance adjustment device, such as... Figure 5 As shown, it includes: The sampling module 501 is used to downsample image frames in the video to be processed based on a preset sampling range to obtain a sampled image; Analysis module 502 is used to analyze the sampled image through the target network model to obtain white balance parameters; The calibration module 503 is used to calibrate the image frames in the video to be processed sequentially based on the white balance parameters to obtain the target video.

[0084] In some alternative implementations, the analysis module 502 includes: The first analysis unit is used to analyze the color shift data of the sampled image through the target network model; The first determining unit is used to determine the gain ratio of each color channel in the sampled image based on color deviation data and preset weights, wherein the preset weights are used to indicate the weights assigned to each color channel in the target network model. The second determining unit is used to determine the white balance parameters based on the gain ratio.

[0085] In some optional implementations, the analysis unit includes: The acquisition subunit is used to extract the brightness values ​​of the sampled image through the target network model; The analysis subunit is used to analyze the color shift values ​​of each color channel of the sampled image at the current brightness value, and obtain color shift data.

[0086] In some optional implementations, the analysis module 502 further includes: The acquisition unit is used to acquire color preference settings and determine preset weights based on the color preference settings; The training unit is used to adjust the model parameters of the network model to be trained based on preset weights, and to train the adjusted network model to obtain the target network model.

[0087] In some optional implementations, the number of sampled images is multiple. The sampled images are analyzed using a target network model to obtain white balance parameters. The analysis module 502 also includes: The second analysis unit is used to analyze multiple sampled images through the target network model to obtain the white balance sub-parameters corresponding to each sampled image. The fitting unit is used to fit the white balance sub-parameters and determine the white balance parameters based on the fitting results.

[0088] In some alternative implementations, the sampling module 501 includes: The third determining unit is used to determine the preset sampling range in the image frame according to the sampling configuration; The fourth determining unit is used to determine the sampling location based on the number of preset sampling ranges; The sampling unit is used to downsample a preset sampling range in the image according to the sampling position to obtain a sampled image.

[0089] In some optional implementations, the fourth determining unit includes: The determination subunit is used to determine the sampling position based on the center position of the image frame when the number of preset sampling ranges is one.

[0090] The calculation subunit is used to calculate the spacing value between the preset sampling ranges based on the size of the preset sampling range and the image frame when there are multiple preset sampling ranges, and to determine the sampling position according to the spacing value. The spacing values ​​between the preset sampling ranges may be the same or different.

[0091] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0092] In this embodiment, the video white balance adjustment device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0093] This invention also provides a computer device having the above-described features. Figure 5 The video white balance adjustment device shown.

[0094] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 6 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 6 Take a processor 10 as an example.

[0095] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0096] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0097] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0098] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0099] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0100] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0101] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0102] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0103] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A video white balance adjustment method, characterized in that, The method includes: The image frames in the video to be processed are downsampled based on a preset sampling range to obtain a sampled image; The white balance parameters are obtained by analyzing the sampled image using a target network model, wherein the target network model is used to analyze the gain of each color channel in the sampled image to obtain the white balance parameters of the image frame. The image frames in the video to be processed are calibrated sequentially based on the white balance parameters to obtain the target video.

2. The method according to claim 1, characterized in that, The step of analyzing the sampled image using a target network model to obtain white balance parameters includes: The color cast data of the sampled image is analyzed using the target network model. Based on the color shift data and preset weights, the gain ratio corresponding to each color channel in the sampled image is determined, wherein the preset weights are used to indicate the weights assigned to each color channel in the target network model; The white balance parameters are determined based on the gain ratio.

3. The method according to claim 2, characterized in that, The step of analyzing the color cast data of the sampled image through the target network model includes: The brightness value of the sampled image is extracted using the target network model; The color cast data is obtained by analyzing the color cast values ​​of each color channel of the sampled image at the current brightness value.

4. The method according to claim 2, characterized in that, The method further includes: Obtain the color tone preference settings and determine the preset weights based on the color tone preference settings; The model parameters of the network model to be trained are adjusted based on the preset weights, and the adjusted network model is trained to obtain the target network model.

5. The method according to claim 1, characterized in that, The number of sampled images is multiple, and the step of analyzing the sampled images through a target network model to obtain white balance parameters further includes: The target network model is used to analyze multiple sampled images to obtain the white balance sub-parameters corresponding to each sampled image. The white balance parameters are fitted based on the white balance sub-parameters, and the white balance parameters are determined based on the fitting results.

6. The method according to claim 1, characterized in that, The step of downsampling image frames in the video to be processed based on a preset sampling range to obtain sampled images includes: The preset sampling range in the image frame is determined according to the sampling configuration; The sampling location is determined based on the number of preset sampling ranges; The image is downsampled according to the sampling position within a preset sampling range to obtain a sampled image.

7. The method according to claim 6, characterized in that, Determining the sampling location based on the number of preset sampling ranges includes: When the number of preset sampling ranges is one, the sampling position is determined based on the center position of the image frame; or When there are multiple preset sampling ranges, the spacing value between the preset sampling ranges is calculated based on the size of the preset sampling range and the image frame, and the sampling position is determined according to the spacing value, wherein the spacing values ​​between the preset sampling ranges are the same or different.

8. A video white balance adjustment device, characterized in that, The device includes: The sampling module is used to downsample image frames in the video to be processed based on a preset sampling range to obtain a sampled image; The analysis module is used to analyze the sampled image through the target network model to obtain white balance parameters; The calibration module is used to calibrate the image frames in the video to be processed sequentially based on the white balance parameters to obtain the target video.

9. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the video white balance adjustment method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the video white balance adjustment method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the video white balance adjustment method according to any one of claims 1 to 7.

Images