Image processing method, device, equipment and medium

Abstract

The present application discloses an image processing method, apparatus, device and medium. By determining the connected domain contained in the image to be processed, the area where the light source is located in the image to be processed is accurately determined, which is conducive to the subsequent addition of glare in the area where the light source is located, and improving the quality of the glare added to the image. The attribute information of the connected domain is obtained, and it is determined whether the attribute information meets the preset screening conditions. When it is determined that the attribute information meets the preset screening conditions, the image area corresponding to the connected domain in the image to be processed is mixed with the preset texture image to obtain a rendered image, thereby avoiding the influence of noise areas such as the wall area, the ring light source and the like in the image to be processed that do not need to add glare on the quality of the rendered image, and also avoiding the generation of too many glare textures in the image to be processed, thereby improving the efficiency of adding glare to the image, and facilitating the real-time addition of glare texture to the image to be processed.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device and medium. Background Art

[0002] Because adding glare to images creates the illusion of viewing high-dynamic-range images, greatly enhancing the image's realism, glare effects are widely used in photorealistic image rendering, gaming, and live streaming filters. Therefore, how to add high-quality glare effects to images is an urgent problem to be solved. Summary of the Invention

[0003] The embodiments of the present application provide an image processing method, apparatus, device, and medium to solve the problem of poor quality when glare is added to an image.

[0004] The present invention provides an image processing method, which includes:

[0005] Determine a connected domain contained in an image to be processed; wherein the image to be processed is a red, green, and blue (RGB) image;

[0006] Determining attribute information of the connected domain;

[0007] If it is determined that the attribute information satisfies a preset screening condition, an image region corresponding to the connected domain in the image to be processed is mixed and rendered with a preset texture image to obtain a rendered image.

[0008] An embodiment of the present application provides an image processing device, the device comprising:

[0009] A first determining unit is configured to determine a connected domain contained in an image to be processed; wherein the image to be processed is a red, green, and blue (RGB) image;

[0010] A second determining unit, configured to determine attribute information of the connected domain;

[0011] The rendering unit is configured to, if it is determined that the attribute information satisfies a preset screening condition, perform mixed rendering on an image area corresponding to the connected domain in the image to be processed and a preset texture image to obtain a rendered image.

[0012] An embodiment of the present application provides an image processing device, which includes a processor, and the processor is used to implement the steps of the image processing method described above when executing a computer program stored in a memory.

[0013] An embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the steps of the image processing method described above are implemented.

[0014] In the process of adding glare to an image, by determining the connected domain contained in the image to be processed, the area where the light source is located in the image to be processed is accurately determined, which is conducive to subsequently adding glare to the area where the light source is located, thereby improving the quality of adding glare to the image. Attribute information of the connected domain is obtained, and it is determined whether the attribute information meets the preset filtering conditions. If it is determined that the attribute information meets the preset filtering conditions, the image area corresponding to the connected domain in the image to be processed is mixed and rendered with a preset texture image to obtain a rendered image, thereby avoiding the impact of noise areas in the image to be processed, such as the area where the wall is located and the ring light source, on the quality of the rendered image, thereby improving the quality of adding glare to the image, avoiding the generation of excessive glare textures in the image to be processed, improving the efficiency of adding glare to the image, and facilitating the real-time addition of glare textures to the image to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0016] Figure 1 A schematic diagram of an image processing process is provided for an embodiment of the present application;

[0017] Figure 2 A schematic diagram of a specific image processing flow provided in an embodiment of the present application;

[0018] Figure 3 A schematic diagram of a binary image provided in an embodiment of the present application;

[0019] Figure 4 A schematic diagram of a binary image after mathematical morphological operation provided in an embodiment of the present application;

[0020] Figure 5 A rendering effect diagram of an image provided in an embodiment of the present application;

[0021] Figure 6 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0022] Figure 7 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The present application will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only a portion of the embodiments of the present application, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present application without creative effort are intended to fall within the scope of protection of the present application.

[0024] Those skilled in the art will appreciate that the embodiments of the present application may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present application may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0025] It should be understood herein that any number of elements in the drawings is for illustration only and not for limitation, and any naming is only for distinction and does not have any limiting meaning.

[0026] For ease of understanding, some concepts involved in the embodiments of this application are explained below:

[0027] Glare: A physical phenomenon where light from bright objects in a scene reflects and scatters between the cornea, lens, and retina of the human eye, or between the optical components of a camera, creating a halo. Glare is primarily composed of three components: a circular halo, a ciliary corona, and bloom.

[0028] Glare Effect: A physical phenomenon in which a bright light source causes light to disperse due to reflection and scattering in the field of view, forming a special pattern.

[0029] Halo: A component of the glare effect, appearing as a series of colored concentric circles surrounding the light source, with the outermost circle being red and the innermost circle being purple.

[0030] Bloom: A component of the glare effect, it refers to the glow around bright objects. It often appears around the light source, wrapping around the light source and spreading outward, reducing the contrast of the surrounding environment.

[0031] Ciliary Corona: A component of the glare effect, appearing as streaks radiating from the center of the light source, often in the shape of a cross or star. Its shape is related to the structure of the observation camera or the human eye.

[0032] High Dynamic Range (HDR): refers to a larger exposure dynamic range than ordinary digital images.

[0033] Billboarding: In 3D graphics, the technique of rendering 2D patches perpendicular to the camera.

[0034] Connected Component Analysis: An algorithm that groups and labels pixels in an image that have similar characteristics and are adjacent to each other.

[0035] Mathematical morphology operation (Morphology): An image processing method developed for binary images based on the set theory method of mathematical morphology.

[0036] Temporal Filtering: A time-based anti-noise algorithm.

[0037] Roundness: refers to the degree to which a measured object is close to a theoretical circle.

[0038] Otsu's Thresholding: Also referred to as Ostu's thresholding method, an algorithm that converts a grayscale image into a binary image to ensure the maximum variance between foreground and background pixels.

[0039] At present, commonly used glare special effects rendering technologies can be mainly divided into the following three categories:

[0040] First, post-processing filtering involves filtering the image using specific operators during the post-processing phase to achieve glare effects. This is primarily used to create bloom and ciliary coronas. Specifically, highlight regions in the original image are first extracted and a highlight map is generated. This highlight map is then downsampled. The downsampled highlight map is then filtered multiple times using a specific directional filter operator. Finally, the filtered results are blended and overlaid with the original image. By repeatedly changing the direction of the directional filter operator, cross-shaped or star-shaped glare effects can be achieved.

[0041] Second, physical simulation involves tracing the propagation of light and simulating its interaction with air and the optical system to calculate the glare effect. For example, physical simulation based on wave optics takes into account the camera aperture, the shape of eyelashes and pupils, and the diffraction of light while constructing a virtual optical system, and uses Fourier transform to calculate the final glare pattern.

[0042] 3. Translucent billboarding, which uses an algorithm that combines threshold and corner detection to determine the position of the light source in the image, and then determines the image area centered on the light source and with the same shape and size as the pre-generated texture map. This image area is then rendered with the texture map to achieve a high-quality glare effect.

[0043] The above methods for adding glare to images do not take into account the presence of noisy areas in the image, such as walls and ring light sources, where glare is not required, which can affect the quality of the rendered image. In addition, the above methods are prone to generating excessive glare in the image, reducing the efficiency of adding glare to the image and failing to meet the demand for adding glare to the image in real time.

[0044] In order to avoid the above situation, the embodiments of the present application provide an image processing method, apparatus, device and medium. Since in the process of adding glare to an image, by determining the connected domain contained in the image to be processed, the area where the light source is located in the image to be processed is accurately determined, which is conducive to the subsequent addition of glare in the area where the light source is located, and improves the quality of adding glare to the image. The attribute information of the connected domain is obtained, and it is determined whether the attribute information meets the preset filtering conditions. When it is determined that the attribute information meets the preset filtering conditions, the image area corresponding to the connected domain in the image to be processed is mixed with the preset texture image to obtain a rendered image, thereby avoiding the presence of noise areas such as the wall area, the ring light source, etc. in the image to be processed that do not need to add glare, which affects the quality of the rendered image, improves the quality of adding glare to the image, and avoids generating too many glare textures in the image to be processed, thereby improving the efficiency of adding glare to the image, and is conducive to the real-time addition of glare texture to the image to be processed.

[0045] It should be understood that the application scenarios listed in the above embodiments are merely for the purpose of illustrating the exemplary scenarios and are not intended to limit the application scenarios of the image processing method, apparatus, device, and medium provided in the embodiments of the present application. Persons skilled in the art will appreciate that as new business scenarios emerge, the technical solutions provided in the embodiments of the present application will also be applicable to similar technical problems.

[0046] Example 1:

[0047] Figure 1 A schematic diagram of an image processing process is provided for an embodiment of the present application, and the process includes:

[0048] S101: Determine a connected domain contained in an image to be processed; wherein the image to be processed is a red, green, and blue (RGB) image.

[0049] The image processing method provided in the embodiments of the present application is applied to an image processing device, which may be an intelligent device, such as a mobile terminal, a computer, etc., or a server, such as an application server, etc.

[0050] In the embodiment of the present application, the image to be processed is an image to which glare needs to be added, and the image to be processed is a red, green, and blue (RGB) image. The image can be an image captured during a live broadcast, an image captured in a game interface, or an image captured in a short video. During the specific implementation, it can be flexibly set according to actual needs and is not specifically limited here.

[0051] For example, during a live broadcast, glare can be added to the captured video image, giving users watching the live broadcast the illusion of viewing a high dynamic range image, greatly enhancing the realism of the image. For example, after the host's smart device captures the video image, it can send the video image to an image processing device, which then adds glare to the video image. The video image with glare added (referred to as a rendered image) is then sent to the host's smart device and the smart devices of all users watching the host's live broadcast.

[0052] It should be noted that the image to be processed may be collected by the image processing device itself, or may be sent by other devices.

[0053] Since the area in the image to be processed where glare needs to be added is generally the area where the light source is located, such as the sun, lamp, etc. Therefore, in the embodiment of the present application, after the image processing device obtains the image to be processed, it can determine the area where the light source is located in the image to be processed (recorded as the highlight area), so as to facilitate the subsequent addition of glare directly to the highlight area.

[0054] Considering that the brightness of the area where the light source is located in the image is higher than the brightness of the area where the light source is not located, in the embodiment of the present application, the image processing device can determine the connected domain contained in the image to be processed based on the brightness of each pixel in the image to be processed. The pixels contained in any connected domain have similar brightness and are adjacent in position.

[0055] In one example, the primary consideration when determining the connected domains contained in the image to be processed is the brightness of each pixel in the image to be processed. Therefore, in an embodiment of the present application, after acquiring the image to be processed, the image processing device obtains a grayscale image of the image to be processed, thereby converting the image to be processed from a three-channel image to a single-channel image, facilitating the subsequent extraction of brightness information for each pixel in the image to be processed. A binary image is then determined based on the grayscale image. Based on the binary image, the connected domains contained in the image to be processed are determined.

[0056] As a possible implementation, when determining a binary image based on the grayscale image, the grayscale value of each pixel contained in the grayscale image can be compared with the grayscale threshold, and the grayscale value of the pixel can be updated based on the comparison result.

[0057] Specifically, if the grayscale value of the pixel is greater than the grayscale threshold, the grayscale value of the pixel is updated according to a first preset grayscale value, such as 255, 250, etc. If the grayscale value of the pixel is not greater than the grayscale threshold, the grayscale value of the pixel is updated according to a second preset grayscale value, such as 0, 5, etc.

[0058] The first preset grayscale value is different from the second preset grayscale value, and the difference between the first preset grayscale value and the second preset grayscale value is as large as possible.

[0059] It should be noted that the grayscale threshold can be set manually based on work experience, or can be determined by a threshold acquisition method such as Ostu's threshold method. In the specific implementation process, it can be flexibly set according to actual needs and is not specifically limited here.

[0060] In one possible embodiment, after acquiring a grayscale image of an image to be processed, the image processing device may first downsample the grayscale image and then determine a binary image of the downsampled grayscale image. This reduces the amount of subsequent computation required, improves the efficiency of acquiring rendered images, and facilitates the real-time addition of glare textures to the image to be processed. For example, the grayscale image may be downsampled at least once according to a preset downsampling factor, such as 1 / 4, 1 / 8, etc. The preset downsampling factor may be one or more.

[0061] For example, the grayscale image may be downsampled according to a preset first downsampling coefficient. The grayscale image that has undergone the first downsampling process may be further downsampled according to a preset second downsampling coefficient. The grayscale image that has undergone the second downsampling process may be further downsampled according to a preset third downsampling coefficient, and so on.

[0062] Since the position of the light source in two adjacent frames may change, when glare is added based on the position of the light source in the two adjacent frames, the glare flashes quickly, thereby reducing the quality of the rendered image. Therefore, in an embodiment of the present application, after the grayscale image of the image to be processed is obtained, the grayscale image can be temporally filtered. Exemplarily, the image processing device saves the filtering result corresponding to the most recently obtained image to be processed. When the image processing device temporally filters the grayscale image of the current image to be processed, the filtering result corresponding to the image to be processed can be determined based on the filtering result and filtering coefficient (recorded as the first filtering coefficient) corresponding to the previous frame of the image to be processed, and the grayscale image and the filtering coefficient (recorded as the second filtering coefficient) corresponding to the grayscale image, and then the grayscale image is updated according to the filtering result.

[0063] Among them, the sum of the first filter coefficient and the second filter coefficient is 1. The first filter coefficient and the second filter coefficient may be the same or different. When setting the filter coefficients (including the first filter coefficient and the second filter coefficient), different values ​​can be set according to different scenes. If you want to ensure the consistency of the light source position in two adjacent frames of the image to be processed as much as possible, you can set the first filter coefficient to be larger, or set the second filter coefficient to be smaller. If you want to avoid a delay in the light source position determined in the current image to be processed, you can set the first filter coefficient to be smaller, or set the second filter coefficient to be larger. In the specific implementation process, it can be flexibly set according to actual needs and is not specifically limited here.

[0064] In a possible implementation, based on the filtering result and the first filtering coefficient corresponding to the previous frame of the image to be processed, and the grayscale image and the corresponding second filtering coefficient, the filtering result corresponding to the image to be processed can be determined by the following formula:

[0065] F t =αF t-1 +(1-α)G

[0066] Among them, F t represents the filtering result corresponding to the t-th frame to be processed image, α represents the first filtering coefficient, 1-α represents the second filtering coefficient, F t-1 Indicates the filtering result corresponding to the previous frame of the image to be processed.

[0067] After the filtering result corresponding to the current image to be processed is obtained, the filtering result corresponding to the previous frame of the image to be processed may be updated according to the filtering result corresponding to the current image to be processed.

[0068] Considering that there is some spatial noise in the image to be processed, which affects the determination of the light source position in the image to be processed, in an embodiment of the present application, after obtaining a binary image of the grayscale image, a mathematical morphological operation can be performed on the binary image to determine some isolated pixels that are not in the area where the light source is located as spatial noise in the image to be processed, and to eliminate this noise, thereby preventing this noise from affecting the subsequent determination of the light source position in the image to be processed.

[0069] The mathematical morphology operation includes an opening operation or a closing operation.

[0070] For example, if the mathematical morphology operation includes an opening operation, a structuring element corresponding to the resolution of the image to be processed can be determined. For example, the structuring element corresponding to the resolution of the image to be processed can be determined based on a pre-configured correspondence between resolution and structuring elements. Then, based on the structuring element, an opening operation is performed on the binary image. Specifically, an erosion operation is first performed on the binary image, and then a dilation operation is performed on the eroded binary image.

[0071] S102: Determine attribute information of the connected domain.

[0072] In one example, the attribute information of the connected domain includes a center position and / or the number of pixels.

[0073] Taking the example where the attribute information of the connected domain includes the center position, after the image processing device determines the connected domain contained in the image to be processed, it can determine the center position of the connected domain based on the coordinate value of each pixel point contained in the connected domain and the grayscale value of each pixel point.

[0074] For example, the center position of the connected domain can be determined according to the coordinate value of each pixel point included in the connected domain and the grayscale value of each pixel point using the following formula:

[0075] X=∑ (x,y)∈Q XI x,y ;

[0076]

[0077] Among them, Q represents all pixels in the connected domain, X represents the horizontal coordinate value of the pixel point with coordinate value (x, y), Y represents the vertical coordinate value of the pixel point with coordinate value (x, y), and I x,y Represents the grayscale value of the pixel with coordinate value (x, y), X Indicates the horizontal coordinate value of the center position, Y The vertical coordinate value indicating the center position.

[0078] S103: If it is determined that the attribute information meets the preset screening condition, the image area corresponding to the connected domain in the image to be processed is mixed and rendered with a preset texture image to obtain a rendered image.

[0079] Considering that there may be high-brightness areas such as walls and ring-shaped light sources in the image to be processed, but no noise areas to which glare does not need to be added. Therefore, in an embodiment of the present application, a screening condition is preset. After the attribute information of the connected domain is obtained, it can be determined whether the attribute information meets the preset screening condition. If it is determined that the attribute information meets the preset screening condition, it means that the connected domain is very likely to be the area where the light source is located, that is, the highlight area, and then the image area corresponding to the connected domain in the image to be processed is determined. The image area is mixed and rendered with the preset texture image to obtain a rendered image, that is, to achieve the addition of glare to the image to be processed.

[0080] As a possible implementation method, considering that the area occupied by non-light source entities such as walls and floors in the image is generally much larger than the area occupied by light sources in the image. Therefore, in an embodiment of the present application, if the attribute information of the connected domain includes the number of pixels, the preset screening condition includes that the ratio of the number of pixels contained in the connected domain to the total number of pixels is not greater than the preset ratio threshold, so that according to the preset screening condition, the connected domains belonging to non-light source entities such as walls and floors can be filtered to avoid adding glare to these non-light source entities in the image to be processed, thereby ensuring the effect and quality of the rendered image obtained and improving the accuracy of adding glare. Among them, the total number of pixels is the number of pixels contained in the image to be processed. Specifically, after the number of pixels of any connected domain is obtained based on the above-mentioned embodiment, the ratio of the number of pixels to the total number of pixels can be determined, and it can be judged whether the ratio is greater than the preset ratio threshold. If it is determined that the ratio is greater than the preset ratio threshold, it means that the connected domain may be an area where non-light source entities such as walls and ground are located, and it is determined that the connected domain does not meet the preset filtering conditions; if it is determined that the ratio is not greater than the preset ratio threshold, it means that the connected domain may be an area where a light source is located, and it is determined that the connected domain meets the preset filtering conditions.

[0081] As another possible implementation, since the glare effect is added to the light source area, the image may contain light sources of special shapes, such as ring-shaped light sources, resulting in the center position determined based on the above embodiment being a non-light source area. Therefore, in this embodiment of the present application, if the attribute information of the connected domain includes the center position, the preset filtering condition includes the grayscale value of the center position being greater than a preset grayscale threshold. Based on this preset filtering condition, connected domains belonging to light sources of special shapes can be filtered out to avoid adding glare to these special-shaped light sources in the processed image and affecting the rendering quality, thereby ensuring the effect and quality of the obtained rendered image and improving the accuracy of the glare addition. Specifically, after the center position of any connected domain is obtained based on the above embodiment, it can be determined whether the grayscale value of the center position is greater than the preset grayscale threshold. If the grayscale value of the center position is determined to be greater than the preset grayscale threshold, it indicates that the center position is a light source area, and the connected domain is determined to meet the preset filtering condition. If the grayscale value of the center position is determined to be less than the preset grayscale threshold, it indicates that the center position is likely a non-light source area, and the connected domain is determined to not meet the preset filtering condition.

[0082] As another possible implementation, if the attribute information of a connected domain includes a center position and a number of pixels, the preset filtering condition may include that the ratio of the number of pixels contained in the connected domain to the total number of pixels is not greater than a preset ratio threshold, and that the grayscale value of the center position is greater than a preset grayscale threshold. After obtaining the center position and the number of pixels of any connected domain based on the above-described embodiment, the ratio of the number of pixels to the total number of pixels can be determined. Then, a determination is made as to whether the ratio is greater than a preset ratio threshold, and whether the grayscale value of the center position is greater than a preset grayscale threshold. If it is determined that the ratio is greater than the preset ratio threshold, or that the grayscale value of the center position is less than the preset grayscale threshold, then the connected domain is determined to not meet the preset filtering condition. If it is determined that the ratio is less than the preset ratio threshold, and the grayscale value of the center position is greater than the preset grayscale threshold, then the connected domain is determined to meet the preset filtering condition.

[0083] After determining that the acquired attribute information of the connected domain satisfies the preset filtering conditions based on the above-mentioned embodiment, the image area corresponding to the connected domain in the image to be processed can be determined, and then the image area is mixed and rendered with the preset texture image to obtain an image with added glare, that is, a rendered image.

[0084] In actual application, when adding glare to a light source that is close to a circle in shape, it is generally added at the center of the light source. Therefore, in an embodiment of the present application, a preset standard roundness, such as 1, 0.99, etc., and a preset roundness threshold, such as 0.3, 0.5, etc. are provided to determine whether the shape of the current connected domain is close to a circle through the preset standard roundness and the preset roundness threshold. When the image area corresponding to the connected domain in the image to be processed is mixed with the preset texture image for rendering, the roundness of the connected domain can be obtained, and it is determined whether the difference between the roundness and the preset standard roundness is greater than the preset roundness threshold. If it is determined that the difference between the roundness and the preset standard roundness is not greater than the preset roundness threshold, it means that the shape of the connected domain is close to a standard circle, then the image area corresponding to the connected domain can be determined in the image to be processed based on the coordinates of all the pixel points contained in the connected domain, and the center position of the connected domain can be determined, and the target pixel point corresponding to the center position of the connected domain in the image area can be determined. Then, a preset rendering algorithm, such as a semi-transparent billboarding algorithm, is used to perform mixed rendering of the image area and a preset texture image with the target pixel point as the center.

[0085] In one example, when a semi-transparent billboarding algorithm is used to blend and render an image region centered on a target pixel with a preset texture image, a billboard image can be determined based on the number of pixels in the connected domain and the preset texture image. The number of pixels in the connected domain can be used to control the transparency or size of the texture image, and the preset texture image can be used to control the texture of the billboard image. Blending this image region with the billboard image can achieve different glare effects, increasing the diversity of glare effects.

[0086] In one possible implementation, when adding glare to some non-circular light sources, at least one glare can be added uniformly in the main axis direction of these light sources. Therefore, in an embodiment of the present application, if it is determined that the difference between the roundness and the preset standard roundness is greater than the preset roundness threshold, it means that the shape of the connected domain is not close to a standard circle. Then, the main axis direction of the connected domain can be obtained, and based on the coordinates of each pixel point contained in the connected domain, the image area corresponding to the connected domain can be determined from the image to be processed. Then, according to the main axis direction, the image area is divided into multiple sub-image areas. For each sub-image area, the sub-image area is mixed and rendered with a preset texture image.

[0087] The image region may be divided into a plurality of sub-image regions according to a preset length, for example, a preset length of a texture image in the main axis direction of the image region, or may be divided into a plurality of sub-image regions according to a preset number of divisions. For example, the image region may be divided into a plurality of sub-image regions evenly according to a preset number of divisions, or may be divided into a plurality of sub-image regions unevenly according to a preset number of divisions.

[0088] In one example, after acquiring multiple sub-image regions, a target pixel at the center of each sub-image region is determined. A pre-defined rendering algorithm, such as a semi-transparent billboarding algorithm, is then used to blend and render the sub-image region with a pre-defined texture image, centered at the target pixel.

[0089] For example, when using a semi-transparent billboarding algorithm to blend the sub-image area with a preset texture image, centered around the target pixel, the billboard image can be determined based on the number of pixels contained in the sub-image area and the preset texture image. The number of pixels in the sub-image area can be used to control the transparency or size of the texture image, and the preset texture image can be used to control the texture of the billboard image. Blending the sub-image area with the billboard image can achieve different glare effects, increasing the diversity of glare effects.

[0090] In an embodiment of the present application, when determining the circularity of a connected domain, the area of ​​the connected domain and the perimeter of the connected domain may be obtained. Based on the area and perimeter, the circularity of the connected domain is determined. For example, the circularity of the connected domain may be determined based on the area and perimeter using the following formula:

[0091]

[0092] Among them, A represents area, P represents circumference, and R represents roundness.

[0093] In one example, the area of ​​the connected domain may be determined according to the number of pixels included in the connected domain.

[0094] In one example, boundary pixels may be determined based on the pixels included in the connected domain, and the perimeter of the connected domain may be determined based on the determined boundary pixels.

[0095] Example 2:

[0096] The image processing method provided by the embodiment of the present application is described in detail below through specific implementation methods. Figure 2 A schematic diagram of a specific image processing process provided in an embodiment of the present application, the process includes:

[0097] S201: Acquire an image to be processed.

[0098] The image to be processed is an RGB image.

[0099] S202: Determine the grayscale image of the image to be processed.

[0100] S203: Downsampling the grayscale image.

[0101] S204: Performing temporal filtering on the grayscale image acquired in S203.

[0102] The specific process of performing temporal filtering on the grayscale image includes:

[0103] Based on the filtering result and the first filtering coefficient corresponding to the previous frame of the image to be processed, and the grayscale image and the corresponding second filtering coefficient, the filtering result corresponding to the image to be processed is determined; based on the filtering result corresponding to the image to be processed, the grayscale image is updated.

[0104] S205: Determine a binary image of the grayscale image obtained in S204.

[0105] Figure 3 A schematic diagram of a binary image provided in an embodiment of the present application. Figure 3 The left picture is the image to be processed. Figure 3 The right image is a binary image.

[0106] S206: Perform mathematical morphological operations on the binary image.

[0107] For example, the binary image is first subjected to an erosion operation to delete the pixels at the edge of the connected domain, and then the binary image after the erosion operation is subjected to an expansion operation to expand the edge of the connected domain, thereby determining some isolated pixels that are not in the area where the light source is located as spatial noise in the image to be processed, and excluding these noises to prevent these noises from affecting the subsequent determination of the light source position in the image to be processed.

[0108] Figure 4 This is a schematic diagram of a binary image processed by mathematical morphology provided in an embodiment of the present application. Figure 3 After the binary image on the right is processed by mathematical morphology, the following is obtained: Figure 4 The binary image shown.

[0109] S207: Acquire the attribute information of the connected domain and determine whether the attribute information meets the preset screening condition. If so, execute S208; otherwise, execute S209.

[0110] The attribute information includes the center position and the number of pixels.

[0111] In one example, specifically, the process of determining whether the attribute information meets the preset filtering conditions includes: determining the total number of pixels based on the number of pixels contained in the image to be processed; if the ratio of the number of pixels in the connected domain to the total number of pixels is not greater than a preset ratio threshold, and the grayscale value of the center position of the connected domain is greater than the preset grayscale threshold, then determining that the attribute information meets the preset filtering conditions; if the ratio of the number of pixels in the connected domain to the total number of pixels is greater than the preset ratio threshold, or the grayscale value of the center position of the connected domain is not greater than the preset grayscale threshold, then determining that the attribute information does not meet the preset filtering conditions.

[0112] S208: Generate a rendering image.

[0113] The specific process of generating a billboard includes:

[0114] The circularity of the connected domain is obtained, and it is determined whether the difference between the circularity of the connected domain and a preset standard circularity is greater than a preset circularity threshold.

[0115] If the difference between the circularity and the preset standard circularity is determined to be no greater than a preset circularity threshold, indicating that the shape of the connected domain is close to a perfect circle, the image region corresponding to the connected domain can be determined in the image to be processed based on the coordinates of all pixels contained in the connected domain, and the center position of the connected domain and the corresponding target pixel in the image region can be determined. Based on the number of pixels in the connected domain and a preset texture image, a billboard image is determined, and the image region is mixed with the billboard image for rendering.

[0116] If the difference between the circularity and the preset standard circularity is greater than a preset circularity threshold, indicating that the shape of the connected domain is not close to a standard circle, the principal axis direction of the connected domain is obtained. Based on the coordinates of each pixel contained in the connected domain, the image region corresponding to the connected domain is determined from the image to be processed. The image region is then divided into multiple sub-image regions according to the principal axis direction. For each sub-image region, a target pixel at the center of the sub-image region is determined. Based on the number of pixels contained in the sub-image region and a preset texture image, a billboard image is determined and mixed with the billboard image for rendering.

[0117] Figure 5 This is a rendering effect diagram provided by the embodiment of the present application. Based on the image processing method provided by the embodiment of the present application, Figure 3 The image to be processed on the left is processed and obtained Figure 5 Rendered image shown.

[0118] S209: Obtain the next connected domain and execute S207.

[0119] In the process of adding glare to an image, the connected domain contained in the image to be processed is determined, thereby accurately determining the area where the light source is located in the image to be processed. This facilitates the subsequent addition of glare to the area where the light source is located, thereby improving the quality of the glare added to the image. Attribute information of the connected domain is obtained, and it is determined whether the attribute information meets a preset filtering condition. If it is determined that the attribute information meets the preset filtering condition, the image area corresponding to the connected domain in the image to be processed is mixed and rendered with a preset texture image to obtain a rendered image. This avoids the impact of noise areas in the image to be processed, such as wall areas and ring light sources, on the quality of the rendered image, which do not require glare addition. This improves the quality of adding glare to the image, avoids generating too many glare textures in the image to be processed, and improves the efficiency of adding glare to the image. This facilitates the real-time addition of glare texture to the image to be processed without the need for complex calculations, reduces the computing power requirements of the image processing equipment, and reduces the difficulty of deployment on the smart device side.

[0120] Example 3:

[0121] The embodiment of the present application provides an image processing device, Figure 6 This is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application, the device comprising:

[0122] A first determining unit 61 is configured to determine a connected domain contained in an image to be processed; wherein the image to be processed is a red, green, and blue (RGB) image;

[0123] A second determining unit 62 is configured to determine attribute information of the connected domain;

[0124] The rendering unit 63 is configured to perform mixed rendering on an image region corresponding to the connected domain in the image to be processed and a preset texture image to obtain a rendered image if it is determined that the attribute information satisfies a preset screening condition.

[0125] Furthermore, the first determining unit 61 is specifically configured to determine a grayscale image of the image to be processed; determine a binary image of the grayscale image; and determine the connected domain based on the binary image.

[0126] Furthermore, the first determining unit 61 is further configured to perform downsampling processing on the grayscale image after determining the grayscale image of the image to be processed and before determining the binary image of the grayscale image.

[0127] Furthermore, the first determination unit 61 is also used to determine the filtering result corresponding to the image to be processed based on the filtering result and first filtering coefficient corresponding to the previous frame of the image to be processed, and the grayscale image and the corresponding second filtering coefficient after determining the grayscale image of the image to be processed and before determining the binary image of the grayscale image; and update the grayscale image based on the filtering result corresponding to the image to be processed.

[0128] Furthermore, the first determining unit 61 is further configured to perform a mathematical morphological operation on the binary image after determining the binary image of the grayscale image and before determining the connected component based on the binary image.

[0129] Furthermore, the rendering unit 63 is specifically used for the attribute information of the connected domain including the center position and the number of pixels. The determination of whether the attribute information meets the preset filtering conditions is as follows: if the ratio of the number of pixels to the total number of pixels is not greater than a preset ratio threshold, and the grayscale value of the center position is greater than a preset grayscale threshold, then it is determined that the attribute information meets the preset filtering conditions; wherein, the total number of pixels is the number of pixels contained in the image to be processed.

[0130] Furthermore, the rendering unit 63 is specifically configured to determine a billboard image based on the number of pixels in the connected domain and the texture image; and perform mixed rendering on the image area and the billboard image.

[0131] Furthermore, the rendering unit 63 is specifically used to obtain the roundness of the connected domain; if the difference between the roundness of the connected domain and the preset standard roundness is greater than a preset roundness threshold, the main axis direction of the connected domain is obtained; according to the main axis direction, the image area is divided into multiple sub-image areas; for the multiple sub-image areas, the sub-image area is mixed and rendered with the texture image.

[0132] Furthermore, the rendering unit 63 is specifically configured to determine a billboard image based on the number of pixels included in the sub-image area and the texture image; and perform mixed rendering on the sub-image area and the billboard image.

[0133] In the process of adding glare to an image, by determining the connected domain contained in the image to be processed, the area where the light source is located in the image to be processed is accurately determined, which is conducive to subsequently adding glare to the area where the light source is located, thereby improving the quality of adding glare to the image. Attribute information of the connected domain is obtained, and it is determined whether the attribute information meets the preset filtering conditions. If it is determined that the attribute information meets the preset filtering conditions, the image area corresponding to the connected domain in the image to be processed is mixed and rendered with a preset texture image to obtain a rendered image, thereby avoiding the impact of noise areas in the image to be processed, such as the area where the wall is located and the ring light source, on the quality of the rendered image, thereby improving the quality of adding glare to the image, avoiding the generation of excessive glare textures in the image to be processed, improving the efficiency of adding glare to the image, and facilitating the real-time addition of glare textures to the image to be processed.

[0134] Example 4:

[0135] Based on the above embodiment, the embodiment of the present application further provides an image processing device, Figure 7 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, it includes: a processor 71, a communication interface 72, a memory 73 and a communication bus 74, wherein the processor 71, the communication interface 72, and the memory 73 communicate with each other through the communication bus 74;

[0136] The memory 73 stores a computer program. When the program is executed by the processor 71, the processor 71 performs the following steps:

[0137] Determine a connected domain contained in an image to be processed; wherein the image to be processed is a red, green, and blue (RGB) image;

[0138] Determining attribute information of the connected domain;

[0139] If it is determined that the attribute information satisfies a preset screening condition, an image region corresponding to the connected domain in the image to be processed is mixed and rendered with a preset texture image to obtain a rendered image.

[0140] Since the principle of solving the problem by the above-mentioned image processing device is similar to that of the image processing method, the implementation of the above-mentioned image processing device can refer to the embodiment of the method, and the repeated parts are not repeated here.

[0141] The communication bus mentioned in the above-mentioned image processing device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but this does not mean that there is only one bus or one type of bus. The communication interface 72 is used for communication between the above-mentioned image processing device and other devices. The memory can include random access memory (RAM) and can also include non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0142] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it can also be a digital signal processing processor (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0143] In the process of adding glare to an image, by determining the connected domain contained in the image to be processed, the area where the light source is located in the image to be processed is accurately determined, which is conducive to subsequently adding glare to the area where the light source is located, thereby improving the quality of adding glare to the image. Attribute information of the connected domain is obtained, and it is determined whether the attribute information meets the preset filtering conditions. If it is determined that the attribute information meets the preset filtering conditions, the image area corresponding to the connected domain in the image to be processed is mixed and rendered with a preset texture image to obtain a rendered image, thereby avoiding the impact of noise areas in the image to be processed, such as the area where the wall is located and the ring light source, on the quality of the rendered image, thereby improving the quality of adding glare to the image, avoiding the generation of excessive glare textures in the image to be processed, improving the efficiency of adding glare to the image, and facilitating the real-time addition of glare textures to the image to be processed.

[0144] Example 5:

[0145] Based on the above embodiments, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program executable by a processor. When the program runs on the processor, the processor implements the following steps:

[0146] Determine a connected domain contained in an image to be processed; wherein the image to be processed is a red, green, and blue (RGB) image;

[0147] Determining attribute information of the connected domain;

[0148] If it is determined that the attribute information satisfies a preset screening condition, an image region corresponding to the connected domain in the image to be processed is mixed and rendered with a preset texture image to obtain a rendered image.

[0149] Since the principle of solving the problem by the above-mentioned computer-readable storage medium is similar to that of the image processing method, the implementation of the above-mentioned computer-readable storage medium can refer to the implementation of the method, and the repeated parts will be omitted.

[0150] In the process of adding glare to an image, by determining the connected domain contained in the image to be processed, the area where the light source is located in the image to be processed is accurately determined, which is conducive to subsequently adding glare to the area where the light source is located, thereby improving the quality of adding glare to the image. Attribute information of the connected domain is obtained, and it is determined whether the attribute information meets the preset filtering conditions. If it is determined that the attribute information meets the preset filtering conditions, the image area corresponding to the connected domain in the image to be processed is mixed and rendered with a preset texture image to obtain a rendered image, thereby avoiding the impact of noise areas in the image to be processed, such as the area where the wall is located and the ring light source, on the quality of the rendered image, thereby improving the quality of adding glare to the image, avoiding the generation of excessive glare textures in the image to be processed, improving the efficiency of adding glare to the image, and facilitating the real-time addition of glare textures to the image to be processed.

[0151] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0152] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0153] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0155] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. An image processing method, characterized in that: The method comprises: Determine a connected domain contained in an image to be processed; wherein the image to be processed is a red, green, and blue (RGB) image; Determining attribute information of the connected domain; wherein the attribute information of the connected domain includes a center position and / or a number of pixels; If it is determined that the attribute information satisfies a preset screening condition, then performing mixed rendering on an image region corresponding to the connected domain in the image to be processed and a preset texture image to obtain a rendered image; If the attribute information of the connected domain includes a center position and a number of pixels, determining whether the attribute information satisfies a preset screening condition includes: If the ratio of the number of pixels to the total number of pixels is not greater than a preset ratio threshold, and the grayscale value of the center position is greater than a preset grayscale threshold, then it is determined that the attribute information meets the preset screening condition; wherein the total number of pixels is the number of pixels contained in the image to be processed; The step of determining a connected domain contained in the image to be processed includes: Determining a grayscale image of the image to be processed; Determining a binary image of the grayscale image; Determining the connected domain based on the binary image; After determining the grayscale image of the image to be processed and before determining the binary image of the grayscale image, the method further includes: Determining a filtering result corresponding to the image to be processed based on a filtering result and a first filtering coefficient corresponding to a previous frame of the image to be processed, and the grayscale image and a corresponding second filtering coefficient, wherein the sum of the first filtering coefficient and the second filtering coefficient is 1; The grayscale image is updated based on the filtering result corresponding to the image to be processed.

2. The method according to claim 1, characterized in that After determining the grayscale image of the image to be processed and before determining the binary image of the grayscale image, the method further includes: Downsampling is performed on the grayscale image.

3. The method according to claim 1, characterized in that After determining the binary image of the grayscale image and before determining the connected component based on the binary image, the method further includes: Perform mathematical morphological operations on the binary image.

4. The method according to claim 1, wherein The step of performing mixed rendering on an image area corresponding to the connected domain in the image to be processed and a preset texture image includes: Determining a billboard image based on the number of pixels in the connected domain and the texture image; The image area is mixed with the billboard image for rendering.

5. The method according to claim 1, wherein The step of performing mixed rendering on an image area corresponding to the connected domain in the image to be processed and a preset texture image includes: Obtaining the circularity of the connected domain; If the difference between the roundness of the connected domain and the preset standard roundness is greater than a preset roundness threshold, the main axis direction of the connected domain is obtained; according to the main axis direction, the image area is divided into multiple sub-image areas; for the multiple sub-image areas, the sub-image areas are mixed and rendered with the texture image.

6. The method according to claim 5, characterized in that The mixed rendering of the sub-image area and the texture image includes: Determining a billboard image based on the number of pixels included in the sub-image area and the texture image; The sub-image area is mixed with the billboard image for rendering.

7. An image processing device, characterized in that The device comprises: A first determining unit is configured to determine a connected domain contained in an image to be processed; wherein the image to be processed is a red, green, and blue (RGB) image; A second determining unit is configured to determine attribute information of the connected domain; wherein the attribute information of the connected domain includes a center position and / or a number of pixels; a rendering unit configured to, if it is determined that the attribute information satisfies a preset screening condition, perform mixed rendering on an image region corresponding to the connected domain in the image to be processed and a preset texture image to obtain a rendered image; The rendering unit is specifically configured to, if the attribute information of the connected domain includes a center position and a number of pixels, determine that the attribute information satisfies a preset screening condition, including: if a ratio of the number of pixels to the total number of pixels is not greater than a preset ratio threshold, and a grayscale value of the center position is greater than a preset grayscale threshold, determine that the attribute information satisfies the preset screening condition; wherein the total number of pixels is the number of pixels included in the image to be processed; The first determining unit is specifically configured to determine a grayscale image of the image to be processed; determine a binary image of the grayscale image; and determine the connected domain based on the binary image. The first determination unit is further configured to determine, after determining the grayscale image of the image to be processed and before determining the binary image of the grayscale image, a filtering result corresponding to the image to be processed based on the filtering result and first filtering coefficient corresponding to the previous frame of the image to be processed, and the grayscale image and the corresponding second filtering coefficient; wherein the sum of the first filtering coefficient and the second filtering coefficient is 1; and update the grayscale image based on the filtering result corresponding to the image to be processed.

8. An image processing device, characterized in that The image processing device comprises a processor, and the processor is configured to implement the steps of the image processing method according to any one of claims 1 to 6 when executing a computer program stored in a memory.

9. A computer-readable storage medium, characterized in that The computer program is stored therein, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 6 are implemented.

Images