Image Data Processing Method, Electronic Device, and Computer Storage Medium

Abstract

Embodiments of the present application provide an image data processing method, an electronic device, and a computer storage medium. Among them, the image data processing method includes: obtaining a to-be-processed image in a preset color space and a background color range matching the to-be-processed image, where the preset color space is a color space that can reflect the hues of the to-be-processed image and the background color range; determining first color data corresponding to the to-be-processed image and second color data corresponding to the background color range, where both the first color data and the second color data include multiple color components corresponding to the color space; obtaining the minimum distance between the color of a pixel in the to-be-processed image and the color of the background color range according to the minimum distance between the first color data and the second color data; and determining the background area of the to-be-processed image according to the minimum distance between the color of a pixel in the to-be-processed image and the color of the background color range.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to an image data processing method, an electronic device, and a computer storage medium. Background Art

[0002] Image processing is a technology that uses a computer to process image information. In image processing, many scenarios rely on the processing of the image background. For example, the recognition of the image background, background segmentation, etc.

[0003] Currently, a relatively widely used method is the method of background segmentation based on a solid color background (such as a green background). In this method, the difference between the background component and the foreground component is calculated pixel by pixel based on the RGB (red, green, blue) components (for example, regarding G as the background component and R and B as the foreground components), and then the background confidence of each pixel is calculated according to a set truncation threshold. Furthermore, background segmentation is performed based on this background confidence.

[0004] However, in this method, it is required that the image background is a solid color (i.e., a certain primary color), and there should be no background color in the image foreground. As a result, the background segmentation effect is poor. Summary of the Invention

[0005] In view of this, the embodiments of the present application provide an image data processing solution to at least partially solve the above problems.

[0006] According to a first aspect of the embodiments of the present application, an image data processing method is provided, including: obtaining a to-be-processed image in a preset color space and a background color range matching the to-be-processed image, where the preset color space is a color space that can reflect the hues of the to-be-processed image and the background color range; determining first color data corresponding to the to-be-processed image and second color data corresponding to the background color range, where both the first color data and the second color data include multiple color components corresponding to the color space; obtaining the minimum distance between the color of a pixel in the to-be-processed image and the color of the background color range according to the minimum distance between the first color data and the second color data; and determining the background area of the to-be-processed image according to the minimum distance between the color of a pixel in the to-be-processed image and the color of the background color range.

[0007] According to a second aspect of the embodiments of the present application, there is provided an image data processing method, including: obtaining an image to be processed and receiving a background color range input for the image to be processed; converting the image to be processed and the background color range into a preset color space, where the preset color space is a color space that can reflect the hues of the image to be processed and the background color range; determining the minimum distance between the color of a pixel in the image to be processed and the color of the background color range based on the color data corresponding to the image to be processed and the background color range in the color space; determining the background area of the image to be processed according to the minimum distance, and performing foreground-background segmentation on the image to be processed according to the determined background area.

[0008] According to a third aspect of the embodiments of the present application, there is provided an electronic device, including: a processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform the operations corresponding to the method described in the first aspect or the second aspect.

[0009] According to a fourth aspect of the embodiments of the present application, there is provided a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method described in the first aspect or the second aspect.

[0010] According to a fifth aspect of the embodiments of the present application, there is provided a computer program product, including computer instructions, and the computer instructions instruct a computing device to perform the operations corresponding to the method described in the first aspect or the second aspect.

[0011] According to the image data processing solution provided by the embodiments of the present application, the determination of whether a pixel in the image to be processed is a background pixel is abstracted as the calculation of the shortest distance between the pixel and the background color range. In this way, the background color range does not need to be a primary color or a pure color, and colors similar to the background color range are also allowed to appear in the foreground object, as long as there is a certain degree of differentiation between the foreground and the background color range. The minimum distance between the first color data corresponding to the image to be processed and the second color data corresponding to the background color range is calculated based on the corresponding single color components among multiple color components in the preset color space, which fully considers the perspective and light and shadow characteristics. The obtained background area will be more accurate, and if background segmentation is performed subsequently, a better background segmentation effect can also be obtained. Description of the Drawings

[0012] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments described in the embodiments of the present application. For those of ordinary skill in the art, other accompanying drawings can also be obtained based on these drawings.

[0013] Figure 1 Schematic diagram of an exemplary system applicable to the image data processing method of the embodiments of the present application;

[0014] Figure 2A Flowchart of the steps of an image data processing method according to Embodiment 1 of the present application;

[0015] Figure 2B For Figure 2A Schematic diagram of an example of a scenario in the illustrated embodiment;

[0016] Figure 3 Flowchart of the steps of an image data processing method according to Embodiment 2 of the present application;

[0017] Figure 4 Flowchart of the steps of an image data processing method according to Embodiment 3 of the present application;

[0018] Figure 5 Schematic diagram of the structure of an electronic device according to Embodiment 4 of the present application. Detailed implementation manners

[0019] In order to enable those in the art to better understand the technical solutions in the embodiments of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the embodiments of the present application, rather than all embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art shall fall within the protection scope of the embodiments of the present application.

[0020] The following further illustrates the specific implementation of the embodiments of the present application in conjunction with the accompanying drawings of the embodiments of the present application.

[0021] Figure 1 Shows an exemplary system applicable to the image data processing method of the embodiments of the present application. As Figure 1 shown, the system 100 may include a server 102, a communication network 104, and / or one or more user devices 106, Figure 1 exemplified as multiple user devices herein.

[0022] Server 102 can be any suitable server for storing information, data, programs, and / or any other suitable type of content. In some embodiments, server 102 can perform any suitable functions. For example, in some embodiments, server 102 can perform image data processing. As an alternative example, in some embodiments, server 102 can be used to determine the background region of an image. As another example, in some embodiments, server 102 can be used to perform subsequent other processing based on the determined background region, such as image background recognition or image segmentation, etc.

[0023] In some embodiments, communication network 104 can be any suitable combination of one or more wired and / or wireless networks. For example, communication network 104 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communication network. User device 106 can be connected to communication network 104 via one or more communication links (e.g., communication link 112), and communication network 104 can be linked to server 102 via one or more communication links (e.g., communication link 114). The communication link can be any communication link suitable for transmitting data between user device 106 and server 102, such as a network link, a dial-up link, a wireless link, a hardwired link, any other suitable communication link, or any suitable combination of such links.

[0024] User device 106 can include any one or more user devices suitable for presenting images. In some embodiments, user device 106 can send the image to be processed to server 102 to request server 102 to determine the background region of the image, and receive the information of the background region feedback by server 102. However, it is not limited thereto. User device 106 can also implement the functions of server 102 locally without the help of the server. That is, the image data processing solution of the embodiments of the present application can be implemented either at server 102 or at user device 106.

[0025] In some embodiments, user device 106 can include any suitable type of device. For example, in some embodiments, user device 106 can include a mobile device, a tablet computer, a laptop computer, a desktop computer, a wearable computer, a game console, a media player, a vehicle entertainment system, and / or any other suitable type of user device.

[0026] Although server 102 is illustrated as one device, in some embodiments, any suitable number of devices may be used to perform the functions performed by server 102. For example, in some embodiments, multiple devices may be used to implement the functions performed by server 102. Alternatively, cloud services may be used to implement the functions of server 102.

[0027] Based on the above system, embodiments of the present application provide an image data processing method, which will be described below through multiple embodiments.

[0028] Embodiment 1

[0029] Referring to Figure 2A , a step flowchart of an image data processing method according to Embodiment 1 of the present application is shown.

[0030] The image data processing method of this embodiment includes the following steps:

[0031] Step S202: Obtain the image to be processed in a preset color space and the background color range matching the image to be processed.

[0032] Wherein, the preset color space is a color space that can reflect the hue of the image to be processed and the background color range.

[0033] In color science, people have established a variety of color models to represent a certain color with one-dimensional, two-dimensional, three-dimensional or even four-dimensional space coordinates. The color range that can be defined by this coordinate system is the color space. More commonly used color spaces include RGB color space, HSV color space, HSL color space, YUV color space, and so on. In the embodiments of the present application, for the convenience of data processing, a color space that can reflect the hue of the image to be processed is selected, such as HSV color space, HSL color space, etc. Those skilled in the art should understand that if the original image to be processed is in the form of other color spaces, before performing the image data processing of the embodiments of the present application, its color space can be converted to a color space that can reflect the hue of the image to be processed. The specific conversion method can refer to the relevant technical description and will not be elaborated here.

[0034] A color space usually consists of multiple components, which are called color components in the embodiments of the present application. For example, in the RGB color space, it includes R component, G component and B component; for another example, in the HSV color space, it includes H component, S component and V component, and so on. The color space that can reflect the hue of the image to be processed needs to have corresponding color components, such as having an H color component, etc. Based on such a color space, subsequent image data processing can be more convenient and the efficiency of image data processing can be improved. Among them, the hue can be measured in degrees, and the value range is 0° to 360°, calculated counterclockwise starting from red, red is 0°, green is 120°, blue is 240°, and so on.

[0035] The background color range in this embodiment matches the image to be processed. In practical applications, the background color range can be specified manually in advance or obtained by performing a rough background detection on the image to be processed. It should be noted that in the embodiments of the present application, the background color range can be a range of background colors, and it supports a certain span of color differences in background objects such as curtains and walls due to wrinkles, stains, light, angles, and exposure. Therefore, it can be understood as the smallest range including the background color.

[0036] Step S204: Determine the first color data corresponding to the image to be processed and the second color data corresponding to the background color range.

[0037] Among them, both the first color data and the second color data include multiple color components corresponding to the color space.

[0038] Taking the HSV color space as an example, it includes the H component, the S component, and the V component. These components are fused together to form HSV model data. In this example, the first color data corresponding to the image to be processed can be the first HSV model data, which includes the H component, the S component, and the V component; similarly, the second color data corresponding to the background color range can be the second HSV model data, which also includes the H component, the S component, and the V component.

[0039] Step S206: Obtain the minimum distance between the color of the pixels in the image to be processed and the color of the background color range according to the minimum distance between the first color data and the second color data.

[0040] First of all, it should be noted that in the embodiments of the present application, "color" means a combination of multiple components in the color space, rather than a single specific color (hue). For example, still taking the HSV color space as an example, the color of a pixel means the HSV model data corresponding to the pixel, rather than only the H component.

[0041] However, when calculating the minimum distance between the first color data and the second color data, the calculation can be reduced to a single color component dimension for calculation. On the one hand, this makes the calculation more accurate, and on the other hand, it can also greatly reduce the computing power resources required for the calculation. After obtaining the minimum distance corresponding to a single color component, fusion can be performed again to form corresponding color data such as HSV model data.

[0042] To effectively calculate the minimum distance between the first color data and the second color data, in a feasible approach, before this step, a non-linear transformation process (referred to as the second non-linear transformation process in this article) is also performed on the saturation component and the brightness component in the first color data, and a third non-linear transformation process is performed on the saturation component and the brightness component in the second color data; the original saturation component and the original brightness component in the first color data are updated using the saturation component and the brightness component after the second non-linear transformation process, and the original saturation component and the original brightness component in the second color data are updated using the saturation component and the brightness component after the third non-linear transformation process to enhance the sensitivity in the medium and low saturation regions and the bright regions. Thus, it also makes the subsequent calculation of the minimum distance based on a single color component more convenient. Among them, the second non-linear transformation process and the third non-linear transformation process can adopt the same processing method or different processing methods. It is preferably to use the same processing method.

[0043] Step S208: Determine the background region of the image to be processed according to the minimum distance between the color of the pixels in the image to be processed and the color in the background color range.

[0044] According to the minimum distance between the color of the pixels in the image to be processed and the color in the background color range, it can be effectively determined whether a certain pixel belongs to the possible background color range. Based on this, the background region of the image to be processed can be determined according to those pixels that are determined to belong to the background color range.

[0045] In a feasible approach, this step can be implemented as: performing a first non-linear transformation process on the minimum distance between the color of the pixels in the image to be processed and the color in the background color range; determining the background color range of the image to be processed according to the result of the first non-linear transformation process; and determining the background region of the image to be processed according to this background color range. Among them, the first non-linear transformation can be implemented by those skilled in the art in an appropriate manner according to actual needs, such as an exponential function, etc., and the embodiments of the present application do not limit this. Through this first non-linear transformation process, it can be ensured that the transition is smoother within the range near the background color to handle some background noise or the problem that the recognition of a small amount of the background color range is not completely accurate.

[0046] In addition, there are semi-transparent regions in some images. To effectively identify such regions, in a feasible approach, when determining the background region of the image to be processed according to the background color range, the truncation threshold of the background color can also be determined according to the background color range; and the background region of the image to be processed is determined according to this truncation threshold. For example, those exceeding this truncation threshold are identified as foreground color pixels, and those not exceeding this truncation threshold may be background color range pixels or foreground + background pixels (i.e., semi-transparent colors). Vice versa.

[0047] Through the above process, the background area in the image to be processed can be effectively determined.

[0048] Next, the above process will be exemplarily described with a scenario example, as Figure 2B shown.

[0049] Figure 2B In this example, it is assumed that the acquired original image is an RGB image, and its corresponding background color range is within the green color system. Exemplarily, in this example, the image is first transformed into the HSV space to make it an HSV image, which serves as the image to be processed. Then, a rough estimation calculation of the background color range is performed on it to obtain the green color system range in the HSV space. In practical applications, the method of manually specifying the background color range is also applicable.

[0050] Next, determine the first color data corresponding to this HSV image, which is shown as "HSV model data corresponding to the image" in the figure, and determine the second color data corresponding to the green color system range in this HSV space, which is shown as "HSV model data corresponding to the green color system" in the figure. Subsequently, based on the "HSV model data corresponding to the image" and the "HSV model data corresponding to the green color system", the minimum distance between the two (denoted as the first minimum distance here) can be determined. Optionally, the minimum distance can be determined based on the H component, S component, and V component in the "HSV model data corresponding to the image" and the corresponding H component, S component, and V component in the "HSV model data corresponding to the green color system". In specific implementation, the first minimum distance can be determined based on the HSV model data corresponding to each pixel. Since the HSV model data of each pixel in this HSV image is in the "HSV model data corresponding to the image", and the HSV model data in the green color system range is in the "HSV model data corresponding to the green color system", therefore, through corresponding processing, the minimum distance between the HSV model data corresponding to each pixel in this HSV image and the HSV model data corresponding to the green color system can be determined. After obtaining the minimum distance corresponding to each pixel in the HSV image, it can be determined whether the pixel belongs to the background color range, and then, according to the determination results of each pixel in the HSV image, the background area of the corresponding image to be processed can be delimited.

[0051] It can be seen that, through this embodiment, the determination of whether a pixel in the image to be processed is a background pixel is abstracted as the calculation of the shortest distance between the pixel and the background color range. In this way, the background color range does not need to be a primary color or a pure color, and colors similar to the background color range are also allowed to appear in the foreground object, as long as there is a certain degree of differentiation between the foreground and the background color range. The minimum distance between the first color data corresponding to the image to be processed and the second color data corresponding to the background color range is calculated based on the corresponding single color components among multiple color components in the preset color space, which fully considers the perspective and lighting characteristics. Thus, the obtained background area will be more accurate. If background segmentation is performed subsequently, a better background segmentation effect can also be obtained.

[0052] Embodiment 2

[0053] Refer to Figure 3 , which shows a flowchart of the steps of an image data processing method according to Embodiment 2 of the present application.

[0054] In this embodiment, with the focus on calculating the minimum distance of color data, the image data method provided by the embodiments of the present application will be described.

[0055] The image data processing method of this embodiment includes the following steps:

[0056] Step S302: Obtain the image to be processed under the preset color space and the background color range matching the image to be processed.

[0057] Among them, the preset color space is a color space that can reflect the hue of the image to be processed and the background color range.

[0058] Step S304: Determine the first color data corresponding to the image to be processed and the second color data corresponding to the background color range.

[0059] Among them, both the first color data and the second color data include multiple color components corresponding to the color space.

[0060] For the specific implementation of the above steps S302 - S304, reference can be made to the description of the relevant parts in the foregoing Embodiment 1, and details will not be repeated here.

[0061] Step S306: Obtain the minimum distance between the color of the pixel in the image to be processed and the color of the background color range according to the minimum distance between the first color data and the second color data.

[0062] In a feasible manner, calculate the minimum distance between the first color data and the second color data on a single color component; based on the minimum distance on the single color component, obtain the minimum distance between the color of the pixel in the image to be processed and the color within the background color range. In this way, when calculating the minimum distance, the calculation with the full components as a whole is respectively reduced to calculations on each component, greatly reducing the time complexity of the calculation and the complexity of the algorithm logic.

[0063] Taking the HSV color space as an example, in this step, the calculation of the minimum distance between HSV model data and HSV model data is reduced to the calculation of the minimum distance between the H component and the H component, between the S component and the S component, and between the V component and the V component.

[0064] In addition, in order to reduce the interference of inconsistent influences of conditions such as light and exposure on the color span of the image, which may affect the accuracy of the calculated minimum distance, in a feasible manner, a scaling factor can also be determined according to the color span corresponding to the background color range. Furthermore, calculate the actual distance between the first color data and the second color data on a single color component; then, scale the actual distance according to the scaling factor to obtain the minimum distance between the first color data and the second color data on the single color component. Thus, while ensuring the accuracy of the minimum distance, the solution of the embodiment of the present application can also adapt to the background color tolerance range and the semi-transparent transition range.

[0065] Through the above process, the accurate calculation of the minimum distance between the first color data and the second color data is achieved.

[0066] Furthermore, based on this minimum distance, the minimum distance between the color of the pixel in the image to be processed and the color within the background color range can be obtained.

[0067] For example, determine the target pixel and the saturation of the target pixel from the image to be processed; according to the minimum distance between the first color data and the second color data on the saturation component, determine the background color in the background color range closest to the target pixel and the saturation of the background color in this background color range; according to the saturation of the target pixel, the saturation of the background color, and the minimum distance between the first color data and the second color data on the hue component, obtain the minimum distance between the hue of the pixel in the image to be processed and the hue of the background color; according to the minimum distance between the hues, obtain the minimum distance between the color of the pixel in the image to be processed and the color within the background color range.

[0068] Among them, the target pixel can be any pixel in the image to be processed. In the case where all pixels are processed, the target pixel can be each pixel in the image to be processed. When determining the minimum distance between hues, the minimum distance between saturations is also combined to fully consider the influence of the visual and light and shadow characteristics of the image to be processed on the hue.

[0069] Similar to the foregoing, in order to reduce the interference of inconsistent influences of conditions such as light and exposure on the color span of the image and affect the accuracy of the calculated minimum distance, in a feasible manner, when implementing this step, the actual minimum distance between the hue of the pixel in the image to be processed and the hue of the background color is obtained according to the saturation of the target pixel, the saturation of the background color, and the minimum distance between the first color data and the second color data in the hue component; the actual minimum distance is scaled according to the scaling factor to obtain the minimum distance between the hue of the pixel in the image to be processed and the hue of the background color. Among them, the scaling factor can be determined according to the color span corresponding to the background color.

[0070] After obtaining the minimum distances in each color component, the minimum distance between the color of the pixel in the image to be processed and the color in the background color range can be obtained based on this. In a feasible manner, the minimum distance between the color of the pixel in the image to be processed and the color in the background color range can be obtained according to the minimum distance between hues, the minimum distance between the first color data and the second color data in the saturation component, and the minimum distance in the brightness component.

[0071] Step S308: Determine the background area of the image to be processed according to the minimum distance between the color of the pixel in the image to be processed and the color in the background color range.

[0072] For the specific implementation of this step, reference can be made to the description of the relevant part in the foregoing Embodiment 1, and details are not described herein again.

[0073] After obtaining the background area of the image to be processed, optionally, the following step S310 can be executed.

[0074] Step S310: Process the background area of the image to be processed.

[0075] Including but not limited to: performing background segmentation, performing background transformation, performing background recognition and performing other processing based on the background recognition result, such as AR (augmented reality) processing, etc.

[0076] Hereinafter, a specific example is used to exemplarily illustrate the above process. The image data processing process of this example includes:

[0077] (A) Convert the image to be processed and the background color range matching the image to be processed into HSV space representation.

[0078] Among them, the background color range can be specified by the user or calculated automatically.

[0079] In this example, the color space is the HSV space as an example. However, those skilled in the art should understand that other color spaces that can reflect hue, such as the HSL space, etc., are also equally applicable.

[0080] (B) Perform non - linear transformation processing on the S component and V component in the HSV model data of the background color range and the HSV model data of the image to be processed, so as to enhance the sensitivity of medium - low saturation and bright regions.

[0081] For example, S = pow(S, a1), V = pow(V, b1), where both a1 and b1 belong to (0.1, 1), and optionally, it can be 0.6 - 0.8. pow() represents the exponential function.

[0082] Or, the S component and V component can be mapped to a hyperbola within the value range, a curve with a larger slope in the target sensitive interval and a smaller slope in the non - sensitive area.

[0083] The transformed S component and V component (including those of the background color range and the image to be processed) are denoted as ST and VT.

[0084] (C) Calculate the minimum distance between the ST / VT of each target pixel in the image to be processed and the corresponding ST / VT range in the background color range respectively.

[0085] For example, the method of calculating the distance of ST can be used, such as abs(target pixel's ST - corresponding ST in the background color range), etc.

[0086] (D) Scale the minimum distance of ST and the minimum distance of VT calculated in (C) according to the span of ST and VT corresponding to the background color range.

[0087] Among them, the span can be obtained by means such as histogram statistics of the corresponding components in the background color range, such as the S component and T component, or the scaled components such as the ST component and VT component. In this example, the span of ST and VT is adopted.

[0088] Specifically,

[0089] Scale the minimum distance of ST to obtain distance_ST = distance_ST / st_range, where st_range = max(pow(max_st_background - min_st_background, a2), MAX_RANGE), the value range of a2 is (0.1, 1.0), MAX_RANGE is used to limit the upper limit of the span, max() represents taking the maximum value, pow() represents the exponential function, max_st_background represents the maximum value of the background color range on the ST component, and min_st_background represents the minimum value of the background color range on the ST component.

[0090] Similarly, scale the minimum distance of VT to obtain distance_VT = distance_VT / vt_range, where vt_range = max(pow(max_vt_background - min_vt_background, a3), MAX_RANGE), the value range of a3 is (0.1, 1.0), MAX_RANGE is used to limit the upper limit of the span, max() represents taking the maximum value, pow() represents the exponential function, max_vt_background represents the maximum value of the background color range on the VT component, and min_vt_background represents the minimum value of the background color range on the VT component.

[0091] (E) Calculate the minimum distance between the H of each pixel in the image to be processed and the corresponding H in the background color range, denoted as distance_H.

[0092] In the HSV color space, H represents its H component, that is, the hue component.

[0093] (F) Scale the distance_H calculated in (E) according to the S (which can also be ST) component of each target pixel in the image to be processed and the S (which can also be ST) component of the background color range, denoted as the minimum distance of the hue distance_Color.

[0094] For example: distance_Color = f(S_target, S_Background) * distance_H;

[0095] Among them, S_target represents the saturation of the target pixel, S_Background represents the saturation of the background color closest to the target pixel, and f(s1, s2) represents the reference saturation of the current foreground / background color. Exemplarily, it can be implemented as max(s1, s2) or (s1 + s2) / 2.

[0096] (G) Scale the minimum distance distance_Color of the hue calculated in (F) according to the span of the H component in the background color range.

[0097] For example: distance_Color = distance_Color / h_range;

[0098] Where h_range = max(pow(max_h_background - min_h_background, a4), MAX_RANGE), the value range of a4 is (0.1, 1.0), MAX_RANGE is used to limit the upper limit of the span, max() represents taking the maximum value, pow() represents the exponential function, max_h_background represents the maximum value of the background color range in the H component, and min_h_background represents the minimum value of the background color range in the H component. Among them, because the H component is cyclic, max_h_background and min_h_background are values considering the rotation direction.

[0099] (H) Converge the minimum distances of the H, S, and V components corresponding to each pixel in the image to be processed, or converge the scaled components corresponding to each pixel, that is, the minimum distances of distance_Color, ST, and VT, to calculate the minimum distance distance of each target pixel from the background color range.

[0100] For example: distance = a5 * distance_Color + distance_ST + distance_VT, where a5 is used to adjust the weight of the hue component and the other two components (ST, VT). Usually, a is much greater than 1.

[0101] (I) Perform a non - linear transformation on distance to ensure that the transition is smoother in the range near the background color, to handle some background noise or the problem that the recognition of a small amount of the background color range is not completely accurate.

[0102] For example: distance = pow(distance, a6), where a6 > 1 is used for non - linear transformation.

[0103] (J) Use the normalization coefficient to determine the semi - transparent region range for the distance calculated in (I), and at the same time perform a truncation process to convert it into an alpha value: alpha = min(distance * b2, 1.0); where b2 is the normalization coefficient to anchor the threshold of 100% alpha (that is, to determine the semi - transparent region range), and min() represents taking the minimum value.

[0104] Through the above examples, (1) the determination of alpha is abstracted as the calculation of the shortest distance between the target pixel and the background color range. In this way, the background color range does not need to be a primary color or a pure color, and colors similar to the background color range are also allowed to appear in the foreground (only requiring a certain degree of differentiation between the foreground and the background color range); (2) the minimum distance calculation is converted to the HSV space (or other color spaces with similar effects) according to the viewing angle and lighting characteristics, and the S component and the V component are non-linearly processed according to the visual and lighting characteristics to ensure reasonable sensitivity. When calculating the minimum distance on the H component, the visual and optical differentiation at different saturations is considered; (3) when calculating the minimum distance, the span of the background color range is also considered, and the background tolerance range and the semi-transparent transition range are adaptively adjusted according to the span of the background color range (that is, the purity of the background); (4) the minimum distance calculation considers the differentiation in three aspects: hue, saturation, and brightness, and the weights can be adjusted; (5) a non-linear transformation is performed on the minimum distance distance, which can ensure a relatively smooth transition effect for the background tolerance interval alpha; (6) after the distance is normalized, it can be directly converted to alpha, and the confidence level of each foreground pixel can be output before truncation, which can be used for post-processing in cases such as color bleeding.

[0105] Embodiment III

[0106] Refer to Figure 4 , which shows a flowchart of the steps of an image data processing method according to Embodiment III of the present application.

[0107] In this embodiment, the image data processing method provided by the embodiments of the present application is described from the specific application level of the above image data processing.

[0108] The image data processing method of this embodiment includes the following steps:

[0109] Step S402: Obtain the image to be processed and receive the background color range input for the image to be processed.

[0110] In this embodiment, the image to be processed is an image to be segmented into foreground and background, which can be a single static image or a video frame image in a video, including but not limited to: video frame images in video conferences, director images in studio broadcasts, live images in video live broadcasts, images to be processed by AR (augmented reality), images for video production, and so on.

[0111] The background color range input for the image to be processed can be a background color range set manually or a background color range obtained by roughly detecting the background color of the image to be processed through an algorithm.

[0112] Step S404: Convert the image to be processed and the background color range to a preset color space.

[0113] The preset color space is a color space that can reflect the hue of the image to be processed and the background color range, including but not limited to the HSV color space, the HSL color space, etc.

[0114] Generally, the image to be processed is mostly an RGB image. Therefore, it needs to be converted to a color space that can reflect the hue, such as the HSV color space or the HSL color space, etc. Correspondingly, the background color range also needs to be consistent with the color space used by the image to be processed, that is, it also needs to be a color space that can reflect the hue.

[0115] Step S406: Based on the color data in the color space corresponding to the image to be processed and the background color range respectively, determine the minimum distance between the color of the pixels in the image to be processed and the color of the background color range.

[0116] For example, the first color data corresponding to the image to be processed and the second color data corresponding to the background color range can be determined first, where both the first color data and the second color data include multiple color components corresponding to the color space; according to the minimum distance between the first color data and the second color data, the minimum distance between the color of the pixels in the image to be processed and the color of the background color range is obtained.

[0117] Step S408: According to the minimum distance, determine the background area of the image to be processed, and perform foreground-background segmentation on the image to be processed according to the determined background area.

[0118] After determining the background area of the image to be processed, foreground-background segmentation can be performed, and further subsequent processing can be carried out based on the result of the foreground-background segmentation.

[0119] It should be noted that the above process is described relatively simply, and the specific implementation of each step can refer to the description of the relevant parts in the foregoing multiple embodiments.

[0120] Hereinafter, taking different scenarios as examples, an exemplary description is given of performing foreground-background segmentation on the image to be processed according to the determined background area.

[0121] Scenario 1: Video conference scenario

[0122] In many video conferences, there are usually requirements for content display, product display, or other background replacement. Based on this, for each video frame image in the video stream of a real-time video conference, the aforementioned image data processing method can be used to determine its background area, and then foreground-background segmentation can be performed based on this background area. Then, the foreground part is retained, such as the image part of the speaker in the conference, etc., and the segmented background part is replaced with an image part including the content to be displayed (such as PPT content), or an image part including the product to be displayed (such as product pictures), or simply replaced with a background image part that conforms to the conference theme, etc. In this way, the effective combination of the conference and the conference content is achieved.

[0123] Scenario 2: Live broadcast or studio director scenario

[0124] Similar to video conferences, in such scenarios, there are also requirements for content display, product display, or scenario display. Taking a travel live broadcast as an example, in addition to introducing the specialties and scenery of the scenic spots, the host will also use the scenic pictures of the scenic spots to attract the audience. Based on this, during the live broadcast, for each video frame image in the live video stream, the aforementioned image data processing method is used to determine its background area, and then foreground-background segmentation is performed based on this background area. Then, the foreground part is retained, such as the image part of the host, etc., and the segmented background part is replaced with an image part including the content of the specialties of the scenic spots to be displayed, or an image part of the well-known scenery of the scenic spots, or an image part of the scenic pictures of the scenic spots, etc. In this way, the effective display of the live broadcast content is achieved. The studio director scenario is similar and will not be elaborated here.

[0125] Scenario 3: Online education scenario

[0126] Similar to video conferences, content display is also required in online education. Therefore, the video of the teacher's teaching process can be recorded first, and then for each video frame image in the video, the aforementioned image data processing method is used to determine its background area, and then foreground-background segmentation is performed based on this background area. Then, the foreground part is retained, such as the image part of the teacher, etc., and the segmented background part is replaced with an image part including the content of the courseware to be displayed, or an image part of the graphics and pictures (dynamic or static) related to the teaching content, etc. In this way, the vivid display of classroom content is achieved.

[0127] Scenario 4: AR scenario

[0128] Whether it is for video frame images in a video stream or for a single static image, there may be a possibility of using AR effects. For example, adding AR objects (such as red envelopes or pets, etc.) to video frame images to interact with video viewers, or adding AR objects (such as text annotations, interesting explanations, or portrait ornaments, etc.) to static images. In these cases, it is necessary to use the aforementioned image data processing method to determine the background area in the video frame image or static image, and add corresponding AR effects based on this background area. However, it is not limited to this. It is also possible to replace the entire determined background area with an AR effect to meet different needs, such as interactive needs or interesting needs, and improve the user experience.

[0129] Scenario Five: Video Production Scenario

[0130] During the process of video production, it is usually found that the background of some images to be used does not meet the requirements and needs to be modified. In such a case, the aforementioned image data processing method can be used to determine the background area in the image to be used, and modify or replace this background area to meet the overall requirements of the video to be produced and achieve an overall improvement in the video effect.

[0131] It can be seen that through this embodiment, on the basis of determining the background area of the image and performing foreground-background segmentation, it can serve various different usage scenarios, greatly meeting the requirements of different usage scenarios and improving the user experience.

[0132] Embodiment Four

[0133] Refer to Figure 5 , which shows a schematic structural diagram of an electronic device according to Embodiment Four of the present application. The specific implementation of the electronic device in the specific embodiments of the present application is not limited.

[0134] As Figure 5 shown, the electronic device may include: a processor 502, a communications interface 504, a memory 506, and a communication bus 508.

[0135] Among them:

[0136] The processor 502, the communications interface 504, and the memory 506 communicate with each other through the communication bus 508.

[0137] The communications interface 504 is used to communicate with other electronic devices or servers.

[0138] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in any of the above image data processing method embodiments.

[0139] Specifically, the program 510 may include program code, which includes computer operation instructions.

[0140] The processor 502 may be a CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the intelligent device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0141] The memory 506 is used to store the program 510. The memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0142] The program 510 may specifically be used to cause the processor 502 to perform the operations corresponding to the image data processing methods described in the foregoing Embodiment 1 or 2 or 3.

[0143] For the specific implementation of each step in the program 510, reference may be made to the corresponding steps and descriptions in the corresponding units in the foregoing image data processing method embodiments, and they have corresponding beneficial effects, which will not be elaborated here. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the devices and modules described above may refer to the corresponding process descriptions in the foregoing method embodiments, which will not be elaborated here.

[0144] The embodiments of the present application also provide a computer program product, including computer instructions, which instruct a computing device to perform the operations corresponding to any one of the image data processing methods in the foregoing multiple method embodiments.

[0145] It should be noted that in multiple embodiments of the present application, the HSV space is taken as an example of the color space. However, those skilled in the art should understand that the determination method of the background color range of other color spaces that can reflect hue can also be implemented with reference to multiple embodiments of the present application, such as the HSL space, the YUV space, etc.

[0146] In addition, multiple embodiments of the present application can be effectively applied to images collected in a preset background scene, such as images collected with a green or other colored curtain, wall, display screen, etc. as the background. However, in the scene of preset background objects such as curtains, walls, and display screens, due to reasons such as curtain wrinkles, light consistency, exposure consistency, and interference, the background color is actually a dynamically changing range, and in many cases, the change and span are large, making it difficult to ensure the accuracy of image background determination. And through the solution of the embodiments of the present application, this problem can be effectively solved. However, it is not limited to this, and images with relatively simple background colors in other non-preset background object scenes can also be equally applicable to the solution of the embodiments of the present application.

[0147] It should be noted that, according to the needs of implementation, each component / step described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0148] The method according to the embodiments of the present application can be implemented in hardware, firmware, or be implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or be implemented as computer code that is originally stored in a remote recording medium or a non-transitory machine-readable medium and downloaded through a network and will be stored in a local recording medium, so that the method described herein can be stored as such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (such as RAM, ROM, flash memory, etc.) that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, the processor, or the hardware, the image data processing method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the image data processing method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the image data processing method shown herein.

[0149] Those of ordinary skill in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0150] The above embodiments are only used to illustrate the embodiments of the present application, rather than to limit the embodiments of the present application. Those of ordinary skill in the relevant technical field can also make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The patent protection scope of the embodiments of the present application shall be defined by the claims.

Claims

1. An image data processing method, including: obtaining a to-be-processed image in a preset color space and a background color range matching the to-be-processed image, where the preset color space is a color space that can reflect the hues of the to-be-processed image and the background color range; determining first color data corresponding to the to-be-processed image and second color data corresponding to the background color range, where both the first color data and the second color data include multiple color components corresponding to the color space; obtaining the minimum distance between the color of a pixel in the to-be-processed image and the color of the background color range according to the minimum distance between the first color data and the second color data; determining the background area of the to-be-processed image according to the minimum distance between the color of a pixel in the to-be-processed image and the color of the background color range.

2. The method according to claim 1, wherein, the obtaining the minimum distance between the color of a pixel in the to-be-processed image and the color of the background color range according to the minimum distance between the first color data and the second color data includes: calculating the minimum distance between the first color data and the second color data on a single color component; obtaining the minimum distance between the color of a pixel in the to-be-processed image and the color of the background color range according to the minimum distance on a single color component.

3. The method according to claim 2, wherein, the calculating the minimum distance between the first color data and the second color data on a single color component includes: calculating the actual distance between the first color data and the second color data on a single color component; performing a scaling process on the actual distance according to a scaling factor to obtain the minimum distance between the first color data and the second color data on a single color component.

4. The method according to claim 3, wherein, the scaling factor is determined according to the color span corresponding to the background color range.

5. The method according to any one of claims 1-4, wherein, the obtaining the minimum distance between the color of a pixel in the to-be-processed image and the color of the background color range according to the minimum distance between the first color data and the second color data includes: determining a target pixel and the saturation of the target pixel from the to-be-processed image; determining the background color in the background color range closest to the target pixel and the saturation of the background color in the background color range according to the minimum distance between the first color data and the second color data on the saturation component; obtaining the minimum distance between the hue of a pixel in the to-be-processed image and the hue of the background color according to the saturation of the target pixel, the saturation of the background color, and the minimum distance between the first color data and the second color data on the hue component; obtaining the minimum distance between the color of a pixel in the to-be-processed image and the color of the background color range according to the minimum distance between the hues.

6. The method according to claim 5, wherein, obtaining a minimum distance between the hue of a pixel in the image to be processed and the hue of the background color according to the saturation of the target pixel, the saturation of the background color, and the minimum distance between the first color data and the second color data in the hue component, includes: obtaining an actual minimum distance between the hue of a pixel in the image to be processed and the hue of the background color according to the saturation of the target pixel, the saturation of the background color, and the minimum distance between the first color data and the second color data in the hue component; performing a scaling process on the actual minimum distance according to a scaling factor to obtain a minimum distance between the hue of a pixel in the image to be processed and the hue of the background color.

7. The method according to claim 5, wherein, obtaining a minimum distance between the color of a pixel in the image to be processed and the color within the background color range according to the minimum distance between the hues, includes: obtaining a minimum distance between the color of a pixel in the image to be processed and the color within the background color range according to the minimum distance between the hues, the minimum distance between the first color data and the second color data in the saturation component, and the minimum distance between the first color data and the second color data in the brightness component.

8. The method according to claim 1, wherein, determining a background region of the image to be processed according to the minimum distance between the color of a pixel in the image to be processed and the color within the background color range, includes: performing a first non-linear transformation process on the minimum distance between the color of a pixel in the image to be processed and the color within the background color range; determining a background color range of the image to be processed according to the result of the first non-linear transformation process; determining a background region of the image to be processed according to the background color range.

9. The method according to claim 8, wherein, determining a background region of the image to be processed according to the background color range, includes: determining a truncation threshold of the background color according to the background color range; determining a background region of the image to be processed according to the truncation threshold.

10. The method according to claim 1, wherein, before obtaining a minimum distance between the color of a pixel in the image to be processed and the color within the background color range according to the minimum distance between the first color data and the second color data, the method further includes: performing a second non-linear transformation process on the saturation component and the brightness component in the first color data, and performing a third non-linear transformation process on the saturation component and the brightness component in the second color data; updating the original saturation component and the original brightness component in the first color data with the saturation component and the brightness component after the second non-linear transformation process, and updating the original saturation component and the original brightness component in the second color data with the saturation component and the brightness component after the third non-linear transformation process.

11. An image data processing method, including: acquiring an image to be processed and receiving a background color range input for the image to be processed; Convert the image to be processed and the background color range to a preset color space, where the preset color space is a color space that can reflect the hues of the image to be processed and the background color range; Based on the color data of the image to be processed and the background color range in the color space respectively, determine the minimum distance between the color of the pixels in the image to be processed and the color of the background color range; According to the minimum distance, determine the background area of the image to be processed, and perform foreground-background segmentation on the image to be processed based on the determined background area.

12. An electronic device, including: A processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction; Wherein, the executable instruction causes the processor to perform the following operations: Obtain an image to be processed in a preset color space and a background color range matching the image to be processed. The preset color space is a color space that can reflect the hues of the image to be processed and the background color range; determine first color data corresponding to the image to be processed and second color data corresponding to the background color range, where the first color data and the second color data both include multiple color components corresponding to the color space; obtain the minimum distance between the color of the pixels in the image to be processed and the color of the background color range according to the minimum distance between the first color data and the second color data; determine the background area of the image to be processed according to the minimum distance between the color of the pixels in the image to be processed and the color of the background color range; Alternatively, the executable instruction causes the processor to perform the following operations: acquire an image to be processed and receive a background color range input for the image to be processed; convert the image to be processed and the background color range to a preset color space, where the preset color space is a color space that can reflect the hues of the image to be processed and the background color range; based on the color data of the image to be processed and the background color range in the color space respectively, determine the minimum distance between the color of the pixels in the image to be processed and the color of the background color range; according to the minimum distance, determine the background area of the image to be processed, and perform foreground-background segmentation on the image to be processed based on the determined background area.

13. A computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the following operations are implemented: Obtain an image to be processed in a preset color space and a background color range matching the image to be processed. The preset color space is a color space that can reflect the hues of the image to be processed and the background color range ； Determine first color data corresponding to the image to be processed and second color data corresponding to the background color range, where, The first color data and the second color data both include a plurality of color components corresponding to the color space; according to the minimum distance between the first color data and the second color data, obtain the minimum distance between the color of the pixel in the image to be processed and the color of the background color range; according to the minimum distance between the color of the pixel in the image to be processed and the color of the background color range, determine the background area of the image to be processed. Alternatively, the executable instructions cause the processor to perform the following operations: obtain an image to be processed and receive an input background color range for the image to be processed. Convert the image to be processed and the background color range to a preset color space, where the preset color space is a color space that can reflect the hues of the image to be processed and the background color range. Based on the color data corresponding to the image to be processed and the background color range respectively in the color space, determine the minimum distance between the color of the pixel in the image to be processed and the color of the background color range; according to the minimum distance, determine the background area of the image to be processed, and perform foreground-background segmentation on the image to be processed according to the determined background area.

14. A computer program product, including computer instructions, the computer instructions instructing a computing device to perform the following operations: Obtain an image to be processed in a preset color space and a background color range matching the image to be processed, where the preset color space is a color space that can reflect the hues of the image to be processed and the background color range ； Determine first color data corresponding to the image to be processed and second color data corresponding to the background color range, where The first color data and the second color data both include a plurality of color components corresponding to the color space; according to the minimum distance between the first color data and the second color data, obtain the minimum distance between the color of the pixel in the image to be processed and the color of the background color range; according to the minimum distance between the color of the pixel in the image to be processed and the color of the background color range, determine the background area of the image to be processed. Alternatively, the executable instructions cause the processor to perform the following operations: obtain an image to be processed and receive an input background color range for the image to be processed. Convert the image to be processed and the background color range to a preset color space, where the preset color space is a color space that can reflect the hues of the image to be processed and the background color range. Based on the color data corresponding to the image to be processed and the background color range respectively in the color space, determine the minimum distance between the color of the pixel in the image to be processed and the color of the background color range; according to the minimum distance, determine the background area of the image to be processed, and perform foreground-background segmentation on the image to be processed according to the determined background area.

Images