Image data processing methods, electronic devices and computer storage media

Abstract

This application provides an image data processing method, an electronic device, and a computer storage medium. The image data processing method includes: obtaining an image to be processed in a preset color space, wherein the preset color space is a color space that reflects the hue of the image to be processed; determining multiple color components corresponding to the background reference color system of the image to be processed, and obtaining at least a portion of individual color components among the multiple color components; obtaining a color range corresponding to each individual color component based on the pixel number distribution of the individual color components; and obtaining a background color range of the image to be processed based on the color range corresponding to each individual color component. This application embodiment makes the determined background color range more accurate.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image data processing method, an electronic device, and a computer storage medium. Background Technology

[0002] Image processing is a technique that uses computers to process image information. In image processing, many scenarios rely on the processing of image backgrounds, such as background recognition and background segmentation.

[0003] Currently, most background segmentation processing works in scenarios with pre-defined background objects, such as green or other colored curtains, walls, or display screens. It calculates the similarity between each target pixel and the corresponding background color or background color range to distinguish the foreground from the background. However, in scenarios with pre-defined background objects like curtains, walls, and displays, the background color is often a dynamically changing range due to factors such as curtain wrinkles, lighting consistency, exposure consistency, and interference. Furthermore, the variations and ranges are often significant, making manual calibration difficult and compromising the accuracy of image background determination.

[0004] Therefore, accurately determining the background color of an image in complex scenarios with the aforementioned preset background objects has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of this application provide an image data processing scheme to at least partially solve the above problems.

[0006] According to a first aspect of the embodiments of this application, an image data processing method is provided, comprising: obtaining an image to be processed in a preset color space, wherein the preset color space is a color space that can reflect the hue of the image to be processed; determining a plurality of color components corresponding to the background reference color system of the image to be processed, and obtaining at least a portion of the single color components among the plurality of color components; obtaining a color range corresponding to the single color component based on the pixel number distribution of the single color component; and obtaining a background color range of the image to be processed based on the color range corresponding to the single color component.

[0007] According to a second aspect of the embodiments of this application, an image data processing method is provided, comprising: acquiring an image to be processed and a preset background reference color system of the image to be processed; converting the image to be processed to a preset color space, and determining multiple color components of the background reference color system in the preset color space based on the image to be processed in the preset color space, wherein the preset color space is a color space that can reflect the hue of the image to be processed; obtaining a background color range of the image to be processed based on the pixel number distribution of at least some single color components of the multiple color components of the background reference color system; and performing background segmentation processing on the image to be processed based on the background color range.

[0008] According to a third aspect of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described in the first or second aspect.

[0009] According to a fourth aspect of the embodiments of this application, a computer storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the method as described in the first or second aspect.

[0010] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including computer instructions that instruct a computing device to perform an operation corresponding to the method described in the first or second aspect.

[0011] According to the image data processing scheme provided in the embodiments of this application, when performing background color recognition on the image to be processed, at least a portion of its multiple color components in the color space is reduced to a single color component based on a pre-given background reference color system (such as the color system corresponding to preset background objects such as curtains, walls, and displays). Then, based on the pixel distribution of the single color component, the color range corresponding to the single color component is obtained, and the background color range is obtained based on the color range corresponding to the single color component. The pixel distribution can represent different background color conditions, such as a pure background color and a large background color range. The color range obtained based on this can effectively adapt to the dynamic changes in background color caused by preset background objects in actual scenes, making the determined background color range more accurate. Furthermore, calculating the color range by reducing the dimensionality to a single color component greatly reduces the computational time complexity and the complexity of the algorithm logic. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0013] Figure 1 This is a schematic diagram of an exemplary system for the image data processing method applicable to the embodiments of this application;

[0014] Figure 2 This is a flowchart illustrating the steps of an image data processing method according to Embodiment 1 of this application;

[0015] Figure 3A This is a flowchart of the steps of an image data processing method according to Embodiment 2 of this application;

[0016] Figure 3B for Figure 3A A process diagram of a specific example in the illustrated embodiment;

[0017] Figure 4 This is a flowchart of the steps of an image data processing method according to Embodiment 3 of this application;

[0018] Figure 5 This is a schematic diagram of the structure of an electronic device according to Embodiment 4 of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0020] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0021] Figure 1 An exemplary system for image data processing methods applicable to embodiments of this application is shown. For example... Figure 1 As shown, the system 100 may include a server 102, a communication network 104, and / or one or more user devices 106. Figure 1 The example in the text shows multiple user devices.

[0022] Server 102 can be any suitable server for storing information, data, programs, and / or any other suitable type of content. In some embodiments, server 102 can perform any suitable function. For example, in some embodiments, server 102 can perform image data processing. As an optional example, in some embodiments, server 102 can be used to determine the background color range of an image. As another example, in some embodiments, server 102 can be used to perform further processing based on the determined background color range, such as image background recognition or image segmentation.

[0023] In some embodiments, communication network 104 may be any suitable combination of one or more wired and / or wireless networks. For example, communication network 104 may include any one or more of the following: the Internet, intranet, wide area network (WAN), local area network (LAN), wireless network, digital subscriber line (DSL) network, frame relay network, asynchronous transfer mode (ATM) network, virtual private network (VPN), and / or any other suitable communication network. User equipment 106 may be connected to communication network 104 via one or more communication links (e.g., communication link 112), and communication network 104 may be linked to server 102 via one or more communication links (e.g., communication link 114). Communication links may be any communication link suitable for transmitting data between user equipment 106 and server 102, such as network links, dial-up links, wireless links, hardwired links, any other suitable communication links, or any suitable combination of such links.

[0024] User equipment 106 may include any one or more user equipments suitable for displaying images. In some embodiments, user equipment 106 may send the image to be processed to server 102 to request server 102 to determine the background color range for the image, and receive background color range information fed back by server 102. However, it is not limited to this; user equipment 106 may also implement the functions of server 102 locally without the aid of a server. That is, the image data processing scheme of this application embodiment can be implemented on either server 102 or user equipment 106.

[0025] In some embodiments, user equipment 106 may include any suitable type of device. For example, in some embodiments, user equipment 106 may include mobile devices, tablet computers, laptop computers, desktop computers, wearable computers, game consoles, media players, vehicle entertainment systems, and / or any other suitable type of user equipment.

[0026] Although server 102 is illustrated as a single device, in some embodiments, any suitable number of devices may be used to perform the functions performed by server 102. For example, in some embodiments, multiple devices may be used to implement the functions performed by server 102. Alternatively, cloud services may be used to implement the functions of server 102.

[0027] Based on the above system, this application provides an image data processing method, which will be described below through several embodiments.

[0028] Example 1

[0029] Reference Figure 2 The diagram shows a flowchart of the steps of an image data processing method according to Embodiment 1 of this application.

[0030] The image data processing method in this embodiment includes the following steps:

[0031] Step S202: Obtain the image to be processed in the preset color space.

[0032] The preset color space is a color space that reflects the hue of the image to be processed.

[0033] In color theory, various color models have been established to represent a color using one-dimensional, two-dimensional, three-dimensional, or even four-dimensional coordinate spaces. The range of colors that can be defined by such a coordinate system is called a color space. Commonly used color spaces are usually three-dimensional, including RGB, HSV, HSL, YUV, and so on. In this embodiment, for the convenience of data processing, a color space that reflects the hue of the image to be processed, such as HSV or HSL, is selected. Those skilled in the art should understand that if the original image to be processed uses a different color space, its color space can be converted to a color space that reflects the hue of the image to be processed before performing image data processing in this embodiment. Specific conversion methods can be found in related technical descriptions and will not be detailed here.

[0034] Color spaces are typically composed of multiple components, referred to as color components in this embodiment. For example, in the RGB color space, it includes R, G, and B components; in the HSV color space, it includes H, S, and V components, and so on. A color space that reflects the hue of the image to be processed needs to have corresponding color components, such as an H color component. Such a color space facilitates subsequent image data processing and improves processing efficiency. Hue can be measured using angles, ranging from 0° to 360°, calculated counter-clockwise starting from red: red is 0°, green is 120°, blue is 240°, and so on.

[0035] Step S204: Based on the background reference color system of the image to be processed, determine multiple color components corresponding to the background reference color system, and obtain at least some of the single color components among the multiple color components.

[0036] As mentioned above, in the solutions of this application embodiment, preset background objects such as curtains, walls, and displays are typically used. Therefore, the background reference color system can be determined based on the colors of these preset background objects. Based on this background reference color system, non-background colors can be quickly filtered out from a general range. In specific implementation, this background reference color system can be determined manually or roughly estimated by color detection of preset areas in the image to be processed (usually the corner areas where the background area is located, etc.). At this time, the background reference color system is used as the initial color system for subsequent more accurate determination of the background color range, providing a relatively good basis for determining the background color for further processing.

[0037] Taking the HSV color space as an example, it includes H components, S components, and V components, which are combined to form HSV model data. In this example, the multiple color components of the image to be processed in the HSV color space constitute the HSV model data of the image to be processed, which includes H components, S components, and V components. Similarly, the multiple color components of the background reference color system in the HSV color space constitute the HSV model data of the background reference color system, which also includes H components, S components, and V components. Furthermore, in this embodiment, "color" refers to a combination of multiple components in the color space, rather than a single specific color (hue). For example, still using the HSV color space as an example, the color of a pixel refers to the HSV model data corresponding to the pixel, rather than just the H component.

[0038] Given a specific color space, the color components corresponding to the background reference color system can also be determined. For example, if the color space is HSV, then the multiple color components corresponding to the background reference color system include the H component, S component, and V component; if the color space is HSL, then the multiple color components corresponding to the background reference color system include the H component, S component, and L component.

[0039] However, in a color space, the various components are integrated into a whole. Processing based on this whole leads to complex algorithms and low accuracy in determining the background color. Therefore, in this embodiment, after determining multiple color components of the background reference color system, these multiple color components can be dimensionality-reduced as a whole. One approach is to obtain some components from the whole, such as obtaining the H and S components in the HSV color space. Another feasible approach is to reduce the dimensionality of multiple color components in the color space as a whole, that is, to reduce the dimensionality of each color component to its corresponding single color component, such as reducing the dimensionality of the HSV color space to individual H, S, and V components. In the specific dimensionality reduction, the data corresponding to each color component can be extracted from the whole, or a weighting method can be used to assign high weights to the desired color components and low weights to other color components. Of course, other dimensionality reduction methods are also applicable.

[0040] Taking the dimensionality reduction of multiple color components corresponding to the background reference color system to their respective single color components as an example, after reducing the dimensionality of multiple color components to their respective single color components, multiple corresponding components will be obtained. For example, after reducing the dimensionality of the HSV of the background reference color system, three corresponding components will be obtained, namely the H component, S component, and V component.

[0041] Step S206: Obtain the color range corresponding to the single color component based on the pixel quantity distribution of the single color component.

[0042] In this embodiment, the color range corresponding to a single color component is obtained based on the pixel quantity distribution of that single color component. If all single color components are obtained in step S204, the color range corresponding to each single color component is obtained according to the pixel data distribution of each single color component. For example, the color range corresponding to each H component, S component, and V component is obtained according to their respective pixel quantity distributions, such as HD, SD, and VD. If only a portion of the single color components are obtained in step S204, the color range corresponding to each single color component is obtained based on the pixel quantity distribution of each single color component within that portion. For example, if only the H component and S component are obtained, the color range corresponding to each H component and S component can be obtained according to their respective pixel quantity distributions, such as HD and SD.

[0043] Taking the H (hue) component in the HSV color space as an example, assuming the background reference color is green, ranging from 60 to 180 degrees, if there are 140 pixels at 110 degrees, 1000 pixels at 115 degrees, and the number of pixels increases with each degree, reaching 1300 pixels at 120 degrees; then, the number of pixels decreases with each degree, reaching 1000 at 130 degrees, 400 at 140 degrees, and 30 at 142 degrees. Therefore, in a rough estimation, based on this distribution, the color range corresponding to the H component can be roughly considered to be 115-130 degrees.

[0044] Furthermore, in practical applications, the number of pixels corresponding to the component values ​​in a single color component can be grouped. As shown in the example above, 140 pixels corresponding to 110 degrees form one group, 1000 pixels corresponding to 115 degrees form another, and so on. The pixels in each group can be sorted according to their component values. For example, sorting by H component value from low to high, the current order is (110-140) → (111-200) → (112-500) → (113-700) → (114-900) → (115-1000) → ...

[0045] In one feasible approach, the color range corresponding to each single color component can be obtained based on the maximum and average values ​​of the pixel number distribution of each single color component.

[0046] When the background color is not a solid color, the maximum and average values ​​of the pixel number distribution can reflect the changes or transitions in the background color. As mentioned earlier, in the embodiments of this application, due to the complexity of the preset background objects used, the background in the image to be processed is likely not a solid color. Therefore, in these embodiments, for each single color component, the maximum and average values ​​of its pixel number distribution are obtained to account for both cases where the background color is relatively pure and cases where the background color has a large range.

[0047] The maximum and average values ​​of the pixel count distribution for each single color component can be obtained by statistically analyzing and calculating the pixels in the respective dimensions of each color component.

[0048] In one feasible approach, this step can be implemented as follows: for each single color component, a first color threshold and a second color threshold are determined based on the maximum value and average value of the pixel number distribution of that single color component, respectively; taking the color corresponding to the maximum value as the center color, the color range corresponding to that single color component is obtained based on the first color threshold and the second color threshold.

[0049] For example, the maximum and average values ​​of the pixel number distribution can be multiplied by a coefficient to determine the first color threshold; after the first color threshold is determined, it can be multiplied by a coefficient to obtain the second color threshold.

[0050] For example, such as:

[0051] threshold1=min(maxVal*a,avgVal*b);

[0052] threshold2 = threshold1 * c

[0053] In this formula, threshold1 represents the first color threshold, threshold2 represents the second color threshold, maxVal represents the maximum value of the pixel number distribution, avgVal represents the average value of the pixel number distribution, min() represents taking the minimum value, and a, b, and c are coefficients. In practical applications, a, b, and c can be set by those skilled in the art according to the actual background color range to be searched. Usually, b is much larger than a and c. For example, a can be 1%-2%, b can be 100%-150%, and c can be 5%-10%, etc. That is, if you want to find a larger background color range, you can set a and b to make threshold1 larger, or you can set c to make threshold2 smaller. Of course, both can be used. It should be noted that the above formula is applicable to all components in the color space.

[0054] This ensures that the pixels in the first and second color threshold ranges cover as much of the possible range of background colors as possible. Furthermore, pixels outside these ranges do not substantially affect the determination of the background color range, so excluding them reduces the corresponding data processing burden and computational load.

[0055] After determining the first color threshold and the second color threshold, the color corresponding to the maximum value of the pixel quantity distribution of a single color component is used as the center color. Based on the first color threshold and the second color threshold, the color range corresponding to that single color component is obtained. In one feasible approach, this operation can be implemented as follows: Method 1: Using the color corresponding to the maximum value of the pixel quantity distribution of a single color component as the center color; obtaining the relationship between the number of pixels corresponding to the center color and the second color threshold, and obtaining the color range corresponding to the single color component in this instance based on the relationship; and / or, Method 2: Obtaining a first ratio between the first color threshold and the number of pixels corresponding to the center color, and a second ratio between the number of pixels corresponding to the center color and the number of pixels of the center color used when determining the color range corresponding to the single color component in the previous instance; obtaining the color range corresponding to the single color component in this instance based on the first ratio and the second ratio.

[0056] In one feasible approach, as mentioned earlier, the individual color components can be grouped, such as by grouping according to component values, and then sorted to form a corresponding sorted sequence. Based on this, the above process can be implemented as a search in two directions, centered on the center color, extending towards both sides of the sorted sequence. If any group on either side does not satisfy the above method, the search is truncated, and the search continues downward.

[0057] In order to make the obtained background color range more accurate, the image data processing in this embodiment of the application is implemented as an iterative loop process. That is, after the background color range is determined once, the background color range will be used as a new background reference color system, and the process will return to step S204 to continue until the iteration termination condition is met, such as iterating to a preset number of times or the difference between the current determined background color range and the previous determined background color range is within a preset small range.

[0058] Based on this, in one iteration, according to Method 1, the color range corresponding to the single color component can be determined by the relationship between the number of pixels corresponding to the center color and the second color threshold. For example, if the number of pixels corresponding to the center color is less than the second color threshold, then the center color can be excluded. This is because the second color threshold itself is a relatively small number, and for most images, the background color is generally relatively concentrated, showing a continuous distribution on the histogram. Therefore, if the number of pixels of the center color is less than the second color threshold, it is less likely to belong to the background color, and the search in that direction can be stopped. This method can accurately and quickly exclude non-background colors, improving the efficiency of determining the background color range.

[0059] In most cases, the center color will not be less than the second color threshold. Therefore, method two can be used to determine whether the center color belongs to the background color range. The first ratio determines the proportion between the number of pixels of the center color and the number of pixels of the background color represented by the first color threshold; the second ratio determines the rate of decrease in the number of pixels corresponding to the center color in two consecutive measurements. If the value determined based on these two factors is greater than a certain cutoff threshold, the center color can be considered to belong to the background color range. This method comprehensively considers both the number of pixels and their decrease rate, allowing for a more effective determination of whether the center color belongs to the background color range and improving the adaptability of background color determination. Generally, the faster the decrease rate, the less likely it is to belong to the background color range.

[0060] In one example, method two can be implemented in the form of the following formula:

[0061] The expression `pow(threshold1 / currentVal, pow(currentVal / lastVal, exp)) > targetThreshold` means that if the result calculated by the above formula is greater than `targetThreshold`, then the current center color belongs to the background color range.

[0062] Step S208: Obtain the background color range of the image to be processed based on the color range corresponding to the single color component.

[0063] After obtaining the color range corresponding to each of the at least some of the single color components, the single color components can be fused together, for example, the H component, S component and V component can be fused together into HSV model data, and the background color range of the image to be processed can be obtained based on this.

[0064] In a practical implementation, one feasible approach is to remove colors from the multiple color components that are outside the color range corresponding to each individual color component; based on the removed color components, the background color range of the image to be processed is obtained. This makes the obtained background color range more accurate.

[0065] In one scenario example, suppose the original image is an RGB image, and the background is a wrinkled green curtain. Then, it is first transformed to HSV space to become an HSV image, which is then used as the image to be processed.

[0066] As shown above, the background reference color system in this HSV image is green in the HSV space. Therefore, the range of the background reference color system of this HSV image can be defined as the neighborhood centered at 120 degrees, such as the range between 60 and 180 degrees. Correspondingly, this range corresponds to the relevant HSV model data. In this example, based on this HSV model data, dimensionality reduction is performed to obtain the H, S, and V components, forming three single-component data.

[0067] Then, the maximum value 1 and average value 1 of the pixel number distribution of the H component, the maximum value 2 and average value 2 of the pixel number distribution of the S component, and the maximum value 3 and average value 3 of the pixel number distribution of the V component are obtained. In this example, for the H component, based on its maximum value 1 and average value 1, its color range 1 is obtained. Then, in the HSV model data, colors other than those corresponding to color range 1 are removed (referred to as the first removal). Next, for the S component in the HSV model data after the first removal, based on its maximum value 2 and average value 2, its color range 2 is obtained. Then, in the HSV model data, colors other than those corresponding to color range 2 are removed (referred to as the second removal). Then, for the V component in the HSV model data after the second removal, based on its maximum value 3 and average value 3, its color range 3 is obtained. Then, in the HSV model data, colors other than those corresponding to color range 3 are removed (referred to as the third removal). The resulting HSV model data, after the third round of elimination, can be used to determine the background color range of the HSV image. The colors corresponding to this range are the background colors of the HSV image.

[0068] It should be noted that the above-described elimination method is only an exemplary method. In practical applications, other methods can also be used to obtain the background color range of an HSV image, such as operating on each individual color component separately to obtain the color range corresponding to each individual color component, and then integrating based on these color ranges. All of these methods are within the protection scope of the embodiments of this application.

[0069] As can be seen, in this embodiment, when identifying the background color of an image, at least a portion of its multiple color components in the color space are reduced to a single color component based on a pre-defined background reference color system (such as the color system corresponding to a pre-defined background object like a curtain, wall, or display screen). Then, based on the pixel distribution of the single color component, the color range corresponding to that single color component is obtained, and the background color range is obtained based on that single color component. The pixel distribution can represent different background color conditions, such as a pure background color or a background color with a large range. The color range obtained based on this can effectively adapt to dynamic changes in background color caused by pre-defined background objects in real-world scenarios, making the determined background color range more accurate. Furthermore, calculating the color range by reducing the dimensionality to a single color component significantly reduces the computational time complexity and the complexity of the algorithm logic.

[0070] Example 2

[0071] Reference Figure 3AThe diagram shows a flowchart of the steps of an image data processing method according to Embodiment 2 of this application.

[0072] In this embodiment, the image data processing method of this application embodiment is described with a focus on how to accurately determine the background reference color system.

[0073] The image data processing method in this embodiment includes the following steps:

[0074] Step S302: Obtain the image to be processed in the preset color space.

[0075] The preset color space is a color space that reflects the hue of the image to be processed.

[0076] The specific implementation of this step can be referred to the relevant description in the aforementioned Embodiment 1, and will not be repeated here.

[0077] Step S304: Determine the background reference color scheme of the image to be processed.

[0078] In one feasible approach, a predetermined background color scheme of the image to be processed can be obtained; a neighborhood color space corresponding to the predetermined background color scheme is determined according to a preset first neighborhood range; and a background reference color scheme of the image to be processed is determined based on the neighborhood color space. For example, if the preset background object is a green curtain, in the HSV space corresponding to the image to be processed, the hue in the neighborhood centered at 120 degrees, such as the range between 60 degrees and 180 degrees, can be determined as the background reference color scheme of the image.

[0079] In this way, on the one hand, based on the background reference color system, non-background colors can be quickly filtered out from a general range; on the other hand, the predetermined background color system can serve as an initial reference for subsequent background color systems; and furthermore, the range of background colors can be constrained to a neighborhood space centered on the corresponding background reference color for subsequent data processing, which can not only ensure the accuracy of the background colors obtained later, but also greatly improve the speed and efficiency of data processing.

[0080] Optionally, when determining the background reference color system of the image to be processed based on the neighborhood color space, the neighborhood color space can be divided into multiple color regions of the first region granularity according to a preset first color region division granularity; the background reference color is determined based on the most densely colored region in each color region; and the background reference color is determined based on the background reference color. This reduces data processing complexity and facilitates faster and more accurate determination of the background center color, ensuring the reliability of the calculated background center color. The first color region division granularity can be set by those skilled in the art according to actual needs. For example, in the HSV color space, the neighborhood color space corresponding to the predetermined background color system can be divided into an nH*nS*nV space, where nH, nS, and nV are the number of regions into which the H, S, and V components of the neighborhood color space are evenly divided, respectively. For example, n can be 30-10.

[0081] However, to further improve the accuracy of background color range determination, in one feasible approach, determining the background reference color system of the image to be processed based on the background reference color can be implemented as follows: Based on the background reference color, determine a candidate background reference color system for the image to be processed; determine the neighborhood color space corresponding to the candidate background reference color system according to a preset second neighborhood amplitude range, wherein the second neighborhood amplitude range is smaller than the aforementioned first neighborhood amplitude range; divide the neighborhood color space corresponding to the candidate background reference color system into multiple color regions with a second region granularity according to a preset second color region division granularity; determine the background reference color system of the image to be processed based on the most densely colored region in each color region. The second color region division granularity can be smaller than or the same as the aforementioned first color region division granularity.

[0082] For example, still using the HSV color space, after determining the candidate background reference color system, the HSV model data corresponding to the candidate background reference color system can be further divided, for example, evenly divided into a space of mH*mS*mV, where mH, mS, and mV are the number of regions into which the H, S, and V components of the neighboring color space are evenly divided, respectively. For example, m can be 20-10.

[0083] Even though the background color range can be determined relatively accurately, it may still contain some noisy data. To make the results more accurate and pure, in one feasible approach, the background reference color of the image to be processed can be determined based on the densest color region in each color region as follows: determine the background center color of the image to be processed based on the densest color region in each color region; using the background center color as the color center, remove colors in the neighboring color space of the candidate background reference color that have a color difference exceeding a preset difference threshold from the color center; and determine the background reference color of the image to be processed based on the removal results.

[0084] As shown above, the background color range is determined by dividing it into multiple levels. While ensuring the reliability of the calculation results, it can also effectively reduce the computational and time complexity caused by using color space model data as a whole for calculation.

[0085] Step S306: Determine multiple color components corresponding to the background reference color system of the image to be processed based on the background reference color system of the image to be processed.

[0086] For example, multiple components of the HSV space or multiple components of the HSL space corresponding to the background reference color system, etc. Specific implementation methods can be found in the descriptions of related technologies, and will not be detailed here.

[0087] Step S308: Reduce the dimensionality of multiple color components to the single color components corresponding to different components.

[0088] Step S310: Obtain the color range corresponding to each single color component based on the pixel quantity distribution of each single color component.

[0089] The specific implementation of the above steps S308-S310 can be referred to the description of the relevant parts in the aforementioned Embodiment 1, and will not be repeated here.

[0090] Step S312: Obtain the background color range of the image to be processed based on the color range corresponding to each single color component.

[0091] The process includes: obtaining the corresponding candidate background color range based on the color range corresponding to each single color component; using the candidate background color range as a new background reference color system and returning the operation of determining multiple color components corresponding to the background reference color system of the image to be processed based on the background reference color system of the image to be processed (step S306) iteratively executing until the iteration termination condition is reached; and using the candidate background color range at the time of iteration termination as the background color range of the image to be processed.

[0092] The iteration termination condition may be that the currently calculated background color range is consistent with the result of the previous round of calculation, or the number of iterations reaches a preset upper limit.

[0093] By using this multi-round iterative calculation method, the detection range for the background color range can be continuously narrowed to approach the true background color distribution range. Furthermore, the complexity of a single calculation is low, and the overall calculation complexity and accuracy are good.

[0094] The following example illustrates the above process, showing the image data processing procedure as follows: Figure 3B As shown.

[0095] The process includes:

[0096] (A) Adjust the resolution of the image to be processed to the preset resolution.

[0097] The specific value of the preset resolution can be set by those skilled in the art according to actual needs; this example does not impose any restrictions on it. If the resolution of the image to be processed already matches the preset resolution, this step is unnecessary.

[0098] (B) Convert the image to be processed to the target color space.

[0099] In this example, the target color space is HSV color space, and the original image is converted into an HSV image. However, those skilled in the art should understand that other color spaces that can reflect the hue of the image to be processed can also be used to implement the corresponding image data processing with reference to this example.

[0100] (C) Define a neighborhood color space based on the specified background reference color system and remove pixels that are significantly too far away from the background reference color system.

[0101] Whether the color distance from the background reference color is too far can be determined by setting an appropriate threshold. For example, if the specified reference background color is green, pixels whose hue is not included in the neighborhood centered at 120 degrees (e.g., 60-180 degrees) or whose saturation is low (less than the preset saturation threshold, such as 10%) will be removed.

[0102] For ease of description, in this example, the defined neighborhood color space will be referred to as the first neighborhood color space.

[0103] (D) According to the granularity of the first color region, the first neighboring color space is divided into uniform regions of the first region granularity, thereby uniformly dividing the color space within the neighborhood into a 3D space of nH*nS*nV. Here, nH, nS, and nV are the number of regions into which the H, S, and V of the neighboring color space are uniformly divided, respectively. For example, n can be 30.

[0104] (E) Calculate the distribution of unremoved pixels (full component) of the HSV image within a uniform region of multiple first region granularities corresponding to the first neighborhood color space, and obtain a relatively reliable background reference color based on the region with the densest distribution.

[0105] For ease of description, the color space in the full component case is referred to as 3D color gradation in the figure.

[0106] Furthermore, if the obtained background reference color has a region with too few pixels (e.g., the number of pixels is less than 2% of the number of pixels in the first neighboring color space), it is treated as an abnormal case (the calculation is terminated and it is determined to be an object without a background).

[0107] (F) defines a smaller color neighborhood space centered on the background reference color.

[0108] In this example, the accurate range of background color is calculated by specifically exploring this smaller color neighborhood space, and pixels outside the neighborhood range are removed.

[0109] The range of this smaller color neighborhood space can be set by those skilled in the art according to the actual situation, and it can be smaller than the neighborhood range in (C) above. For example, with the background reference color as green as the center, pixels whose hue is not included in the neighborhood centered at 120 degrees (e.g., 39 degrees-150 degrees) are removed.

[0110] In addition, in this example, for ease of distinction, the background reference color scheme corresponding to this smaller color neighborhood space is referred to as the candidate background reference color scheme.

[0111] (G) Divide the smaller color neighborhood space into multiple uniform intervals according to the granularity of the second color region.

[0112] For example, the smaller color neighborhood space can be uniformly divided into a color space of mH*mS*mV, where mH, mS, and mV are the number of regions into which the smaller neighborhood color space is uniformly divided (H, S, and V, respectively). Here, mH / mS / mV can be equal to nH / nS / nV in (D), or can be adjusted to different values ​​as needed.

[0113] (H) In this smaller color neighborhood space, the distribution of the pixels retained after removing pixels in (F) is calculated (full component), and the background center color is calculated more accurately based on the densest distribution area.

[0114] (I) Using the more accurate center color of the background as the center, remove colors that are too far away from the center color of the background.

[0115] This step can be performed directly on the pixel distribution result in the color neighborhood space calculated by (H). For example, the removal method can use Manhattan distance or Chebyshev distance, etc.

[0116] A color distance threshold can be set. If the color distance is greater than the threshold, it is considered to be too far from the center color of the background. This threshold can be set by those skilled in the art based on actual conditions, and this embodiment does not impose any limitations on it. For example, the threshold can be a hue value of 10-15 degrees, etc.

[0117] (J) To reduce computational complexity, the full components of HSV are reduced to single components.

[0118] For example, in HSV, after dimensionality reduction, you will obtain an H component, an S component, and a V component, all of which are one-dimensional color levels.

[0119] For ease of description, only the H component is used as an example in the figure, but those skilled in the art should understand that the processing of other components can refer to the processing of the H component.

[0120] (K) The minimum value obtained by multiplying the maximum and average values ​​of each one-dimensional color level by the corresponding coefficient is determined as the first color threshold. The first color threshold is then reduced by a certain factor to determine the second color threshold.

[0121] For example, threshold1 = min(maxVal*a, avgVal*b);

[0122] threshold2 = threshold1 * c;

[0123] In this formula, threshold1 represents the first color threshold, threshold2 represents the second color threshold, maxVal represents the maximum value, avgVal represents the average value, min() represents the minimum value, and a, b, and c are coefficients. In practical applications, a, b, and c can be set by those skilled in the art according to the actual background color range to be searched. Usually, b is much larger than a and b. For example, a can be 1%-2%, b can be 100%-150%, and c can be 5%-10%, etc. That is, if you want to find a larger background color range, you can set a and b to make threshold1 larger, or you can set c to make threshold2 smaller. Of course, both can be used. It should be noted that the above formula is applicable to all components in the color space.

[0124] It should be noted that each component has its corresponding threshold1 and threshold2.

[0125] (L) For each component, based on threshold1 and threshold2, the distribution range of each component is centered on the maximum value corresponding to each component in two directions.

[0126] For example, taking the H component as an example, based on the threshold1 and threshold2 of the H component, the distribution range of the H component is searched with the maximum value of the H component as the center.

[0127] The search logic is as follows: if the number of pixels corresponding to the current maximum value is less than the second color threshold, then it is truncated, that is, excluded from the possible background color range.

[0128] or,

[0129] If the number of pixels corresponding to the current maximum value is less than the first color threshold, and the first color threshold, the number of pixels corresponding to the current maximum value, and the number of pixels corresponding to the maximum value used in the previous search satisfy the following conditions:

[0130] pow(threshold1 / currentVal,pow(currentVal / lastVal,exp))>targetThreshold,

[0131] Then it can be categorized into the possible background color range.

[0132] Where currentVal represents the number of pixels corresponding to the current maximum value, lastVal represents the number of pixels corresponding to the maximum value used in the previous search, exp is a specific exponent that is generally significantly greater than 1, and targetThreshold represents the specified truncation threshold.

[0133] This process yields a possible range of background colors, which in this example is referred to as the suspected background color range for a specific component. For example, the suspected background color range for component H. Similarly, performing the above processing on each component will obtain the suspected background color range for each component.

[0134] (M) Based on the suspected background color range of each component, the part outside the suspected background color range in the neighboring color space where color removal has been performed in (I) is removed.

[0135] In one feasible approach, the removal can be performed directly on the 3D color scale.

[0136] At this point, the range of background colors for this round of calculations can be obtained.

[0137] (N) After completing the first round of processing (C)-(M), continue with the second round of processing until the background color range obtained in a certain calculation is consistent with the result of the previous round or the number of calculation repetitions reaches the preset upper limit.

[0138] The background color range at the end of the iteration can be ultimately determined as the background color range of the image to be processed.

[0139] In this embodiment, (1) the background color range is constrained to search within the neighborhood space centered on a certain reference background color, which improves the search speed and efficiency; (2) the full component model data of the background reference color system is calculated at multiple levels and different regional granularities, which ensures the reliability of the calculation of the background center color and greatly reduces the computational complexity of the color space and the time complexity of subsequent searches; (3) when calculating the background color range, the dimension is reduced to search on the components, which greatly reduces the computational time complexity and the complexity of the algorithm logic; (4) when searching the background color range on a single component, the maximum value and the average value are used to take into account the cases where the background color is relatively pure and the range is large; when searching in two directions with the maximum value as the center, the proportion of the number of pixels corresponding to the current maximum value in the suspected background range and the rate of decrease of the number of pixels are considered at the same time, and the adaptive ability of various scenarios is good; (5) through multiple rounds of iterative calculation, the search range is continuously narrowed to approach the real background color distribution range, and the single calculation complexity is low, and the overall calculation complexity and accuracy are good.

[0140] Example 3

[0141] Reference Figure 4 The flowchart illustrates the steps of an image data processing method according to Embodiment 3 of this application.

[0142] Step S402: Obtain the image to be processed and the preset background reference color scheme of the image to be processed.

[0143] In this embodiment, the image to be processed is the image to be segmented into foreground and background. It can be a single static image or a video frame image in a video, including but not limited to: video frame images in a video conference, director images in a studio broadcast, live images in a live video broadcast, images to be processed by AR (augmented reality), images used for video production, etc.

[0144] The background reference color scheme for the image to be processed can be set manually, for example, it can be preset based on the color of the background objects in the image to be processed.

[0145] Step S404: Convert the image to be processed to a preset color space and determine the multiple color components of the background reference color system in the preset color space.

[0146] The preset color space is a color space that can reflect the hue of the image to be processed, including but not limited to HSV color space, HSL color space, etc.

[0147] Typically, the images to be processed are RGB images. Therefore, they need to be converted to a color space that reflects hue, such as the HSV or HSL color space. Correspondingly, the background reference color scheme also needs to be consistent with the color space used by the image to be processed, that is, it also needs to be a color space that reflects hue.

[0148] Step S406: Obtain the background color range of the image to be processed based on the pixel number distribution of at least some of the single color components among the multiple color components of the background reference color system.

[0149] In one feasible approach, at least some of the single color components among multiple color components of the background reference color system can be obtained; the color range corresponding to the single color component is obtained according to the pixel number distribution of the single color component; and the background color range of the image to be processed is obtained based on the color range corresponding to the single color component.

[0150] Step S408: Perform background segmentation processing on the image to be processed based on the background color range.

[0151] After determining the background color range of the image to be processed, its background region can be identified, and then foreground and background segmentation can be performed. Further processing can then be carried out based on the results of the foreground and background segmentation.

[0152] It should be noted that the above process is described in a relatively simple manner, and the specific implementation of each step can be referred to the description of the relevant parts in the foregoing embodiments.

[0153] The following examples illustrate background segmentation of images to be processed, using different scenarios as examples.

[0154] Scenario 1: Video Conferencing

[0155] In many video conferences, there is often a need for content presentations, product demonstrations, or other background replacements. Therefore, when hosting a video conference, a background object can be set up in the physical meeting space beforehand, and images of the video conference can be captured against this background. Then, for each frame of the real-time video stream, the aforementioned image data processing method is used to determine its background color range, and based on this, its background area is determined. Foreground and background segmentation is then performed based on this background area. The foreground portion, such as the speaker's image, is retained, and the segmented background portion is replaced with an image that includes the content to be presented (such as PowerPoint content), or an image that includes the product to be presented (such as product images), or simply a background image that matches the meeting theme. This effectively integrates the meeting with its content.

[0156] Scenario 2: Live broadcast or studio directing scenario

[0157] Similar to video conferencing, these scenarios also involve content, product, or scene demonstrations. Taking tourism live streaming as an example, in addition to introducing local specialties and scenery, the host will use images of the scenic spots to attract viewers. Based on this, a background object can be pre-set in the physical live streaming room, and the live stream can be conducted against this background. During the live stream, for each frame of the video stream, the aforementioned image data processing method is used to determine its background color range, and based on this, its background area is determined. Then, foreground and background segmentation is performed based on this background area. Then, the foreground portion, such as the host's image, is retained, and the segmented background portion is replaced with images containing the local specialties to be displayed, images of famous scenic spots, or images of scenic views, etc. This achieves effective display of the live stream content. The studio directing scenario is similar and will not be elaborated further.

[0158] Scenario 3: Online Education Scenario

[0159] Similar to video conferencing, online education also requires content presentation. Therefore, a background object can be pre-set in a physical space, such as a physics classroom, and the lecture can be conducted against this background. The teacher's lecture is recorded as a video. Then, for each frame of the video, the aforementioned image data processing methods are used to determine its background color range, and based on this, the background area is defined. Foreground and background segmentation is then performed based on this background area. Next, the foreground portion, such as the teacher's image, is retained, and the segmented background portion is replaced with images containing the courseware content to be presented, or images of graphics (dynamic or static) related to the lecture content. This achieves a vivid and engaging presentation of classroom content.

[0160] Scenario 4: AR Scenario

[0161] Whether targeting video frames within a video stream or a single still image, there's always the possibility of using AR effects. For example, adding AR objects (such as red envelopes or pets) to video frames allows for interaction with viewers, or adding AR objects (such as text annotations, fun descriptions, or figurative decorations) to still images. In these cases, if the image to be processed has relatively few background colors, the corresponding background reference color scheme can be determined first. Then, using the aforementioned image data processing methods, the background color range can be determined, and the background area in the video frame or still image can be identified. Based on this background area, appropriate AR effects can be added. However, this is not limited to this; the entire identified background area can also be replaced with AR effects to meet different needs, such as interactive or fun requirements, thus improving the user experience.

[0162] Scenario 5: Video Production Scenario

[0163] During video production, it's common to find that the backgrounds of some images don't meet requirements and need modification. In such cases, if the image to be processed has relatively few background colors, we can first determine the background reference color system corresponding to that background color, then use the aforementioned image data processing methods to determine its background color range, and finally determine the background area in the image to be used. This background area can then be modified or replaced to meet the overall requirements of the video being produced, thus improving the overall video quality.

[0164] As can be seen, this embodiment can provide services for various usage scenarios by accurately identifying the background area of ​​the image and performing foreground-background segmentation, greatly satisfying the needs of different usage scenarios and improving the user experience.

[0165] Example 4

[0166] Reference Figure 5 The diagram shows a structural schematic of an electronic device according to Embodiment 4 of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0167] like Figure 5 As shown, the electronic device may include: a processor 502, a communications interface 504, a memory 506, and a communications bus 508.

[0168] in:

[0169] The processor 502, communication interface 504, and memory 506 communicate with each other via communication bus 508.

[0170] Communication interface 504 is used to communicate with other electronic devices or servers.

[0171] The processor 502 is used to execute program 510, which can specifically execute the relevant steps in any of the above-described image data processing method embodiments.

[0172] Specifically, program 510 may include program code that includes computer operation instructions.

[0173] The processor 502 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0174] Memory 506 is used to store program 510. Memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0175] Specifically, program 510 can be used to cause processor 502 to perform the operations corresponding to the image data processing method described in the foregoing embodiments one, two or three.

[0176] The specific implementation of each step in program 510 can be found in the corresponding steps and units described in the above-described image data processing method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the aforementioned method embodiments, and will not be repeated here.

[0177] This application also provides a computer program product, including computer instructions that instruct a computing device to perform an operation corresponding to any of the image data processing methods in the above-described multiple method embodiments.

[0178] It should be noted that in many embodiments of this application, the color space is HSV space as an example. However, those skilled in the art should understand that the determination of the background color range of other color spaces that can reflect hue can also be implemented with reference to many embodiments of this application, such as HSL space, YUV space, etc.

[0179] It should be noted that, depending on the implementation needs, the components / steps described in the embodiments of this application can be split into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0180] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored as software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the image data processing methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the image data processing methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the image data processing methods shown herein.

[0181] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0182] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. An image data processing method, comprising: obtaining a to-be-processed image in a preset color space, the preset color space being a color space capable of reflecting a hue of the to-be-processed image; determining a plurality of color components corresponding to a background reference color system of the to-be-processed image according to the background reference color system, and obtaining at least part of single-color components in the plurality of color components; obtaining a color range corresponding to a single-color component according to a pixel quantity distribution of the single-color component, the pixel quantity distribution of the single-color component being a distribution of quantities of pixels corresponding to a plurality of color ranges included in the single-color component; obtaining a background color range of the to-be-processed image based on the color range corresponding to the single-color component; the obtaining of the color range corresponding to the single-color component according to the pixel quantity distribution of the single-color component comprises: determining a first color threshold and a second color threshold according to a maximum value and an average value of the pixel quantity distribution of each single-color component in the at least part of single-color components; obtaining the color range corresponding to the single-color component according to the first color threshold and the second color threshold with a color corresponding to the maximum value as a center color.

2. The method of claim 1, wherein, the obtaining of the color range corresponding to the single-color component according to the first color threshold and the second color threshold comprises: obtaining a relationship between a quantity of pixels corresponding to the center color and the second color threshold, and obtaining the color range corresponding to the single-color component this time according to the relationship; and / or, obtaining a first ratio of quantities of pixels corresponding to the first color threshold and the center color and a second ratio of quantities of pixels corresponding to the center color and a center color used when a color range corresponding to the single-color component was determined last time; and obtaining the color range corresponding to the single-color component this time according to the first ratio and the second ratio.

3. The method of claim 1 or 2, wherein, the obtaining of the background color range of the to-be-processed image based on the color range corresponding to the single-color component comprises: excluding other colors in the plurality of color components except for the color range corresponding to each single-color component; obtaining the background color range of the to-be-processed image according to the plurality of color components after the exclusion.

4. The method of claim 1, wherein, before the determining of the plurality of color components corresponding to the background reference color system according to the background reference color system of the to-be-processed image, the method further comprises: obtaining a predetermined background color system of the to-be-processed image; determining a neighborhood color space corresponding to the predetermined background color system according to a preset first neighborhood amplitude range, and determining the background reference color system of the to-be-processed image according to the neighborhood color space.

5. The method of claim 4, wherein, the determining of the background reference color system of the to-be-processed image according to the neighborhood color space comprises: dividing the neighborhood color space into a plurality of color regions of a first region granularity according to a preset first color region division granularity; determining a background reference color according to a color densest region in each color region; determining the background reference color system of the to-be-processed image according to the background reference color.

6. The method of claim 5, wherein, the determining of the background reference color of the to-be-processed image according to the background reference color comprises: determining a candidate background reference color system of the image to be processed according to the background reference color; determining a neighborhood color space corresponding to the candidate background reference color system according to a preset second neighborhood magnitude range, wherein the second neighborhood magnitude range is smaller than the first neighborhood magnitude range; dividing the neighborhood color space corresponding to the candidate background reference color system into a plurality of color regions of a second region granularity according to a preset second color region division granularity; determining a background reference color system of the image to be processed according to a color densest region in each color region.

7. The method of claim 6, wherein, The determining a background reference color system of the image to be processed according to a color densest region in each color region comprises: determining a background center color of the image to be processed according to a color densest region in each color region; excluding colors in the neighborhood color space corresponding to the candidate background reference color system that have a color difference from a color center color exceeding a preset difference threshold, taking the color center color as the color center; determining a background reference color system of the image to be processed according to the exclusion result.

8. The method of claim 1, wherein, The obtaining a background color range of the image to be processed based on the color range corresponding to the single color component comprises: obtaining a corresponding candidate background color range based on the color range corresponding to the single color component; taking the candidate background color range as a new background reference color system, and returning to the operation of determining the plurality of color components corresponding to the background reference color system of the image to be processed according to the background reference color system of the image to be processed, and iteratively executing until an iteration termination condition is reached; taking the candidate background color range at the iteration termination as the background color range of the image to be processed.

9. An image data processing method, comprising: obtaining an image to be processed and a preset background reference color system of the image to be processed; converting the image to be processed to a preset color space, and determining a plurality of color components of the background reference color system in the preset color space, wherein the preset color space is a color space that can reflect the hue of the image to be processed; obtaining at least part of single color components in the plurality of color components of the background reference color system; obtaining a color range corresponding to the single color component according to a pixel number distribution of the single color component; obtaining a background color range of the image to be processed based on the color range corresponding to the single color component; the pixel number distribution of the single color component is a distribution of the number of pixels corresponding to a plurality of color ranges included in the single color component; performing background segmentation processing on the image to be processed based on the background color range. The obtaining a color range corresponding to the single color component according to the maximum value and the average value of the pixel number distribution of the single color component comprises: determining a first color threshold and a second color threshold according to the maximum value and the average value of the pixel number distribution of each single color component in the at least part of single color components, respectively; obtaining a color range corresponding to the single color component according to the first color threshold and the second color threshold, taking a color corresponding to the maximum value as a center color.

10. An electronic device comprising: a processor, a memory, a communication interface, and a communication bus, which enable communication among the processor, the memory, and the communication interface through the communication bus; the memory is configured to store at least one executable instruction, which causes the processor to perform operations corresponding to the method in any one of claims 1-9.

11. A computer storage medium having stored thereon a computer program, which, when executed by a processor, implements the method in any one of claims 1-9.

12. A computer program product comprising computer instructions, which instruct a computing device to perform operations corresponding to the method in any one of claims 1-9.

Images