Image processing method and device, electronic equipment and storage medium

Abstract

The invention relates to an image processing method and device, electronic equipment and a storage medium. The image processing method comprises the steps that a to-be-processed image is acquired; performing image segmentation processing on the to-be-processed image according to colors to obtain a plurality of segmented regions; based on region image information of each segmented region, performing region merging processing on the plurality of segmented regions to obtain a plurality of merged regions, the region image information including color information of the segmented regions; and determining main color information of the to-be-processed image based on each merged region. The image segmentation processing is performed according to the colors, and the region merging processing is performed according to the image information of the segmented regions, so that the accuracy of the segmentation of the image regions with different colors and the accuracy of the merging process of the image regions with approximate colors are ensured, and the region segmentation and region merging processes are simplified; the extraction efficiency of the main color information and the accuracy of the extraction result are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] As one of the main features of an image, color is an important factor that affects the overall visual effect of the image. Extracting the main color of an image has gradually become a key technology in the field of image processing. The main color extraction result can be obtained by extracting the main color of an image. Image classification, image retrieval, image fusion and other functions can be realized based on the main color extraction result.

[0003] However, the extraction of the main color of an image using the image processing method of the related art has problems such as a complex extraction process and poor color merging effect, resulting in low extraction efficiency and deviation in the extraction results. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides an image processing method, device, electronic device and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, the image processing method comprising:

[0006] Get the image to be processed;

[0007] Performing image segmentation processing on the image to be processed according to color to obtain multiple segmented areas;

[0008] Based on the regional image information of each of the segmented regions, performing region merging processing on the multiple segmented regions to obtain multiple merged regions, wherein the regional image information includes color information of the segmented regions;

[0009] Based on each of the merged areas, main color information of the image to be processed is determined.

[0010] In some embodiments of the present disclosure, the image to be processed is segmented according to color to obtain multiple segmented areas, including:

[0011] Performing flood filling processing on the image to be processed to obtain the multiple segmented areas.

[0012] In some embodiments of the present disclosure, performing image segmentation processing on the image to be processed according to color to obtain a plurality of segmented regions further comprises:

[0013] The region identifier of the segmented region is determined according to the color value of the color filled in each segmented region by the flood filling process.

[0014] In some embodiments of the present disclosure, the region merging process is performed on the plurality of segmented regions based on the region image information of each segmented region, including:

[0015] Converting the image to be processed from the first color space to the second color space to obtain a converted image;

[0016] Acquire the color information of each of the segmented areas from the converted image;

[0017] Based on the color information of each of the segmented regions, the segmented regions whose color differences meet a preset merging condition are merged to obtain the multiple merged regions.

[0018] In some embodiments of the present disclosure, based on the color information of each of the segmented regions, merging the segmented regions whose color differences meet a preset merging condition includes:

[0019] The segmented regions are used as candidate regions, and adjacent candidate regions that meet the preset merging condition are merged into new candidate regions until all candidate regions meet the preset merging stop condition;

[0020] Determine each of the candidate regions after merging stops as the merging region;

[0021] The preset merging condition includes that a color difference value determined based on color information of the candidate region is less than a first threshold value, and the preset merging stop condition includes that the color difference values ​​between the adjacent candidate regions are greater than or equal to the first threshold value.

[0022] In some embodiments of the present disclosure, the regional image information further includes the number of pixels of the segmented regions. Before merging the segmented regions whose color differences meet preset merging conditions based on the color information of each segmented region, the regional merging process of the multiple segmented regions based on the regional image information of each segmented region further includes:

[0023] The segmented area with the number of pixels less than the second threshold is removed.

[0024] In some embodiments of the present disclosure, the first color space is an RBG color space, the second color space is a LAB color space, and acquiring the color information of each segmented area from the converted image includes:

[0025] The channel mean of each channel of each segmented area in the LAB color space is obtained, and the channel mean is used as the color information.

[0026] In some embodiments of the present disclosure, determining the main color information of the image to be processed based on each of the merged regions includes:

[0027] Determine at least one main color area based on the number of pixels in each of the merged areas;

[0028] Determine a channel mean of each channel of the at least one primary color region in the RBG color space as the primary color information.

[0029] In some embodiments of the present disclosure, the step of obtaining an image to be processed includes:

[0030] The original image is subjected to denoising processing to obtain the image to be processed.

[0031] In some embodiments of the present disclosure, the denoising process is performed on the original image to obtain the image to be processed, including:

[0032] The original image is processed by a mean shift filter to obtain the image to be processed.

[0033] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing device, the image processing device comprising:

[0034] An acquisition module, wherein the acquisition module is used to acquire an image to be processed;

[0035] A segmentation module, the segmentation module is used to perform image segmentation processing on the image to be processed according to color to obtain multiple segmentation areas;

[0036] A merging module, the merging module is used to perform region merging processing on the multiple segmented regions based on the region image information of each of the segmented regions to obtain multiple merged regions, wherein the region image information includes color information of the segmented regions;

[0037] A determination module is used to determine the main color information of the image to be processed based on each of the merged areas.

[0038] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, the electronic device comprising:

[0039] processor;

[0040] a memory for storing processor-executable instructions;

[0041] Wherein, the processor is configured to:

[0042] Get the image to be processed;

[0043] Performing image segmentation processing on the image to be processed according to color to obtain multiple segmented areas;

[0044] Based on the regional image information of each of the segmented regions, performing region merging processing on the multiple segmented regions to obtain multiple merged regions, wherein the regional image information includes color information of the segmented regions;

[0045] Based on each of the merged areas, main color information of the image to be processed is determined.

[0046] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform an image processing method. The image processing method includes:

[0047] Get the image to be processed;

[0048] Performing image segmentation processing on the image to be processed according to color to obtain multiple segmented areas;

[0049] Based on the regional image information of each of the segmented regions, performing region merging processing on the multiple segmented regions to obtain multiple merged regions, wherein the regional image information includes color information of the segmented regions;

[0050] Based on each of the merged areas, main color information of the image to be processed is determined.

[0051] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: performing image segmentation processing according to color, and performing region merging processing according to image information of the segmented regions, thereby ensuring the accuracy of the segmentation of image regions of different colors and the merging of image regions of similar colors, simplifying the region segmentation and region merging processes, and improving the efficiency of extracting primary color information and the accuracy of the extraction results.

[0052] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0054] Figure 1 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0055] Figure 2 is a schematic diagram of an image to be processed according to an exemplary embodiment.

[0056] Figure 3 is a schematic diagram showing a segmented area according to an exemplary embodiment.

[0057] Figure 4 The flowchart is a flowchart of performing region merging processing on multiple segmented regions based on region image information of each segmented region according to an exemplary embodiment.

[0058] Figure 5 The flowchart is a flowchart of merging the segmented regions whose color differences meet the preset merging condition based on the color information of each segmented region according to an exemplary embodiment.

[0059] Figure 6 is a schematic diagram of a conversion image according to an exemplary embodiment.

[0060] Figure 7 The present invention is a flowchart showing a method of determining the main color information of an image to be processed based on each merged area according to an exemplary embodiment.

[0061] Figure 8 is a schematic diagram of an original image according to an exemplary embodiment.

[0062] Fig. 9 is a flowchart of an image processing method according to another exemplary embodiment.

[0063] Fig.10 This is a comparison diagram of the results of extracting the main color from an original image. Fig.10 a is a schematic diagram of an original image. Fig.10 b is Fig.10 a Schematic diagram of the main color extraction result using the image processing method of the related technology, Fig.10 c is Fig.10 a Schematic diagram of the main color extraction result using the image processing method of the exemplary embodiment of the present application.

[0064] Fig.11 This is a comparison diagram of the results of main color extraction from another original image. Fig.11 a is a schematic diagram of another original image, Fig.11 b is Fig.11 a Schematic diagram of the main color extraction result using the image processing method of the related technology, Fig.11 c is Fig.11 a Schematic diagram of the main color extraction result using the image processing method of the exemplary embodiment of the present application.

[0065] Fig.12 This is a comparison diagram of the results of main color extraction from another original image. Fig.12 a is a schematic diagram of another original image, Fig.12 b is Fig.12 a Schematic diagram of the main color extraction result using the image processing method of the related technology, Fig.12 c is Fig.12 a Schematic diagram of the main color extraction result using the image processing method of the exemplary embodiment of the present application.

[0072] Fig.13 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0073] Fig.14 is a block diagram of an electronic device according to an exemplary embodiment.

[0074] In the figure:

[0075] 10-acquisition module; 20-segmentation module; 30-merging module; 40-determination module; 101-processing component; 102-memory; 103-power component; 104-multimedia component; 105-audio component; 106-input / output interface; 107-sensor component; 108-communication component; 109-processor. DETAILED DESCRIPTION

[0076] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0077] Image color, as one of the main features of describing an image, is the main factor affecting the overall visual effect of the image, making color play a very important role in fields such as visual design. In order to achieve image retrieval functions that screen images with dominant colors that meet search requirements, as well as image fusion functions that use foreground dominant colors and harmonized background tones, it is usually necessary to extract the dominant color of the image, making dominant color extraction gradually become a key technology in the field of image processing.

[0078] In order to extract the main color, it is necessary to classify and merge image areas of different colors. For example, the K-means clustering algorithm can be used to extract the main color, cluster the image pixels into multiple clusters, and use the cluster center of each cluster as the extracted image main color according to the cluster size.

[0079] However, when using the K-means clustering algorithm to extract the main color of the image, the number of clusters is a preset value. If the number of clusters is set too small, pixels with dissimilar colors will be incorrectly merged. If the number of clusters is set too large, the color merging effect will not be significant. In addition, the pixel clustering process takes too long, and there are problems such as complex extraction process and poor color merging effect, resulting in low extraction efficiency and deviation in extraction results.

[0080] Based on this, the exemplary embodiment of the present disclosure provides an image processing method, which can extract the main color information according to the obtained merged area by performing image segmentation processing on the image to be processed and performing region merging processing on multiple segmented areas. The image segmentation processing is performed according to color, and the region merging processing is performed according to the image information of the segmented areas, which ensures the accuracy of the segmentation of image areas of different colors and the merging process of image areas of similar colors, simplifies the region segmentation and region merging process, and improves the extraction efficiency of the main color information and the accuracy of the extraction result.

[0081] In an exemplary embodiment, an image processing method is provided, referring to Figure 1 As shown, the image processing method includes:

[0082] S100: Acquire an image to be processed.

[0083] In step S100, firstly, an image to be processed is obtained, and the image to be processed is an image that needs to be subjected to primary color extraction. For example, the image to be processed may be an original image obtained by photographing or drawing, or may be an image obtained after certain preprocessing of the original image. When the image to be processed is an image obtained after certain preprocessing of the original image, the primary color information of the image to be processed that is finally determined is also the primary color information of the original image.

[0084] S200 , performing image segmentation processing on the image to be processed according to color to obtain a plurality of segmented regions.

[0085] In step S200, the image to be processed is segmented according to color. For example, the image to be processed can be divided into multiple segmented areas according to the color of each pixel in the image to be processed, so that the pixels in the same segmented area have the same or similar color.

[0086] S300 , based on the regional image information of each segmented area, performing region merging processing on multiple segmented areas to obtain multiple merged areas, wherein the regional image information includes color information of the segmented areas.

[0087] In step S300, after the image segmentation process is performed, different segmented regions have different color differences. To ensure the differences between the main colors of the extracted multiple images, segmented regions with larger color differences need to be merged while retaining the segmented regions with smaller color differences. Each segmented region has corresponding regional image information, and the regional image information includes color information of the segmented region. Exemplarily, the color information of the segmented region may be, for example, the channel value of the color channel of each pixel in the segmented region.

[0088] Based on the regional image information of each segmented area, a plurality of segmented areas can be merged to obtain a plurality of merged areas, wherein the number of merged areas is smaller than the number of segmented areas. By merging segmented areas with smaller color differences, the number of image areas is reduced, making the image area corresponding to the main color of the image more prominent.

[0089] S400: Determine the main color information of the image to be processed based on each merged area.

[0090] In step S400, there are significant color differences between the multiple merged areas obtained after image segmentation and area merging. Exemplarily, at least some of the merged areas can be selected from the multiple merged areas as the main color areas of the image to be processed, and information representing the colors of these merged areas can be used as the main color information of the image to be processed.

[0091] It should be noted that, unlike the related art in which the number of clusters is limited, the image processing method disclosed in the present invention does not limit the number of segmented regions when performing image segmentation processing, and does not limit the number of merged regions when performing region merging processing. Instead, the image segmentation processing and region merging processing are completed based on the actual color of the image to be processed and the regional image information of the segmented regions, thereby ensuring the color difference between the merged regions while achieving the color merging of the segmented regions.

[0092] In this embodiment, by performing image segmentation processing on the image to be processed and performing region merging processing on multiple segmented regions, the main color information can be extracted according to the obtained merged region. The image segmentation processing is performed according to color, and the region merging processing is performed according to the image information of the segmented regions, which ensures the accuracy of the segmentation of image regions of different colors and the merging process of image regions of similar colors, simplifies the region segmentation and region merging process, and improves the extraction efficiency of the main color information and the accuracy of the extraction result.

[0093] In some embodiments, the image to be processed can be segmented by image segmentation methods such as binarization segmentation. In other embodiments, the image to be processed is segmented according to color to obtain multiple segmented areas, including: the image to be processed is flood-filled to obtain multiple segmented areas.

[0094] Flood fill processing is a method of filling connected areas with specific colors. Different filling effects are achieved by selecting connected areas composed of pixels and filling them with different colors. Adjacent pixels with similar colors are divided into the same connected area and the pixels in the connected area are given the same color, thereby realizing the division of multiple segmented areas. The filling colors of different connected areas can be randomly selected or pre-set.

[0095] For example, Figure 2 The image to be processed as shown in the figure is subjected to flood filling processing. A seed point can be arbitrarily selected in the image to be processed, and the difference between the pixel values ​​of other pixels in the neighborhood of the seed point and the pixel value of the seed point is calculated. The pixel points whose difference is less than a certain preset value are added to the area where the seed point is located, and the newly added pixel points are used as new seed points to repeatedly perform the above steps until no new pixel points are added to the area. Finally, the same color is given to each pixel in the area, and the following is obtained. Figure 3 Multiple segmented areas are shown.

[0096] It should be noted that the filling colors of different segmented regions may be different from the actual colors of the segmented regions in the image to be processed, and the filling colors of the segmented regions only reflect the segmentation effects of the different segmented regions.

[0097] In this embodiment, by performing flood filling processing on the image to be processed, the image to be processed can be segmented according to color, thereby realizing the determination of multiple segmented areas. The flood filling processing divides the image to be processed into multiple segmented areas according to color, which can ensure that there is a certain color difference between the segmented areas, and fills the same segmented area with a uniform color, which is convenient for the subsequent use of the colors filled in different segmented areas as area identifiers of the segmented areas. In addition, the flood filling processing has a fast response speed, which can improve the efficiency of extracting the main color information.

[0098] In some embodiments, performing image segmentation processing on the image to be processed according to color to obtain a plurality of segmented areas further comprises: determining area identifiers of the segmented areas according to color values ​​of the colors filled in each segmented area by flood filling processing.

[0099] After the image to be processed is flood-filled to obtain multiple segmented areas, the region identifier of each segmented area can be determined according to the color value of the color filled in each segmented area, so as to obtain or search for the region image information of the segmented area according to the region identifier of the segmented area. For example, a dictionary can be created for the multiple segmented areas, the region identifier of each segmented area is used as the dictionary key, and the region image information corresponding to the segmented area is used as the dictionary value corresponding to the dictionary key.

[0100] For example, the color value of the color filled in a segmented area by the flood filling process may be, for example, the channel value (r, b, g) of the RBG channel of the segmented area in the RBG color space. Then, the region identifier key of the segmented area may be determined according to the color value (r, b, g) of the color filled in the segmented area, key=r+g*256+b*256 2 Since each segmented area is filled with a different color, the value of the area identifier of each segmented area is different. According to the area identifier of the segmented area as the dictionary key, the area image information of the segmented area as the dictionary value can be obtained or searched in the created dictionary.

[0101] In this embodiment, the region identifier of the segmented region is determined by determining the color value of the color filled in each segmented region according to the flood filling process, so that a region identifier corresponding to each segmented region can be obtained, thereby realizing numerical marking of each segmented region, facilitating subsequent acquisition or search of the region image information corresponding to the segmented region according to the region identifier of the segmented region, and helping to improve the efficiency of extracting the main color information.

[0102] In some embodiments, reference Figure 4 As shown, based on the regional image information of each segmented area, a region merging process is performed on multiple segmented areas, including:

[0103] S310: Convert the image to be processed from the first color space to the second color space to obtain a converted image.

[0104] In step S310, the image to be processed has different color channels in different color spaces, and the richness of the color and the difference between the colors of the image to be processed in the first color space are different from those in the second color space. For example, the richness of the color and the difference between the colors of the image to be processed in the first color space are higher than those in the second color space, so as to merge the segmented areas with similar colors as much as possible to avoid too many redundant segmented areas affecting the extraction of the main color. Exemplarily, the first color space may be, for example, an RGB color space, and the second color space may be, for example, a LAB color space.

[0105] S320: Acquire color information of each segmented area from the converted image.

[0106] In step S320, each segmented area has the same shape and size in the image to be processed and in the converted image. After obtaining the converted image, the color information of the segmented area in the converted image is obtained to further reduce the color difference between the segmented areas represented by the color information. Exemplarily, the color information of the segmented area can be, for example, the mean channel value of each channel of each pixel point in the segmented area in the second color space.

[0107] It can be understood that after obtaining the color information of each segmented area from the converted image, a dictionary can be created in the manner described above, and the color information of the segmented area can be recorded as part of the dictionary value corresponding to the area identifier of the segmented area, so as to facilitate finding the color information of the segmented area according to the area identifier of the segmented area.

[0108] S330 , based on the color information of each segmented area, merging the segmented areas whose color differences meet the preset merging condition to obtain a plurality of merged areas.

[0109] In step S330, the colors of the segmented areas represented by the color information have different color differences. According to the color information of the segmented areas, the segmented areas whose color differences meet the preset merging conditions can be merged to merge the segmented areas with smaller color differences. This makes the color difference between the merged areas larger, thereby ensuring the difference between the main colors of the multiple images finally extracted.

[0110] In this embodiment, by converting the image to be processed from the first color space to the second color space to obtain a converted image, and obtaining the color information of each segmented area from the converted image, it is possible to merge the segmented areas whose color differences meet the preset merging conditions according to the color information of each segmented area, thereby achieving the determination of the merged area. The region merging process performed in the above manner ensures the accuracy of merging the segmented areas with small color differences, so that there is a large color difference between the merged areas, and improves the extraction efficiency of the main color information and the accuracy of the extraction results.

[0111] In some embodiments, reference Figure 5 As shown, based on the color information of each segmented area, the segmented areas whose color differences meet the preset merging conditions are merged, including:

[0112] S331 , taking the segmented regions as candidate regions, and merging adjacent candidate regions that meet a preset merging condition into new candidate regions, until all candidate regions meet a preset merging stop condition.

[0113] S332: Determine each candidate region after merging stops as a merged region.

[0114] In step S331 to step S332, the merging process may be performed once or multiple times. Each segmented region is used as a candidate region for the first merging process, and adjacent candidate regions that meet the preset merging conditions are merged to complete the current merging process. The number of candidate regions merged at one time may be a positive integer greater than or equal to two. The merged region and the unmerged segmented regions are used as new candidate regions for the next merging process, and the merging process is repeated until each candidate region meets the preset merging stop condition. After the merging process is completed, each candidate region that meets the preset merging stop condition is used as the merged region obtained by the region merging process.

[0115] It should be noted that whether the candidate area meets the preset merging conditions and whether each candidate area meets the preset merging stop conditions need to be determined based on the color information of the candidate area. In the first merging process, the color information of the segmented area can be used as the color information of the candidate area to determine whether the candidate area meets the preset merging conditions. In the subsequent merging process, it is necessary to re-determine the color information of the new candidate area in the converted image (the color information of the segmented area that has not been merged can be determined without determining the color information, and the previously determined color information can be used), and determine whether the new candidate area meets the preset merging conditions and whether it meets the preset stop merging conditions based on the updated color information of the new candidate area.

[0116] The preset merging condition includes that the color difference value determined based on the color information of the candidate area is less than the first threshold value, and the preset merging stop condition includes that the color difference values ​​between the adjacent candidate areas are greater than or equal to the first threshold value. There is a color difference value between the two adjacent candidate areas, and the color difference value is determined based on the color information of the two adjacent candidate areas. The color difference value can represent the color difference between the two candidate areas, and the first threshold value can be, for example, an empirical value.

[0117] Exemplarily, the second color space is the LAB color space, and the color information of the candidate area in the converted image is the channel value mean (l, a, b) of the LAB channel of the candidate area in the LAB color space, where l is the channel value mean of each pixel in the candidate area in the L channel, a is the channel value mean of each pixel in the candidate area in the A channel, and b is the channel value mean of each pixel in the candidate area in the B channel. The color information of two adjacent candidate areas is (l1, a1, b1) and (l2, a2, b2) respectively, then the color difference dist between the two candidate areas can be determined according to the color information of the two candidate areas, and the color difference dist can be determined as the following formula:

[0118]

[0119] Among them, d l It can be determined by the following formula:

[0120]

[0121] d a It can be determined by the following formula:

[0122] d a =a1-a2

[0123] d b It can be determined by the following formula:

[0124] d b =b1-b2

[0125] The first threshold value may be 10, for example. If the color difference dist between adjacent candidate areas is less than 10, the candidate areas are merged, and the color information of the merged area in the converted image is re-determined, and then the preset merging conditions are judged and the merging process is performed until the color difference between each candidate area and each adjacent candidate area is greater than or equal to 10.

[0126] In this embodiment, by merging adjacent candidate areas that meet the preset merging conditions into new candidate areas until all candidate areas meet the preset merging stop conditions, and determining each candidate area after the merging stops as the merged area, the segmented areas whose color differences meet the preset merging conditions are merged, and a plurality of merged areas with large color differences are obtained. The color difference is determined according to the color information of the candidate area, and it is judged whether the preset merging condition and the preset merging stop condition are met according to the relationship between the color difference and the first threshold, which provides a basis for the merging process, ensures the color merging effect of the area merging process, and improves the extraction efficiency of the main color information and the accuracy of the extraction results.

[0127] It is understandable that the segmented areas obtained through image segmentation processing have certain color differences, and different segmented areas have different shapes and sizes. If segmented areas of various areas are all used as candidate areas, it may cause the segmented areas with very small areas to fail to meet the preset merging conditions and be determined as the final merged area alone. However, the small area of ​​the merged area determines that its corresponding color cannot be used as the main color of the image to be processed, which increases the meaningless calculation amount in the region merging process and affects the efficiency of extracting the main color information.

[0128] Based on this, in some embodiments, the regional image information also includes the number of pixels in the segmented area. Before merging the segmented areas whose color differences meet the preset merging conditions based on the color information of each segmented area, the regional merging processing of multiple segmented areas based on the regional image information of each segmented area also includes: removing the segmented areas whose number of pixels is less than the second threshold.

[0129] The regional image information of the segmented regions includes the number of pixels in the segmented regions in addition to the color information of the segmented regions. Before merging the segmented regions whose color differences meet the preset merging conditions, the number of pixels in each segmented region can be obtained while obtaining the color information of the segmented regions, and the segmented regions whose number of pixels is less than a second threshold are removed. The removed segmented regions are no longer used as candidate regions for subsequent region merging. The second threshold can be determined, for example, based on the total number of pixels in the image to be processed.

[0130] Exemplarily, for example, 0.5% of the total number of pixels in the image to be processed can be used as the second threshold, and the number of pixels in each segmented area is compared with the second threshold. If the number of pixels in a segmented area is less than the second threshold, the segmented area is removed and no longer participates in the subsequent area merging as a candidate area.

[0131] In this embodiment, the number of pixels in the segmented area is taken as part of the regional image information. By comparing the number of pixels in the segmented area with the second threshold, the segmented area with the number of pixels less than the second threshold can be removed, thereby removing the segmented areas with smaller areas in the segmented area, avoiding increasing the meaningless amount of calculation in the region merging process, and improving the extraction efficiency of the main color information and the accuracy of the extraction results.

[0132] In some embodiments, the first color space is an RBG color space, the second color space is a LAB color space, and color information of each segmented area is obtained from the converted image, including: obtaining the channel mean of each channel of each segmented area in the LAB color space, and using the channel mean as color information.

[0133] The first color space is the RBG color space. The image to be processed has three primary color channels in the RBG color space, namely, the R channel, the B channel, and the G channel. Each pixel unit has channel values ​​corresponding to the R channel, the B channel, and the G channel. The second color space is the LAB color space. The converted image has a brightness channel L channel and two color channels A channel and B channel in the LAB color space. Each pixel unit has channel values ​​corresponding to the L channel, the A channel, and the B channel. The channel mean of each channel in each segmented area in the LAB color space is obtained, that is, the channel mean of all pixel units in the segmented area corresponding to the L channel, the A channel, and the B channel is calculated respectively as the color information of the segmented area.

[0134] For example, the image to be processed is Figure 2 As shown, the image to be processed is in RBG color space. The image to be processed is converted from RBG color space to LAB color space to obtain a converted image. The converted image is as shown in Figure 6As shown. In the conversion image, the number of pixels in a segmented area is n, where n is a positive integer greater than 3, and the channel values ​​of the three channels of each pixel in the segmented area are (L1, A1, B1), (L2, A2, B2), ..., (Ln, An, Bn), respectively. Then, the channel mean l of the L channel = (L1+L2+...+Ln) / n, the channel mean a of the A channel = (A1+A2+...+An) / n, and the channel mean b of the B channel = (B1+B2+...+Bn) / n can be calculated, and the channel mean (l, a, b) is used as the color information of the segmented area. After determining the color information of each segmented area, the subsequent area merging can be performed according to the color information of the segmented area in the conversion image.

[0135] In this embodiment, the first color space is the RBG color space, and the second color space is the LAB color space. The channel mean of each channel of each segmented area in the LAB color space can be used as the color information of the segmented area in the converted image, thereby realizing the acquisition of color information. The richness of color and the difference between colors of the image to be processed in the first color space are higher than the richness of color and the difference between colors in the second color space. The color difference between each segmented area in the converted image is smaller than the color difference between each segmented area in the image to be processed, which reduces the color difference between each segmented area represented by the color information, so that the color difference between the merged areas obtained after the subsequent regional merging processing according to the color information is larger, thereby improving the extraction efficiency of the main color information and the accuracy of the extraction results.

[0136] In some embodiments, reference Figure 7 As shown, based on each merged area, determining the main color information of the image to be processed includes:

[0137] S410: Determine at least one main color area based on the number of pixels in each merged area.

[0138] In step S410, after completing the region merging process to obtain multiple merged regions, one or more primary color regions can be determined according to the number of pixels in each merged region. The number of primary color regions can be determined according to the number of primary colors to be extracted.

[0139] S420: Determine a channel mean value of each channel of at least one primary color region in an RBG color space as primary color information.

[0140] In step S420, the channel mean of each channel of the main color area in the RBG color space is used as the final main color information, so that the main color information can represent the color of the main color area through the RBG channel value and thus represent the color of the extracted main color.

[0141] For example, if it is necessary to extract four primary colors in the image to be processed, the merged areas can be sorted in descending order according to the number of pixels in each merged area, and the four merged areas with the first four pixel numbers are selected as the primary color areas. Then, the channel means corresponding to the three primary color channels R channel, B channel and G channel in the image to be processed are calculated respectively, and the RBG channel means of each primary color area are used as the primary color information to characterize the color of the primary color area, thereby realizing the extraction of the primary color.

[0142] In this embodiment, the primary color area is determined according to the number of pixels in each merged area, and the channel mean of each channel of each primary color area in the RBG color space is used as the primary color information. The determination of the primary color information is completed, so that the primary color information can represent the color of the primary color area with the largest area in the image to be processed, thereby realizing the primary color extraction of the image.

[0143] The original image is an image obtained by photographing or drawing, and has image noise. If the original image is directly segmented, the noise in the original image will interfere with the image segmentation process, resulting in inaccurate segmentation of the segmented area.

[0144] In some embodiments, obtaining the image to be processed includes: performing denoising processing on the original image to obtain the image to be processed.

[0145] In this embodiment, the original image is denoised to obtain the image to be processed, which can remove image noise in the original image to achieve noise filtering of the original image, thereby preventing the noise pixels in the original image from affecting the image segmentation process and the region merging process, causing deviations in the main color extraction results, thereby improving the extraction efficiency of the main color information and the accuracy of the extraction results.

[0146] In some embodiments, performing denoising processing on the original image to obtain the image to be processed includes: performing mean shift filtering processing on the original image to obtain the image to be processed.

[0147] Mean shift filtering is a clustering algorithm used to smooth and segment images. Its essence is to replace the original pixel value with the pixel value of the convergence point through iteration, so that some locally similar textures can be merged with each other, but the features with large differences such as edges can still be preserved. For example, the original image is Figure 8 As shown, we can arbitrarily select a pixel point in the original image, delineate a region with the pixel point as the center point, and find the centroid of the pixels in the region, that is, the pixel point with the maximum density, and then continue to perform the above iterative process with the pixel point as the center until it converges to obtain the following Figure 2 The image to be processed is shown.

[0148] In this embodiment, by performing mean shift filtering on the original image, smoothing filtering and denoising are performed on the original image at the color level, which can neutralize colors with similar colors and eliminate noise and noise in the original image, so as to achieve the effect of image color purification and preliminary color segmentation, further improving the extraction efficiency of the main color information and the accuracy of the extraction results.

[0149] It should be noted that before the original image is processed by the mean shift filter, the image size of the original image can be adjusted. For example, the pixels in the image can be merged by bilinear interpolation, and the width of the original image can be uniformly scaled to a preset width value, which can be 512, for example, and the height of the original image is scaled according to the initial aspect ratio of the original image. Scaling the original image before the mean shift filter can reduce the amount of calculation in the subsequent image segmentation and region merging processes to improve the efficiency of extracting the primary color information.

[0150] In some embodiments, reference Fig. 9 As shown, an image processing method is provided, and the image processing method includes:

[0151] S1, scaling the original image;

[0152] S2, performing mean shift filtering on the scaled original image to obtain an image to be processed;

[0153] S3, performing flood filling processing on the image to be processed to obtain multiple segmented areas;

[0154] S4, determining the region identification of the segmented region according to the color value of the color filled in each segmented region by the flood filling process;

[0155] S5, converting the image to be processed from the RBG color space to the LAB color space to obtain a converted image;

[0156] S6, obtaining the channel mean of each channel of each segmented area in the LAB color space, and using the channel mean as color information;

[0157] S7, removing the segmented areas where the number of pixels is less than the second threshold;

[0158] S8, taking the segmented regions as candidate regions, and merging adjacent candidate regions that meet preset merging conditions into new candidate regions, until all candidate regions meet preset merging stop conditions;

[0159] S9, determining each candidate region after merging stops as a merged region;

[0160] S10, determining at least one main color area based on the number of pixels in each merged area;

[0161] S11. Determine a channel mean value of each channel of at least one primary color region in an RBG color space as primary color information.

[0162] In this embodiment, by performing flood filling processing on the image to be processed and performing region merging processing on multiple segmented regions, the extraction of primary color information can be realized according to the obtained merged region. The flood filling processing on the image to be processed realizes image segmentation processing, and performs region merging processing according to the color information of the segmented regions, thereby ensuring the accuracy of the segmentation of image regions of different colors and the merging process of image regions of similar colors, and simplifying the region segmentation and region merging process, thereby improving the extraction efficiency of primary color information and the accuracy of the extraction result.

[0163] It should be noted that the reference Figures 10 to 12 As shown, Fig.10 In Fig.10 a is an original image, Fig.10 In Fig.10 b is the four main colors extracted from the original image using the image processing method of related technology, which takes 23.64 seconds. Fig.10 In Fig.10 c is the four main colors extracted from the original image using the above image processing method, which takes 5.95 seconds. Fig.11 In Fig.11 a is another original image, Fig.11 In Fig.11 b is the four main colors extracted from the original image using the image processing method of related technology, which takes 12.86 seconds. Fig.10 In Fig.10 c is the four main colors extracted from the original image using the above image processing method, which takes 5.69 seconds. Fig.12 In Fig.12 a is another original image, Fig.12 In Fig.12 b is the four main colors extracted from the original image using the image processing method of related technology, which takes 15.79 seconds. Fig.10 In Fig.10 c is the four main colors extracted from the original image using the above image processing method, which takes 4.80 seconds. From the above results, it can be seen that the use of the above image processing method for main color extraction greatly improves the efficiency of main color extraction, and the main color extraction result is closer to the real result.

[0164] In an exemplary embodiment, an image processing apparatus is provided, referring to Fig.13As shown, the image processing device includes an acquisition module 10, a segmentation module 20, a merging module 30 and a determination module 40. The acquisition module 10 is used to acquire the image to be processed. The segmentation module 20 is used to perform image segmentation processing on the image to be processed according to color to obtain multiple segmented regions. The merging module 30 is used to perform region merging processing on the multiple segmented regions based on the region image information of each segmented region to obtain multiple merged regions, wherein the region image information includes the color information of the segmented regions. The determination module 40 is used to determine the main color information of the image to be processed based on each merged region.

[0165] In this embodiment, the image to be processed is acquired by the acquisition module 10, the image to be processed is segmented by the segmentation module 20, and the multiple segmented regions are merged by the merging module 30, and the main color information can be extracted according to the merged region obtained by the determination module 40. The image segmentation is performed according to color, and the region merging is performed according to the image information of the segmented regions, which ensures the accuracy of the segmentation of image regions of different colors and the merging of image regions of similar colors, simplifies the region segmentation and region merging processes, and improves the extraction efficiency of the main color information and the accuracy of the extraction results.

[0166] In one embodiment, the acquisition module 10 is further used to: perform denoising on the original image to obtain an image to be processed.

[0167] In one embodiment, the acquisition module 10 is further used to: perform mean shift filtering on the original image to obtain an image to be processed.

[0168] In one embodiment, the segmentation module 20 is further used to perform flood filling processing on the image to be processed to obtain a plurality of segmented areas.

[0169] In one embodiment, the segmentation module 20 is further used to determine the region identification of the segmented region according to the color value of the color filled into each segmented region by the flood filling process.

[0170] In one embodiment, the merging module 30 is also used to: convert the image to be processed from the first color space to the second color space to obtain a converted image; obtain color information of each segmented area from the converted image; and based on the color information of each segmented area, merge the segmented areas whose color differences meet the preset merging conditions to obtain multiple merged areas.

[0171] In one embodiment, the merging module 30 is further configured to: use the segmented area as a candidate area, merge adjacent candidate areas that meet the preset merging conditions into new candidate areas, until all candidate areas meet the preset merging stop conditions; and determine each candidate area after the merging stops as the merged area. The preset merging conditions include that the color difference value determined based on the color information of the candidate area is less than a first threshold value, and the preset merging stop conditions include that the color difference value between each adjacent candidate area is greater than or equal to the first threshold value.

[0172] In one embodiment, the regional image information further includes the number of pixels in the segmented region, and the merging module 30 is further configured to remove the segmented region with the number of pixels being less than a second threshold.

[0173] In one embodiment, the first color space is an RBG color space, the second color space is a LAB color space, and the merging module 30 is further used to: obtain the channel mean of each channel of each segmented area in the LAB color space, and use the channel mean as color information.

[0174] In one embodiment, the determination module 40 is further used to: determine at least one primary color area based on the number of pixels in each merged area; determine the channel mean of each channel of at least one primary color area in the RBG color space as the primary color information.

[0175] In an exemplary embodiment, an electronic device is provided, referring to Fig.14 As shown, the electronic device may include one or more of the following components: a processing component 101 , a memory 102 , a power component 103 , a multimedia component 104 , an audio component 105 , an input / output (I / O) interface 106 , a sensor component 107 , and a communication component 108 .

[0176] The processing component 101 generally controls the overall operation of the electronic device, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 101 may include one or more processors 109 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 101 may include one or more modules to facilitate the interaction between the processing component 101 and other components. For example, the processing component 101 may include a multimedia module to facilitate the interaction between the multimedia component 104 and the processing component 101.

[0177] The memory 102 is configured to store various types of data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc. The memory 102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0178] The power component 103 provides power to various components of the electronic device. The power component 103 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0179] The multimedia component 104 includes a screen that provides an output interface between the electronic device and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 104 includes a front camera and / or a rear camera. When the electronic device is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0180] The audio component 105 is configured to output and / or input audio signals. For example, the audio component 105 includes a microphone (MIC), and when the electronic device is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 102 or sent via the communication component 108. In some embodiments, the audio component 105 also includes a speaker for outputting audio signals.

[0181] I / O interface 106 provides an interface between processing component 101 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0182] The sensor assembly 107 includes one or more sensors for providing various aspects of status assessment for the electronic device. For example, the sensor assembly 107 can detect the on / off state of the electronic device, the relative positioning of components, such as the display and keypad of the electronic device, and the sensor assembly 107 can also detect the position change of the electronic device or a component of the electronic device, the presence or absence of user contact with the electronic device, the orientation or acceleration / deceleration of the electronic device and the temperature change of the electronic device. The sensor assembly 107 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 107 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 107 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0183] The communication component 108 is configured to facilitate wired or wireless communication between the electronic device and other devices. The device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 108 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 108 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0184] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned image processing method applied to the electronic device.

[0185] In an exemplary embodiment, a non-temporary computer-readable storage medium including instructions is also provided, such as a memory 102 including instructions, and the above instructions can be executed by a processor 109 of an electronic device to complete the above-mentioned image processing method applied to the electronic device. For example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, and an optical data storage device. When the instructions in the storage medium are executed by the processor 109 of the electronic device, the electronic device is enabled to perform the image processing method shown in the above embodiment.

[0186] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0187] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: The image processing method comprises: Get the image to be processed; Performing image segmentation processing on the image to be processed according to color to obtain multiple segmented areas; Based on the regional image information of each of the segmented regions, performing region merging processing on the multiple segmented regions to obtain multiple merged regions, wherein the regional image information includes color information of the segmented regions; Based on each of the merged areas, main color information of the image to be processed is determined.

2. The image processing method according to claim 1, characterized in that: The image to be processed is segmented according to color to obtain a plurality of segmented areas, including: Performing flood filling processing on the image to be processed to obtain the multiple segmented areas.

3. The image processing method according to claim 2, characterized in that: The performing image segmentation processing on the image to be processed according to color to obtain a plurality of segmented areas further comprises: The region identifier of the segmented region is determined according to the color value of the color filled in each segmented region by the flood filling process.

4. The image processing method according to claim 1, characterized in that: The performing region merging processing on the plurality of segmented regions based on the region image information of each segmented region comprises: Converting the image to be processed from the first color space to the second color space to obtain a converted image; Acquire the color information of each of the segmented areas from the converted image; Based on the color information of each of the segmented regions, the segmented regions whose color differences meet a preset merging condition are merged to obtain the multiple merged regions.

5. The image processing method according to claim 4, characterized in that: The merging of the segmented regions whose color differences meet a preset merging condition based on the color information of each segmented region includes: The segmented regions are used as candidate regions, and adjacent candidate regions that meet the preset merging condition are merged into new candidate regions until all candidate regions meet the preset merging stop condition; Determine each of the candidate regions after merging stops as the merging region; The preset merging condition includes that a color difference value determined based on color information of the candidate region is less than a first threshold value, and the preset merging stop condition includes that the color difference values ​​between the adjacent candidate regions are greater than or equal to the first threshold value.

6. The image processing method according to claim 4, characterized in that: The regional image information also includes the number of pixels of the segmented regions. Before merging the segmented regions whose color differences meet the preset merging conditions based on the color information of the segmented regions, the regional merging process of the plurality of segmented regions based on the regional image information of the segmented regions further includes: The segmented area with the number of pixels less than the second threshold is removed.

7. The image processing method according to any one of claims 4 to 6, characterized in that: The first color space is an RBG color space, the second color space is a LAB color space, and the acquiring the color information of each segmented area from the converted image includes: The channel mean of each channel of each segmented area in the LAB color space is obtained, and the channel mean is used as the color information.

8. The image processing method according to claim 7, characterized in that: Determining the main color information of the image to be processed based on each of the merged areas includes: Determine at least one main color area based on the number of pixels in each of the merged areas; Determine a channel mean of each channel of the at least one primary color region in the RBG color space as the primary color information.

9. The image processing method according to any one of claims 1 to 6, characterized in that: The step of obtaining the image to be processed comprises: The original image is subjected to denoising processing to obtain the image to be processed.

10. The image processing method according to claim 9, characterized in that: The denoising process is performed on the original image to obtain the image to be processed, including: The original image is processed by a mean shift filter to obtain the image to be processed.

11. An image processing device, characterized in that: The image processing device comprises: An acquisition module, wherein the acquisition module is used to acquire an image to be processed; A segmentation module, the segmentation module is used to perform image segmentation processing on the image to be processed according to color to obtain multiple segmentation areas; A merging module, the merging module is used to perform region merging processing on the multiple segmented regions based on the region image information of each of the segmented regions to obtain multiple merged regions, wherein the region image information includes color information of the segmented regions; A determination module is used to determine the main color information of the image to be processed based on each of the merged areas.

12. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Get the image to be processed; Performing image segmentation processing on the image to be processed according to color to obtain multiple segmented areas; Based on the regional image information of each of the segmented regions, performing region merging processing on the multiple segmented regions to obtain multiple merged regions, wherein the regional image information includes color information of the segmented regions; Based on each of the merged areas, main color information of the image to be processed is determined.

13. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform an image processing method, the image processing method comprising: Get the image to be processed; Performing image segmentation processing on the image to be processed according to color to obtain multiple segmented areas; Based on the regional image information of each of the segmented regions, performing region merging processing on the multiple segmented regions to obtain multiple merged regions, wherein the regional image information includes color information of the segmented regions; Based on each of the merged areas, main color information of the image to be processed is determined.

Images