Image processing method, device, apparatus, and computer-readable storage medium

Abstract

The present invention discloses an image processing method, apparatus, device and computer-readable storage medium, wherein the method comprises: obtaining an initial segmentation mask image of a preset shooting scene in an image to be processed; performing region growth processing on the initial segmentation mask image to obtain a grown segmentation mask image; determining multiple target connected domains in a difference binary image corresponding to both the initial segmentation mask image and the grown segmentation mask image; when the maximum area of ​​the multiple target connected domains meets a preset condition, generating a target segmentation mask image according to the initial segmentation mask image; processing the image to be processed according to the target segmentation mask image, wherein the target segmentation mask image clearly displays the details at the boundary between different regions in the image, and is easy to process more finely. The image processing method provided by the present invention improves the accuracy of image processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing method, apparatus, device and computer-readable storage medium. Background Art

[0002] Currently, the best performing image processing technology is based on deep learning image processing technology, but the boundary segmentation between different areas in the image is not very precise, which means that the accuracy of image processing needs to be improved. Summary of the Invention

[0003] The main purpose of the present invention is to provide an image processing method, device, equipment and computer-readable storage medium, aiming to solve the problem of low image segmentation accuracy.

[0004] The present invention provides an image processing method, comprising:

[0005] Obtaining an initial segmentation mask image of a preset shooting scene in the image to be processed;

[0006] Perform region growing processing on the initial segmentation mask image to obtain a grown segmentation mask image;

[0007] Determine a plurality of target connected regions in a difference binary image corresponding to both the initial segmentation mask image and the grown segmentation mask image;

[0008] When the maximum area of ​​multiple target connected domains meets the preset conditions, a target segmentation mask image is generated according to the initial segmentation mask image;

[0009] Process the image to be processed according to the target segmentation mask image.

[0010] In addition, to achieve the above-mentioned object, the present invention further provides an image processing device, comprising:

[0011] An acquisition module is used to acquire an initial segmentation mask image of a preset shooting scene in the image to be processed;

[0012] A region growing processing module is used to perform region growing processing on the initial segmentation mask image to obtain a grown segmentation mask image;

[0013] A determination module, configured to determine a plurality of target connected domains in a difference binary image corresponding to both the initial segmentation mask image and the growth segmentation mask image;

[0014] A generation module is used to generate a target segmentation mask image according to the initial segmentation mask image when the maximum area of ​​multiple target connected domains meets a preset condition;

[0015] The processing module is used to process the image to be processed according to the target segmentation mask image.

[0016] In addition, to achieve the above-mentioned purpose, the present invention also provides an image processing device, which includes a memory, a processor, and an image processing program stored in the memory and runnable on the processor. When the processor executes the image processing program, the steps of the image processing method as described above are implemented.

[0017] In addition, to achieve the above-mentioned purpose, the present invention further provides a computer-readable storage medium, on which an image processing program is stored. When the image processing program is executed by a processor, the steps of the above-mentioned image processing method are implemented.

[0018] The present invention obtains an initial segmentation mask image of a preset shooting scene in an image to be processed, then performs region growing processing on the initial segmentation mask image, determines a target connected domain in a difference binary map corresponding to both the initial segmentation mask image and the grown segmentation mask image obtained by the region growing processing, generates a target segmentation image based on the initial segmentation mask image when the maximum area of ​​multiple target connected domains meets a preset condition, and finally processes the image to be processed by processing the target segmentation mask image. The target segmentation mask image clearly shows the details at the boundary between different areas in the image, and is easy to process more finely. The image processing method provided by the present invention improves the accuracy of image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of the hardware structure of a device for implementing an embodiment of the present invention;

[0020] Figure 2 Schematic diagram of the process of an image processing method according to an embodiment of the present invention;

[0021] Figure 3 is the initial segmentation mask map of the present invention;

[0022] Figure 4 The segmentation mask image of the present invention is increased;

[0023] Figure 5 It is the cluster segmentation mask map of the present invention;

[0024] Figure 6 The statistical segmentation mask map of the present invention;

[0025] Figure 7 The target segmentation mask map of the present invention;

[0026] Figure 8 Increase the fine segmentation mask map for the present invention;

[0027] Figure 9 This is a schematic diagram of the functional modules of an image processing device provided by the present invention.

[0028] The purpose, features and advantages of the present invention will be described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0029] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0030] The present invention provides an image processing device, referring to Figure 1 , Figure 1 It is a schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present invention.

[0031] It should be noted that Figure 1 The image processing device of the embodiment of the present invention can be a PC (Personal Computer), a portable computer, a server or other equipment.

[0032] like Figure 1 As shown, the image processing device may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally also be a storage device independent of the aforementioned processor 1001.

[0033] Optionally, the image processing device may further include an RF (Radio Frequency) circuit, a sensor, a WiFi module, and the like.

[0034] Those skilled in the art will understand that Figure 1 The structure of the image processing device shown in the figure does not constitute a limitation of the image processing device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0035] like Figure 1As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and an image processing program. The operating system is a program that manages and controls the hardware and software resources of the image processing device and supports the operation of the image processing program and other software or programs.

[0036] Figure 1 The image processing method and device shown can be used to solve the problem of low image segmentation accuracy. The user interface 1003 is mainly used to detect or output various information, such as inputting an initial segmentation mask image and outputting a refined segmentation mask image. The network interface 1004 is mainly used to interact and communicate with the backend server. The processor 1001 can be used to call the image processing program stored in the memory 1005 and perform the following operations:

[0037] Obtaining an initial segmentation mask image of a preset shooting scene in the image to be processed;

[0038] Perform region growing processing on the initial segmentation mask image to obtain a grown segmentation mask image;

[0039] Determine a plurality of target connected regions in a difference binary image corresponding to both the initial segmentation mask image and the grown segmentation mask image;

[0040] When the maximum area of ​​multiple target connected domains meets the preset conditions, a target segmentation mask image is generated according to the initial segmentation mask image;

[0041] The image to be processed is processed according to the target segmentation mask image.

[0042] The present invention obtains an initial segmentation mask image of a preset shooting scene in an image to be processed, then performs region growing processing on the initial segmentation mask image, determines a target connected domain in a difference binary map corresponding to both the initial segmentation mask image and the grown segmentation mask image obtained by the region growing processing, generates a target segmentation image based on the initial segmentation mask image when the maximum area of ​​multiple target connected domains meets a preset condition, and finally processes the image to be processed by processing the target segmentation mask image. The target segmentation mask image clearly shows the details at the boundary between different areas in the image, and is easy to process more finely. The image processing method provided by the present invention improves the accuracy of image processing.

[0043] The specific implementation of the mobile terminal of the present invention is basically the same as the embodiments of the following image processing method, and will not be repeated here.

[0044] Based on the above structure, various embodiments of the image processing method of the present invention are proposed.

[0045] The present invention provides an image processing method.

[0046] Reference Figure 2 , Figure 2 Schematic diagram of the image processing method according to an embodiment of the present invention.

[0047] In this embodiment, an embodiment of an image processing method is provided. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown here.

[0048] In this embodiment, the image processing method includes:

[0049] Step S10, obtaining an initial segmentation mask image of a preset shooting scene in the image to be processed;

[0050] The image processing method provided in this embodiment is used to segment an area representing a specific object from a captured image, such as segmenting a sky area from a captured image. The image to be processed can be any frame of the image captured by the camera, and the preset shooting scene is a scene including a specific object in the image to be processed. The mask image is an image containing only two kinds of pixels. The pixel values ​​of these two kinds of pixels are represented by RGB color values. The pixel values ​​are represented by grayscale levels. There are 256 grayscale levels in total. Different grayscale levels represent different brightness. The pixel value can be any integer in the closed interval of [0, 256]. One kind of pixel in the mask image is a pixel with an RGB pixel value of 0, and the other kind of pixel is a pixel with an RGB pixel value of 225. The mask image observed by the human eye is a black and white image. The area of ​​the specific object to be segmented is displayed in only one color in the initial segmented image. Figure 3 , Figure 3 The preset shooting scene represented by is a sky scene, which is the initial segmentation image for segmenting the sky area. Figure 3 The white area is composed of pixels with a value of 255, i.e., the sky area, and the black area is composed of pixels with a value of 0, i.e., the surface area, such as buildings and trees. The initial segmentation mask image is obtained by preprocessing the preset shooting scene in the image to be processed.

[0051] In one embodiment, step S10 includes:

[0052] Step a, using a high-resolution network to process the preset shooting scene in the image to be processed to obtain the original segmentation mask image;

[0053] Step b: transferring the original segmentation mask image to a preset color space for display to obtain an initial segmentation mask image.

[0054] This embodiment uses the HRNet (High Resolution Net) method to roughly segment the preset shooting scene in the processed image. The resulting mask image is also a raw segmentation mask image. The raw segmentation mask image is displayed in RGB color mode. The raw segmentation mask image is converted to a preset color mode for display, that is, transferred to a preset color space for display, to obtain an initial segmentation mask image. The preset color space can be a Lab color space (the preset color space will be described as a Lab color space below). Research has found that image processing on a mask image displayed in a preset color mode can achieve better segmentation results than a mask image displayed in RGB color mode.

[0055] Step S20, performing region growing processing on the initial segmentation mask image to obtain a grown segmentation mask image;

[0056] Region growing is the process of developing groups of pixels or regions into larger regions. The initial segmentation mask image is a coarse segmentation image. By grouping the pixels in the initial segmentation mask image and combining the pixels in the same group into a region, objects in the scene image can be re-segmented. It should be noted that the region growing process provided in this embodiment is performed on the initial segmentation mask image displayed in Lab color mode. The initial segmentation mask image after region growing is called the grown segmentation mask image.

[0057] Specifically, step S20 includes:

[0058] Step c, extracting a seed point from all pixels of the initial segmentation mask image, and determining a first similarity between the seed point and its adjacent pixels, wherein the seed point is a pixel point selected from the initial segmentation mask image according to a preset rule;

[0059] Step d, determining the pre-increase pixel points according to the first similarity;

[0060] The preset rule can be to evenly set seed points in the initial segmentation mask image. It can be understood that the seed points are pixel points. For example, a seed point is determined every 5 pixels. The pixel value of the seed point represented by the Lab color value and the pixel value of the pixel points around the seed point (i.e., the adjacent pixel points) represented by the Lab color value are substituted into the first formula to calculate the similarity between the seed point and the adjacent pixel points in terms of pixel value, i.e., the first similarity. Then, based on the first pixel point, it is determined whether the adjacent pixel points and the seed point are grouped together, i.e., whether the adjacent pixel points are determined as pre-growth pixel points. The first formula is d s Represents the first similarity, (i, j) is the position coordinate of the seed point in the initial segmentation mask image, (i_n, j_n) is the position coordinate of the adjacent pixel point in the initial segmentation mask image, L(i,j) 、a (i,j) 、b (i,j) are the pixel values ​​of the seed points, L (i_n,j_n) 、a (i_n,j_n) 、b (i_n,j_n) is the pixel value of the adjacent pixel.

[0061] Specifically, step d also includes:

[0062] Step d1: if the first similarity is greater than or equal to a first preset similarity, then determining the adjacent pixel points of the seed point as pre-growth pixel points;

[0063] The first preset similarity is determined based on experimental research. Generally, the first preset similarity is set to ensure a good segmentation effect. The first preset similarity can be set to a value of 5. When the first similarity is greater than or equal to the first preset similarity, it indicates that the adjacent pixel point is very similar to the seed point. To a large extent, the seed point and the adjacent pixel point belong to the area covered by the same object in the scene image, so the adjacent pixel point is determined as a pre-increment pixel point. When the first similarity is less than the first preset similarity, it indicates that the adjacent pixel point is less similar to the seed point. To a large extent, the seed point and the adjacent pixel point do not belong to the area covered by the same object in the scene image, so the adjacent pixel point is not a pre-increment pixel point.

[0064] Step e: If the growth error is less than or equal to the preset error, the target area to which the seed point belongs is covered with the preset growth pixel points to obtain a growth segmentation mask image.

[0065] When the adjacent pixel point is a pre-growth pixel point, in order to ensure the error growth in the region growing process, that is, to further judge whether the adjacent pixel point and the seed point belong to the area covered by the same object in the scene image, the pixel value a represented by the preset color value of the pre-growth pixel point is (i_n,j_n) and b (i_n,j_n) , the average value of all pixel values ​​a of the pre-growth pixel points and the average value of all pixel values ​​b of the pre-growth pixel points are substituted into the second formula to obtain the growth error of all pre-growth pixel points. The second formula is d t =(m a -a (i_n,j_n) )+(m b -b (i_n,j_n) ).

[0066] The preset error is determined by researchers based on experimental research. Generally, the preset error is set to ensure a good segmentation effect. The preset error can be set to 40. When the growth error is less than or equal to the preset error, it means that the preset growth point belongs to the region where the seed point belongs in the scene image to a large extent, that is, the target region. The preset growth point is placed in the region where the seed point is located in the initial segmentation mask image to obtain the grown segmentation mask image.

[0067] by Figure 3 For example, Figure 3 The boundary between the white and black areas is very fuzzy and does not show details, such as the treetops. Figure 3 The seed points are evenly set in the image, for example, the seed points are set to a 10*8 arrangement, the first similarity between each seed point and its adjacent pixel points is calculated, the adjacent pixel points corresponding to the first similarity greater than or equal to the first preset similarity are determined as pre-growth pixel points, and the growth error of the pre-growth pixel points is calculated, and the pre-growth pixel points whose growth error is less than or equal to the preset error are included in the target area to which the seed point belongs. If the seed point is in the white area, the pre-growth pixel points corresponding to the seed point are included in the white area, so as to realize the regional growth of the seed point in the initial segmentation mask image and obtain the grown segmentation mask image. Figure 4 .

[0068] Step S30, determining a plurality of target connected regions in the difference binary image corresponding to both the initial segmentation mask image and the grown segmentation mask image;

[0069] A binary image refers to an image with only two pixel values. In this embodiment, a difference binary image refers to an image related to the difference in pixel values ​​between the initial segmentation mask image and the growth segmentation mask image. A connected domain is a continuous area composed of pixel points with the same pixel values ​​in the difference binary image. A target connected domain is a continuous area composed of pixel points with the same pixel values ​​and selected by researchers. Researchers can select the area corresponding to the pixel value as the target connected domain by inputting the pixel value. It can be understood that a difference binary image can contain multiple target connected domains.

[0070] Specifically, step S30 includes:

[0071] Step f, determining the pixel value difference between each group of corresponding pixels in the initial segmentation mask image and the grown segmentation mask image;

[0072] Step g, drawing a difference binary map according to the absolute value of the pixel value difference, where the absolute value is the pixel value of the pixel point in the difference binary map;

[0073] Step h, determining a continuous region composed of pixels with the same pixel value in the difference binary image as a connected region;

[0074] In step i, a connected domain in each preset pixel value region is determined as a target connected domain.

[0075] The pixel values ​​in the initial segmentation mask image and the grown segmentation mask image are represented by RGB color values. The corresponding points in the initial segmentation mask image and the grown segmentation mask image are the set of pixels whose positions in the initial segmentation mask image and the grown segmentation mask image are the same. First, the difference in the pixel values ​​of each set of corresponding pixels is calculated, and the absolute value of the difference is used as the pixel value to generate a difference binary image. If the pixel values ​​in the initial segmentation mask image and the grown segmentation mask image are only 0 and 255, then the absolute value of the difference is only 0 and 255. If the pixel values ​​of a set of corresponding pixels are the same, then the pixel with the pixel value calculated from the pixel values ​​of the set of corresponding pixels is displayed black in the difference binary image; if the pixel values ​​of a set of corresponding pixels are different, then the pixel with the pixel value calculated from the pixel values ​​of the set of corresponding pixels is displayed white in the difference binary image.

[0076] The difference binary image is a black and white image. The continuous area composed of pixels with the same pixel value is determined as a connected domain. It can be understood that the difference binary image is an image composed of black and white connected domains. The preset pixel value is set by the researchers. The area where the pixel value of the pixel point in the difference binary image is equal to the preset pixel value is determined as the preset pixel value area. The connected domain in the preset pixel value area is the target connected domain. It can be understood that the preset pixel value area is the sum of multiple target connected domains. Step S40, when the maximum value among the areas of multiple target connected domains meets the preset conditions, the initial segmentation mask image is segmented according to the preset statistical rules to obtain the target segmentation mask image, and the target segmentation mask image is used for image processing.

[0077] The area of ​​a target connected domain can be represented by the number of pixels in the target connected domain. The number of pixels in each target connected domain is counted and the number of pixels in the target connected domain is used as the area of ​​the target connected domain.

[0078] Compare the areas of the target connected domains one by one, obtain the maximum value of the area of ​​the target connected domain, and judge whether the maximum value meets the preset conditions. If it does, it means that the growth segmentation mask image is an unreasonable growth result, so the initial segmentation mask image is segmented according to the preset statistical rules to obtain the target segmentation mask image for image processing.

[0079] In one embodiment, before step S40, the method further includes:

[0080] Step j: when the ratio of the maximum area of ​​the plurality of target connected regions to the area of ​​the corresponding preset pixel value region is greater than or equal to a preset ratio, it is determined that the maximum area of ​​the plurality of target connected regions meets a preset condition.

[0081] The area of ​​the preset pixel value region is equal to the sum of the areas of all target connected domains. The preset ratio is input by researchers based on experimental research to ensure that the value of the preset ratio can achieve a good segmentation effect. The preset ratio can be 0.05. When the ratio of the maximum area of ​​the target connected domain to the area of ​​the corresponding preset pixel value region is greater than or equal to the preset ratio, it means that the area of ​​the largest target connected domain meets the preset conditions; when the ratio of the area of ​​the largest target connected domain to the area of ​​the preset pixel value region is less than the preset ratio, it means that the area of ​​the largest target connected domain does not meet the preset conditions.

[0082] In some embodiments, step S40 includes:

[0083] Step k, clustering the pixels in the initial segmentation mask image using the K-means clustering algorithm to obtain a clustered segmentation image;

[0084] Step 1, calculating the second similarity between the pixel points in the cluster segmentation image and each category according to a pre-stored third formula;

[0085] Step m, performing maximum inter-class variance adaptive threshold segmentation on the initial segmentation mask image according to the maximum second similarity corresponding to each pixel in the cluster segmentation image to obtain a statistical segmentation mask image;

[0086] Step n: perform an OR operation on the statistical segmentation mask image and the initial segmentation mask image to obtain a target segmentation mask image.

[0087] When the maximum area of ​​multiple target connected domains meets the preset conditions, the growth segmentation mask image is discarded, and the initial segmentation mask image is clustered using the K-means clustering algorithm. The initial segmentation mask image is divided into k categories to obtain a clustered segmentation mask image. Figure 5 That is Figure 3 The cluster segmentation mask image is generated. The parameter k is determined by researchers based on actual conditions. For example, k = 3. The third formula is used to calculate the similarity between each pixel in the initial segmentation mask image and each category, that is, the second similarity. Each pixel has k second similarities. The third formula is:

[0088] X represents the feature vector, a and b are the pixel values ​​of the pixels in the cluster segmentation image, d represents the feature dimension, u represents the mean of the pixel values, and Σ represents the covariance matrix of the pixel values.

[0089] According to the second largest similarity of each pixel, the cluster segmentation mask image is segmented by the maximum inter-class variance method adaptive threshold to obtain a statistical segmentation mask image. Figure 6, then perform an OR operation on the statistical segmentation mask image and the initial segmentation mask image. The OR operation process is to merge a group of pixels corresponding to the positions of the statistical segmentation mask image and the initial segmentation mask image. The pixel value of the merged pixel is the sum of the pixel values ​​of the previous group of pixels. The image obtained by performing an OR operation on the statistical segmentation mask image and the initial segmentation mask image is the target segmentation mask image. Figure 7 , it can be observed Figure 7 Shows the gaps between the trees.

[0090] Step S50: Processing the image to be processed according to the target segmentation mask image.

[0091] The processing of the image to be processed is achieved by processing the target segmentation mask image. The processing of the target segmentation mask image can be a segmentation process,

[0092] In some embodiments, when the preset shooting scene is a sky scene, a target segmentation mask image including the sky scene is acquired; and a sky area of ​​the image to be processed is determined according to the target segmentation mask image.

[0093] Reference Figures 3 to 8 , is the process of processing the image to be processed with the preset shooting scene being the sky scene. The sky area in the target segmentation mask image is the sky area in the image to be processed. Figure 7 The details in the image are clearly displayed, so the sky segmentation effect is greatly improved.

[0094] It should be noted that although this embodiment only takes the segmentation of the sky area as an example, it does not limit the scope of application of the image processing method provided by this embodiment. The image processing method provided by this embodiment can also be used to segment images of other scenes, such as surface buildings, oceans, etc.

[0095] This embodiment obtains an initial segmentation mask image of a preset shooting scene in the image to be processed, then performs region growing processing on the initial segmentation mask image, determines the target connected domain in the difference binary map corresponding to both the initial segmentation mask image and the grown segmentation mask image obtained by the region growing processing, and generates a target segmentation mask image based on the initial segmentation mask image when the maximum area of ​​multiple target connected domains meets the preset conditions. The target segmentation mask image is used to process the image to be processed. Specifically, the initial segmentation mask image is clustered by the K-means clustering algorithm in sequence, and then the maximum inter-class variance method is used for adaptive threshold segmentation to obtain a statistical segmentation mask image. Finally, the statistical segmentation mask image and the initial segmentation mask image are ORed to obtain the target segmentation mask image. The target segmentation mask image clearly shows the details at the boundary between different areas in the image, and is easy to segment more finely. The image processing method provided by the present invention improves the accuracy of image processing.

[0096] Furthermore, a second embodiment of the image processing method of the present invention is proposed. The difference between the second embodiment of the image processing method and the first embodiment of the image processing method is that the image processing method further includes:

[0097] Step o: when the maximum area of ​​multiple target connected domains does not meet the preset conditions, perform an OR operation on the initial segmentation mask image and the growth segmentation mask image to obtain a growth fine segmentation mask image;

[0098] Step p: Segment the growing fine segmentation mask image according to preset statistical rules to obtain a target segmentation mask image.

[0099] This embodiment proposes an image processing method when the maximum area of ​​multiple target connected domains does not meet the preset conditions. When the area of ​​the largest target connected domain does not meet the preset conditions, it means that the growth segmentation mask image is the result of reasonable growth, and the initial segmentation mask image and the growth segmentation mask image are subjected to an OR operation. A group of pixel points corresponding to the positions in the initial segmentation mask image and the growth segmentation mask image are merged, and the pixel values ​​of the pixel points obtained after the merger are the sum of the pixel values ​​of the previous group of pixel points, that is, an OR operation is performed on the two images to obtain a new image, namely, the growth fine segmentation mask image, see Figure 8 .

[0100] The growing fine segmentation mask image is segmented according to preset statistical rules. The segmentation process is the same as that of Example 1, except that the segmentation object is different. Example 1 segments the initial segmentation mask image, while this embodiment segments the growing fine segmentation mask image. Specifically, when the area of ​​the largest target connected domain does not meet the preset conditions, the pixels in the growing fine segmentation mask image are clustered using the K-means clustering algorithm to obtain a cluster segmentation image; the second similarity between the pixels in the cluster segmentation image and each category is calculated according to the third formula; the growing fine segmentation mask image is segmented using the maximum inter-class variance method based on the largest second similarity of each pixel in the cluster segmentation image to obtain a statistical segmentation mask image; the statistical segmentation mask image and the growing fine segmentation mask image are ORed to obtain a target segmentation mask image.

[0101] In this embodiment, when the area of ​​the largest target connected domain does not meet the preset conditions, an OR operation is performed on the initial segmentation mask image and the growing segmentation mask image to obtain a growing fine segmentation mask image, and then the growing fine segmentation mask image is segmented according to preset statistical rules to obtain a target segmentation mask image, which greatly improves the image segmentation accuracy and demonstrates a good segmentation effect.

[0102] In addition, an embodiment of the present invention further provides an image processing method and apparatus, including:

[0103] An acquisition module 910 is configured to acquire an initial segmentation mask image of a preset shooting scene in an image to be processed;

[0104] A region growing processing module 920 is configured to perform region growing processing on the initial segmentation mask image to obtain a grown segmentation mask image;

[0105] A determination module 930 is configured to determine a plurality of target connected components in a difference binary image corresponding to both the initial segmentation mask image and the grown segmentation mask image;

[0106] A generating module 940 is configured to generate a target segmentation mask image according to the initial segmentation mask image when the maximum area of ​​the plurality of target connected components meets a preset condition;

[0107] The processing module 950 is configured to process the image to be processed according to the target segmentation mask image.

[0108] The implementation of the image processing method and apparatus of the present invention is basically the same as the above-mentioned embodiments of the image processing method, and will not be described in detail here.

[0109] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which an image processing program is stored. When the image processing program is executed by a processor, each step of the above image processing method is implemented.

[0110] It should be noted that the computer-readable storage medium may be provided in the image processing device.

[0111] The specific implementation of the computer-readable storage medium of the present invention is basically the same as the above-mentioned image processing embodiments, and will not be repeated here.

[0112] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0113] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0114] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0115] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Obtaining an initial segmentation mask image of a preset shooting scene in the image to be processed; Performing region growing processing on the initial segmentation mask image to obtain a grown segmentation mask image; Determining a plurality of target connected regions in a difference binary image corresponding to both the initial segmentation mask image and the grown segmentation mask image; When the maximum area of ​​the plurality of target connected domains meets a preset condition, generating a target segmentation mask image according to the initial segmentation mask image; Processing the image to be processed according to the target segmentation mask image; Generating a target segmentation mask image according to the initial segmentation mask image includes: Performing clustering on the pixels in the initial segmentation mask image using a K-means clustering algorithm to obtain a clustered segmentation image; Calculating a second similarity between a pixel point in the cluster segmentation image and each pre-stored category according to a pre-stored third formula; performing maximum inter-class variance method adaptive threshold segmentation on the initial segmentation mask image according to the maximum second similarity corresponding to each pixel point in the cluster segmentation image to obtain a statistical segmentation mask image; An OR operation is performed on the statistical segmentation mask image and the initial segmentation mask image to obtain a target segmentation mask image.

2. The method according to claim 1, wherein The performing region growing processing on the initial segmentation mask image to obtain a grown segmentation mask image includes: Extracting a seed point from all pixels of the initial segmentation mask image, and determining a first similarity between the seed point and pixels adjacent to the seed point, wherein the seed point is a pixel point selected from the initial segmentation mask image according to a preset rule; Determining pre-increment pixel points according to the first similarity; If the growth error of the pre-growth pixel point is less than or equal to the preset error, the pre-growth pixel point is covered with the target area to which the seed point belongs to obtain a growth segmentation mask image.

3. The method according to claim 2, wherein A first similarity between the seed point and its adjacent pixels is determined according to a first formula, wherein the first formula is: , represents the first similarity, is the position coordinate of the seed point in the initial segmentation mask image, are the position coordinates of the adjacent pixel points in the initial segmentation mask image, 、 、 are the pixel values ​​of the seed points, 、 、 is the pixel value of the adjacent pixel point; The growth error is determined according to a second formula, which is: , represents the growth error, Indicates the pixel value of the pre-incremented pixel The average value of Indicates the pixel value of the pre-incremented pixel The average value of .

4. The method according to claim 2, wherein The determining of the pre-increment pixel points according to the first similarity includes: If the first similarity is greater than or equal to a first preset similarity, the adjacent pixel points of the seed point are determined as pre-growth pixel points.

5. The method according to any one of claims 1 to 4, characterized in that The determining of a plurality of target connected regions in the difference binary image corresponding to both the initial segmentation mask image and the grown segmentation mask image comprises: Determining a pixel value difference between each group of corresponding pixels in the initial segmentation mask image and the grown segmentation mask image; Draw a difference binary map according to the absolute value of the pixel value difference, wherein the absolute value is the pixel value of the pixel point in the difference binary map; Determine a continuous region composed of pixels with the same pixel value in the difference binary image as a connected region; The connected domain in each preset pixel value region is determined as the target connected domain.

6. The method according to claim 5, wherein Before the step of generating a target segmentation mask image according to the initial segmentation mask image, the method further comprises: When the ratio of the maximum area of ​​the plurality of target connected domains to the area of ​​the corresponding preset pixel value region is greater than or equal to a preset ratio, it is determined that the maximum area of ​​the plurality of target connected domains meets a preset condition.

7. The method according to claim 1, wherein The method further comprises: When the maximum area of ​​the plurality of target connected components does not meet a preset condition, performing an OR operation on the initial segmentation mask image and the growing segmentation mask image to obtain a growing fine segmentation mask image; Segmenting the growing fine segmentation mask image according to a preset statistical rule to obtain the target segmentation mask image; The step of segmenting the growing fine segmentation mask image according to a preset statistical rule to obtain the target segmentation mask image includes: When the area of ​​the largest target connected domain does not meet the preset conditions, clustering the pixels in the growing fine segmentation mask image using a K-means clustering algorithm to obtain a clustered segmentation image; Calculate the second similarity between the pixel points in the cluster segmentation image and each category according to the third formula; Based on the maximum second similarity of each pixel in the cluster segmentation image, the maximum inter-class variance method adaptive threshold segmentation is performed on the growing fine segmentation mask image to obtain a statistical segmentation mask image; An OR operation is performed on the statistical segmentation mask image and the growth fine segmentation mask image to obtain the target segmentation mask image.

8. The method according to any one of claims 1 to 4, characterized in that The step of obtaining an initial segmentation mask image of a preset shooting scene in the image to be processed includes: Use the high-resolution network to process the preset shooting scene in the image to be processed to obtain the original segmentation mask image; The original segmentation mask image is transferred to a preset color space for display to obtain an initial segmentation mask image.

9. The method according to claim 1, wherein include: When the preset shooting scene is a sky scene, obtaining a target segmentation mask image including the sky scene; The sky area of ​​the image to be processed is determined according to the target segmentation mask image.

10. An image processing device, characterized in that: include: An acquisition module is used to acquire an initial segmentation mask image of a preset shooting scene in the image to be processed; A region growing processing module, configured to perform region growing processing on the initial segmentation mask image to obtain a grown segmentation mask image; A determination module, configured to determine a plurality of target connected regions in a difference binary image corresponding to both the initial segmentation mask image and the grown segmentation mask image; a generating module, configured to generate a target segmentation mask image according to the initial segmentation mask image when the maximum area of ​​the plurality of target connected domains meets a preset condition; Generating a target segmentation mask image according to the initial segmentation mask image includes: Performing clustering on the pixels in the initial segmentation mask image using a K-means clustering algorithm to obtain a clustered segmentation image; Calculating a second similarity between a pixel point in the cluster segmentation image and each pre-stored category according to a pre-stored third formula; performing maximum inter-class variance method adaptive threshold segmentation on the initial segmentation mask image according to the maximum second similarity corresponding to each pixel point in the cluster segmentation image to obtain a statistical segmentation mask image; Performing an OR operation on the statistical segmentation mask image and the initial segmentation mask image to obtain a target segmentation mask image; A processing module is used to process the image to be processed according to the target segmentation mask image.

11. An image processing device, characterized in that: The image processing device includes a memory, a processor, and an image processing program stored in the memory and executable on the processor. When the processor executes the image processing program, the steps of the image processing method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that An image processing program is stored on the computer-readable storage medium, and when the image processing program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 9 are implemented.

Images