Image foreground segmentation method, apparatus, device and computer-readable storage medium

Abstract

This application discloses a method, apparatus, device, and computer-readable storage medium for image foreground segmentation, particularly relating to foreground segmentation of circular regions in images displaying circular regions of different colors. The method establishes a labeled information database based on the circular region image with labeled information, processes the information to obtain a threshold range corresponding to the target pixel region in the color space, and then accurately separates the monochrome circular particle foreground images from the user-provided image based on this threshold range. Each monochrome particle image is segmented separately to obtain monochrome particle foreground segmentation results. Finally, the multiple monochrome circular particle foreground segmentation results are fused to obtain the foreground segmentation result of the multi-dimensional circular region image. This method has higher recognition efficiency and better noise tolerance compared to the traditional watershed algorithm.

Description

Technical Field

[0001] This application relates to the field of image foreground segmentation technology, specifically to image foreground segmentation methods, apparatus, devices, and computer-readable storage media for images with circular foregrounds, and particularly to foreground segmentation of circular regions in images displaying circular regions of different colors. Background Technology

[0002] Image processing is commonly used in industrial, agricultural, and pharmaceutical production for intuitive monitoring of processes, especially for monitoring the mixing of near-circular particles. In image detection, it's necessary to segment the contours of near-circular particles in the image. The watershed algorithm is one of the most widely used image segmentation algorithms. This algorithm uses the similarity of neighboring pixels as an important reference, and its main steps are as follows: First, the input image is converted to grayscale; then, pixels with similar spatial locations and grayscale values ​​(or gradient values) are connected to form closed contour lines, thus achieving image segmentation. As can be seen from the steps, directly using the traditional watershed algorithm to segment complex color images results in the loss of much important color information during grayscale processing, leading to poor segmentation results. This has significantly limited the algorithm's application. Figure 1 (B) shows the foreground image segmentation using the traditional watershed algorithm, compared to... Figure 1 (A) It can be seen that the algorithm only achieves segmentation of some red particles, and the segmentation effect on other colored particles is poor. Therefore, the traditional watershed algorithm still has difficulty in segmenting multi-dimensional circular regions in the mixing process. Summary of the Invention

[0003] To at least partially solve one of the aforementioned technical problems, this application provides a method, segmentation method, apparatus, device, and computer-readable storage medium for color stripping of the foreground of a grainy image.

[0004] Therefore, the embodiments of this application disclose at least the following technical solutions:

[0005] This application discloses a foreground segmentation method, applied to segmenting a foreground image of a circular region from an image displaying circular regions of different colors, wherein the foreground segmentation method includes:

[0006] In response to a user's annotation operation on a circular region with a target color in a circular region image, annotation information is obtained, wherein the annotation information is in response to the color space information of the target color in the target pixel region of the user's annotation operation;

[0007] Sending a foreground image acquisition request to the system, the acquisition request being a foreground segmentation instruction, the foreground segmentation instruction causing the system to perform the step of segmenting and acquiring the foreground image from the roughly circular region image; and

[0008] Obtain the foreground image output by the system.

[0009] The step of obtaining the foreground image from the circular region image includes:

[0010] Establish a labeling information database, wherein the labeling information database stores one or more of the labeling information;

[0011] The threshold range for stripping the target color from the target pixel region is determined based on the annotation information database;

[0012] The target color is extracted from the near-circular region image based on the threshold range to obtain a monochrome stripped foreground image;

[0013] Watershed segmentation and image fusion are performed on one or more of the monochrome stripped foreground images to obtain the segmented foreground image.

[0014] A foreground segmentation apparatus is used to segment a foreground image of a roughly circular region from an image displaying roughly circular regions of different colors, the foreground segmentation apparatus comprising:

[0015] The acquisition unit is used to acquire images of circular regions displaying different colors and annotation information, wherein the annotation information includes color space information of the target pixel region in response to the user's annotation operation;

[0016] The database establishment unit is used to establish a database for the input, storage, transformation, dumping, recovery, reorganization, performance monitoring and analysis of the labeled information;

[0017] The determining unit is configured to determine, based on the annotation information database, a threshold range for stripping the target color from the target pixel region;

[0018] A monochrome stripping unit is used to strip the target color from the target pixel region according to the threshold range to obtain a monochrome stripped foreground image;

[0019] A segmentation and merging unit is used to perform watershed segmentation and image fusion on one or more of the monochrome stripped foreground images to obtain the segmented foreground image.

[0020] This application also discloses an image segmentation device, which includes:

[0021] processor;

[0022] The memory stores computer programs that can run on the processor;

[0023] When the computer program is executed by the processor, it implements the foreground segmentation method provided in the above embodiments.

[0024] This application also discloses a computer-readable storage medium storing a computer program, wherein the computer program, when executed, implements the foreground segmentation method provided in the above embodiments.

[0025] This application provides a foreground segmentation method in which the system collects and establishes a database of annotation information based on a circular region image with annotation information. The annotation information is then processed, and a threshold corresponding to the color space of the target pixel region determined by the user is selected. Based on this threshold range, all monochrome particles of the target color determined by the user in the image provided by the user can be accurately extracted. Based on one or more monochrome particle foreground images obtained by the above color extraction method, the watershed algorithm is used to segment the single particle images respectively, merge the regions, and merge the segmented foreground images of single particles of different colors to obtain a segmented foreground image of a multi-dimensional circular region image.

[0026] The technical solution provided in this application can arbitrarily determine the color image of the quasi-circular region to be segmented by the user. Furthermore, it can reduce the difficulty of image processing, avoid the use of complex network modeling, obtain monochrome particle foreground images through color stripping methods, greatly shorten the image recognition time, have fast calculation speed, occupy small storage units, have higher recognition efficiency than traditional watershed algorithms, and have better noise tolerance.

[0027] Furthermore, embodiments of the present invention also provide corresponding implementation devices, equipment, and computer-readable storage media for segmenting multi-dimensional circular region images, further making the method more practical, and the devices, equipment, and computer-readable storage media have corresponding advantages. Attached Figure Description

[0028] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0029] Figure 1 This is the segmentation effect of the traditional watershed algorithm provided by existing technology (A is the original image, B is the segmented foreground image).

[0030] Figure 2 These are schematic diagrams of image segmentation systems provided in some embodiments.

[0031] Figure 3 These are schematic diagrams of the foreground segmentation method provided in some embodiments.

[0032] Figure 4 This is a flowchart illustrating a method for determining a threshold range for color feature values, provided in some embodiments. The dashed lines in the diagram represent optional steps, not necessarily mandatory ones.

[0033] Figure 5 This is a flowchart illustrating a method for "sorting the first interval in descending order to obtain the second interval" provided in some embodiments.

[0034] Figure 6 This is a schematic diagram of a method flow for "using a combination of multiple second intervals that meet preset conditions as a first color feature value threshold interval" provided in some embodiments.

[0035] Figure 7 These are schematic diagrams illustrating the division of a first interval (A) and a second interval (B) as provided in some embodiments.

[0036] Figure 8 These are four monochrome stripped foreground images provided in some embodiments (A. green grains B. red grains C. blue grains D. yellow grains).

[0037] Figure 9 Yes Figure 8 The four monochrome stripped foreground images are obtained by segmenting the four monochrome stripped foreground images in the image (A. green B. red C. blue D. yellow).

[0038] Figure 10 Yes Figure 9 The foreground image is obtained by fusing four monochrome stripped foreground images.

[0039] Figure 11 These are schematic diagrams of the foreground segmentation method provided in some embodiments.

[0040] Figure 12 These are user-machine interfaces (A) that prompt the user to input annotation information, (B) that display the annotation operation, and (C) that prompt the user whether to end the annotation operation in the user terminal input unit provided in some embodiments.

[0041] Figure 13 These are schematic diagrams of the virtual device structure of the foreground segmentation apparatus provided in some embodiments.

[0042] Figure 14 These are hardware structure block diagrams of image segmentation devices provided in some embodiments.

[0043] Figure 15 The foreground images are obtained by segmenting multiple color grain images using the traditional watershed algorithm and by segmenting them using the method provided in the embodiments of this application, with each column representing an image segmentation process.

[0044] It should be noted that, Figure 1 , Figure 7 , Figure 8 , Figure 9 , Figure 10 , Figure 11 , Figure 12 ,and Figure 15 All actual images are color images. To meet the requirements for drawings in the Patent Law and its Implementing Regulations, the drawings provided in the embodiments of this invention will... Figure 1 , Figure 7 , Figure 8 , Figure 9 , Figure 10 , Figure 11 , Figure 12 ,and Figure 15 The real image was processed into grayscale. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. Reagents not specifically described in detail herein are all conventional reagents and are commercially available; methods not specifically described in detail are all conventional experimental methods and can be learned from the prior art.

[0046] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor do they substantially limit the technical features thereafter. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0047] Some embodiments provide foreground segmentation methods and systems for color images displaying circular regions of different colors, in order to solve the problem that traditional watershed algorithms in the prior art are difficult to segment images of multi-dimensional circular regions.

[0048] In this application, "circular-like region" refers to the outline of an object that is displayed as a circle in an image, picture, video, video screenshot or visual information, such as a circle, sphere, pie, ring, polygon, hemisphere or semicircle, etc.

[0049] In this application, "color space," also known as a color model or color space, is used to describe colors in a generally acceptable way under certain standards. Essentially, a color space is a description of a coordinate system and subspaces, where each color is represented by a single point. Most currently used color spaces are hardware-oriented or application-oriented; hundreds have existed since their inception, but most are only partial modifications or specific to a particular field. In reflectance reconstruction, commonly used color spaces include RGB, CIE XYZ, Lab, and HSV. "Color characteristic value" refers to the numerical value established in a "color space" for digitally representing color, such as the coordinate value corresponding to the "R" of red brightness in the "RGB color space."

[0050] Figure 2 A block diagram of an image segmentation system 100 according to some embodiments is shown. System 100 may include a user terminal 110 and a system terminal 120. User terminal 110 mainly includes an input unit 111, an annotation unit 112, and an output unit 113. System terminal 120 mainly includes an acquisition unit 121, a database establishment unit 122, a determination unit 123, a monochrome stripping unit 124, and a segmentation and merging unit 125. Each device in system 100 can communicate with one or more other devices in system 100 via a suitable communication network (e.g., the Internet, a wide area network, or a local area network). In some embodiments, one or more units, computers, or computing systems in system 100 may be the same device and / or included in the same computing system.

[0051] The communication network can take many forms, including, for example, cellular telephone networks, terrestrial telephone networks, packet-switched networks such as the Internet, and / or combinations of such networks. Other examples are also possible. The communication network can be configured to perform various operations, including, for example, facilitating communication between input unit 111, annotation unit 112, output unit 113, acquisition unit 121, database establishment unit 122, determination unit 123, monochrome stripping unit 124, and segmentation and merging unit 125 using one or more protocols.

[0052] For illustrative purposes, communication networks in Figure 1 The input unit 111, annotation unit 112, output unit 113, acquisition unit 121, database establishment unit 122, determination unit 123, monochrome stripping unit 124, and segmentation and merging unit 125 may include one or more servers, one or more processors and / or one or more computers, one or more user terminals (e.g., mobile phones), and / or each of the input unit 111, annotation unit 112, output unit 113, acquisition unit 121, database establishment unit 122, determination unit 123, monochrome stripping unit 124, and segmentation and merging unit 125 may be included within a computing system including one or more servers, one or more processors, and / or one or more servers. Each of the acquisition unit 121, database establishment unit 122, determination unit 123, monochrome stripping unit 124, and segmentation and merging unit 125 may include one or more memories and / or one or more computer-readable media configured to store instructions that, when executed by one or more processors, cause to perform one or more actions disclosed herein. However, it is worth noting that a communication network may include two or more separate communication networks, each configured to facilitate communication between selected systems or devices.

[0053] Input unit 111 can generate or receive input datasets. Input unit 111 may include or communicate with input sensor 115. Examples of input sensor 115 may include a camera (e.g., a CCD color camera), a camera (e.g., connected to a microscope), a microphone, a spectrometer, or some other sensor capable of recording data. In some embodiments, input sensor 115 may be internal to input unit 111 and record internal data such as that used by the CPU. For example, input unit 111 may be a production monitoring system, which may include a system or component (5G remote image monitoring component or system) for monitoring colorized particle-like outline images generated during the production process.

[0054] In some embodiments, input unit 111 may receive input data from other sources, such as by scraping images from a website, downloading sensor data from a government database (e.g., the NIH database), and / or receiving images and / or annotation data from a user terminal or user device (e.g., a mobile phone). Input unit 111 may store the input data for later retrieval.

[0055] Those skilled in the art will understand that the dataset used by input unit 111 for input can be collected at one or more times and / or from one or more sources. For example, a first input dataset (e.g., associated with corresponding labeled data) can be collected at a first time via input sensor 115 and / or from a user terminal, and a second input dataset (e.g., not associated with real labeled data) can be collected at a later second time from the user terminal. In some cases, each input dataset includes a set of digital images, for example, in which color spaces are represented by HSV values.

[0056] Those skilled in the art will understand that the input unit 111 can also preprocess the data by, for example, normalizing the data, removing noise, and standardizing the data size. The input unit 111 can send the input dataset to the annotation module acquisition unit 121, and the database establishment unit 122 can establish a set of digital images with annotation information, including color spaces in the set of digital images characterized by HSV values.

[0057] In some embodiments, annotation unit 112 identifies annotations for at least some of the input data in the input dataset. The annotations generated by annotation unit 112 can be dense annotations, such that every part of the data is labeled (e.g., in a particle image displaying multiple colors, particles with certain colors are labeled as either part of the foreground of that type of circular particle outline (i.e., a monochrome stripped foreground image)). In some embodiments, annotation unit 112 may have I / O devices for presenting input data to an operator and receiving annotations input by the operator. For example, annotation unit 112 may display an image, and a human operator can type or select appropriate annotations for that image.

[0058] In some embodiments, the annotation unit 112 can automatically determine annotations. In some embodiments, the annotation unit 112 can determine annotations by receiving annotations from a source (e.g., a government database). The annotation unit 112 can receive input data to be annotated from the input unit 111. The annotation unit 112 can transmit the annotation dataset to the system terminal 120 for processing (e.g., processing via a neural network). In some embodiments, the annotation unit 112 can send both the annotation dataset and the input dataset.

[0059] In some embodiments, the database building unit 122 may build a neural network and train it using one or more input datasets and corresponding annotation information. The neural network may include a convolutional neural network and / or a dense neural network. Some input data and corresponding annotations may be used for validation and / or testing. Some input data (e.g., not associated with annotations) may be processed by (e.g., a trained) neural network that can generate predictive annotations. For each of one or more types of objects or structures, the annotations may identify (e.g.) which pixels within each input image are known or predicted to be associated with the object or structure type. In some cases, the annotations may further identify individual instances of a given object or structure type. For example, the annotations may identify a first set of pixels predicted to be associated with a first target pixel and a second set of pixels predicted to be associated with a different second target pixel, specifically, a pixel displayed as a circular region of a different color.

[0060] In some embodiments, the determining unit 123 can calculate the threshold space representing the color space in these digital image sets based on the set of annotation information to determine the threshold space for the target color to be stripped from the target pixel region. The determining unit 123 can generate dense threshold intervals, such as determining the segmentation boundary of a circular region in a grainy image that displays different target colors, where the target color feature is reflected in the color emission spectrum, or identifying words in a recording. In some embodiments, the determining unit 123 may include annotation information corresponding to the color space to be stripped determined by the user and associated with a preset mapping, segmenting to obtain multiple threshold intervals, calculating the number of pixels whose feature values ​​in the standard color space fall into the threshold intervals, and statistically analyzing the number of pixels to determine the final threshold space. The execution code for these execution codes is then used by a computer or system to perform these operations.

[0061] In some embodiments, the monochrome stripping unit 124 can generate a monochrome stripped foreground image based on the output of a threshold range determined by the determining unit 123. For example, the monochrome stripping unit 124 can strip the target color of the target pixel region corresponding to the annotation information. The specific stripping process can be as follows: the monochrome stripping unit 124 judges the color space representation value of the target pixel region of the read-in circular region image. If it is within the threshold range, the pixel is processed as a foreground pixel (e.g., marked with a label different from the background pixel). Then, all foreground pixel regions that meet the threshold range are extracted, which is the monochrome stripped foreground image. For example, stripping a color image displaying spherical particles of different colors can yield multiple monochrome stripped foreground images of particles of different colors.

[0062] In some embodiments, the segmentation and merging unit is used to perform watershed segmentation and image fusion on one or more monochrome stripped foreground images to obtain a foreground image. For example, after stripping a color image displaying spherical particles of different colors, multiple monochrome stripped foreground images of different colored particles can be obtained. At this time, the watershed algorithm is used to segment each monochrome stripped foreground image separately, resulting in multiple monochrome stripped foreground images. Finally, the multiple monochrome stripped foreground images are merged to obtain the final foreground image.

[0063] Figure 3 A foreground segmentation method according to some embodiments is illustrated. This method can be executed by a computing system including one or more computers (e.g., executed by a system endpoint) and applied to foreground segmentation of circular regions in an image displaying circular regions of different colors.

[0064] As shown in box 210, in some embodiments, the system acquires images of circular regions displaying different colors and annotation information. The annotation information includes color space information of the target color in the target pixel region in response to the user's annotation operation. For example, the color space information is preprocessed data that has been normalized, noise removed, and data size standardized. This preprocessed data is a dataset that digitizes the obtained color space information, and the annotation information is the annotation of these datasets.

[0065] As shown in box 220, in some embodiments, the system establishes a labeling information database, which stores one or more types of labeling information. In some embodiments, the system can also input, store, transform, dump, restore, reorganize, monitor performance, and analyze the labeling information in the labeling information database to analyze and manage the database. In some embodiments, the system can also build a neural network and train it using one or more input datasets and corresponding labeling information. The neural network may include convolutional neural networks and / or dense neural networks. Some input data and corresponding labels can be used for verification and / or testing. Some input data (e.g., not associated with labels) can be processed by (e.g., trained) a neural network that can generate predictive labels. For each of one or more types of objects or structures, the labels can identify (e.g.) which pixels within each input image are known or predicted to be associated with the object or structure type. In some cases, the labels can further identify individual instances of a given object or structure type. For example, the annotation can identify a first set of pixels predicted to be associated with a first target pixel and a second set of pixels predicted to be associated with a different second target pixel, specifically, the target pixel being a pixel displayed as a circular area of ​​a different color.

[0066] As shown in box 230, in some embodiments, the system determines a threshold range for stripping the target color from the target pixel region based on a labeling information database. For example, the target pixel region may be colored particles with different colors that need to be stripped, stored in the labeling information database. The system calculates the threshold space for the target color to be stripped from the target pixel region by characterizing the color spaces in these digital image sets. Alternatively, it may generate dense threshold ranges, such as determining the segmentation boundaries of circular regions in a particle image that display different target colors, where the target color feature is reflected in the color emission spectrum or identifies words in a recording. In some embodiments, the system may include labeling information corresponding to the color space to be stripped determined by the user and associated with a preset mapping, segmenting to obtain multiple threshold ranges, calculating the number of pixels whose feature values ​​in the standard color space fall within the threshold range, and statistically analyzing the number of pixels to determine the final threshold space's operating code.

[0067] As shown in box 240, in some embodiments, the system extracts the target color from the target pixel region according to the threshold range to obtain a monochrome stripped foreground image. For example, the specific stripping process can be as follows: the system judges the color space representation value of the target pixel region of the read-in circular region image. If it is within the threshold range, the pixel is processed as a foreground pixel (e.g., marked with a different label from the background pixel). Then, all foreground pixel regions that meet the threshold range are extracted, which is the monochrome stripped foreground image. For example, stripping a color image displaying spherical particles of different colors can yield multiple monochrome stripped foreground images of particles of different colors.

[0068] As shown in box 250, in some embodiments, the system performs watershed segmentation and image fusion on one or more of the monochrome stripped foreground images to obtain a foreground image. For example, after stripping a color image displaying spherical particles of different colors, multiple monochrome stripped foreground images of different colored particles can be obtained. At this point, a watershed algorithm is used to segment each monochrome stripped foreground image separately, resulting in multiple monochrome stripped foreground images. Finally, these multiple monochrome stripped foreground images are merged to obtain the final foreground image. For example, OpenCV implements a label-based watershed algorithm. Based on the annotation information, i.e., the label assigned by the user to the foreground region to distinguish it from the background region, the watershed algorithm is used to segment the foreground, for example, using the well-known Cv2.wAtersheD() function, to obtain the segmented monochrome foreground images. Then, merging the monochrome foreground images yields the foreground image of the original input image. In these embodiments, fusion mainly refers to pixel-level image fusion. Pixel-level image fusion methods can employ, for example, color space fusion methods.

[0069] In some embodiments, the color space information in the target pixel region is characterized by color feature values. In this application embodiment, the color space is selected from any one of the RGB color space, CIE XYZ color space, Lab color space, or HSV color space, preferably HSV or RGB color space. In this application embodiment, the color feature values ​​are selected from at least one of the H, S, and V values ​​in the HSV color space; or at least one of the R, G, and B values ​​in the RGB color space; or at least one of the X, Y, and Z values ​​in the CIE XYZ color space; or at least one of the L, a, and b values ​​in the Lab color space; preferably at least one of the H, S, and V values ​​in the HSV color space or at least one of the R, G, and B values ​​in the RGB color space.

[0070] In some embodiments, there are three color feature values, so the determination of the threshold interval (box 230) includes the determination of the first color feature value threshold interval (box 231), the determination of the second color feature value threshold interval (box 232), and the determination of the third color feature value threshold interval (box 233).

[0071] In some embodiments, at least one of the first color feature value threshold interval, the second color feature value threshold interval, and the third color feature value threshold interval is configured according to... Figure 4 Steps 310-330 shown are executed. For example, the method for determining the first color feature value threshold interval (box 231) includes: obtaining a plurality of first intervals in response to a preset mapping (box 310), wherein the preset mapping is a correspondence between the first color feature value of a target color pixel region and a range of natural numbers; sorting the plurality of first intervals in descending order to obtain a plurality of second intervals (box 320); and using a combination of the plurality of second intervals that meet preset conditions as the first color feature value threshold interval (box 330).

[0072] In the above embodiments, such as Figure 5 As shown, “sorting the first interval in descending order to obtain the second interval” (box 320) specifically includes: obtaining a first value (box 321) of the number of pixels with the first color feature value of the target color pixel region that are in the first interval; sorting the first interval in descending order according to the first value to obtain the second interval (box 322).

[0073] In the above embodiments, such as Figure 6 As shown, "using a combination of multiple second intervals that meet preset conditions as the H-value threshold interval" (box 330) specifically includes:

[0074] Obtain a second value (box 331) of the number of pixels whose first color feature value of the target color pixel region is within the second interval;

[0075] Select a third and a fourth sequence of numbers arranged in descending order of multiple second values, wherein the number of second values ​​in the third sequence is one less than the number of second values ​​in the fourth sequence (box 332);

[0076] Determine whether 95% of the sum of all the first values ​​is greater than or equal to the sum of all the second values ​​in the third sequence and less than or equal to the first color feature value threshold range (box 334). Further, if not, execute steps 332 to 333 until condition 333 is met.

[0077] In a specific embodiment of the foreground segmentation method, the foreground segmentation method includes:

[0078] (1) The system reads any annotation information from the annotation information database and reads the H value of each pixel in the HSV color space of the digital image corresponding to the processed annotation information.

[0079] (2) Negative values ​​are set to 0, and a preset mapping relationship is established between the H value of each pixel and the value range. For example, the threshold range is [0, 180]. This divides the first interval into multiple first intervals (N) with the same number of pixels in the pixel area corresponding to the annotation information. For example, Figure 7 (A) shows {A1:[0,180 / N], A2:[(180 / N)+1,180×2 / N],…, AN:[180×(N-1) / N]+1,180]}.

[0080] (3) Then, count the number of pixels corresponding to the H values ​​falling into each of the first intervals. This is the first value, which is nA1, nA2, ..., nAN, and nA1 + nA2 + ..., nAN = n, which is the total number of target pixels in the target pixel region. This corresponds to all the pixel regions of the circular regions marked in the foreground image, such as all the spheres of a certain color in the image marked by the user. Based on the first value from largest to smallest, sort the first intervals in descending order to obtain the second interval, for example, as shown below. Figure 7 As shown in (B) {B1, B2, ... BN}, the statistical results of each pixel point corresponding to each second interval are obtained as nB1, nB2, ... nBN (that is, the second value), and where nB1≧nB2≧...≧nBN.

[0081] (4) Select a third and a fourth sequence of numbers arranged in descending order of multiple second values, wherein the number of second values ​​in the third sequence is one less than the number of second values ​​in the fourth sequence; determine whether 95% of the sum of all first values ​​is greater than or equal to the sum of all second values ​​in the third sequence and less than or equal to the sum of all second values ​​in the fourth sequence: if so, merge all corresponding second intervals in the third sequence as the H value threshold interval.

[0082] For example, selecting nB1+nB2+...+nBi-1≦n×p%≦nB1+nB2+...+nBi, where p% is a preset percentage, then the union of the intervals B1, B2...Bi (performing a logical AND operation) is taken as the threshold range of the H value for all pixel regions (e.g., all spheres of a certain color in a user-annotated image) corresponding to the circular region annotations in the foreground image. Figure 7 The information in B is labeled as green particles. The number N in the first interval is 18, the preset percentage is 95%, nB1+nB2≦n×95%≦nB1+nB2+nB3. By counting, we can see that B1UB2 (i.e. A6UA7) is the threshold range of H value.

[0083] (5) The ranges of S and V values ​​in the HSV color space of all pixel regions of the annotated circular areas (e.g., all spheres of a certain color in the user-annotated image) are mapped to value intervals respectively; secondly, the S and V values ​​in the obtained data are determined according to the H interval threshold determination process. Figure 8 In the embodiments, taking the HSV color space as an example, the H value threshold mapping range is [50, 70], the S value threshold range is [100, 255], and the V value threshold range is [90, 250].

[0084] (6) Based on the threshold range determined above, for example... Figure 1 Image A is segmented by color to obtain, as shown below. Figure 8 The four monochrome foreground images shown are then segmented using the watershed segmentation algorithm and fused with color space images to obtain the following results: Figure 9 The segmented foreground image shown. This segmented foreground image is compared with... Figure 1 Compared to B, it can be used for Figure 1 The four colors of the particles were repeatedly segmented, and no over-segmentation occurred during noise segmentation. The segmented image clearly displays the edges and contours of the colored particles, with sharp lines. Additionally, as shown... Figure 15As shown, compared with the traditional watershed segmentation method, this image segmentation result can not only segment colored particles and obtain a foreground image with clear lines and distinct contours, but also greatly shorten the image recognition time, has a fast calculation speed, occupies a small storage unit, and has a higher recognition efficiency than the traditional watershed algorithm.

[0085] Figure 12 Some embodiments of a foreground segmentation method are illustrated. This method can be executed by a computing system including one or more computers (e.g., by a user terminal, such as a mobile phone), and is applied to foreground segmentation of circular regions in an image displaying circular regions of different colors. As shown in box 410, the user annotates the color and type of the circular region to be segmented. The user terminal responds to this operation to obtain annotation information, thereby simultaneously sending the annotation information and a foreground image acquisition request to the system. Specifically, the annotation information is the color space information of the target color in the target pixel region in response to the user's annotation operation. As shown in box 420, the user terminal sends the foreground image acquisition request containing the annotation information to the system. The foreground image acquisition request is a foreground segmentation instruction that causes the system to perform the step of acquiring a foreground circular image from the circular region image. Specifically, the foreground image acquisition request is a foreground segmentation instruction that causes the system to perform the step of acquiring a foreground circular image from the circular region image, for example... Figures 2-11 The steps provided in the illustrated embodiment are as follows. As shown in box 430, the user terminal obtains the foreground image output by the system terminal.

[0086] For example, loading such on the user's end Figure 1 The image, wherein the input unit 111 includes a human-computer interaction interface, as shown in the image... Figure 12 As shown in Figure A, the user is prompted to input annotation information. The user can perform annotation operations, such as "drag from the edge of the particle towards the center of the particle." The human-computer interaction interface displays the following: Figure 12 As shown in Figure B (where the bolded circle represents the labeled sphere), repeating these annotation operations allows for the labeling of particles of various sizes and colors in an image as different types of particles, and the annotation information is saved on the user's device. In some embodiments, such as Figure 12 C. The client can also generate prompts on the human-computer interaction interface to ask the user whether to end the annotation operation.

[0087] Figure 13 This illustration shows a foreground segmentation apparatus according to an embodiment of this application, applied to foreground segmentation of a circular region in an image displaying circular regions of different colors. For example... Figure 13As shown, the foreground segmentation device includes an acquisition module 510, a database establishment module 520, a determination module 530, a monochrome stripping module 540, and a segmentation and merging module 550. The acquisition module 510 acquires images of roughly circular regions displaying different colors and annotation information, including color space information of target pixel regions in response to user annotation operations. The database establishment module 520 establishes a database for inputting, storing, converting, dumping, restoring, reorganizing, monitoring, and analyzing the color space information of the target colors. The determination module 530 determines a threshold range for stripping the target color from the target pixel regions based on the annotation information database. The monochrome stripping module 540 strips the target color from the target pixel regions according to the threshold range to obtain a monochrome stripped foreground image. The segmentation and merging module 550 performs watershed segmentation and image fusion on one or more of the monochrome stripped foreground images to obtain a foreground image.

[0088] Since the functions of the foreground segmentation device 500 are the same as those described in the system description above, detailed descriptions of the same content are omitted here for simplicity. Furthermore, in addition to these four units, the foreground segmentation device 500 may also include other components; however, since these components are irrelevant to the embodiments of this application, their illustrations and descriptions are omitted here.

[0089] Figure 14 A hardware structure block diagram of the image segmentation device provided in an embodiment of this application is shown, with reference to... Figure 5 The hardware structure of the image partitioning device 600 may include: at least one processor 610 and at least one memory 620, on which a computer program executable by the processor is stored, the computer program implementing, when executed by the processor, as follows: Figure 2 The method described in any one of the figures or diagrams.

[0090] Furthermore, the image segmentation apparatus (e.g., image processing apparatus, etc.) according to embodiments of this application can also be used by means of Figure 14 The architecture of the exemplary computing device shown is used to implement this. Figure 14 A schematic diagram of the architecture of an exemplary computing device according to an embodiment of this application is shown. Figure 14As shown, the image segmentation device 600 may include a bus 630, one or more CPUs (processors) 610, a read-only memory (ROM) 621, a random access memory (RAM) 622, a communication port 640 connected to a network, an input / output component 650, a hard disk 660, etc. The storage devices in the image segmentation device 600, such as the ROM 6230 or the hard disk 660, can store various data or files used for computer processing and / or communication, as well as program instructions executed by the CPU. The image segmentation device 600 may also include a user interface 670. Of course, Figure 14 The architecture shown is merely exemplary and can be omitted as needed when implementing different devices. Figure 14 One or more components in the computing device shown.

[0091] The embodiments of this application can also be implemented as a computer-readable storage medium. A computer-readable storage medium according to an embodiment of this application stores computer-readable instructions. When the computer-readable instructions are executed by a processor, the methods described with reference to the above figures according to the embodiments of this application can be performed. The computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (CAChe). Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0092] According to embodiments of this application, a computer program product or computer program is also provided, which includes computer-readable instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer-readable instructions from the computer-readable storage medium and execute the computer-readable instructions, causing the computer device to perform the training method and image processing method described in the various embodiments above.

[0093] Those skilled in the art will understand that the content disclosed in this application can be varied and modified in many ways. For example, the various devices or components described above can be implemented by hardware, or by software, firmware, or a combination of some or all of the three.

[0094] Furthermore, as shown in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. The terms "first," "second," and similar terms used in this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms "comprising" or "including" mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, without excluding other elements or objects. The terms "connected" or "linked" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0095] Furthermore, flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously. Additionally, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0096] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in a common dictionary shall be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0097] The present application has been described in detail above; however, it will be apparent to those skilled in the art that the present application is not limited to the embodiments described herein. The present application can be implemented in modified and altered ways without departing from the spirit and scope of the present application as defined by the claims. Therefore, the description in this specification is for illustrative purposes only and is not intended to be restrictive in any way.

Claims

1. A foreground segmentation method, applied to segment a foreground image of a circular region from an image displaying circular regions of different colors, wherein, The foreground segmentation method includes: Acquire images of circular regions displaying different colors and annotation information, wherein the annotation information includes color space information of the target pixel region in response to user annotation operations; Establish a labeling information database, wherein the labeling information database stores one or more of the labeling information; The threshold range for stripping the target color in the target pixel region is determined based on the annotation information database. The threshold range is the range of color feature values ​​of the target pixel region. The color feature values ​​are numerical values ​​that represent colors digitally in the color space. The target color is extracted from circular regions of different colors according to the threshold range to obtain a monochrome stripped foreground image; watershed segmentation and image fusion are performed on one or more of the monochrome stripped foreground images to obtain the foreground image.

2. The foreground segmentation method according to claim 1, wherein, The color space information of the target pixel region is represented by color feature values, and the threshold range is the range of values ​​for the color feature values.

3. The foreground segmentation method according to claim 2, wherein, The method for determining the threshold range includes: Obtain a plurality of first intervals in response to a preset mapping, wherein the preset mapping is a correspondence between the color feature value of the target color pixel region and a range of natural number values; The plurality of first intervals are sorted in descending order to obtain a plurality of second intervals; Multiple second interval combinations that meet preset conditions are used as the threshold intervals for the color feature values.

4. The foreground segmentation method according to claim 3, wherein, "Sort the first interval in descending order to obtain the second interval" specifically includes: Obtain a first value representing the number of pixels whose color feature values ​​of the target color pixel region fall within the first interval; The second interval is obtained by sorting the first interval in descending order based on the first value.

5. The foreground segmentation method according to claim 4, wherein, "Using multiple second interval combinations that meet preset conditions as the color feature value threshold interval" specifically includes: A second value is obtained representing the number of pixels whose color feature values ​​of the target color pixel region fall within the second interval. Select a third and a fourth sequence of numbers, which are composed of multiple second values ​​arranged in descending order, wherein the number of second values ​​in the third sequence is one less than the number of second values ​​in the fourth sequence; Determine whether the preset percentage of the sum of all the first values ​​is greater than or equal to the sum of all the second values ​​in the third sequence, and less than or equal to the sum of all the second values ​​in the fourth sequence: If so, then all corresponding second intervals in the third sequence are merged as the color feature value threshold interval.

6. The foreground segmentation method according to claim 2, wherein, The color feature value is a numerical value representing the color digitally in the color space; The color space is selected from any one of RGB space, CIEXYZ color space, Lab color space or HSV color space, and can be selected as HSV color space or RGB space. The color feature value is selected from at least one of the H, S and V values ​​in the HSV color space; or at least one of the R, G and B values ​​in the RGB color space; or at least one of the X, Y and Z values ​​in the CIEXYZ color space; or at least one of the L, a and b values ​​in the Lab color space.

7. The foreground segmentation method according to claim 6, wherein, The color feature value is at least one of the H, S and V values ​​in the HSV color space or at least one of the R, G and B values ​​in the RGB color space.

8. A foreground segmentation method, applied to segment a foreground image of a circular region from an image displaying circular regions of different colors, wherein, The foreground segmentation method includes: In response to a user's annotation operation on a circular region with a target color in a circular region image, annotation information is obtained, the annotation information including color space information of the target pixel region in response to the user's annotation operation; Sending a foreground image acquisition request to the system, the acquisition request being a foreground segmentation instruction, the foreground segmentation instruction causing the system to perform the step of segmenting and acquiring the foreground image from the roughly circular region image; and Obtain the foreground image output by the system. The step of segmenting and obtaining the foreground image from the circular region image includes: Establish a labeling information database, wherein the labeling information database stores one or more of the labeling information; The threshold range for stripping the target color in the target pixel region is determined based on the annotation information database. The threshold range is the range of color feature values ​​of the target pixel region. The color feature values ​​are numerical values ​​that represent colors digitally in the color space. The target color is extracted from the near-circular region image based on the threshold range to obtain a monochrome stripped foreground image; Watershed segmentation and image fusion are performed on one or more of the monochrome stripped foreground images to obtain the foreground image.

9. A foreground segmentation apparatus, used to segment a foreground image of a roughly circular region from an image displaying roughly circular regions of different colors, the foreground segmentation apparatus comprising: The acquisition unit is used to acquire images of circular regions displaying different colors and annotation information, wherein the annotation information includes color space information of the target pixel region in response to the user's annotation operation; The database establishment unit is used to establish a database for the input, storage, transformation, dumping, recovery, reorganization, performance monitoring and analysis of the labeled information; The determining unit is configured to determine, based on the annotation information database, a threshold range for stripping the target color from the target pixel region, wherein the threshold range is the range of color feature values ​​for the target pixel region, and the color feature values ​​are numerical values ​​that represent colors digitally in a color space. A monochrome stripping unit is used to strip the target color from the target pixel region according to the threshold range to obtain a monochrome stripped foreground image; The segmentation and merging unit is used to perform watershed segmentation and image fusion on one or more of the monochrome stripped foreground images to obtain the foreground image of the quasi-circular region image.

10. An image segmentation device, comprising: processor; The memory stores computer programs that can run on the processor; The computer program, when executed by the processor, implements the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, wherein, When the computer program is executed, it implements the method as described in any one of claims 1 to 8.

Images