Image segmentation based on multi-agent deep reinforcement learning

Abstract

Access a digital image frame containing an object of interest and receive object selection instructions for pixels associated with the object of interest. Assign a software agent to each pixel, whereby each software agent applies an iterative deep reinforcement learning process to its pixel to estimate the probability that the pixel is one of the pixels associated with the object of interest, and classifies whether the pixel is one of the pixels associated with the object of interest based on the estimated probability. Using multiple software agents and other software agents associated with regions of the object of interest to communicate or collaborate enables the software agents to compute more accurate estimated probabilities. For example, probability estimation errors can be more effectively avoided in cases where pixel values ​​are occluded (e.g., the boundary region between skin and hair).

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to image segmentation based on multi-agent deep reinforcement learning, as well as related devices, methods, and computer programs. Background Technology

[0002] In computer-implemented image processing, image segmentation involves dividing a digital image frame into two or more segments, such as pixel regions or sets of image objects. Image segmentation is commonly used to locate objects and / or boundaries in an image. For example, image segmentation can be used to separate objects of interest from the background of a digital image frame, such as a specific person. Summary of the Invention

[0003] The present invention is provided to introduce, in a simplified form, some concepts further described in the following detailed description. The summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0004] The object of this invention is to enable image segmentation based on multi-agent deep reinforcement learning. The above and other objects are achieved through the features claimed in the independent claims. Other implementations are apparent from the dependent claims, the specification, and the drawings.

[0005] According to a first aspect of the present invention, an image segmentation apparatus is provided. The image segmentation apparatus is configured to: access a digital image frame depicting a scene containing an object of interest; and receive an object selection indication of a portion of pixels of the digital image frame associated with the object of interest. The image segmentation apparatus is further configured to: assign a software agent to each pixel of the digital image frame. Each software agent is configured to apply an iterative deep reinforcement learning process to the pixel to estimate the probability that the pixel is one of the pixels associated with the object of interest, and classify whether the pixel is one of the pixels associated with the object of interest based on the estimated probability. Each iteration step of the iterative deep reinforcement learning process is executed according to a software agent-specific policy and maps a pixel state dataset to a corresponding action. Each pixel state dataset includes the color value of each pixel and the object selection indication. Each action includes setting the estimated probability to one of a predefined discrete value. The image segmentation apparatus is further configured to: after each iteration step, receive a classification result from each assigned software agent; and determine the object of interest after each iteration step based on the received classification result. The present invention enables image segmentation based on multi-agent deep reinforcement learning. Using multiple software agents allows the software agents to compute accurate estimated probabilities. For example, probability estimation errors can be avoided more effectively when pixel values ​​are occluded (e.g., in the boundary region between skin and hair).

[0006] In one implementation of the first aspect, each software agent is further configured to communicate with at least some other software agents sharing the same strategy to exchange information related to the pixel state dataset and to further classify the pixels based on the exchanged information. This invention enables image segmentation based on multi-agent deep reinforcement learning. By using multiple software agents and other software agents related to the region of the object of interest to communicate or collaborate, the software agents are further able to compute more accurate estimated probabilities. For example, probability estimation errors can be more effectively avoided when pixel values ​​are occluded (e.g., the boundary region between skin and hair).

[0007] In one implementation of the first aspect, the pixel state dataset for a given iteration step also includes an estimated probability from the previous iteration step. Including the estimated probability from the previous iteration step enables the software agent to calculate a more accurate probability estimate.

[0008] In one implementation of the first aspect, each software agent is further configured to adjust the estimated probability of each iteration step according to a reward metric; the reward metric having a positive value indicates an increase in the estimated probability compared to the previous iteration step; or, the reward metric having a negative value indicates a decrease in the estimated probability compared to the previous iteration step. Adjusting the estimated probability according to the reward metric can make the current estimated probability exceed the previous estimated probability, thereby making the iterative deep reinforcement learning process more efficient.

[0009] In one implementation of the first aspect, the image segmentation device is further configured to update the object selection indicator after one or more of the iterative steps based on user feedback. Updating the object selection indicator based on user feedback, such as user input, can improve the image segmentation result in the next iterative step.

[0010] In one implementation of the first aspect, each software agent is further configured to use an Advantageous Action Commentary (A2C) network when setting the estimated probabilities. The A2C network can be used as input to the pixel state dataset and to select the action to be taken based on the policy for each pixel, thereby forming a probability map.

[0011] In one implementation of the first aspect, each software agent is further configured to perform maximum a posteriori probability (MAP) estimation on the estimated probability results of the A2C network. Performing MAP estimation on the estimated probability results of the A2C network allows the generation of a final segmentation mask from the probability map generated by the A2C network.

[0012] In one implementation of the first aspect, the A2C network includes a graph convolutional network (GCN) for facilitating communication between the software agents. Using the GCN enables the software agents to efficiently communicate or collaborate with other software agents related to the region of the object of interest, thereby allowing the software agents to compute more accurate probability estimates.

[0013] In one implementation of the first aspect, the GCN is further configured to take the pixel state dataset of the pixels and a graph of pixel similarity based on the digital image frame as input. Using the GCN enables a given software agent to communicate or collaborate with other software agents that are not near the given software agent but are still related to a region of the object of interest.

[0014] In one implementation of the first aspect, the similarity of the pixels in the digital image frame includes top-k nearest neighbors in one of a feature space, a color space, or a distance space. The GCN enables a given software agent to communicate or collaborate with other software agents that are not near the given software agent but are still related to the region of the object of interest.

[0015] According to a second aspect of the present invention, an image segmentation method is provided. The method includes: a processor accessing a digital image frame depicting a scene containing an object of interest; the processor receiving an object selection indication of a subset of pixels of the digital image frame associated with the object of interest. The method further includes: the processor assigning a software agent to each pixel of the digital image frame, each software agent applying an iterative deep reinforcement learning process to the pixel to estimate the probability that the pixel is one of the pixels associated with the object of interest, and classifying whether the pixel is one of the pixels associated with the object of interest based on the estimated probability; wherein each iteration step of the iterative deep reinforcement learning process is executed according to a software agent-specific policy and maps a pixel state dataset to a corresponding action. Each pixel state dataset includes the color value of each pixel and the object selection indication, and each action includes setting the estimated probability to one of a predefined discrete value. The method further includes: after each iteration step, the processor receiving a classification result from each assigned software agent; the processor determining the object of interest after each iteration step based on the received classification result. The present invention enables image segmentation based on multi-agent deep reinforcement learning. Using multiple software agents allows the software agents to compute accurate estimated probabilities. For example, probability estimation errors can be avoided more effectively when pixel values ​​are occluded (e.g., in the boundary region between skin and hair).

[0016] In one implementation of the second aspect, the method further includes: each software agent communicating with at least some other software agents sharing the same policy to exchange information related to the pixel state dataset, and further classifying the pixels based on the exchanged information. This invention enables image segmentation based on multi-agent deep reinforcement learning. By using multiple software agents and other software agents related to the region of the object of interest to communicate or collaborate, the software agents are further able to compute more accurate estimated probabilities. For example, probability estimation errors can be avoided more effectively when pixel values ​​are occluded (e.g., the boundary region between skin and hair).

[0017] In one implementation of the second aspect, the pixel state dataset for a given iteration step also includes an estimated probability from the previous iteration step. Including the estimated probability from the previous iteration step enables the software agent to calculate a more accurate probability estimate.

[0018] In one implementation of the second aspect, the method further includes: each software agent adjusting the estimated probability of each iteration step according to a reward metric; the reward metric having a positive value indicates an increase in the estimated probability compared to the previous iteration step; or, the reward metric having a negative value indicates a decrease in the estimated probability compared to the previous iteration step. Adjusting the estimated probability according to the reward metric can make the current estimated probability exceed the previous estimated probability, thereby making the iterative deep reinforcement learning process more efficient.

[0019] In one implementation of the second aspect, the method further includes updating the object selection indicator after one or more iteration steps based on user feedback. Updating the object selection indicator based on user feedback such as user input can improve the image segmentation result in the next iteration step.

[0020] In one implementation of the second aspect, the method further includes: each software agent using an Advantageous Action Commentary (A2C) network when setting the estimated probabilities. The A2C network can be used as input to the pixel state dataset and to select the action to be taken according to the policy of each pixel, thereby forming a probability map.

[0021] In one implementation of the second aspect, the method further includes: each software agent performing maximum a posteriori probability (MAP) estimation on the estimated probability results of the A2C network. Performing MAP estimation on the estimated probability results of the A2C network can generate a final segmentation mask from the probability map generated by the A2C network.

[0022] In one implementation of the second aspect, the A2C network includes a graph convolutional network (GCN) used to facilitate communication between the software agents. Using the GCN enables the software agents to efficiently communicate or collaborate with other software agents related to the region of the object of interest, thereby allowing the software agents to compute more accurate estimated probabilities.

[0023] In one implementation of the second aspect, the GCN is further configured to take the pixel state dataset of the pixels and a graph of pixel similarity based on the digital image frame as input. Using the GCN enables a given software agent to communicate or collaborate with other software agents that are not near the given software agent but are still related to the region of the object of interest.

[0024] In one implementation of the second aspect, the similarity of the pixels in the digital image frame includes top-k nearest neighbors in one of a feature space, color space, or distance space. The GCN enables a given software agent to communicate or collaborate with other software agents that are not near the given software agent but are still related to the region of the object of interest.

[0025] According to a third aspect of the present invention, a computer program product is provided. The computer program product includes program code that, when executed on a computer, performs the method according to the second aspect. The present invention enables image segmentation based on multi-agent deep reinforcement learning. Using multiple software agents allows these agents to compute accurate estimated probabilities. For example, probability estimation errors can be avoided more effectively when pixel values ​​are occluded (e.g., in the boundary region between skin and hair).

[0026] Many features will become easier to understand and more readily apparent with reference to the following detailed description taken in conjunction with the accompanying drawings. Attached Figure Description

[0027] Exemplary embodiments are described in more detail below with reference to the accompanying drawings.

[0028] Figure 1 A block diagram of the device provided in an embodiment of the present invention is shown;

[0029] Figure 2 A diagram illustrating an overview of the end-to-end architecture provided by an exemplary embodiment;

[0030] Figure 3 A diagram illustrating an overview of a graph-based A2C network provided by an exemplary embodiment;

[0031] Figure 4 A flowchart of the method provided by an embodiment of the present invention is shown.

[0032] In the following text, the same reference numerals refer to the same or at least functionally equivalent features. Detailed Implementation

[0033] The following description is in conjunction with the accompanying drawings, which are an integral part of this invention and illustrate specific aspects of the invention through illustrative means. It will be understood that other aspects can be utilized and structural or logical changes can be made without departing from the scope of the invention. Therefore, the following detailed description is not intended to be limiting, and the scope of the invention is defined by the appended claims.

[0034] It is understood that the disclosure relating to the described method also applies to the apparatus or system corresponding to performing the method, and vice versa. For example, if specific method steps are described, the corresponding apparatus may include units that perform the described method steps, even if such units are not explicitly described or shown in the drawings. On the other hand, for example, if a specific device is described based on functional units, the corresponding method may include steps that perform the described functions, even if such steps are not explicitly described or shown in the drawings. Furthermore, it should be understood that features of the various exemplary aspects described herein can be combined with each other unless otherwise explicitly stated.

[0035] As will be discussed in detail below, the present invention provides a method that formulates interactive image segmentation as a sequential decision problem based on multi-agent reinforcement learning: each pixel is treated as an agent cooperating with another agent, sharing the same policy, and user interaction is iteratively introduced in each refinement step to enhance performance. At least some of the disclosed embodiments also include agents cooperating with each other via a graph convolutional network, the similarity of which is defined as a combination of appearance and distance to encode global contextual information. At least some of the disclosed embodiments also provide rewards, defined as positive rewards for improvement and negative rewards for decline at each step, so that new predictions can be better than previous predictions. Furthermore, a dominant action commenting algorithm can be used to train the policy.

[0036] Next, based on Figure 1 An exemplary embodiment of the image segmentation device 100 is described. Some features of the device are optional features that provide further advantages.

[0037] Figure 1 A block diagram of an image segmentation device 100 provided in an embodiment of the present invention is shown. In one embodiment, the device 100 may include... Figure 1The client device shown can be any type of device that is directly used by an end-user entity and is capable of communicating in a wireless network, such as a user equipment (UE). Such devices include, but are not limited to, smartphones, tablets, smartwatches, laptops, Internet of Things (IoT) devices, etc. While embodiments can be described based on client devices, this is by way of example only and not as a limitation.

[0038] Device 100 may include at least one processor or processing unit 102 and at least one memory 104 coupled to at least one processor 102, which may be used to implement the functions described in more detail below. Device 100 may also include a digital camera 106 capable of forming digital image files from captured images.

[0039] For example, at least one processor 102 may include one or more of various processing devices (e.g., coprocessors, microprocessors, controllers, digital signal processors (DSPs), processing circuitry with or without an accompanying DSP) or various other processing devices including integrated circuits (e.g., application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontroller units (MCUs), hardware accelerators, dedicated computer chips, etc.).

[0040] The memory 104 can be used to store computer programs, etc. The memory may include one or more volatile storage devices, one or more non-volatile storage devices, and / or a combination of one or more volatile and non-volatile storage devices. For example, the memory can be implemented as a magnetic storage device (e.g., hard disk drive, floppy disk, magnetic tape, etc.), an optical-magnetic storage device, and a semiconductor memory (e.g., mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash memory ROM, random access memory (RAM), etc.).

[0041] Digital camera 106 can be used to capture digital image frames depicting a scene containing an object of interest. Digital camera 106 communicates with processor 102 and / or other components of device 100 and is used to capture digital images, video, and / or other graphic media. Digital camera 106 may include one or more image sensors, including but not limited to complementary metal-oxide semiconductor (CMOS) image sensors, charge-coupled device (CCD) image sensors, etc.

[0042] Device 100 is used to access the digital image frame. The digital image frame depicts a scene containing an object of interest (e.g., a person). The digital image frame may be a still image. Alternatively, the digital image frame may be an image frame from a video sequence. The digital image frame may be captured using digital camera 106. Alternatively, the digital image frame may be captured using a digital camera external to device 100. The digital image frame may be stored in at least one memory 104. Alternatively, the digital image frame may be stored in a memory or storage device external to device 100, such as a remote server or cloud-based storage.

[0043] Device 100 is also configured to receive an object selection instruction. The object selection instruction indicates a portion of pixels in the digital image frame associated with the object of interest. That is, the object selection instruction is provided (e.g., by a user of device 100) to select at least a portion of the object of interest (e.g., a face, hand, or stomach). The object selection instruction may include input by the user using a finger, stylus, etc., on the touchscreen of device 100. Figure 1 (not shown in the image) or one or more doodles or strokes or any other gestures provided on it.

[0044] The device 100 is also configured to assign a software agent to each pixel of the digital image frame. Each software agent is configured to apply an iterative deep reinforcement learning process to the pixel to estimate the probability that the pixel is one of the pixels associated with the object of interest, and to classify whether the pixel is one of the pixels associated with the object of interest based on the estimated probability.

[0045] Figure 2 Figure 200 illustrates an overview of an end-to-end architecture provided by an exemplary embodiment. For example, for a dataset containing N pixels x = (x1, ..., x2)... N For an image x, the software agent (or intelligent agent) can be defined as follows:

[0046] Intelligent agent: Each pixel x i It can be viewed as an intelligent agent, whose strategy is in, This indicates action 203. This indicates state 201.

[0047] Each iteration step of the iterative deep reinforcement learning process is executed according to a software agent-specific policy, mapping the pixel state dataset to the corresponding action. For example, for the above process containing N pixels x = (x1, ..., x... N The image x, pixel state dataset (or state) 201, and action 203 can be defined as follows:

[0048] Status: In one embodiment, the image frame pixel values ​​x can be concatenated. i Estimated probabilities (or predictions) prior to 2012 2013 and object selection indicator (or user intent) diagram 2011 defines state 201, that is... In other words, a longer vector is generated for each agent by concatenating multiple vectors, namely the image frame pixel value vector (e.g., RGB values), the previous prediction vector (probability), and the user intent vector or scalar (e.g., 0 or 1).

[0049] Action: The set A of actions 203 = {A k Let k = 1, ..., K contain K actions that enable the agent to adjust the probability to one of K discrete values, for example, within the interval 0 to 1. In one embodiment, the value of K may be uniformly distributed within the interval.

[0050] Each pixel state dataset includes the color value of each pixel and the object selection indicator. In at least part of the disclosed exemplary embodiments, the pixel state dataset for a given iteration step may also include an estimated probability from the previous iteration step. Furthermore, each action includes setting the estimated probability to one of predefined discrete values. For example, as... Figure 2 As shown in Figure 200, the pixel state dataset 201 for each iteration step (or time step) t may include an object selection indicator (or user intent) 2011, a pixel color value 2012, and an estimated probability 2013 for the previous iteration step (or time step) t-1. In one embodiment, the object selection indicator 2011 may include a binary mask obtained by recording user input 207, indicating the location of the expected object. That is, user input 207 (e.g., a pen stroke or finger tap on an image) may be recorded as "1", etc., indicating a portion of the expected object. In another embodiment, the object selection indicator 2011 may be initialized with the probability of an object detector (e.g., a face detector or a human detector) specifying the desired object category.

[0051] The device 100 is further configured to, after each iteration step, receive from each assigned software agent a classification result regarding whether the pixel is one of the pixels associated with the object of interest. The device 100 is also configured to, based on the received classification result, determine the object of interest after each iteration step.

[0052] In one embodiment of device 100, each software agent can also communicate with at least some other software agents sharing the same strategy to exchange information related to the pixel state dataset and further classify the pixels based on the exchanged information. In other words, in at least some embodiments, each agent can observe the states of similar agents through communication, which in turn enables the agents to collaborate when performing pixel classification. Using multiple software agents that can collaborate with each other allows the software agents to calculate more accurate estimated probabilities, thereby performing more accurate pixel classification. For example, probability estimation errors can be avoided more effectively when pixel values ​​are occluded (e.g., the boundary region between skin and hair).

[0053] In one embodiment of device 100, each software agent can also be used to adjust the estimated probability of each iteration step according to a reward metric. For example, the reward metric can have a positive value, indicating an improvement in the estimated probability compared to the previous iteration step; or, the reward metric can have a negative value, indicating a decrease in the estimated probability compared to the previous iteration step. The "reward metric" or "reward" can include signals passed from the environment of the deep reinforcement learning process to the agent, etc. For example, the reward metric can include a number for each iteration step or time step. The agent's objective can be defined as maximizing the total amount of reward it receives. More specifically, this may involve maximizing long-term cumulative rewards rather than immediate rewards. Continuing with the above, containing N pixels x = (x1, ..., x... N The image x of ) Figure 2 As shown in the example, the reward 2062 can be defined as L in a given cross-entropy loss (L in Equations 1 and 2 below). i In the case of 2063, the relative improvement from the previous step to the current step, the cross-entropy loss 2063 measures the amount of misclassification:

[0054]

[0055]

[0056] Among them, y i Represents pixel x i The truth label is 2061, and the total reward for the interaction sequence is:

[0057]

[0058] Here, γ represents the discount factor, with a value between 0 and 1. In this paper, the term "truth value" refers to the act of checking the accuracy of machine learning results against the real world. In other words, the term represents a reality check of the machine learning algorithm. In one embodiment, the truth value label 2061 can be 0 (background) or 1 (object), etc., or vice versa.

[0059] In an embodiment of device 100, each software agent may also be used to employ an advantage action commenting algorithm (A2C) network when setting the estimated probabilities. For example, the A2C network may include a graph convolutional network (GCN) used to facilitate communication between the software agents. When the A2C network includes a GCN, the A2C network is a graph-based A2C network. Figure 2 An example of such a graph-based A2C network is shown. Each software agent can also be used to perform maximum a posteriori (MAP) probability estimation on the estimated probability results of the A2C network. Continuing from the above... Figure 2 As shown in the example, the A2C network 202 can take state 201 as input and predict action 203 based on the policy to be taken for each pixel, thereby forming a probability map 204 indicating the probability of the expected object. In some embodiments, a final segmentation mask 205 can be generated by performing a MAP.

[0060] In one embodiment of device 100, device 100 can also be used to update the object selection indication after one or more said iteration steps based on user feedback. For example, based on the quality of the final segmentation mask 205, the user can provide further input 207, which in turn causes the updated intent to constitute a new state to improve segmentation in the next time step.

[0061] The GCN can also be used to take the pixel state dataset of the pixels and a graph based on the similarity of the pixels in the digital image frame as input. For example, the similarity of the pixels in the digital image frame includes the top-k nearest neighbors (i.e., the k nearest neighbors) in a feature space, color space, or distance space. The feature space can include handcrafted features or features extracted by a neural network. The color space can include RGB (red, green, blue) space, HSV (hue, saturation, brightness) space, etc. The distance space can include Euclidean distance or geodesic distance, etc.

[0062] Figure 3 An exemplary embodiment of an A2C network is shown (e.g., Figure 2Figure 300 shows an overview of the graph-based A2C network 202. A standard graph convolutional network (GCN) 303 can be used, taking state 301 and a graph 302 defined according to pixel similarity as input. As mentioned above, in some embodiments, similarity can be defined as the top-k nearest neighbors in a feature space, color space, or distance space. The GCN 303 transforms state 301 into features fed into two sub-networks: a policy head 304 (including convolutional blocks 3041-3043, an upsampling layer 3044, and a softmax layer 3045) and a value head 305 (including convolutional blocks 3051-3053 and an upsampling layer 3054). The convolutional blocks (3041-3043, 3051-3053) can include convolutional layers, non-linear activation layers, and batch normalization layers. The upsampling layers (3044, 3054) may include bilinear upsampling layers or transposed convolutional layers, which expand the feature map size due to downsampling or pooling operations. The softmax layer 3045 can output a vector representing the probability distribution of a list of potential outcomes. In other words, the policy head 304 predicts the likelihood of adjustments to the previous probability map at each scale. The value head 305 estimates the value of the current state 301. θ g θ p and θ v These are the model parameters for GCN 303, Policy Head 304, and Value Head 305, respectively. In this paper, the term "model parameter" refers to the underlying network weights that transform the input and produce the output.

[0063] For example, during training, θ p and θ v The gradient can be calculated using the following equation:

[0064]

[0065]

[0066] in, r represents the average excess reward of all pixels. i These two heads can be trained together, for example, in an end-to-end manner.

[0067] Figure 4 A flowchart 400 of the image segmentation method provided in an embodiment of the present invention is shown.

[0068] In operation 401, the processor accesses a digital image frame depicting a scene containing an object of interest.

[0069] In operation 402, the processor receives an object selection indication. The object selection indication indicates a subset of pixels in the digital image frame associated with the object of interest. The object selection indication may be updated after one or more iterations based on user feedback.

[0070] In operation 403, the processor assigns a software agent to each pixel of the digital image frame.

[0071] In operation 404, each software agent applies an iterative deep reinforcement learning process to the pixel to estimate the probability that the pixel is one of the pixels associated with the object of interest. Furthermore, each software agent classifies whether the pixel is one of the pixels associated with the object of interest based on the estimated probability. Each iteration step of the iterative deep reinforcement learning process is executed according to a software agent-specific policy and maps a pixel state dataset to a corresponding action. Each pixel state dataset includes the color value of each pixel and the object selection indication, and each action includes setting the estimated probability to one of a predefined discrete value. Additionally, each software agent can selectively communicate with at least some other software agents sharing the same policy to exchange information related to the pixel state dataset and further classify the pixels based on the exchanged information. Furthermore, each software agent can adjust the estimated probability at each iteration step according to a reward metric; the reward metric has a positive value indicating an improvement in the estimated probability compared to the previous iteration step; or the reward metric has a negative value indicating a decrease in the estimated probability compared to the previous iteration step.

[0072] In operation 405, after each iteration step, the processor receives from each assigned software agent the result of classifying whether the pixel is one of the pixels associated with the object of interest.

[0073] In operation 406, the processor determines the object of interest after each iteration step based on the received classification result. If it is determined in operation 407 that the object of interest has been sufficiently determined, method 400 can proceed to operation 408 and terminate. Otherwise, method 400 returns to operation 404 to begin another iteration step of the deep reinforcement learning process.

[0074] Method 400 can be executed by device 100. For example, operations 401-408 can be executed by at least one processor 102 and memory 104. Other features of method 400 are directly derived from the functions and parameters of device 100, and therefore will not be described further here. Method 400 can be executed by a computer program.

[0075] The functions described herein may be performed at least in part by one or more computer program product components (e.g., software components). Alternatively or additionally, the functions described herein may be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that may be used include, but are not limited to, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), and graphics processing units (GPUs).

[0076] Any ranges or device values ​​given herein can be extended or changed without loss of the desired effect. Furthermore, unless expressly prohibited, any embodiment can be combined with other embodiments.

[0077] Although the subject matter of the invention has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as examples of implementing the claims, and other equivalent features and actions are intended to be included within the scope of the claims.

[0078] It should be understood that the above advantages and benefits may relate to one embodiment or several embodiments. The embodiments are not limited to embodiments that solve any or all of the described problems, nor are they limited to embodiments that have any or all of the described advantages and benefits. Furthermore, it should be understood that a reference to "one" item may refer to one or more of these items.

[0079] The steps of the methods described herein can be performed in any suitable order, or simultaneously where appropriate. Furthermore, individual blocks can be removed from any of the methods without departing from the spirit and scope of the subject matter described herein. Aspects of any of the above embodiments can be combined with aspects of any other described embodiments to form further embodiments without loss of the desired effects.

[0080] The term “comprising” is used herein to mean including identified methods, blocks or elements, but such blocks or elements do not include an exclusive list, and methods or apparatus may include additional blocks or elements.

[0081] It should be understood that the above description is provided by way of example only, and various modifications can be made by those skilled in the art. The foregoing specification, examples, and data provide a complete description of the structure and application of exemplary embodiments. Although the embodiments have been described above with a degree of specificity or in combination with one or more individual embodiments, those skilled in the art can make various modifications to the disclosed embodiments without departing from the scope of this specification.

Claims

1. An image segmentation device (100), characterized in that, The image segmentation device (100) is used for: Access digital image frames depicting a scene containing objects of interest; Receive an object selection instruction that indicates a portion of pixels of the digital image frame associated with the object of interest; A software agent is assigned to each pixel of the digital image frame. Each software agent applies an iterative deep reinforcement learning process to the pixel to estimate the probability that the pixel is one of the pixels associated with the object of interest, and classifies whether the pixel is one of the pixels associated with the object of interest based on the estimated probability. Each iteration step of the iterative deep reinforcement learning process is executed according to a software agent-specific policy and maps a pixel state dataset to a corresponding action. Each pixel state dataset includes the color value of each pixel and the object selection indication, and each action includes setting the estimated probability to one of a predefined discrete value. After each iteration step, the classification results are received from each assigned software agent; Based on the received classification results, the object of interest is determined after each iteration step; Each software agent is also used to communicate with at least some other software agents sharing the same policy to exchange information related to the pixel state dataset and to further classify the pixels based on the exchanged information; the shared policy includes shared pixel values, previous estimated probabilities, object selection instructions, and actions.

2. The device (100) according to claim 1, characterized in that, The pixel state dataset for a given iteration step also includes the estimated probability of the previous iteration step.

3. The device (100) according to any one of claims 1 to 2, characterized in that, Each software agent is also used to adjust the estimated probability of each iteration step according to a reward metric; the reward metric having a positive value indicates that the estimated probability is higher than that of the previous iteration step; or, the reward metric having a negative value indicates that the estimated probability is lower than that of the previous iteration step.

4. The device (100) according to any one of claims 1 to 3, characterized in that, The device (100) is also used to update the object selection indication after one or more of the iteration steps based on user feedback.

5. The device (100) according to any one of claims 1 to 4, characterized in that, Each software agent is also used to employ the Advantage Action Commentary (A2C) network when setting the estimated probabilities.

6. The device (100) according to claim 5, characterized in that, Each software agent is also used to perform maximum a posteriori probability estimation on the estimated probability results of the A2C network.

7. The device (100) according to claim 5 or 6, characterized in that, The A2C network includes a graph convolutional network (GCN), which is used to facilitate communication between the software agents.

8. The device (100) according to claim 7, characterized in that, The GCN is also used to take the pixel state dataset of the pixel and a graph of the similarity of the pixel based on the digital image frame as input.

9. The device (100) according to claim 8, characterized in that, The similarity of the pixels in the digital image frame includes the top-k nearest neighbors in one of the feature space, color space, or distance space.

10. An image segmentation method (400), characterized in that, The image segmentation method (400) includes: The processor accesses (401) a digital image frame depicting a scene containing objects of interest; The processor receives (402) an object selection instruction indicating a portion of pixels of the digital image frame associated with the object of interest; The processor assigns (403) a software agent to each pixel of the digital image frame. Each software agent applies (404) an iterative deep reinforcement learning process to the pixel to estimate the probability that the pixel is one of the pixels associated with the object of interest, and classifies whether the pixel is one of the pixels associated with the object of interest based on the estimated probability. Each iteration step of the iterative deep reinforcement learning process is executed according to a software agent-specific policy and maps a pixel state dataset to a corresponding action. Each pixel state dataset includes the color value of each pixel and the object selection indication. Each action includes setting the estimated probability to one of a predefined discrete value. After each iteration step, the processor receives (405) classification results from each assigned software agent; The processor determines (406) the object of interest after each iteration step based on the received classification results; The method (400) further includes: each software agent communicating with at least some other software agents sharing the same policy to exchange information related to the pixel state dataset, and further classifying the pixels based on the exchanged information; the policy includes pixel values, previously estimated probabilities, and object selection indications; the shared policy includes shared pixel values, previously estimated probabilities, object selection indications, and actions.

11. The method (400) according to claim 10, characterized in that, The pixel state dataset for a given iteration step also includes the estimated probability of the previous iteration step.

12. The method (400) according to any one of claims 10 to 11, characterized in that, The method (400) further includes: each software agent adjusting the estimated probability of each iteration step according to a reward metric; the reward metric having a positive value indicates that the estimated probability is higher than that of the previous iteration step; or, the reward metric having a negative value indicates that the estimated probability is lower than that of the previous iteration step.

13. A computer program product, characterized in that, The computer program product includes program code that, when executed on a computer, performs the method according to any one of claims 10 to 12.

Images