Method and electronic device for recognizing object based on mask update
The mask update-based object recognition method using an MRF model enhances object recognition by iteratively refining mask parameters, addressing the limitations of conventional methods in resource-constrained environments and low-shot learning scenarios.
Patent Information
- Application Number
- KR1020220162470
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-11-29
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2042-11-29
AI Technical Summary
Conventional object recognition methods require sophisticated AI models and large training datasets, often leading to incorrect results, especially in resource-constrained environments or low-shot learning scenarios, and there is a need for improving recognition results without additional training data or model modifications.
A mask update-based object recognition method using a Markov Random Field (MRF) model iteratively refines object recognition by distinguishing foreground and background, updating mask heads based on objective functions, and enhancing boundary matching without modifying the recognition model.
Improves object recognition results through faster convergence by updating mask parameters, achieving clearer object boundaries and reducing errors in resource-constrained environments.
Smart Images

Figure 112022127741684-PAT00078_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a mask update-based object recognition method and an electronic device. Background Technology
[0002] Instance recognition is an essential field in visual intelligence research for understanding images in a manner similar to humans. The object recognition task involves identifying the pixel-level location of each individual object appearing in an input image. Due to its importance, the results of object recognition can be directly applied to technologies such as autonomous vehicles and robots, or used as a preprocessing step to understand images in detail, much like a human.
[0003] Meanwhile, conventional object recognition methods require an excellent artificial intelligence model or a large amount of training data to obtain high-quality object recognition results, and even when these conditions are met, there are often cases where incorrect results are produced depending on the difficulty of the object recognition task.
[0004] In addition, there is a problem in obtaining high-quality object recognition results under restrictive conditions such as embedded environments or in training environments where it is difficult to obtain large amounts of training data. Prior art literature
[0005] Published Patent Application No. 10-2018-0060977 (June 7, 2018) The problem to be solved
[0006] The problem to be solved by the present invention is to provide a mask update-based object recognition method and an electronic device that iteratively improves object recognition results by utilizing a Markov Random Field (MRF) model when performing object recognition tests using an artificial intelligence model such as a deep artificial neural network.
[0007] One embodiment of the present invention can derive improved object recognition results by iteratively improving the initial object recognition results during the test phase, without the need to modify the object recognition model, such as by preparing a large amount of additional training datasets, retraining, or fine-tuning, in order to improve performance during this process.
[0008] However, the problems that the present invention aims to solve are not limited to those described above, and other problems may exist. means of solving the problem
[0009] A mask update-based object recognition method according to a first aspect of the present invention for solving the above-described problem comprises: a step of generating first result information in which the foreground and background are distinguished for a predetermined input image; a step of generating second result information in which the boundary portion of an object is matched with the object recognition result for the input image; a step of calculating an objective function to be applied to each mask head based on the first and second result information and parameter information for the same; a step of updating the entire mask head based on the objective function; and a step of performing object recognition based on the updated mask.
[0010] In some embodiments of the present invention, the step of generating first result information in which foreground and background are distinguished for a predetermined input image may include: a step of calculating a first similarity between a feature vector obtained by passing through the first layer of a mask head at each pixel of the input image and a foreground prototype; a step of calculating a second similarity between the feature vector and a background prototype; and a step of generating the first result information by applying the initial object recognition result at each pixel of the input image to the first and second similarities.
[0011] In some embodiments of the present invention, the step of calculating a second similarity between the feature vector and the background prototype may include: a step of calculating a foreground error for pixels located around the perimeter of the bounding box of each object; a step of extracting a set of pixels for a predetermined number in order of the calculated foreground error; and a step of calculating a background prototype based on the feature vector for the extracted set of pixels.
[0012] In some embodiments of the present invention, the step of generating the first result information by applying the initial object recognition result at each pixel of the input image to the first and second similarities may further include the step of applying the set information of pixels in which the minimum values of foreground error and background error exceed a certain threshold to the calculated first similarity.
[0013] In some embodiments of the present invention, the step of generating second result information that matches the object recognition result for the input image with the boundary portion of the object may include: generating a set of neighbor pixels for a predetermined pixel that enters as input to the mask head; calculating a third similarity between the predetermined pixel and neighbor pixels within the generated set; and matching the object recognition result with the boundary portion based on whether the calculated third similarity exceeds a preset threshold.
[0014] Additionally, an electronic device for object recognition according to a second aspect of the present invention comprises: a communication unit for receiving a predetermined image; a memory storing a program for updating a mask based on a predetermined objective function for the input predetermined image; and a processor that, by executing the program stored in the memory, generates first result information in which the foreground and background are distinguished for the input image, generates second result information in which the object recognition result for the input image matches the boundary portion of the object, calculates an objective function to be applied to each mask head based on the first and second result information and parameter information for the same, updates the entire mask head, and performs object recognition based on the updated mask.
[0015] In some embodiments of the present invention, the processor may calculate a first similarity between a feature vector obtained by passing through the first layer of a mask head at each pixel of the input image and a foreground prototype, calculate a second similarity between the feature vector and a background prototype, and then apply the initial object recognition result at each pixel of the input image to the first and second similarities to generate the first result information.
[0016] In some embodiments of the present invention, the processor may calculate a foreground error for pixels located around the perimeter of the bounding box of each object included in the image, extract a set of pixels for a predetermined number in order of the calculated foreground error, and then calculate a background prototype based on a feature vector for the extracted set of pixels.
[0017] In some embodiments of the present invention, the processor may apply set information of pixels in which the minimum values of foreground error and background error exceed a certain threshold to the calculated first similarity.
[0018] In some embodiments of the present invention, the processor generates a set of neighbor pixels for a predetermined pixel that enters as input to the mask head, calculates a third feature similarity between the predetermined pixel and neighbor pixels within the generated set, and then matches the boundary portion with the object recognition result based on whether the calculated third similarity exceeds a preset threshold.
[0019] Additionally, a mask update-based object recognition method according to a third aspect of the present invention comprises the steps of: receiving an image containing a predetermined object to be recognized; passing the input image through a backbone network to extract a first feature map; passing the first feature map through a feature pyramid network to extract a second feature map; updating a mask for object recognition based on a controller head corresponding to the second feature map; and performing object recognition for the second feature map based on the updated mask. In this case, the step of updating the mask for object recognition based on a controller head included in the second feature map involves updating the mask based on an objective function that produces first result information in which the foreground and background are distinguished for the input image, and second result information in which the boundary portion of the object is matched with the object recognition result for the input image.
[0020] A computer program according to another aspect of the present invention for solving the above-described problem executes a mask update-based object recognition method and is stored in a computer-readable recording medium.
[0021] Other specific details of the present invention are included in the detailed description and drawings. Effects of the invention
[0022] According to one embodiment of the present invention, an improved object recognition result can be generated by utilizing a feature map-based MRF model, which is an intermediate result of an object recognition model. Furthermore, since the object recognition result is improved through parameter updates of the mask head rather than through the improvement of the object recognition result itself, there is an advantage in expecting a faster convergence speed.
[0023] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing
[0024] FIG. 1 is a drawing for explaining an electronic device (100) according to one embodiment of the present invention. FIG. 2 is a flowchart of an object recognition method according to one embodiment of the present invention. FIG. 3 is a drawing for explaining an example of an object recognition method according to an embodiment of the present invention. FIG. 4 is a flowchart of an object recognition method in another embodiment of the present invention. FIG. 5 is a diagram illustrating an application example of an object recognition model in one embodiment of the present invention. Figure 6a is a diagram illustrating the MRF-based test unit optimization process at the feature map level in the present invention. Figure 6b is a diagram showing the visualization results for the input image and the input feature map of the mask head. FIG. 6c is a drawing illustrating the result of object recognition applied to an embodiment of the present invention. Figure 7a is a diagram illustrating the results of object recognition based on low-shot learning. FIG. 7b is a diagram illustrating the result of object recognition applied to an embodiment of the present invention. Specific details for implementing the invention
[0025] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined only by the scope of the claims.
[0026] The terms used in this specification are for describing embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more. Although terms such as "first," "second," etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, the first component mentioned below may be the second component within the technical scope of the invention.
[0027] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.
[0028] Hereinafter, to aid the understanding of those skilled in the art, the background of the present invention will be explained, and then an embodiment of the present invention will be described in detail.
[0029] Existing approaches for performing object recognition tasks are as follows.
[0030] First, a sophisticated AI model for object recognition and a large amount of training dataset to use when training the AI model Prepare the training dataset. At this time, Each element of is an image Wow, written by a human Object recognition results for pair of It consists of.
[0031] A predefined set of recognition target classes Regarding, object recognition results is an image Appearing in Each of the objects belongs to a class ( Pixel-level recognition information regarding the location within the image( Pair of ) ( It consists of ).
[0032] In the training phase, for the initial AI model, the training dataset Each image of With as input, the object recognition result output by the model is a human-written object recognition result The AI model is repeatedly updated to produce results close to the desired outcome. Using the AI model trained in this way, an object recognition task is performed on an arbitrary image during the actual test phase.
[0033] Conventional technology requires sophisticated artificial intelligence models and large training datasets to obtain high-quality object recognition results. However, in conventional technology, the difficulty of the object recognition task itself is high, which can often result in incorrect object recognition results.
[0034] Furthermore, it may be difficult to obtain AI models or training datasets of the required standard due to issues such as time and cost. For example, on hardware with constraints, such as embedded hardware, only small-sized object recognition models may be usable. In this case, it may be difficult to achieve high-quality object recognition results due to the limitations of the AI model itself.
[0035] As another example, generating training datasets is inherently labor-intensive, making it difficult to create datasets of the required standard. Representative examples include low-shot learning, which trains on a very small number of examples to recognize objects of a new class that the AI model has not previously learned, and weakly-supervised learning, which uses training datasets that contain noise or provide only limited information. These methods inevitably result in lower object recognition performance.
[0036] In situations where it is inevitably difficult to obtain high-quality object recognition results, it is necessary to improve the results by introducing additional improvement processes during the testing phase.
[0037] Hereinafter, an electronic device (100) for object recognition according to an embodiment of the present invention will be described with reference to FIG. 1.
[0038] FIG. 1 is a drawing for explaining an electronic device (100) according to one embodiment of the present invention.
[0039] An electronic device (100) according to one embodiment of the present invention includes an input unit (110), a communication unit (120), a display unit (130), a memory (140), and a processor (150).
[0040] The input unit (110) generates input data in response to user input of the electronic device (100). User input may include user input regarding data that the electronic device (100) intends to process. The input unit (110) includes at least one input means. The input unit (110) may include a keyboard, a key pad, a dome switch, a touch panel, a touch key, a mouse, a menu button, etc.
[0041] The communication unit (120) transmits and receives data between internal components or performs communication with external devices such as external servers. That is, the communication unit (120) can receive a predetermined image from a camera, an external server, or a device, or transmit and receive other necessary data. Such a communication unit (120) may include both a wired communication module and a wireless communication module. The wired communication module may be implemented as a power line communication device, a telephone line communication device, a cable home (MoCA), Ethernet, IEEE1294, an integrated wired home network, and an RS-485 control device. Additionally, the wireless communication module may be composed of a module for implementing functions such as WLAN (wireless LAN), Bluetooth, HDR WPAN, UWB, ZigBee, Impulse Radio, 60GHz WPAN, Binary-CDMA, wireless USB technology and wireless HDMI technology, as well as 5G (5th generation communication), LTE-A (long term evolution-advanced), LTE (long term evolution), and Wi-Fi (wireless fidelity).
[0042] The display unit (130) displays display data according to the operation of the electronic device (100). The display unit (130) includes a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a micro electro mechanical systems (MEMS) display, and an electronic paper display. The display unit (130) can be combined with the input unit (110) to be implemented as a touch screen.
[0043] The memory (140) stores programs for updating a mask based on a predetermined objective function for a given input image. Here, the memory (140) is a general term for non-volatile storage devices and volatile storage devices that retain stored information even when power is not supplied. For example, the memory (140) may include NAND flash memory such as compact flash (CF) cards, SD (secure digital) cards, memory sticks, solid-state drives (SSDs), and micro SD cards, magnetic computer memory devices such as hard disk drives (HDDs), and optical disc drives such as CD-ROMs and DVD-ROMs.
[0044] The processor (150) can execute software, such as a program, to control at least one other component (e.g., hardware or software component) of the client device (100) and can perform various data processing or operations.
[0045] Hereinafter, with reference to FIGS. 2 to 5, a mask update-based object recognition method performed by an electronic device (100) according to an embodiment of the present invention will be described.
[0046] FIG. 2 is a flowchart of an object recognition method according to an embodiment of the present invention. FIG. 3 is a diagram illustrating an example of an object recognition method according to an embodiment of the present invention.
[0047] One embodiment of the present invention utilizes test-time optimization, which iteratively refines object recognition results using a Markov Random Field (MRF) model to more clearly distinguish the foreground and background of an image. Here, test-time optimization refers to a process for matching the boundary portions of the corresponding objects without modifying the object recognition model, such as through retraining or fine-tuning.
[0048] Specifically, one embodiment of the present invention defines the following objective function as Equation 1 for the optimization of the test phase.
[0049] [Equation 1]
[0050]
[0051] At this time, in Equation 1 class is two detailed terms and It is a parameter for balancing. Objective function The parameters of the mask head It is used to update.
[0052] Specifically, one embodiment of the present invention first provides first result information in which the foreground and background are distinguished for a predetermined input image ( Generates ) (S210).
[0053] [Equation 2]
[0054]
[0055] In order to generate the first result information in Equation 2, one embodiment of the present invention may calculate the first similarity and the second similarity, respectively, and generate the first result information by applying the initial object recognition result at each pixel of the image input to the first and second similarities.
[0056] Here, the first similarity refers to the similarity between the feature vector obtained by passing through the first layer of the mask head at each pixel of the input image and the foreground prototype, and the second similarity refers to the similarity between the feature vector and the background prototype.
[0057] If we examine this based on Equation 2, in Equation 2 is an image Each pixel of, represents a parameter for balancing, represents the initial object recognition result.
[0058] is pixels Feature vector obtained by passing through the first layer of the mask head Wow, foreground prototype It means the first degree of similarity between them.
[0059] At this time, the foreground prototype can be represented as Equation 3.
[0060] [Equation 3]
[0061]
[0062] In addition, the first similarity is calculated as in Equation 5 using a similarity function as in Equation 4.
[0063] [Equation 4]
[0064]
[0065] [Equation 5]
[0066]
[0067] is pixels Feature vector obtained by passing through the first layer of the mask head and background prototype It is the second similarity with, and the second similarity can be expressed as Equation 6.
[0068] [Equation 6]
[0069]
[0070] At this time, the background prototype in Equation 6 It can be calculated as follows. First, the bounding box of each object Assuming that background pixels exist on the perimeter (side), the bounding box Pixels located around the perimeter The foreground error for is calculated using the following Equation 7.
[0071] [Equation 7]
[0072]
[0073] Next, a set of pixels for a predetermined number is extracted in order of highest calculated foreground error, and a background prototype is calculated based on the feature vectors for the extracted pixel set.
[0074] In one embodiment, a set of the top 5 pixels with high foreground error When that is the case, the background prototype can be calculated as in Equation 8.
[0075] [Equation 8]
[0076]
[0077] Also, in Equation 2 represents the set of pixels where both the foreground error in Equation 7 and the background error in Equation 9 have high values, and the minimum value of the foreground and background errors This certain threshold It consists of a set of pixels exceeding . The set information of the pixels can be applied to the first similarity to finally generate the first result information.
[0078] [Equation 9]
[0079]
[0080] Next, second result information that matches the object recognition result for the input image with the boundary portion of the object ( Creates (S220).
[0081] The second result information This is intended to obtain a precise recognition result by ensuring that the object recognition result matches the boundary of the object, and can be expressed as shown in the following Equation 10.
[0082] [Equation 10]
[0083]
[0084] In one embodiment, the second result information can be generated by generating a set of neighbor pixels for a predetermined pixel that enters as input to a mask head, calculating a third similarity between a predetermined pixel and neighbor pixels within the generated set, and then matching the boundary portion with the object recognition result based on whether the third similarity exceeds a preset threshold.
[0085] To explain this with reference to Equation 10 above, in Equation 10 is a certain pixel 8 neighbor pixels for ( It represents the set of ). And is two pixels and It refers to a weight value determined according to the feature similarity (third similarity) between them. At this time, the third similarity is calculated as shown in Equation 11 below, and the weight value can be determined according to Equation 12.
[0086] [Equation 11]
[0087]
[0088] [Equation 12]
[0089] ( Pre-set threshold (e.g., 0.5))
[0090]
[0091] At this time, is the pixel entering as input to the mask head It means the feature vector of.
[0092] Next, first and second result information and parameter information for this ( Objective function according to Equation 1 for application to each mask head based on ) Calculates (S230).
[0093] Subsequently, the entire mask head is updated based on the objective function (S240). That is, based on the defined objective function, the initial mask head ( Starting from ), iteratively update the mask head.
[0094] The entire mask heads obtained by repeating this process are ensembled to perform object recognition based on the updated masks (S250). That is, the entirety of multiple masks By considering all of them together to update the mask and performing object recognition based on the mask, the object recognition results can be improved.
[0095] FIG. 4 is a flowchart of an object recognition method in another embodiment of the present invention. FIG. 5 is a diagram illustrating an application example of an object recognition model in one embodiment of the present invention.
[0096] An object recognition method according to another embodiment of the present invention can perform object recognition by applying it to the object recognition model shown in FIG. 5.
[0097] First, when an image containing a predetermined target object for recognition is input (S410), the input image is passed through a backbone network to extract a first feature map (S420). In one embodiment, a ResNet-50-based backbone network, etc., may be applied in step S420. The first feature map extracted in step S420 corresponds to C3, C4, and C5 in FIG. 5. The first feature maps, C3, C4, and C5, represent feature maps of different resolutions that can be extracted from the backbone network.
[0098] Next, the first feature map is passed through a Feature Pyramid Network (FPN) to extract a second feature map (S430). The second feature map extracted in step S430 corresponds to P3 to P7 in FIG. 5.
[0099] In one embodiment, the second feature map P5 is in the first feature map C5 It can be extracted by applying a convolution layer. In addition, the second feature maps P3 and P4 are respectively to the first feature maps C3 and C4. It can be extracted by applying the results of upsampling the convolution layer and the second feature maps P4 and P5. In addition, the second feature maps P6 and P7 are respectively to the second feature maps P5 and P6. It can be extracted by applying a convolution layer.
[0100] Next, a mask for object recognition is updated based on a controller head corresponding to the second feature map (S440), and object recognition for the second feature map is performed based on the updated mask (S450).
[0101] That is, the controller head for object recognition, which is added to each of the second feature maps P3 to P7 obtained in Fig. 5, can be used to set the parameters of the mask head for object recognition. Through these parameter settings, the mask can be updated.
[0102] In addition, the result of upsampling the second feature maps P4 and P5 and summing them to P3 is passed through multiple convolution layers and instance-wise mask heads to output the result of recognizing each object appearing in the image.
[0103] Meanwhile, in step S240 of updating the mask in one embodiment of the present invention, as described in FIGS. 2 and 3, the mask can be updated based on an objective function that produces first result information in which the foreground and background are distinguished for an input image, and second result information in which the boundary portion of an object is matched with the object recognition result for the input image.
[0104] Meanwhile, in the above description, steps S210 to S450 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Also, some steps may be omitted as necessary, and the order between steps may be changed. Furthermore, even if other omitted details are omitted, the details described in FIG. 1 and the details described in FIG. 2 to FIG. 5 may be mutually applicable.
[0105] Hereinafter, the effects of applying the present invention will be explained with reference to FIGS. 6a to 7b.
[0106] FIG. 6a is a diagram illustrating the MRF-based test unit optimization process at the feature map level in the present invention. FIG. 6b is a diagram illustrating the visualization results for the input feature map of the input image and the mask head. FIG. 6c is a diagram illustrating the object recognition results applying an embodiment of the present invention.
[0107] One embodiment of the present invention, as shown in FIG. 6a, uses MRF at the feature map level rather than the pixel level of the image to obtain the parameters of the mask head It repeatedly optimizes. The method according to the present invention has two advantages.
[0108] First, the feature map, which is an intermediate result of the object recognition model, clearly preserves the boundaries of each object, so the feature map-based MRF model can yield improved object recognition results compared to the pixel-based MRF model. Figure 6b visualizes the input image and the input feature map of the mask head. In the original image, the boundaries between different objects often become indistinct due to changes in color, brightness, saturation, etc., but in the feature map, it can be seen that the boundaries of each object appear clearly even in the indistinct boundary areas.
[0109] Secondly, since the object recognition result is improved by updating the parameters of the mask head rather than directly improving the object recognition result itself, it has the effect of accelerating the convergence speed. Figure 6c illustrates the object recognition result to which the present invention is applied. In the initial result (iter 0) obtained from the object recognition model, it can be seen that cases where the object is over-recognized (the red box area on the left of the sofa and bus) or recognized as smaller than it actually is (the box area on the right of the bus) are improved through the present invention. In Figure 6c, most of the modifications are made within 5 steps, which indicates that the object recognition improvement speed of the present invention is very fast.
[0110] FIG. 7a is a drawing illustrating the result of object recognition based on low-shot learning, and FIG. 7b is a drawing illustrating the result of object recognition with an embodiment of the present invention applied.
[0111] As shown in Fig. 7a, since the low-shot learning-based object recognition model is trained with only a very small number of examples, the performance of the object recognition results is inevitably bound to be poor. In the case of Fig. 7a, as shown in the red box area, it can be seen that cases occur where objects are over-recognized or recognized as small.
[0112] In contrast, FIG. 7b shows the improved result according to the present invention, and it can be seen that a high level of object recognition result can be obtained without the phenomenon of being perceived as excessively large or small.
[0113] The mask update-based object recognition method according to one embodiment of the present invention described above may be implemented as a program (or application) and stored on a medium to be executed in combination with a computer, which is hardware.
[0114] The aforementioned program may include code encoded in computer languages such as C, C++, JAVA, Ruby, and machine language, which can be read by the computer's processor (CPU) through the computer's device interface, in order for the computer to read the program and execute the methods implemented in the program. Such code may include functional code related to functions that define the necessary functions for executing the methods, and may include control code related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure. Additionally, such code may further include memory reference code regarding where (address) additional information or media necessary for the computer's processor to execute the functions should be referenced in the computer's internal or external memory. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the above functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to transmit or receive during communication.
[0115] The above-mentioned storage medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the above-mentioned storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the above-mentioned program may be stored on various recording media on various servers that the computer can access, or on various recording media on the user's computer. Additionally, the above-mentioned medium may be distributed across networked computer systems, and computer-readable code may be stored in a distributed manner.
[0116] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.
[0117] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols
[0118] 100: Electronic device 110: Input section 120: Communications Department 130: Display unit 140: Memory 150: Processor
Claims
Claim 1 A mask update-based object recognition method comprising: a step of generating first result information in which foreground and background are distinguished for a predetermined input image, wherein the object recognition result for the input image is matched with the boundary portion of the object; a step of calculating an objective function to be applied to each mask head based on the first and second result information and parameter information for the same; a step of updating the entire mask head by optimizing the parameters of the mask head by applying a Markov Random Field (MRF) at the feature map level rather than the pixel level based on the objective function; and a step of performing object recognition based on the updated mask. Claim 2 A mask update-based object recognition method according to claim 1, wherein the step of generating first result information in which foreground and background are distinguished for a predetermined input image comprises: a step of calculating a first similarity between a feature vector obtained by passing through the first layer of a mask head at each pixel of the input image and a foreground prototype; a step of calculating a second similarity between the feature vector and a background prototype; and a step of generating the first result information by applying the initial object recognition result at each pixel of the input image to the first and second similarities. Claim 3 In claim 2, the step of calculating a second similarity between the feature vector and the background prototype comprises: a step of calculating a foreground error for pixels located around the perimeter of the bounding box of each object; a step of extracting a set of pixels for a predetermined number in order of the calculated foreground error; and a step of calculating a background prototype based on the feature vector for the extracted set of pixels. Claim 4 In paragraph 2, the step of generating the first result information by applying the initial object recognition result at each pixel of the input image to the first and second similarities further includes the step of applying the set information of pixels in which the minimum values of foreground error and background error exceed a certain threshold to the calculated first similarity, a mask update-based object recognition method. Claim 5 A mask update-based object recognition method according to claim 1, wherein the step of generating second result information that matches the object recognition result for the input image with the boundary portion of the object comprises: generating a set of neighbor pixels for a predetermined pixel that enters as input to the mask head; calculating a third similarity between the predetermined pixel and the neighbor pixels within the generated set; and matching the object recognition result with the boundary portion based on whether the calculated third similarity exceeds a preset threshold. Claim 6 A communication unit that receives a predetermined image; a memory storing a program for updating a mask based on a preset objective function for the input predetermined image; and, by executing the program stored in the memory, generating first result information in which the foreground and background are distinguished for the input image, and generating second result information in which the object recognition result for the input image matches the boundary portion of the object, and then calculating an objective function to be applied to each mask head based on the first and second result information and parameter information for the same, and optimizing the parameters of the mask head by applying MRF (Markov Random Field) at the feature map level rather than the pixel level. An electronic device for object recognition comprising a processor that updates an entire mask head and performs object recognition based on the updated mask. Claim 7 An electronic device for object recognition according to claim 6, wherein the processor calculates a first similarity between a feature vector obtained by passing through the first layer of a mask head at each pixel of the input image and a foreground prototype, calculates a second similarity between the feature vector and a background prototype, and then applies the initial object recognition result at each pixel of the input image to the first and second similarities to generate the first result information. Claim 8 An electronic device for object recognition according to claim 7, wherein the processor calculates a foreground error for pixels located around the perimeter of the bounding box of each object included in the image, extracts a set of pixels for a predetermined number in order of highest foreground error, and then calculates a background prototype based on a feature vector for the extracted set of pixels. Claim 9 An electronic device for object recognition according to claim 7, wherein the processor applies set information of pixels in which the minimum values of foreground error and background error exceed a certain threshold to the calculated first similarity. Claim 10 An electronic device for object recognition according to claim 6, wherein the processor generates a set of neighbor pixels for a predetermined pixel that enters as input to the mask head, calculates a third feature similarity between the predetermined pixel and neighbor pixels within the generated set, and matches the boundary portion with the object recognition result based on whether the calculated third similarity exceeds a preset threshold. Claim 11 A mask update-based object recognition method comprising: receiving an image containing a predetermined object to be recognized as input by a computer; passing the input image through a backbone network to extract a first feature map; passing the first feature map through a feature pyramid network to extract a second feature map; updating a mask for object recognition based on a controller head corresponding to the second feature map; and performing object recognition for the second feature map based on the updated mask, wherein the step of updating the mask for object recognition based on a controller head included in the second feature map is to update the mask by optimizing the parameters of the mask head by applying a Markov Random Field (MRF) at the feature map level rather than the pixel level, based on an objective function that produces first result information in which the foreground and background are distinguished for the input image and second result information in which the boundary portion of the object is matched with the object recognition result for the input image.
Citation Information
Patent Citations
Foreground extraction using iterated graph cuts
US20050271273A1
