An image segmentation method and apparatus

Abstract

This application provides an image segmentation method and apparatus. The method includes: when the specified segmentation object corresponding to the specified segmentation image is inconsistent with the current segmentation object corresponding to the image segmentation model, obtaining an image annotation template corresponding to the specified segmentation image; recording the features of the image annotation template through a memory module to obtain memory segmentation features; and segmenting the specified segmentation image through the image segmentation model and the memory segmentation features to obtain an image segmentation result. Using this scheme, new segmentation tasks can be quickly expanded without retraining the image segmentation model, simplifying the process.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image segmentation method and apparatus. Background Technology

[0002] In industrial inspection tasks, changes in production lines, incoming materials, and design updates often necessitate the inspection of different products. Image segmentation is a crucial step in product inspection, enabling the identification of product regions within an image, thus facilitating subsequent product inspection.

[0003] Currently, deep learning models used for image segmentation often require collecting a large amount of training data corresponding to the new product, labeling the data, and training again to obtain a model for segmenting the product. The retraining process is complex and time-consuming. Summary of the Invention

[0004] This application provides an image segmentation method and apparatus to at least solve the above-mentioned technical problems existing in the prior art.

[0005] According to a first aspect of the embodiments of this application, an image segmentation method is provided, the method comprising: when a specified segmentation object corresponding to a specified segmentation image is inconsistent with the current segmentation object corresponding to an image segmentation model, obtaining an image annotation template corresponding to the specified segmentation image; recording features of the image annotation template through a memory module to obtain memory segmentation features; and segmenting the specified segmentation image through the image segmentation model and the memory segmentation features to obtain an image segmentation result.

[0006] In one possible implementation, obtaining the image annotation template corresponding to the specified segmented image includes: registering the specified annotation template and its corresponding index information through a registration module; searching the memory module according to the index information to determine historical annotation templates that belong to the same specified segmentation object as the specified annotation template; and determining the image annotation template based on the specified annotation template and the historical annotation templates.

[0007] In one embodiment, the step of recording features of the image annotation template through a memory module to obtain memory segmentation features includes: determining an image mask corresponding to the image annotation template; encoding the image annotation template through the image mask to obtain a foreground encoding feature map and a background encoding feature map; and recording the memory segmentation features through the foreground encoding feature map and the background encoding feature map, wherein the foreground encoding feature map is used to record foreground segmentation features, and the background encoding feature map is used to record background segmentation features.

[0008] In one possible implementation, encoding the image annotation template using the image mask to obtain a foreground encoded feature map and a background encoded feature map includes: determining a region of interest based on the annotation information corresponding to the image annotation template; and encoding the region of interest using the image mask to obtain a foreground encoded feature map and a background encoded feature map.

[0009] In one embodiment, obtaining the memory segmentation features further includes: when the number of image annotation templates is not unique, integrating the memory segmentation features corresponding to each image annotation template to obtain integrated segmentation features.

[0010] In one possible implementation, the step of segmenting the specified segmented image using the image segmentation model and the memory segmentation features to obtain an image segmentation result includes: extracting features from the specified segmented image to obtain specified segmentation features; cross-matching the specified segmentation features and the memory segmentation features using a cross-attention module to obtain attention features; and determining the image segmentation result based on the attention features.

[0011] In one possible implementation, the step of obtaining attention features by cross-matching the specified segmentation feature and the memory segmentation feature through a cross-attention module includes: performing similarity matching on the specified segmentation feature and the memory segmentation feature to determine a matching similarity value corresponding to the specified segmentation feature; and filtering the specified segmentation feature based on the matching similarity value to obtain attention features.

[0012] In one possible implementation, after obtaining the image segmentation result, the method further includes: if the image segmentation result is an incorrect segmentation result, annotating the specified segmented image through an update module to obtain an incorrect annotation template; recording the incorrect segmentation features corresponding to the incorrect annotation template through a memory module; and integrating the incorrect segmentation features with the memory segmentation features to obtain the updated memory segmentation features.

[0013] In one embodiment, the method further includes: obtaining image training samples corresponding to different segmentation objects; training the original segmentation model using the image training samples to obtain the image segmentation model; wherein the image training samples corresponding to different segmentation objects may or may not contain image training samples corresponding to the specified segmentation object.

[0014] According to a second aspect of the embodiments of this application, an image segmentation apparatus is provided, the apparatus comprising: an acquisition module, configured to acquire an image annotation template corresponding to the specified segmentation image when the specified segmentation object corresponding to the specified segmentation image is inconsistent with the current segmentation object corresponding to the image segmentation model; a recording module, configured to record features of the image annotation template through a memory module to obtain memory segmentation features; and a segmentation module, configured to segment the specified segmentation image through the image segmentation model and the memory segmentation features to obtain an image segmentation result.

[0015] In one embodiment, the acquisition module includes: a registration submodule, used to register a specified annotation template and corresponding index information through the registration module; a search submodule, used to search the memory module according to the index information to determine a historical annotation template that belongs to the same specified segmentation object as the specified annotation template; and a first determination submodule, used to determine the image annotation template according to the specified annotation template and the historical annotation template.

[0016] In one embodiment, the recording module includes: a second determining submodule, configured to determine an image mask corresponding to the image annotation template; an encoding submodule, configured to encode the image annotation template using the image mask to obtain a foreground encoding feature map and a background encoding feature map; and a recording submodule, configured to record the memory segmentation features using the foreground encoding feature map and the background encoding feature map, wherein the foreground encoding feature map is used to record foreground segmentation features, and the background encoding feature map is used to record background segmentation features.

[0017] In one embodiment, the encoding submodule is further configured to determine the region of interest based on the annotation information corresponding to the image annotation template; and to encode the region of interest using the image mask to obtain a foreground encoded feature map and a background encoded feature map.

[0018] In one embodiment, the recording module is further configured to integrate the memory segmentation features corresponding to each image annotation template to obtain integrated segmentation features when the number of image annotation templates is not unique.

[0019] In one embodiment, the segmentation module includes: an extraction submodule for extracting features from the specified segmented image to obtain specified segmentation features; a matching submodule for cross-matching the specified segmentation features and the memory segmentation features through a cross-attention module to obtain attention features; and a third determination submodule for determining the image segmentation result based on the attention features.

[0020] In one embodiment, the matching submodule includes: performing similarity matching on the specified segmentation feature and the memory segmentation feature to determine a matching similarity value corresponding to the specified segmentation feature; and filtering the specified segmentation feature based on the matching similarity value to obtain attention features.

[0021] In one embodiment, the apparatus further includes: an annotation module, configured to annotate the specified segmented image through an update module to obtain an error annotation template if the image segmentation result is an erroneous segmentation result; the recording module is further configured to record the erroneous segmentation features corresponding to the error annotation template through a memory module; and an integration module, configured to integrate the erroneous segmentation features and the memory segmentation features to obtain the updated memory segmentation features.

[0022] In one embodiment, the apparatus further includes: an obtaining module for obtaining image training samples corresponding to different segmentation objects; and a training module for training the original segmentation model using the image training samples to obtain the image segmentation model; wherein the image training samples corresponding to different segmentation objects may or may not include image training samples corresponding to the specified segmentation object.

[0023] According to a third aspect of the present application, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in the present application.

[0024] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in the present application.

[0025] The image segmentation method and apparatus provided in this application record features of the image annotation template corresponding to a specified segmentation object using a memory module to obtain memory segmentation features. These memory segmentation features, along with an image segmentation model, are then used to segment the specified image corresponding to the specified segmentation object, thereby obtaining the image segmentation result. By applying this method, when the specified segmentation object needs to be changed, there is no need to retrain the image segmentation model based on the specified segmentation object. The memory module can be used to achieve image segmentation for different segmentation objects without retraining the image segmentation model.

[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0027] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, wherein:

[0028] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0029] Figure 1 A schematic diagram illustrating the implementation flow of an image segmentation method according to an embodiment of this application is shown;

[0030] Figure 2 This paper shows a schematic diagram of an implementation module of an image segmentation apparatus according to an embodiment of the present application;

[0031] Figure 3 A schematic diagram illustrating the implementation flow of an image segmentation method according to another embodiment of this application is shown;

[0032] Figure 4 A schematic diagram of an implementation module of an image segmentation apparatus according to another embodiment of this application is shown;

[0033] Figure 5 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0034] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] Figure 1 The diagram illustrates the implementation flow of an image segmentation method according to an embodiment of this application.

[0036] See Figure 1According to a first aspect of the embodiments of this application, an image segmentation method is provided, the method comprising: operation 101, when the specified segmentation object corresponding to the specified segmentation image is inconsistent with the current segmentation object corresponding to the image segmentation model, obtaining an image annotation template corresponding to the specified segmentation image; operation 102, recording the features of the image annotation template through a memory module to obtain memory segmentation features; and operation 103, segmenting the specified segmentation image through the image segmentation model and the memory segmentation features to obtain an image segmentation result.

[0037] The image segmentation method provided in this application uses a memory module to record features of the image annotation template corresponding to a specified segmentation object, obtaining memory segmentation features. These memory segmentation features, along with an image segmentation model, are then used to segment the specified image corresponding to the specified segmentation object, thereby obtaining the image segmentation result. Applying this method, when the specified segmentation object needs to be changed, there is no need to retrain the image segmentation model based on the specified segmentation object. The memory module records the memory segmentation features of the specified segmentation object, and the recorded memory segmentation features, along with the image segmentation model, are used to segment the specified image. This allows for image segmentation of different segmentation objects without retraining the image segmentation model, simplifying the process for segmenting different segmentation objects.

[0038] The image segmentation method provided in this application is applicable to segmenting items within the same field. For example, it can segment various industrial items. This method can be applied to image segmentation equipment or industrial inspection equipment used in industrial inspection. Industrial items include, but are not limited to, nuts, screws, and lenses. Taking industrial production as an example, industrial production requires the inspection of products obtained from production. The inspection process typically involves first photographing the product, then determining the target segmentation region for the product through model segmentation, and finally performing detection and analysis on the target segmentation region to obtain the detection result. Industrial inspection tasks often require the detection of different segmentation objects, i.e., image segmentation of different segmentation objects is necessary.

[0039] In operation 101 of this method, the current segmentation object represents the object that the image segmentation model can currently segment. The specified segmentation object represents the object that the image segmentation model will segment according to the actual needs of the working conditions. The current segmentation object and the specified segmentation object are not the same object. For example, the current segmentation object can be a bolt, and the specified segmentation object can be a lens. The specified segmentation image can be obtained by acquiring an image of the specified segmentation object through an acquisition device. According to the actual working conditions of industrial production, when the specified segmentation object that needs to be inspected in industrial production is inconsistent with the current segmentation object corresponding to the image segmentation model, an image annotation template corresponding to the specified segmentation image is obtained. The image annotation template and the specified segmentation image belong to the same specified segmentation object. Unlike the specified segmentation image, the image annotation template has feature annotations for the specified segmentation object. The image annotation template can be pre-annotated and stored in the device, and the device obtains the image annotation template by reading the storage module. When the specified segmentation object is a new object, the image annotation template can be obtained by acquiring an image of the specified segmentation object, then annotating the acquired image, and uploading the annotated image annotation template to the device, thereby enabling the device to obtain the image annotation template.

[0040] Unlike model training samples used to update model parameters, image annotation templates are few in number and do not update model parameters. Usually only a few or dozens of image annotation templates are needed. When there are few annotation features and / or the image segmentation model itself has good segmentation performance, only 1 to 2 image annotation templates are needed.

[0041] In operation 102 of this method, the model is equipped with a memory module. This memory module records the features annotated by the image annotation template, thereby obtaining memory segmentation features for a specified segmentation object. Whenever the device adjusts the segmentation object, the memory segmentation features recorded in the memory module for the current segmentation object are adjusted to record the memory segmentation features for the specified segmentation object, so that the memory segmentation features recorded in the memory module correspond to the specified segmentation object that needs to be segmented. Therefore, in operation 103, the specified segmentation image can be segmented using the image segmentation model and the memory segmentation features for the specified segmentation object, obtaining an image segmentation result that meets the segmentation accuracy requirements. Using this method, there is no need to retrain the model when the segmentation object is adjusted, thus eliminating the need for training data collection, training data annotation, cloud training updates, and other operations for training the model. This saves time that would otherwise be required to retrain and adjust the model when the segmentation object is adjusted, improving segmentation efficiency.

[0042] To facilitate a further understanding of the above implementation methods, a specific implementation scenario is provided below. In this scenario, the task of an industrial product inspection production line is to inspect "screws," and the image segmentation model was originally used to segment the object "screws." As the task of the industrial product inspection production line changes, the task is adjusted to inspect "lenses," and the segmentation object of the image segmentation model needs to be adjusted to "lenses." In this case, this method obtains 1-2 image annotation templates corresponding to "lenses" and inputs them into a memory module. The memory module records features based on the annotation information of the image annotation templates to obtain memory segmentation features. After the memory module completes the recording, the production line begins to perform the inspection task of "lenses." An image acquisition device is used to acquire images of the "lenses" to be inspected, obtaining a specified segmented image. Then, features are extracted from the specified segmented image to obtain specified segmentation features. The image segmentation model performs attention segmentation on the specified segmentation features and the memory segmentation features, enabling the image segmentation model to segment the "lenses." Furthermore, the use of memory segmentation features improves the accuracy of the image segmentation model in segmenting the "lenses," thus helping to improve the accuracy of subsequent lens inspection results.

[0043] For a new segmentation object (shot), a small number of shot images and corresponding annotations are registered in the memory module. When the production line is actually running, the features of the shot images will be matched with the corresponding features in the memory module through the attention module. The matched features are decoded and output to achieve the segmentation of the new segmentation object (shot).

[0044] In one possible implementation, operation 101, obtaining an image annotation template corresponding to a specified segmented image, includes: first, registering the specified annotation template and its corresponding index information through a registration module; then, searching the memory module based on the index information to determine historical annotation templates belonging to the same specified segmentation object as the specified annotation template; and finally, determining an image annotation template based on the specified annotation template and the historical annotation templates.

[0045] This method also includes a registration module for storing image annotation templates and corresponding image masks for various types of industrial items, and for adding image annotation templates and corresponding image masks for new types of industrial items not yet registered in the registration module. Each type of industrial item's image annotation template also stores corresponding index information, which can be used to find the corresponding image annotation template. The index information also includes number information, ROI location information, etc. This method can also store the location information of industrial items in the image within the registration module. If the specified segmentation object to be segmented is a new type of industrial item not registered in the registration module, this method can upload a specified annotation template and image mask corresponding to the specified segmentation object through the registration module, define the corresponding index information, and use the uploaded specified annotation template as the image annotation template to perform operation 102, recording features of the specified segmentation object's image annotation template.

[0046] If the specified segmentation object to be segmented is an industrial item already registered in the registration module, the stored historical annotation template is searched in the registration module using the index information. The historical annotation template that matches the index information is determined as the image annotation template for feature recording in operation 102. Furthermore, during the segmentation of the same type of industrial item, erroneous segmentation results can be collected, annotated, and added to the memory module as erroneous segmentation templates. This is integrated with the memory segmentation features to increase the information content of the memory segmentation features, thereby further improving the segmentation accuracy of the image segmentation model for the specified segmentation object. It should be added that, in actual working conditions, this method can also add, delete, modify, or replace the image annotation templates corresponding to the index information based on the index information, making the memory segmentation features corresponding to the image annotation templates more accurate.

[0047] In one possible implementation, operation 102 involves recording the features of an image annotation template using a memory module to obtain memory segmentation features. This includes: first, determining an image mask corresponding to the image annotation template; then, encoding the image annotation template using the image mask to obtain a foreground encoding feature map and a background encoding feature map; and finally, recording the memory segmentation features using the foreground encoding feature map and the background encoding feature map, wherein the foreground encoding feature map is used to record foreground segmentation features and the background encoding feature map is used to record background segmentation features.

[0048] When the current segmentation object of the image segmentation model needs to be replaced with a specified segmentation object, the index information corresponding to the specified segmentation object is input through the registration module. The memory module can then obtain the image mask and image annotation template corresponding to the specified segmentation object. The memory module then encodes the image annotation template using the image mask, thereby distinguishing the specified segmentation object from the background in the image annotation template, obtaining a foreground encoded feature map containing the specified segmentation object and a background encoded feature map not containing the specified segmentation object. The memory module then records the features of both the foreground and background encoded feature maps to obtain the memorized segmentation features. This process ensures that the memorized segmentation features include both foreground and background encoded feature maps, providing subsequent image segmentation with more information-rich segmentation criteria, reducing confusion between foreground and background memorized segmentation features, and improving the accuracy of image segmentation. When this method is applied to industrial inspection tasks, which are usually assembly line inspection operations, the background of the acquired specified segmented image is usually the same assembly line background in the same area. Therefore, the background in the specified segmented image is usually similar or even unchanged. Based on this, by distinguishing the foreground and background of the specified segmented image, this method can further improve the segmentation accuracy.

[0049] In one possible implementation, encoding an image annotation template using an image mask to obtain a foreground encoded feature map and a background encoded feature map includes: first, determining a region of interest based on the annotation information corresponding to the image annotation template; and then, encoding the region of interest using an image mask to obtain a foreground encoded feature map and a background encoded feature map.

[0050] The annotation information is manually annotated and used to distinguish the foreground and background in the image. The region of interest is used to represent the area in the image where the specified segmented object is located. In industrial scenarios, the specified segmented images captured have similar backgrounds and similar segmentation positions. Therefore, the region of interest can be encoded using an image mask to obtain foreground encoded feature maps and background encoded feature maps.

[0051] In one embodiment, operation 102, obtaining the memory segmentation features, further includes: when the number of image annotation templates is not unique, integrating the memory segmentation features corresponding to each image annotation template to obtain integrated segmentation features. When the number of image annotation templates is not unique, this method integrates the memory segmentation features, making the integrated segmentation features have a larger amount of information and able to reflect the information of all image annotation templates, thereby improving the segmentation accuracy of the image segmentation model.

[0052] In one possible implementation, operation 103 involves segmenting a specified segmented image using an image segmentation model and memory segmentation features to obtain an image segmentation result. This includes: first, extracting features from the specified segmented image to obtain specified segmentation features; then, performing similarity matching between the specified segmentation features and memory segmentation features to determine a matching similarity value corresponding to the specified segmentation features; next, filtering the specified segmentation features based on the matching similarity value to obtain attention features; and finally, determining the image segmentation result based on the attention features.

[0053] Specifically, this method utilizes an attention mechanism to segment images using memory segmentation features and specified segmentation features. Since the memory segmentation features in this application distinguish between foreground and background, the foreground and background features of the memory segmentation features can further bias the segmentation results of the image segmentation model towards the specified segmentation object. Through similarity value matching, the image segmentation model can output the image segmentation result with the highest similarity to the specified segmentation object in the memory segmentation features, thereby achieving accurate segmentation of the specified segmentation image by the image segmentation model.

[0054] In one possible implementation, after obtaining the image segmentation result in operation 103, the method further includes: first, if the image segmentation result is an incorrect segmentation result, an error annotation template is obtained by an update module to annotate the specified segmented image; then, the error segmentation features corresponding to the error annotation template are recorded by a memory module; and finally, the error segmentation features and the memory segmentation features are integrated to obtain the updated memory segmentation features.

[0055] The image segmentation results can be judged manually or by machine to determine if they are segmentation errors. The determination of an error is based on whether the segmentation result corresponds to the specified segmentation object. If the image segmentation result shows a non-specified segmentation object or an incomplete specified segmentation object, it is determined to be an incorrect segmentation result. If the image segmentation result is determined to be incorrect, the segmented image corresponding to the incorrect segmentation result is labeled manually or by machine to obtain an error labeling template. The update module adds the error labeling template to the memory module, allowing the memory module to record the features of the error segmentation template and obtain error segmentation features. By integrating the error segmentation features with the memory segmentation features, the updated memory segmentation features contain more and more accurate segmentation information, thereby improving the segmentation accuracy of the image segmentation model for the specified segmentation object.

[0056] In one embodiment, the method further includes: first, obtaining image training samples corresponding to different segmentation objects; then, training the original segmentation model using the image training samples to obtain an image segmentation model; wherein the image training samples corresponding to different segmentation objects may or may not contain image training samples corresponding to a specified segmentation object.

[0057] The image segmentation model presented in this method is pre-trained using a large number of image training samples of different types of industrial products. This allows the model to learn comprehensive technical features related to industrial products, resulting in a well-rounded learning and segmentation capability, strong scalability, and the ability to segment various industrial products. Furthermore, since different types of industrial products often share some similar technical features, the image segmentation model trained using this method can segment objects that are included or not included in the image training samples, provided that the segmentation object and the training objects in the image training samples belong to the same product category. For example, if the image training samples include bolts, nuts, screws, and washers of type A, the segmentation object could be bolts, lead screws, etc. of type B.

[0058] Figure 2 A schematic diagram of an implementation module of an image segmentation apparatus according to an embodiment of this application is shown.

[0059] See Figure 2 To facilitate further understanding of the above embodiments, a specific implementation scenario is provided below. This scenario involves an image segmentation device on an industrial inspection production line, which is communicatively connected to an image acquisition device. The image segmentation device includes: a memory module, a registration module, an update module, and a segmentation module.

[0060] Memory module: Used to record the image annotation template and corresponding image mask of the industrial items to be segmented. The image annotation template is marked with foreground, background and ROI.

[0061] The registration module is the control module for the memory module. It is used to store the image annotation templates of registered industrial items, and to perform operations such as modification, deletion, and replacement on the image annotation templates of recorded industrial items. It is also used to add image annotation templates of unrecorded industrial items.

[0062] The update module is used to add incorrectly segmented samples to the memory module, so that the memory module records the incorrectly segmented samples.

[0063] The segmentation module is used to perform cross-matching segmentation between the image to be segmented and the image annotation templates recorded in the memory module.

[0064] Figure 3The diagram illustrates a scenario of implementing an image segmentation method according to another embodiment of this application.

[0065] See Figure 3 When an image segmentation device is needed to segment item A, but the current segmentation object of the image segmentation model is item B, the registration module is searched using the index number corresponding to item A to determine whether the registration module stores an image annotation template corresponding to item A.

[0066] If the registration module does not store the image annotation template corresponding to item A, upload the image annotation template and image mask corresponding to item A through the registration module, and enter the index number corresponding to item A to register. The image annotation template and image mask corresponding to item A will be stored in the memory module.

[0067] The image annotation template and image mask corresponding to item A are encoded using the memory module to determine the foreground encoding feature map containing item A and the background encoding feature map not containing item A. The memory segmentation features are then determined based on the foreground encoding feature map and the background encoding feature map.

[0068] The image acquisition device acquires the image of item A that needs to be segmented, obtains the specified segmented image, encodes the specified segmented image to obtain the specified segmentation features, and performs foreground and background attention cross-matching segmentation on the specified segmentation features and the memory segmentation features to obtain the image segmentation result. The image segmentation result is used to characterize the segmentation of the region where item A is located in the specified segmented image from other regions.

[0069] After obtaining the image segmentation results, the results can be reviewed manually or by intelligent devices. If the review determines that the image segmentation results have not segmented item A, the image segmentation results are identified as incorrect segmentation results. The update module then adds the specified segmented image of item A to the memory module, so that the memory module records the incorrect segmentation features corresponding to the incorrect segmentation results.

[0070] The memory module integrates erroneous segmentation features and remembered segmentation features to obtain integrated segmentation features. These integrated segmentation features are then used for cross-matching with specified segmentation features of the next segmented image, thereby gradually improving the segmentation accuracy of the image segmentation model.

[0071] Figure 4 A schematic diagram of an image segmentation apparatus implementation module according to another embodiment of this application is shown.

[0072] See Figure 4According to a second aspect of the embodiments of this application, an image segmentation apparatus is provided, the apparatus comprising: an acquisition module 401, configured to acquire an image annotation template corresponding to a specified segmentation image when the specified segmentation object corresponding to a specified segmentation image is inconsistent with the current segmentation object corresponding to an image segmentation model; a recording module 402, configured to record features of the image annotation template through a memory module to obtain memory segmentation features; and a segmentation module 403, configured to segment the specified segmentation image through an image segmentation model and memory segmentation features to obtain an image segmentation result.

[0073] In one embodiment, the acquisition module 401 includes: a registration submodule 4011, used to register a specified annotation template and corresponding index information through the registration module; a search submodule 4012, used to search the memory module according to the index information to determine a historical annotation template that belongs to the same specified segmentation object as the specified annotation template; and a first determination submodule 4013, used to determine an image annotation template according to the specified annotation template and the historical annotation template.

[0074] In one embodiment, the recording module 402 includes: a second determining submodule 4021, used to determine an image mask corresponding to the image annotation template; an encoding submodule 4022, used to encode the image annotation template through the image mask to obtain a foreground encoding feature map and a background encoding feature map; and a recording submodule 4023, used to record memory segmentation features through the foreground encoding feature map and the background encoding feature map, wherein the foreground encoding feature map is used to record foreground segmentation features and the background encoding feature map is used to record background segmentation features.

[0075] In one embodiment, the encoding submodule 4022 is further configured to determine the region of interest based on the annotation information corresponding to the image annotation template; and to encode the region of interest using an image mask to obtain a foreground encoding feature map and a background encoding feature map.

[0076] In one embodiment, the recording module 402 is further configured to integrate the memory segmentation features corresponding to each image annotation template to obtain integrated segmentation features when the number of image annotation templates is not unique.

[0077] In one embodiment, the segmentation module 403 includes: an extraction submodule 4031, used to extract features from a specified segmented image to obtain specified segmentation features; a matching submodule 4032, used to cross-match the specified segmentation features and the memory segmentation features through a cross-attention module to obtain attention features; and a third determination submodule 4033, used to determine the image segmentation result based on the attention features.

[0078] In one embodiment, the matching submodule 4032 includes: performing similarity matching on specified segmentation features and memory segmentation features to determine the matching similarity value corresponding to the specified segmentation features; and filtering the specified segmentation features based on the matching similarity value to obtain attention features.

[0079] In one embodiment, the apparatus further includes: a labeling module 404, used to label a specified segmented image through an update module to obtain an error labeling template if the image segmentation result is an error segmentation result; a recording module 402, used to record the error segmentation features corresponding to the error labeling template through a memory module; and an integration module, used to integrate the error segmentation features and the memory segmentation features to obtain updated memory segmentation features.

[0080] In one embodiment, the apparatus further includes: an acquisition module 405, configured to acquire image training samples corresponding to different segmentation objects; and a training module 406, configured to train the original segmentation model using the image training samples to obtain an image segmentation model; wherein the image training samples corresponding to different segmentation objects may or may not contain image training samples corresponding to a specified segmentation object.

[0081] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0082] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0083] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0084] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0085] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as an image segmentation method. For example, in some embodiments, an image segmentation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of an image segmentation method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform an image segmentation method by any other suitable means (e.g., by means of firmware).

[0086] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0087] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0088] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0089] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0090] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0091] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0092] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0093] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0094] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image segmentation method, the method comprising: When the specified segmentation object corresponding to the specified segmentation image is inconsistent with the current segmentation object corresponding to the image segmentation model, obtain the image annotation template corresponding to the specified segmentation image; The image annotation template is used to record features through a memory module to obtain memory segmentation features; The specified segmentation image is segmented using the image segmentation model and the memory segmentation features to obtain the image segmentation result; The step of obtaining an image annotation template corresponding to the specified segmentation object includes: obtaining a pre-stored image annotation template with feature annotations; The memory module records features of the image annotation template to obtain memory segmentation features, including: recording the features annotated on the image annotation template to obtain the memory segmentation features; The image segmentation process involves segmenting the specified image using the image segmentation model and the memory segmentation features to obtain an image segmentation result. This includes: extracting features from the specified image segmentation to obtain specified segmentation features; cross-matching the specified segmentation features and the memory segmentation features using a cross-attention module to obtain attention features; and determining the image segmentation result based on the attention features.

2. The method according to claim 1, wherein obtaining the image annotation template corresponding to the specified segmented image includes: Register the specified annotation template and corresponding index information through the registration module; The memory module is searched based on the index information to determine the historical annotation template that belongs to the same specified segmentation object as the specified annotation template; The image annotation template is determined based on the specified annotation template and the historical annotation template.

3. The method according to claim 1, wherein the step of recording features of the image annotation template through a memory module to obtain memory segmentation features includes: Determine the image mask corresponding to the image annotation template; The image annotation template is encoded using the image mask to obtain a foreground encoded feature map and a background encoded feature map. The memory segmentation features are recorded using the foreground encoding feature map and the background encoding feature map, wherein the foreground encoding feature map is used to record foreground segmentation features and the background encoding feature map is used to record background segmentation features.

4. The method according to claim 3, wherein encoding the image annotation template through the image mask to obtain a foreground encoded feature map and a background encoded feature map comprises: The region of interest is determined based on the annotation information corresponding to the image annotation template; The region of interest is encoded using the image mask to obtain a foreground encoded feature map and a background encoded feature map.

5. The method according to any one of claims 1 to 4, wherein obtaining the memory segmentation features further comprises: When the number of image annotation templates is not unique, the memory segmentation features corresponding to each image annotation template are integrated to obtain integrated segmentation features.

6. The method according to claim 1, wherein obtaining attention features by cross-matching the specified segmentation features and the memory segmentation features through a cross-attention module includes: Perform similarity matching on the specified segmentation feature and the memory segmentation feature to determine the matching similarity value corresponding to the specified segmentation feature; The specified segmentation features are filtered based on the matching similarity values ​​to obtain attention features.

7. The method according to claim 1, further comprising, after obtaining the image segmentation result: If the image segmentation result is an incorrect segmentation result, the specified segmented image is annotated by the update module to obtain an incorrect annotation template; The memory module records the erroneous segmentation features corresponding to the erroneous annotation template. The erroneous segmentation features and the memory segmentation features are integrated to obtain the updated memory segmentation features.

8. The method according to claim 1, further comprising: Obtain image training samples corresponding to different segmentation objects; The original segmentation model is trained using the image training samples to obtain the image segmentation model; The image training samples corresponding to different segmentation objects may or may not contain image training samples corresponding to the specified segmentation object.

9. An image segmentation apparatus, the apparatus comprising: The acquisition module is used to acquire an image annotation template corresponding to the specified segmentation image when the specified segmentation object corresponding to the specified segmentation image is inconsistent with the current segmentation object corresponding to the image segmentation model. The recording module is used to record features of the image annotation template through the memory module to obtain memory segmentation features; The segmentation module is used to segment the specified segmentation image using the image segmentation model and the memory segmentation features to obtain the image segmentation result; The acquisition module is used to: acquire the image annotation template with feature annotations that has been stored in advance; The recording module is used to: record the features annotated on the image annotation template to obtain the memory segmentation features; The segmentation module is used to: extract features from the specified segmented image to obtain specified segmentation features; cross-match the specified segmentation features and the memory segmentation features through the cross-attention module to obtain attention features; and determine the image segmentation result based on the attention features.

Images