Image labeling method, system and device based on adaptive threshold semantic segmentation

Abstract

The application discloses an image labeling method, system and device based on adaptive threshold semantic segmentation, wherein the method comprises the following steps: when detecting that a first control on an operation interface is triggered, displaying a to-be-processed image in a preset region of the operation interface, and then converting the to-be-processed image into an OpenCV image; when detecting that a color list combination control on the operation interface occurs multiple valid triggering events, in response to each valid triggering event, marking a color curve produced by a user performing a long-press drag operation of a left mouse button on the to-be-processed image, and obtaining corresponding curve information to give a color label in the OpenCV image; and when detecting that a hidden save control on the operation interface is triggered, using a watershed algorithm to perform adaptive pixel filling on each region in the current OpenCV image according to the color label given in the region, obtaining a final mask image, and saving the final mask image to a specified path. The application can improve the image labeling efficiency.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an image annotation method, system, and device based on adaptive threshold semantic segmentation. Background Technology

[0002] Currently, image annotation tools based on semantic segmentation mainly include Labelme and Sprite Annotation Assistant, but both types of image annotation tools have certain drawbacks, as follows: First, when there are objects with complex shapes and structures in the image, using Labelme requires constantly clicking the mouse to draw multiple annotation points along the outer perimeter of the object, making the entire annotation process cumbersome. Furthermore, it does not directly generate the required mask image output after completing the entire image annotation task. Second, when there are objects with complex shapes and structures in the image, using Sprite Annotation Assistant requires constantly moving the mouse to completely erase and annotate the interior of the object. Sometimes, it is also necessary to adjust the brush thickness appropriately at the object boundary to avoid incorrectly marking other objects, making the image annotation efficiency relatively low. Summary of the Invention

[0003] This invention provides an image annotation method, system, and device based on adaptive threshold semantic segmentation to solve one or more technical problems existing in the prior art, and at least provides a beneficial option or creates conditions.

[0004] Firstly, an image annotation method based on adaptive threshold semantic segmentation is provided, including:

[0005] When a trigger event is detected in the first control on the operation interface, the image to be processed is displayed in a preset area of ​​the operation interface;

[0006] The image to be processed is converted to a format to obtain an OpenCV image;

[0007] When multiple valid trigger events are detected in the color list combination control on the operation interface, in response to each valid trigger event, the color curve generated when the user performs each left-click, long-press and drag operation is marked on the image to be processed and the corresponding curve information is obtained. Then, the curve information is used to assign color labels to the OpenCV image.

[0008] When a hidden save control on the operation interface is detected to have been triggered, the watershed algorithm is used to adaptively fill each region of the current OpenCV image with pixels according to the color labels assigned to them, so as to obtain the final mask image and save it to the specified path.

[0009] Furthermore, before detecting multiple valid trigger events of the color list combination control on the operation interface, or during the process of responding to each valid trigger event, the method further includes:

[0010] When a trigger event is detected in the second control on the operation interface, a first copy image corresponding to the image to be processed is generated;

[0011] Contour extraction is performed on the first copy image to obtain multiple foreground contour lines and their position information on the first copy image is recorded;

[0012] The multiple foreground contour lines are marked on the image to be processed according to the recorded position information.

[0013] Furthermore, before detecting multiple valid trigger events of the color list combination control on the operation interface, the method also includes:

[0014] When a trigger event is detected in the third control on the operation interface, a second copy image corresponding to the image to be processed is generated;

[0015] The watershed algorithm is used to randomly fill each region of the second copy image with adaptive pixels according to different color labels to obtain the initial mask image;

[0016] The initial mask image is converted into a four-channel image, and then the transparency value in the four-channel image is adjusted to a predetermined threshold and overlaid on the image to be processed.

[0017] Furthermore, after adjusting the transparency values ​​in the four-channel image to a predetermined threshold and overlaying them onto the image to be processed, the process further includes:

[0018] When a trigger event is detected in the hidden save control on the operation interface, the initial mask image is directly used as the final mask image and saved to the specified path.

[0019] Furthermore, the step of using the watershed algorithm to randomly fill each region of the second copy image with adaptive pixels according to different color labels to obtain the initial mask image includes:

[0020] The second copy image is converted into a single-channel grayscale image, and then the single-channel grayscale image is converted into a binarized image;

[0021] The binarized image is subjected to opening and closing operations in sequence to obtain a denoised image;

[0022] The denoised image is subjected to dilation to extract the background region, resulting in the first image;

[0023] The denoised image is subjected to Euclidean distance transformation to segment the connected foreground regions into multiple disconnected foreground regions, thus obtaining a second image.

[0024] The first image and the second image are subjected to a difference operation to extract the undetermined areas to be filled, resulting in a third image;

[0025] The first image, the second image, and the third image are merged to obtain a fourth image;

[0026] Different color labels are randomly assigned to each region of the fourth image. Then, the watershed algorithm is used to adaptively fill each region of the current fourth image with pixels according to the color labels assigned to it, so as to obtain the initial mask image.

[0027] Furthermore, the step of marking the colored curves generated when the user performs each left-click, long-press, and drag operation on the image to be processed and obtaining the corresponding curve information in response to each valid trigger event includes:

[0028] Based on each valid triggering event, a corresponding colored brush is automatically generated in the preset area;

[0029] When the user performs any left-click and drag operation, the colored pen is simultaneously controlled to draw a colored curve on the image to be processed. During the entire drawing process, the two-dimensional coordinates of the mouse when it completes each preset movement distance, as well as the two-dimensional coordinates of the mouse when it starts and ends dragging, are recorded and encapsulated together with the color label associated with the colored pen to obtain the corresponding curve information.

[0030] Furthermore, the detection of a hidden save control on the user interface triggering an event includes:

[0031] In response to a right-click operation performed by the user, a menu bar is displayed within the preset area;

[0032] A save control is set in the menu bar, and left mouse clicks applied to the save control are detected.

[0033] Furthermore, the menu bar also includes undo, clear, and redo controls;

[0034] The undo control is used to delete the colored curve and its corresponding curve information generated by the most recent left-click drag operation, and at the same time, perform associated deletion in the OpenCV image.

[0035] The clear control is used to delete all colored curves and their corresponding curve information generated by all left-click, long-press, and drag operations after they are completed, and at the same time delete all marker traces in the OpenCV image.

[0036] The restore control is used to restore the deletion operation that was previously completed by the undo control or the clear control.

[0037] Secondly, an image annotation system based on adaptive threshold semantic segmentation is provided, including:

[0038] The display module is used to display the image to be processed within a preset area of ​​the operation interface when a trigger event is detected in the first control on the operation interface;

[0039] The conversion module is used to convert the format of the image to be processed to obtain an OpenCV image;

[0040] The marking module is used to mark the colored curves generated when the user performs each left-click, long-press and drag operation on the image to be processed when multiple valid trigger events are detected in the color list combination control on the operation interface, and to obtain the corresponding curve information. Then, the color label is assigned to the OpenCV image using the curve information.

[0041] The generation module is used to adaptively fill each region of the current OpenCV image with pixels according to the color labels assigned to it when a hidden save control on the operation interface is detected to trigger an event, thereby obtaining the final mask image and saving it to a specified path.

[0042] Thirdly, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the image annotation method based on adaptive threshold semantic segmentation as described in the first aspect.

[0043] This invention offers at least the following advantages: By drawing curves of different colors within the regions of each object on the imported image to be processed, and then using the watershed algorithm to directly fill pixel thresholds within each object's region based on assigned color labels, the required mask image output is automatically generated. Compared to existing image annotation tools, this eliminates the need to manually plot points along the outer contours of each object's region or completely erase its interior, significantly improving image annotation efficiency. Furthermore, by automatically annotating the outer contours of each object on the imported image to be processed, users can perform curve drawing operations more quickly and accurately. Finally, by automatically generating an initial mask image corresponding to the imported image to be processed and overlaying it onto the image according to predetermined requirements, users can obtain more satisfactory mask image output. Attached Figure Description

[0044] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0045] Figure 1 This is a flowchart illustrating an image annotation method based on adaptive threshold semantic segmentation in an embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram of the operation interface in an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the curve marking result of the first image to be processed in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of the final mask image corresponding to the first image to be processed in an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram of the outer contour line marking result of the first image to be processed in an embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram of the automatically labeled mask image corresponding to the first image to be processed in an embodiment of the present invention;

[0051] Figure 7 This is a schematic diagram of the composition of an image annotation system based on adaptive threshold semantic segmentation in an embodiment of the present invention;

[0052] Figure 8 This is a schematic diagram of the hardware structure of the computer device in an embodiment of this disclosure. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0054] It should be noted that although functional modules are divided in the system diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first," "second," "third," "fourth," etc., used in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed and are inherent to these processes, methods, products, or apparatuses.

[0055] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an image annotation method based on adaptive threshold semantic segmentation provided by an embodiment of the present invention. The method includes the following:

[0056] Step S110: When a trigger event is detected in the first control on the operation interface, the image to be processed is displayed in the preset area of ​​the operation interface;

[0057] Step S120: Convert the format of the image to be processed to obtain an OpenCV image;

[0058] Step S130: When multiple valid trigger events are detected in the color list combination control on the operation interface, in response to each valid trigger event, mark the colored curve generated when the user performs each left-click long-press drag operation on the image to be processed and obtain the corresponding curve information, and then use the curve information to assign color labels to the OpenCV image.

[0059] Step S140: When the hidden save control on the operation interface is triggered, the watershed algorithm is used to adaptively fill each region in the current OpenCV image with pixels according to the color labels assigned to them, so as to obtain the final mask image and save it to the specified path.

[0060] In this embodiment of the invention, the operation interface mentioned in step S110 is a GUI (Graphical User Interface) created by a technician using relevant functions in the open-source tkinter library provided with Python software. Its preferred size is 1200*650. A dynamically adjustable preset area is added to the operation interface, and at least a first control, a second control, a third control, and a color list combination control are provided on the operation interface. Specifically, the first control is named "Select Image Path" control, the second control is named "Image Hint" control, and the third control is named "Auto-Annotation" control. The color list combination control includes, but is not limited to, a red selection control, a brown selection control, a yellow selection control, a green selection control, a blue selection control, and a pink selection control. See details... Figure 2 As shown, the present invention can also name any of the above selection controls using corresponding font colors to facilitate intuitive selection by the user.

[0061] In this embodiment of the invention, the first control allows the user to easily access any folder on the currently running device. The specific implementation of step S110 includes: when the user positions the mouse over the first control and performs a left-click operation, the system determines that a trigger event has occurred on the first control. Then, after the user selects and imports an image to be processed from any folder, the system first adjusts the size of the preset area according to the size of the image to be processed, and then displays the image to be processed completely within the preset area. See details... Figure 2 As shown.

[0062] In this embodiment of the invention, the detection of a valid trigger event for the color list combination control mentioned in step S130 is specifically manifested as follows: when a user manipulates the mouse to locate a selection control in the color list combination control and performs a left-click operation, and the user's next step is to perform a long-press and drag operation within the preset area, the system determines that a valid trigger event has been detected for the color list combination control; however, when a user manipulates the mouse to locate a selection control in the color list combination control and performs a left-click operation, but the user's next step is still to manipulate the mouse to locate other selection controls in the color list combination control and perform a left-click operation, the system determines that an invalid trigger event has previously occurred for the color list combination control.

[0063] In this embodiment of the invention, the specific implementation process for responding to any valid triggering event mentioned in step S130 above includes the following:

[0064] Step S131: When the system detects that any selection control in the color list combination control has a valid trigger event, a colored brush is automatically generated in the preset area according to the color label pointed to by the selection control, and the colored brush is limited to performing drawing operations only in the preset area.

[0065] Step S132: When the user performs any left-click, long-press and drag operation within the preset area, the system will respond synchronously to the left-click, long-press and drag operation to control the colored brush to follow the mouse movement path in real time to draw on the image to be processed to generate a colored curve.

[0066] Throughout the curve drawing process, the system will synchronously record the two-dimensional coordinates of the mouse when it starts dragging, the two-dimensional coordinates of the mouse when it completes each preset movement distance, and the two-dimensional coordinates of the mouse when it ends dragging. Furthermore, it needs to encapsulate each synchronously recorded two-dimensional coordinate value with the color label indicated by the selection control, and finally obtain the curve information associated with the left-click and long-press drag operation.

[0067] Step S133: Since the OpenCV image and the image to be processed have the same size, for each two-dimensional coordinate value and its associated color label recorded in the curve information, the two-dimensional coordinate value is used to locate the corresponding position in the OpenCV image, and the color label is assigned at the corresponding position.

[0068] Repeat steps S132 to S133 until the system detects a valid trigger event in another selection control in the color list combo control.

[0069] In this embodiment of the invention, the specific implementation process for responding to any valid triggering event mentioned in step S130 above may further include the following:

[0070] Step S130.1: When the system detects that any selection control in the color list combination control has a valid trigger event, a colored brush is automatically generated in the preset area according to the color label indicated by the selection control, and the colored brush is restricted to performing drawing operations only in the preset area.

[0071] Step S130.2: When the user performs any left-click, long-press and drag operation within the preset area, the system will respond synchronously to the left-click, long-press and drag operation to control the colored brush to follow the mouse movement path in real time to draw on the image to be processed to generate a colored curve.

[0072] Throughout the curve drawing process, the system will record the two-dimensional coordinates of the two endpoints of the mouse at each preset movement distance in real time, and connect the two endpoints using colored line segments. Since the OpenCV image and the image to be processed are the same size, the system will synchronously position itself at two corresponding positions in the OpenCV image based on the recorded two-dimensional coordinates of the two endpoints in real time, and assign the color label associated with the selection control to these two corresponding positions. This process will continue until the left-click and drag operation of the mouse ends.

[0073] The above step S130.2 is executed repeatedly until the system detects that another selection control in the color list combo control has triggered a valid event.

[0074] It should be noted that when the system detects that the left-click drag operation has ended, but the distance between the current position of the mouse at the end of the drag and the latest recorded endpoint is less than the preset movement distance, it still needs to record the two-dimensional coordinate value of the mouse at the end of the drag and connect the current position of the mouse at the end of the drag with the latest recorded endpoint using colored line segments.

[0075] In a specific implementation of the present invention, by selecting and importing an image to be processed (hereinafter referred to as the first image to be processed) and performing a series of curve drawing operations using the above-mentioned step S130, the following can be obtained: Figure 3 The corresponding implementation results shown here are for... Figure 3 The specific marking process in the non-grayscale state is explained as follows: In the first image to be processed, green apples, yellow apples, and red apples are displayed in order from left to right. After the user clicks the brown selection control with the mouse, a corresponding brown brush is generated, and multiple brown curves are drawn relatively randomly within the area where the green apple is located. After the user clicks the blue selection control with the mouse, a corresponding blue brush is generated, and multiple blue curves are drawn relatively randomly within the area where the yellow apple is located. After the user clicks the yellow selection control with the mouse, a corresponding yellow brush is generated, and multiple yellow curves are drawn relatively randomly within the area where the red apple is located. After the user clicks the pink selection control with the mouse, a corresponding pink brush is generated, and multiple pink curves are drawn relatively randomly within the background area of ​​the image. This allows for the use of different colors to mark different objects in the first image to be processed, as well as the use of the same color to mark the same object.

[0076] In this embodiment of the invention, regarding the implementation process of responding to any valid triggering event mentioned in step S130 above, the system automatically creates three types of lists to complete the data storage operation, specifically including the following:

[0077] (1) Drawing list: named Draw list, used to cache all three-dimensional vectors associated with color curves obtained by the system in real time when responding to each left-click, long-press and drag operation of the mouse synchronously; wherein, any three-dimensional vector is denoted as point = [x, y, color_index], (x, y) is a single two-dimensional coordinate value recorded by the system, and color_index is the color label associated with the left-click, long-press and drag operation; it should be noted that when the left-click, long-press and drag operation ends and the color label assignment operation on the OpenCV image has been completed, the system will automatically clear all currently cached three-dimensional vectors in the Draw list.

[0078] (2) Undo list: Named Revoke list, it is used to automatically create several sub-lists in the order of operation time from first to last. Any one of the sub-lists is used to store all the three-dimensional vectors and their associated colored curves cached by the Draw list when the system synchronously responds to a single left-click long-press drag operation, so as to facilitate undoing or clearing operations when curve marking errors occur in subsequent operations.

[0079] (3) Recovery List: Named the Recover list, it is used to automatically create several sub-lists in the order of operation time from first to last. Any one of the sub-lists is used to store all the three-dimensional vectors and their associated colored curves cached by the Draw list when the system synchronously responds to a single left-click long-press drag operation, so that it can be restored to the state before the undo or clearing when an error occurs in subsequent operations.

[0080] Based on this, the present invention also provides a hidden menu bar on the operation interface. The menu bar is only displayed within the preset area after the user performs a right-click operation. The menu bar includes at least undo, redo, clear, and save controls. See details... Figure 3 As shown, the functions of the above four types of controls are described below:

[0081] (1) Undo control, which deletes a single sublist in the Revoke list associated with the most recent left-click-long-press drag operation that has been completed. This allows the system to delete the color curve generated on the image to be processed and the color label assigned to the OpenCV image when synchronously responding to the left-click-long-press drag operation.

[0082] (2) Clear control, which deletes all sublists stored in the Revoke list. This allows the system to delete all colored curves generated on the image to be processed and all color labels assigned to the OpenCV image when all left-click, long-press and drag operations have been completed in the synchronous response.

[0083] (3) The restore control is used to restore the deletion operation completed by the undo control in the previous step, and add the single sublist associated with the most recent left-click-long-press-drag operation that has been completed back to the Revoke list; or, restore the deletion operation completed by the clear control in the previous step, and add all sublists associated with all left-click-long-press-drag operations that have been completed back to the Revoke list.

[0084] (4) Save control, which detects that the user has completed the curve marking operation on the image to be processed. At this time, the OpenCV image that has been assigned color labels is saved, and the watershed algorithm is automatically called to continue to perform the region pixel filling operation on the saved OpenCV image.

[0085] In this embodiment of the invention, the specific implementation process of step S140 includes: by calling the cv::watershed function contained in OpenCV (OpenSource Computer Vision Library, a cross-platform computer vision library), for each region in the current OpenCV image (i.e., the OpenCV image that has been assigned color labels), all pixels in the region are reassigned according to the color threshold corresponding to the color label originally assigned to that region, so that the pixel value corresponding to each pixel is the color threshold, thereby realizing the region segmentation processing of the current OpenCV image; then the processed OpenCV image is converted from BGR channel order to RGB channel order to form the final mask image, and the final mask image is saved to a specified path. At the same time, a display sub-interface pops up on the operation interface to intuitively display the final mask image to the user.

[0086] In a specific implementation of the present invention, after performing a color label assignment operation on the OpenCV image (hereinafter referred to as the first OpenCV image) corresponding to the first image to be processed using the above-described step S130, and then performing a region pixel filling operation on the current first OpenCV image using the above-described step S140, the following can be obtained: Figure 4 The corresponding implementation result shown is that the final mask image corresponding to the first image to be processed has been displayed in the pop-up display sub-interface.

[0087] When the image to be processed imported by the user is relatively complex, that is, when the image to be processed contains multiple objects and the dividing line between two adjacent objects is not obvious, the present invention also proposes a first auxiliary marking function to enable the user to perform each left-click long-press drag operation more quickly and accurately, that is, to mark different areas for the user as a prompt; the first auxiliary marking function is only activated after the user controls the mouse to position to the second control and performs a left-click operation, and can be executed before the system detects a valid trigger event in any selection control in the color list combination control (that is, before executing the above step S130), or when the system responds to any valid trigger event (that is, during the execution of the above step S130).

[0088] In this embodiment of the invention, the first auxiliary marking function is specifically manifested as follows:

[0089] Step A1: Copy the image to be processed to obtain a first copy image;

[0090] Step A2: Convert the first copy image into a binary image, and then use the Moore-Neighbor Tracing algorithm to extract the outer contour lines of each foreground object from the binary image. Specifically, since many images contain outer contour lines and multiple inner contour lines, that is, the objects enclosed by the outer contour lines still have textures and other contour parts inside. The region marking function that this invention wants to achieve does not need to focus on the internal texture of a single object in the image. At this time, it is only necessary to call the cv::findContours function included in OpenCV to find the outer contour lines of each foreground object from the binary image, and at the same time record the specific position information of the outer contour lines of each foreground object in the binary image.

[0091] Step A3: Since the binary image and the image to be processed have the same size, based on the specific position information of the outer contour lines of each foreground object found, the outer contour lines of each foreground object are marked on the image to be processed by calling the cv::drawContours function included in OpenCV. This marking result will be directly displayed in the preset area for the user to view, but no corresponding marking will be generated in the OpenCV image.

[0092] In a specific implementation of the present invention, before performing a series of curve drawing operations on the first image to be processed using the above step S130, the user can obtain the following when triggering the first auxiliary marking function: Figure 5 The corresponding implementation results are shown.

[0093] The present invention also proposes a second auxiliary marking function to improve the user's marking efficiency of the imported image to be processed, that is, to prompt the user to select appropriate colors to mark different areas; the second auxiliary marking function is only activated after the user controls the mouse to position the third control and performs a left mouse click operation, and can only be executed before the system detects that any selection control in the color list combination control has a valid trigger event (that is, before executing the above step S130).

[0094] In this embodiment of the invention, the second auxiliary marking function is specifically manifested as follows:

[0095] Step B1: Copy the image to be processed to obtain a second copy image;

[0096] Step B2: Based on all the color tags associated with the color list combination control, perform an adaptive pixel filling operation on each region of the second copy image by randomly selecting one color tag from all the color tags using the watershed algorithm to obtain the initial mask image.

[0097] Step B3: Perform channel conversion on the initial mask image to obtain a four-channel image, wherein the four-channel image contains R channel image data, G channel image data, B channel image data and alpha channel image data. Then, increase the alpha value in the four-channel image from 0% to a predetermined threshold (preferably set to 50% in this invention) and then directly overlay it onto the image to be processed. This overlay result will be directly displayed in the preset area for the user to view, but no overlay mark will be generated in the OpenCV image accordingly.

[0098] Step B4: When the user believes that it is unnecessary to modify the annotation range and annotation color of a certain area within the initial mask image, the user first performs a right-click operation to display the menu bar within the preset area. Then, after the user positions the mouse to the save control and performs a left-click operation, the initial mask image obtained in step B2 can be directly defined as the final mask image and saved to the specified path, and displayed in the pop-up display sub-interface.

[0099] Alternatively, when the user believes that the annotation range and annotation color of some areas within the initial mask image need to be modified, step S130 can be continued based on the existing annotation results within the initial mask image. At this time, the same annotation color is still used to perform curve drawing operation for the satisfactory areas, while other appropriate annotation colors are used to perform curve drawing operation for the unsatisfactory areas. Then, step S140 is continued to obtain the desired final mask image.

[0100] In a specific implementation of the present invention, before performing a series of curve drawing operations on the first image to be processed using the above-described step S130, the user can trigger the second auxiliary marking function to obtain the following: Figure 6 The corresponding implementation results are shown.

[0101] More specifically, the implementation process of step B2 above includes the following:

[0102] Step B2.1: Perform channel conversion on the second copy image to obtain a single-channel grayscale image;

[0103] Step B2.2: Obtain a suitable binarization threshold by calling the cv::THRESH_OTSU function included in OpenCV, and then perform binarization processing on the single-channel grayscale image by calling the cv::threshold function included in OpenCV and using the binarization threshold as a reference to obtain a binarized image.

[0104] Step B2.3: Perform an opening operation on the binarized image to eliminate external noise in the image, and then perform a closing operation on the processed binarized image to eliminate internal noise in the image, to obtain a denoised image;

[0105] Step B2.4: Perform a dilation operation on the denoised image to dilate the foreground region while compressing the background region, thereby obtaining a first image with a more obvious background region;

[0106] Step B2.5: By calling the cv::distanceTransform function included in OpenCV, Euclidean distance transformation is performed on the denoised image to calculate the distance between each pixel in the denoised image and the nearest zero pixel. Then, an appropriate Euclidean distance threshold is selected to binarize the transformed denoised image, thereby obtaining the center position of each foreground region in the denoised image. This achieves the segmentation of the entire connected foreground region in the denoised image into multiple unconnected foreground regions, and thus a second image with more obvious foreground regions can be obtained.

[0107] Step B2.6: Perform a difference operation on the first image and the second image to obtain a third image, and only represent the undetermined areas to be filled in the third image;

[0108] Step B2.7: Perform an image fusion operation on the first image, the second image, and the third image to obtain a fourth image, so that the fourth image can simultaneously represent the background area, multiple foreground areas, and areas to be filled. Then, according to all the color labels associated with the color list combination control, randomly assign different color labels to each of the represented areas.

[0109] Step B2.8: By calling the cv::watershed function included in OpenCV, for each region in the current fourth image (i.e., the fourth image that has already been assigned color labels), all pixels in that region are reassigned according to the color threshold corresponding to the original color label in that region, so that the pixel value corresponding to each pixel is the color threshold, thereby realizing the region segmentation processing of the current fourth image; then the current processed fourth image is converted from BGR channel order to RGB channel order to form the initial mask image.

[0110] In this embodiment of the invention, by drawing curves of different colors within the regions of each object on the imported image to be processed, and then using the watershed algorithm to directly fill the pixel thresholds within each object's region according to the assigned color labels, the required mask image output is automatically generated. Compared to existing image annotation tools, this eliminates the need to manually manipulate the mouse to sequentially plot the outer contours of each object's region or to completely erase its interior, significantly improving image annotation efficiency. By manually annotating the outer contours of each object on the imported image to be processed, users can perform curve drawing operations more quickly and accurately. By automatically generating an initial mask image corresponding to the imported image to be processed and overlaying it on the image according to predetermined requirements, users can obtain a more satisfactory mask image output.

[0111] Please refer to Figure 7 , Figure 7 This is a schematic diagram illustrating the composition of an image annotation system based on adaptive threshold semantic segmentation provided in an embodiment of the present invention. The system includes:

[0112] The display module 210 is used to display the image to be processed in a preset area set on the operation interface when a trigger event is detected in the first control set on the operation interface.

[0113] The conversion module 220 is used to perform a format conversion operation on the image to be processed in order to obtain a corresponding OpenCV image;

[0114] The marking module 230 is used to respond to each valid trigger event when it detects that the color list combination control set on the operation interface has multiple valid trigger events. That is, when the user completes each left-click, long-press and drag operation, the colored curve is adaptively marked on the image to be processed and the associated curve information is recorded. The color label assignment operation is performed on the OpenCV image according to the recorded curve information.

[0115] The generation module 240 is used to perform adaptive pixel filling operation on each region of the current OpenCV image according to the currently assigned color label in its internal region when a trigger event is detected that the hidden setting save control on the operation interface is currently triggered, and finally save the final mask image to a specified path.

[0116] The content of the above method embodiments is applicable to this system embodiment. The functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are the same as those in the above method embodiments. Therefore, they will not be repeated here.

[0117] Furthermore, embodiments of the present invention also provide a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the image annotation method based on adaptive threshold semantic segmentation described in the above embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, the storage device includes any medium on which a device (e.g., a computer, mobile phone, etc.) stores or transmits information in a readable form, and can be a read-only memory, a disk, or an optical disk, etc.

[0118] also, Figure 8 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. The computer device includes components such as a processor 320, a memory 330, an input unit 340, and a display unit 350. Those skilled in the art will understand that... Figure 8The illustrated device structure is not intended to limit all devices and may include more or fewer components than shown, or combine certain components. The memory 330 can be used to store the computer program 310 and various functional modules. The processor 320 runs the computer program 310 stored in the memory 330, thereby performing various functional applications and data processing of the device. The memory can be internal memory or external memory, or include both internal and external memory. Internal memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. External memory may include hard disks, floppy disks, ZIP disks, USB flash drives, magnetic tapes, etc. The memory 330 disclosed in the embodiments of this invention includes, but is not limited to, these types of memory. The memory 330 disclosed in the embodiments of this invention is only an example and not a limitation.

[0119] Input unit 340 is used to receive signal input and user-input keywords. Input unit 340 may include a touch panel and other input devices. The touch panel can collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel) and drive the corresponding connection device according to a pre-set program; other input devices may include, but are not limited to, one or more of physical keyboards, function keys (such as play control buttons, power buttons, etc.), trackballs, mice, joysticks, etc. Display unit 350 can be used to display user-input information or information provided to the user, as well as various menus of the terminal device. Display unit 350 may be in the form of a liquid crystal display, organic light-emitting diode, etc. Processor 320 is the control center of the terminal device, connecting various parts of the entire device through various interfaces and lines, performing various functions and processing data by running or executing software programs and / or modules stored in memory 320, and calling data stored in memory.

[0120] As one embodiment, the computer device includes a processor 320, a memory 330, and a computer program 310, wherein the computer program 310 is stored in the memory 330 and configured to be executed by the processor 320, and the computer program 310 is configured to perform the image annotation method based on adaptive threshold semantic segmentation in the above embodiment.

[0121] Although the description of this application has been quite detailed and particularly focused on several of the described embodiments, it is not intended to limit itself to any of these details or embodiments or any particular embodiment. Rather, it should be considered as effectively covering the intended scope of this application by referring to the appended claims and taking into account the prior art, which provides for a broad possible interpretation of these claims. Furthermore, the foregoing description of this application with respect to embodiments foreseeable by the inventors is intended to provide a useful description, and non-substantial modifications to this application that have not yet been foreseen may still represent equivalent modifications.

Claims

1. An image annotation method based on adaptive threshold semantic segmentation, characterized in that, include: When a trigger event is detected in the first control on the operation interface, the image to be processed is displayed in a preset area of ​​the operation interface; The image to be processed is converted to a format to obtain an OpenCV image; When multiple valid trigger events are detected in the color list combination control on the operation interface, in response to each valid trigger event, the color curve generated when the user performs each left-click, long-press and drag operation is marked on the image to be processed and the corresponding curve information is obtained. Then, the curve information is used to assign color labels to the OpenCV image. When a hidden save control on the operation interface is detected to be triggered, the watershed algorithm is used to adaptively fill each region in the current OpenCV image with pixels according to the color labels assigned to them, so as to obtain the final mask image and save it to the specified path. The method further includes, before detecting multiple valid trigger events of the color list combination control on the operation interface, or during the process of responding to each valid trigger event: When a trigger event is detected in the second control on the operation interface, a first copy image corresponding to the image to be processed is generated; Contour extraction is performed on the first copy image to obtain multiple foreground contour lines and their position information on the first copy image is recorded; The multiple foreground contour lines are marked on the image to be processed according to the recorded position information.

2. The image annotation method based on adaptive threshold semantic segmentation according to claim 1, characterized in that, Before detecting multiple valid trigger events of the color list combination control on the user interface, the method further includes: When a trigger event is detected in the third control on the operation interface, a second copy image corresponding to the image to be processed is generated; The watershed algorithm is used to randomly fill each region of the second copy image with adaptive pixels according to different color labels to obtain the initial mask image; The initial mask image is converted into a four-channel image, and then the transparency value in the four-channel image is adjusted to a predetermined threshold and overlaid on the image to be processed.

3. The image annotation method based on adaptive threshold semantic segmentation according to claim 2, characterized in that, The step of using the watershed algorithm to randomly fill each region of the second copy image with adaptive pixels according to different color labels to obtain the initial mask image includes: The second copy image is converted into a single-channel grayscale image, and then the single-channel grayscale image is converted into a binarized image; The binarized image is subjected to opening and closing operations in sequence to obtain a denoised image; The denoised image is subjected to dilation to extract the background region, resulting in the first image; The denoised image is subjected to Euclidean distance transformation to segment the connected foreground regions into multiple disconnected foreground regions, thus obtaining a second image. The first image and the second image are subjected to a difference operation to extract the undetermined areas to be filled, resulting in a third image; The first image, the second image, and the third image are merged to obtain a fourth image; Different color labels are randomly assigned to each region of the fourth image. Then, the watershed algorithm is used to adaptively fill each region of the current fourth image with pixels according to the color labels assigned to it, so as to obtain the initial mask image.

4. The image annotation method based on adaptive threshold semantic segmentation according to claim 1, characterized in that, The step of responding to each valid trigger event by marking the colored curves generated when the user performs each left-click, long-press, and drag operation on the image to be processed and obtaining the corresponding curve information includes: Based on each valid triggering event, a corresponding colored brush is automatically generated in the preset area; When the user performs any left-click and drag operation, the colored pen is simultaneously controlled to draw a colored curve on the image to be processed. During the entire drawing process, the two-dimensional coordinates of the mouse when it completes each preset movement distance, as well as the two-dimensional coordinates of the mouse when it starts and ends dragging, are recorded and encapsulated together with the color label associated with the colored pen to obtain the corresponding curve information.

5. The image annotation method based on adaptive threshold semantic segmentation according to claim 1, characterized in that, The event triggered by detecting the hidden save control on the user interface includes: In response to a right-click operation performed by the user, a menu bar is displayed within the preset area; A save control is set in the menu bar, and left mouse clicks applied to the save control are detected.

6. The image annotation method based on adaptive threshold semantic segmentation according to claim 5, characterized in that, The menu bar also includes undo, clear, and redo controls. The undo control is used to delete the colored curve and its corresponding curve information generated by the most recent left-click drag operation, and at the same time, perform associated deletion in the OpenCV image. The clear control is used to delete all colored curves and their corresponding curve information generated by all left-click, long-press, and drag operations after they are completed, and at the same time delete all marker traces in the OpenCV image. The restore control is used to restore the deletion operation that was previously completed by the undo control or the clear control.

7. An image annotation system based on adaptive threshold semantic segmentation, characterized in that, include: The display module is used to display the image to be processed within a preset area of ​​the operation interface when a trigger event is detected in the first control on the operation interface; The conversion module is used to convert the format of the image to be processed to obtain an OpenCV image; The marking module is used to mark the colored curves generated when the user performs each left-click, long-press and drag operation on the image to be processed when multiple valid trigger events are detected in the color list combination control on the operation interface, and to obtain the corresponding curve information. Then, the color label is assigned to the OpenCV image using the curve information. The generation module is used to, when a trigger event is detected in the hidden save control on the operation interface, use the watershed algorithm to adaptively fill each region in the current OpenCV image with pixels according to the color labels assigned to it, to obtain the final mask image and save it to the specified path. The method further includes, before detecting multiple valid trigger events of the color list combination control on the operation interface, or during the process of responding to each valid trigger event: When a trigger event is detected in the second control on the operation interface, a first copy image corresponding to the image to be processed is generated; Contour extraction is performed on the first copy image to obtain multiple foreground contour lines and their position information on the first copy image is recorded; The multiple foreground contour lines are marked on the image to be processed according to the recorded position information.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, The processor executes the computer program to implement the image annotation method based on adaptive threshold semantic segmentation as described in any one of claims 1 to 6.

Images