Image labeling method, model training method, device, equipment and medium
Through interactive image annotation method and pre-trained image segmentation model, the problems of long manual annotation time and low accuracy of model output are solved, and efficient image annotation and model training are achieved.
Patent Information
- Application Number
- CN202311110466.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-30
- Publication Date
- 2025-02-28
AI Technical Summary
In the prior art, the number of images required for manually labeled image segmentation models is huge, resulting in a long time for manual labeling and a low output accuracy of the model trained with only a small amount of labeling data, which cannot effectively improve the accuracy of the full amount of training data.
Through the interactive image annotation method, users only need to click on the target subject to the annotation, use the pre-trained image segmentation model to generate image segmentation results, and add label information to realize automatic image annotation.
It effectively improves the efficiency of image annotation, reduces the time and workload of manual annotation, and automatically generates point-level supervision information, reduces the cost of model training and improves the accuracy of model output.
Smart Images
Figure CN119536594A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to an image annotation method, a model training method, a device, a equipment and a medium. Background Art
[0002] In the process of training existing image segmentation models, a large number of images with annotation information are needed as training data, and these images used as training data need to be manually annotated in advance, such as selecting target contours or target annotation boxes, etc. Since the number of images that need to be annotated is very large, the manual annotation time is also very long. Although some people have proposed a method of pre-training the image segmentation model with only a small amount of annotated data, and then using the model to automatically segment and annotate the remaining images to obtain the full amount of training data; however, the output accuracy of the image segmentation model obtained by training only a small amount of annotated data is low, which in turn leads to the low accuracy of the training data obtained by subsequent automatic segmentation and annotation. The output accuracy of the model trained using such full amount of training data cannot be guaranteed.
[0003] Therefore, how to reduce the workload and time of manual annotation in the process of manual annotation becomes an urgent problem to be solved. Summary of the invention
[0004] In order to solve the above technical problems, the present disclosure provides an image annotation method, a model training method, an apparatus, a device and a medium to improve the efficiency of image annotation.
[0005] In a first aspect, an embodiment of the present disclosure provides an image annotation method, comprising:
[0006] Displaying an original image, wherein the original image includes a target subject to be annotated;
[0007] In response to a click operation of a user on the target subject, determining at least one target position corresponding to the click operation in the original image;
[0008] Inputting the original image with the target location information into a pre-trained image segmentation model;
[0009] Obtaining an image segmentation result for the target subject output by the image segmentation model;
[0010] The label information corresponding to the target subject is added to the image segmentation result to obtain the annotation result of the original image.
[0011] In some embodiments, after displaying the original image, the method further includes:
[0012] In response to a user's label input operation for a target subject in the original image, label information corresponding to the target subject is acquired.
[0013] In some embodiments, adding label information corresponding to the target subject to the image segmentation result to obtain the annotation result of the original image includes:
[0014] Adding label information corresponding to the target subject to the image segmentation result to obtain a semantic mask image of the original image;
[0015] The format of the semantic mask image is converted to obtain a labeling result of the original image.
[0016] In some embodiments, after displaying the original image, the method further includes:
[0017] Determining whether automatic image annotation is required according to an image processing task corresponding to the original image;
[0018] If yes, the original image is input into a pre-trained image detection model;
[0019] Obtain the annotation result of the original image output by the image detection model.
[0020] In a second aspect, the present disclosure provides a model training method, including:
[0021] Acquire a training image and a target area in the training image;
[0022] Generate annotation information of the training image according to the target area, the annotation information at least including point information of at least one first annotation point in the target area and point information of at least one second annotation point outside the target area;
[0023] The training image and the annotation information of the training image are used as inputs of an image segmentation model, and the target area is used as output of the image segmentation model to train the image segmentation model.
[0024] In some embodiments, generating the annotation information of the training image according to the target area includes:
[0025] Generating a preset number of preset annotation points in the training image, wherein the preset annotation points include at least one first annotation point in the target area and at least one second annotation point outside the target area;
[0026] For a target marking point among the preset marking points, calculating the relative distance between the target marking point and other preset marking points to obtain a relative distance vector corresponding to the target marking point;
[0027] The relative distance vector is used as point information corresponding to the target marked point.
[0028] In a third aspect, an embodiment of the present disclosure provides an image annotation device, including:
[0029] A display module, used to display an original image, wherein the original image includes a target subject to be annotated;
[0030] A first determining module, configured to determine, in response to a user's click operation on the target subject, at least one target position corresponding to the click operation in the original image;
[0031] An input module, used for inputting the original image with the target position information into a pre-trained image segmentation model;
[0032] A first acquisition module, used to acquire the image segmentation result of the target subject output by the image segmentation model;
[0033] The second acquisition module is used to add label information corresponding to the target subject to the image segmentation result to obtain the annotation result of the original image.
[0034] In a fourth aspect, an embodiment of the present disclosure provides a model training device, including:
[0035] A fourth acquisition module, used to acquire a training image and a target area in the training image;
[0036] A generating module, configured to generate annotation information of the training image according to the target area, wherein the annotation information at least includes point information of at least one first annotation point in the target area and point information of at least one second annotation point outside the target area;
[0037] The training module is used to use the training image and the annotation information of the training image as input of the image segmentation model, and use the target area as output of the image segmentation model to train the image segmentation model.
[0038] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including:
[0039] Memory;
[0040] Processor; and
[0041] Computer programs;
[0042] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect or the second aspect.
[0043] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method described in the first aspect or the second aspect.
[0044] The image annotation method, model training method, device, equipment and medium provided by the embodiments of the present disclosure use an interactive annotation method, in which the user only needs to click on the target subject to be annotated to complete the segmentation and annotation of the target subject, thereby effectively improving the efficiency of the image annotation method. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0047] Figure 1 A flow chart of an image annotation method provided by an embodiment of the present disclosure;
[0048] Figure 2 A flow chart of a model training method provided in an embodiment of the present disclosure;
[0049] Figure 3 A schematic diagram of a preset marking point provided in an embodiment of the present disclosure;
[0050] Figure 4 A flow chart of an image annotation method provided by another embodiment of the present disclosure;
[0051] Figure 5 A schematic diagram of an automatic annotation system interface provided by an embodiment of the present disclosure;
[0052] Figure 6 A schematic diagram of the structure of an image annotation device provided in an embodiment of the present disclosure;
[0053] Figure 7 A schematic diagram of the structure of a model training device provided in an embodiment of the present disclosure;
[0054] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0055] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0056] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0057] The embodiment of the present disclosure provides an image annotation method, which is introduced below in conjunction with a specific embodiment.
[0058] Figure 1 This is a flow chart of the image annotation method provided in the embodiment of the present disclosure. The method can be applied to any terminal device with data processing function, wherein the terminal device can be a car computer, a smart phone, a PDA, a tablet computer, a laptop, an all-in-one machine, an intelligent driving device, etc. It is understandable that the image annotation method provided in the embodiment of the present disclosure can also be applied in other scenarios.
[0059] Below Figure 1 The image annotation method shown in FIG. 1 is introduced, and the specific steps of the method are as follows:
[0060] S101: displaying an original image, wherein the original image includes a target subject to be labeled.
[0061] The original image is an image that has not been labeled yet. The original image includes at least one subject, including at least one target subject that needs to be labeled.
[0062] Specifically, the original image can be obtained from a library of images to be annotated, or the original image can be input by a user, etc., which is not limited in the embodiments of the present disclosure. The subject can be the relevant parts on the vehicle, and the target subject can be the parts to be annotated in the subject; in addition, the subject can also be other objects or areas of interest, etc., which are not limited here.
[0063] S102: In response to a click operation of a user on the target subject, determining at least one target position corresponding to the click operation in the original image.
[0064] The user can perform a click operation on the target subject area that needs to be marked in the original image. In response to the user's click operation, the terminal device collects the target position corresponding to the click operation in the original image.
[0065] Specifically, the target position includes one or several pixel points covered by the cursor when the user clicks, or a pixel area with a preset radius extending outward from any pixel point among the one or several pixel points covered by the cursor as the center, and the embodiments of the present disclosure are not limited to this.
[0066] In some embodiments, the user can perform multiple click operations on the same target subject, and the terminal device obtains the target position corresponding to each click operation.
[0067] In some embodiments, each target position corresponding to each click operation of the user in the original image may be located within the target body area, or may be located outside the target body area (for example, due to an operation error, the target position corresponding to the click operation is located outside the target body area and close to the boundary of the target body). It is only necessary to ensure that at least one of the above target positions is located within the target body area, and the embodiments of the present disclosure are not limited to this.
[0068] S103: inputting the original image with the target position information into a pre-trained image segmentation model.
[0069] S104: Obtain an image segmentation result for the target subject output by the image segmentation model.
[0070] The target position information includes the position information of one or more target positions mentioned above, which may specifically be the coordinate information of the target position in the original image. The original image with the target position information is input into a pre-trained image segmentation model to obtain the image segmentation result for the target subject output by the image segmentation model. Specifically, the image segmentation result of the target subject may be the original image containing the circumscribed rectangular annotation box of the target subject, or the circumscribed rectangular image of the target subject, etc., which is not limited in the embodiments of the present disclosure.
[0071] Figure 2 A flow chart of a model training method provided in an embodiment of the present disclosure, the image segmentation model mentioned in S103 or S104 can be Figure 2 The training is performed using the method shown. Figure 2 As shown, the model training method includes the following steps:
[0072] S201: Acquire a training image and a target area in the training image.
[0073] The training image refers to any training image in the training set used for model training. The training image is manually segmented in advance through manual annotation to determine the target area in the training image. The target area is the area corresponding to a subject in the training image.
[0074] The above processing is performed on each training image in the training set used for model training, and finally multiple training images and the target area in each training image are obtained.
[0075] S202: Generate annotation information of the training image according to the target area, where the annotation information at least includes point information of at least one first annotation point in the target area and point information of at least one second annotation point outside the target area.
[0076] Specifically, a preset number of preset annotation points are generated in the training image, and the preset annotation points include at least one first annotation point in the target area and at least one second annotation point outside the target area; for a target annotation point among the preset annotation points, a relative distance between the target annotation point and other preset annotation points is calculated to obtain a relative distance vector corresponding to the target annotation point; and the relative distance vector is used as point information corresponding to the target annotation point.
[0077] For a training image, point-level supervision information for model training is automatically generated based on the target area that has been determined in the training image, that is, the boundary of the target area can be used to randomly generate at least one first annotation point in the target area, and at least one second annotation point can be randomly generated outside the target area, and the point information of each first annotation point and the point information of each second annotation point are used as the annotation information of the training image. The embodiment of the present disclosure does not limit the number of first annotation points and second annotation points, and can be adjusted according to the needs of model training, for example, 12 first annotation points and 12 second annotation points are generated.
[0078] Among them, for a target marking point (which can be any first marking point or any second marking point among the preset marking points) among a preset number of preset marking points (including all first marking points and all second marking points), the point information of the target marking point is obtained; each preset marking point is taken as the target marking point in turn, and the point information of each preset marking point, that is, the point information of each first marking point and the point information of each second marking point, can be obtained. Taking the target marking point as the first marking point as an example, Figure 3 A schematic diagram of a preset marking point provided in an embodiment of the present disclosure. Figure 3 As shown in the figure, in a training image, the box 31 is a predetermined target area, and there are 12 preset annotation points (in Figure 3 There are 12 first marking points in the figure), taking the preset marking point 32 as the target marking point as an example, according to the position of each preset marking point, the relative distance between point 32 and each other preset marking point is calculated, and a relative distance vector with a length of 12 can be obtained, that is, the point information corresponding to the preset marking point 32.
[0079] S203: Using the training image and the annotation information of the training image as input of an image segmentation model, using the target area as output of the image segmentation model, and training the image segmentation model.
[0080] Specifically, the image segmentation model can be implemented using a convolutional neural network model, a machine learning model, etc., and can be implemented using models such as DeepLabV3, HRNet, OCR, etc., or using any other model that can achieve the above functions, which is not limited in the embodiments of the present disclosure.
[0081] After the above S201 to S203, a pre-trained image segmentation model can be obtained, and the original image with the target position information is input into the pre-trained image segmentation model to obtain the image segmentation result for the target subject.
[0082] S105: Add label information corresponding to the target subject to the image segmentation result to obtain a labeling result of the original image.
[0083] The label information corresponding to the target subject can be the name of the target subject, the category of the target subject, the serial number of the target subject, and any one or more identification information related to the target subject. By adding the label information to the image segmentation result of the target subject, the annotation result of the original image where the target subject is located can be obtained.
[0084] The disclosed embodiment displays an original image, which includes a target subject to be annotated; in response to a user's click operation on the target subject, determines at least one target position corresponding to the click operation in the original image; inputs the original image with the target position information into a pre-trained image segmentation model; obtains an image segmentation result for the target subject output by the image segmentation model; adds label information corresponding to the target subject to the image segmentation result to obtain an annotation result of the original image. By adopting an interactive annotation method, the user only needs to click on the target subject to be annotated to complete the segmentation and annotation of the target subject, which effectively improves the efficiency of the image annotation method.
[0085] In addition, since the image segmentation model in the embodiment of the present disclosure adopts a model training method based on automatically generating point-level supervision information, only a small amount of annotation is required to achieve model training, which greatly reduces the cost of model training and further improves the efficiency of the image annotation method.
[0086] Figure 4 This is a flow chart of an image annotation method provided by another embodiment of the present disclosure. The method can be applied to Figure 5 The automatic annotation system shown below Figure 5The schematic diagram of the automatic annotation system interface shown in Figure 4 The method shown is introduced. Figure 4 As shown, the method includes the following steps:
[0087] S401: Display an original image, wherein the original image includes a target subject to be labeled.
[0088] Specifically, the implementation process and principle of S401 and S101 are the same, and will not be repeated here.
[0089] For example, in Figure 5 In the operation interface shown, in response to the original image loaded by the user through "Loadimage", the original image is displayed in the "Image" area.
[0090] S402, judging whether automatic image annotation is required according to the image processing task corresponding to the original image; if so, executing S409; if not, executing S403.
[0091] S403: In response to the user's label input operation for the target subject in the original image, obtain label information corresponding to the target subject.
[0092] The user can enter the corresponding label for the target subject in the original image. Figure 5 In the interface shown, the user can provide the label information of the target entity entered in the text box to the terminal device by clicking the "Label an ID" button in the Click annotation function area. The terminal device obtains the label information corresponding to the target entity based on the label information entered by the user.
[0093] S404: In response to a click operation of the user on the target subject, determine at least one target position corresponding to the click operation in the original image.
[0094] S405: Input the original image with the target position information into a pre-trained image segmentation model.
[0095] S406: Obtain the image segmentation result of the target subject output by the image segmentation model.
[0096] Specifically, the implementation process and principle of S404 to S406 are consistent with those of S102 to S104, and will not be described in detail here.
[0097] S407: Add label information corresponding to the target subject to the image segmentation result to obtain a semantic mask image of the original image.
[0098] The mask is a template of an image filter, which filters the pixels of the image through a matrix to highlight the areas of interest in the image. As described in the above embodiment, the original image may include multiple target subjects. By looping through S403 to S406 for the multiple target subjects, the image segmentation result of each target subject in the original image can be obtained, and the corresponding label information is added to the image segmentation result of each target subject, that is, the semantic mask of the entire original image is obtained, in which each target subject corresponds to an image area with label information.
[0099] S408: Convert the format of the semantic mask image to obtain a labeling result of the original image.
[0100] In some embodiments, the semantic mask image is converted into a Json (JavaScript Object Notation) file, and the Json file is used as the annotation result of the original image. Json is a lightweight data exchange format that uses a text format that is completely independent of the programming language to store and represent data. It has good readability and is easy to write quickly, and can exchange data between different platforms. Json data uses characters to describe relevant information in the image, such as update time, project name, data set name, color information, point coordinate information, completion time, label name, etc., to informatize the image and effectively extract key information from the image data.
[0101] In such Figure 5 In the interface shown, the Datas convert function area is used to support users in data format conversion, supports converting semantic mask images into Json files, and can also support conversion between Json files and Coco file formats.
[0102] S409, inputting the original image into a pre-trained image detection model;
[0103] S410: Obtain the annotation result of the original image output by the image detection model.
[0104] When the original image needs to be automatically annotated, that is, interactive click annotation is not required, the original image can be directly input into a pre-trained image detection model to obtain the annotation result of the original image. The image detection model can be any existing model with automatic image detection, segmentation, and annotation functions, which is not limited in the embodiments of the present disclosure.
[0105] In such Figure 5 In the interface shown, users can select and use existing image detection models in the Infer images function area to complete automatic detection, segmentation, and labeling.
[0106] In some embodiments, the user can also manually adjust the annotation result of the original image, for example, adjust the area corresponding to the target subject. Figure 5 In the interface shown, users can check and adjust the Json file in the Correct annotations ribbon.
[0107] In some embodiments, the user can also adjust the accuracy of image segmentation according to actual needs. Figure 5 In the interface shown, the user can set the accuracy of image segmentation in the Predictions threshold function area.
[0108] In some embodiments, Figure 5 The automatic annotation system shown also supports functions such as viewing the previous / next image and saving the image.
[0109] The disclosed embodiment integrates different image annotation methods for users to choose from, and can adjust the image annotation results according to different needs of users, thereby further improving the flexibility of the image annotation method.
[0110] In addition, the automatic annotation system provided by the embodiment of the present disclosure can implement the above-mentioned image annotation method through different function buttons, which is simple to operate and requires no learning cost, thereby further improving the efficiency of image annotation.
[0111] Figure 6 Schematic diagram of the structure of the image annotation device provided in the embodiment of the present disclosure. The image annotation device may be the terminal device as described in the above embodiment, or the image annotation device may be a component or assembly in the terminal device. The image annotation device provided in the embodiment of the present disclosure may execute the processing flow provided in the image annotation method embodiment, such as Figure 6 As shown, the image annotation device 60 includes: a display module 61, a first determination module 62, an input module 63, a first acquisition module 64, and a second acquisition module 65; wherein the display module 61 is used to display the original image, wherein the original image includes a target subject to be annotated; the first determination module 62 is used to determine at least one target position corresponding to the click operation in the original image in response to a user's click operation on the target subject; the input module 63 is used to input the original image with the target position information into a pre-trained image segmentation model; the first acquisition module 64 is used to obtain the image segmentation result for the target subject output by the image segmentation model; the second acquisition module 65 is used to add the label information corresponding to the target subject to the image segmentation result to obtain the annotation result of the original image.
[0112] Optionally, the image annotation device 60 further includes a third acquisition module 66, which is used to acquire label information corresponding to the target subject in response to a label input operation of a user for the target subject in the original image.
[0113] Optionally, the second acquisition module 65 includes an acquisition unit 651 and a conversion unit 652; the acquisition unit 651 is used to add label information corresponding to the target subject to the image segmentation result to obtain a semantic mask map of the original image; the conversion unit 652 is used to convert the format of the semantic mask map to obtain the annotation result of the original image.
[0114] Optionally, the image annotation device 60 also includes a second determination module 67, which is used to determine whether automatic image annotation is required based on the image processing task corresponding to the original image; the input module 63 is also used to input the original image into a pre-trained image detection model when automatic image annotation is required; the first acquisition module 64 is also used to obtain the annotation result of the original image output by the image detection model.
[0115] Figure 6 The image annotation device of the illustrated embodiment can be used to implement the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail here.
[0116] Figure 7 The schematic diagram of the structure of the model training device provided in the embodiment of the present disclosure. The model training device provided in the embodiment of the present disclosure can be used for the processing flow provided in the model training method embodiment, such as Figure 7 As shown, the model training device 70 includes: a fourth acquisition module 71, a generation module 72, and a training module 73; wherein the fourth acquisition module 71 is used to acquire a training image and a target area in the training image; the generation module 72 is used to generate annotation information of the training image according to the target area, and the annotation information at least includes point information of at least one first annotation point in the target area, and point information of at least one second annotation point outside the target area; the training module 73 is used to use the training image and the annotation information of the training image as input of an image segmentation model, and use the target area as output of the image segmentation model, to train the image segmentation model.
[0117] Optionally, the generation module 72 includes a generation unit 721, a calculation unit 722, and a determination unit 723; the generation unit 721 is used to generate a preset number of preset annotation points in the training image, and the preset annotation points include at least one first annotation point in the target area and at least one second annotation point outside the target area; the calculation unit 722 is used to calculate the relative distance between the target annotation point and other preset annotation points for the target annotation point among the preset annotation points, and obtain the relative distance vector corresponding to the target annotation point; the determination unit 723 is used to use the relative distance vector as the point information corresponding to the target annotation point.
[0118] Figure 7 The image annotation device of the illustrated embodiment can be used to implement the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail here.
[0119] Figure 8 The electronic device provided in the embodiment of the present disclosure may be a terminal device as described in the above embodiment. The electronic device provided in the embodiment of the present disclosure may execute the processing flow provided in the embodiment of the image annotation method, or the processing flow provided in the embodiment of the model training method, such as Figure 8 As shown, the electronic device 80 includes: a memory 81, a processor 82, a computer program and a communication interface 83; wherein the computer program is stored in the memory 81 and is configured so that the processor 82 executes the image annotation method or model training method as described above.
[0120] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the image annotation method or model training method described in the above embodiment.
[0121] In addition, an embodiment of the present disclosure also provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the image annotation method or model training method as described above is implemented.
[0122] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0123] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0124] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0125] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An image annotation method, characterized in that: The method comprises: Displaying an original image, wherein the original image includes a target subject to be annotated; In response to a click operation of a user on the target subject, determining at least one target position corresponding to the click operation in the original image; Inputting the original image with the target location information into a pre-trained image segmentation model; Obtaining an image segmentation result for the target subject output by the image segmentation model; The label information corresponding to the target subject is added to the image segmentation result to obtain the annotation result of the original image.
2. The method according to claim 1, characterized in that After displaying the original image, the method further includes: In response to a user's label input operation for a target subject in the original image, label information corresponding to the target subject is acquired.
3. The method according to claim 1, characterized in that The step of adding label information corresponding to the target subject to the image segmentation result to obtain a labeling result of the original image includes: Adding label information corresponding to the target subject to the image segmentation result to obtain a semantic mask image of the original image; The format of the semantic mask image is converted to obtain a labeling result of the original image.
4. The method according to claim 1, characterized in that After displaying the original image, the method further includes: Determining whether automatic image annotation is required according to an image processing task corresponding to the original image; If yes, the original image is input into a pre-trained image detection model; Obtain the annotation result of the original image output by the image detection model.
5. A model training method, characterized in that: The model is applied to the image annotation method according to any one of claims 1 to 4, and the model training method comprises: Acquire a training image and a target area in the training image; Generate annotation information of the training image according to the target area, the annotation information at least including point information of at least one first annotation point in the target area and point information of at least one second annotation point outside the target area; The training image and the annotation information of the training image are used as inputs of an image segmentation model, and the target area is used as output of the image segmentation model to train the image segmentation model.
6. The method according to claim 5, characterized in that Generating the annotation information of the training image according to the target area includes: Generating a preset number of preset annotation points in the training image, wherein the preset annotation points include at least one first annotation point in the target area and at least one second annotation point outside the target area; For a target marking point among the preset marking points, calculating the relative distance between the target marking point and other preset marking points to obtain a relative distance vector corresponding to the target marking point; The relative distance vector is used as point information corresponding to the target marked point.
7. An image annotation device, characterized in that: The device comprises: A display module, used to display an original image, wherein the original image includes a target subject to be annotated; A first determining module, configured to determine, in response to a user's click operation on the target subject, at least one target position corresponding to the click operation in the original image; An input module, used for inputting the original image with the target position information into a pre-trained image segmentation model; A first acquisition module, used to acquire the image segmentation result of the target subject output by the image segmentation model; The second acquisition module is used to add label information corresponding to the target subject to the image segmentation result to obtain the annotation result of the original image.
8. A model training device, characterized in that: The device comprises: A fourth acquisition module, used to acquire a training image and a target area in the training image; A generating module, configured to generate annotation information of the training image according to the target area, wherein the annotation information at least includes point information of at least one first annotation point in the target area and point information of at least one second annotation point outside the target area; The training module is used to use the training image and the annotation information of the training image as input of the image segmentation model, and use the target area as output of the image segmentation model to train the image segmentation model.
9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 4 or claims 5 to 6.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 or claims 5 to 6 is implemented.