Image-based target labeling method and device, equipment and storage medium

Abstract

The embodiment of the invention provides an image-based target labeling method and device, equipment and a storage medium, and belongs to the technical field of financial science and technology. The method comprises the steps of obtaining an original image, an unlabeled target and initial labeling data of the unlabeled target, wherein the initial labeling data comprises a type labeling text and labeling action data; performing regional image extraction on the original image based on the annotation action data to obtain a target regional image; performing image coding on the original image to obtain original image features; performing image coding on the target area image to obtain area image features; performing segmentation mask generation on the regional image features and the original image features based on a segmentation mask generation model to obtain a target segmentation mask; and performing data combination on the target segmentation mask and the type annotation text to obtain target annotation data of the unannotated target. According to the embodiment of the invention, the labeling efficiency and accuracy of the newly added unlabeled target in the image can be improved.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to an image-based target labeling method and apparatus, device, and storage medium. Background Art

[0002] Image-based target annotation refers to the process of adding labels to specific targets in an image. These labels can be text, borders, or other forms of markers. Model training based on target-annotated images enables the model to quickly identify targets in the image that match the annotation type. In the self-service car damage claims scenario of financial technology, the segmentation model trained based on target-annotated images can quickly segment targets in the image, such as auto parts, and further detect whether the target is damaged and the degree of damage, which can serve as the basis for insurance claims.

[0003] At present, the related art object annotation methods based on images usually annotate rectangular boxes or arbitrary polygons in the image. The rectangular boxes are generally used for detection, and the area surrounded by arbitrary polygons is used for segmentation. However, the types of annotations in the existing annotation sample set are limited, and because the manual actions required for rectangular and arbitrary polygon annotation in the image are relatively complicated, when the annotator annotates the newly added refined targets in the original image, it takes a long time and is prone to problems such as missing labels, wrong labels, and inaccurate annotation areas. Therefore, how to provide a method that can improve the efficiency and accuracy of annotating newly added unlabeled targets in an image has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose an image-based target labeling method and apparatus, device, and storage medium, which can improve the labeling efficiency and accuracy of newly added unlabeled targets in the image.

[0005] To achieve the above object, a first aspect of an embodiment of the present application proposes an image-based object labeling method, the method comprising:

[0006] Acquire an original image and an unlabeled object in the original image;

[0007] Acquire initial annotation data of the unlabeled target, wherein the initial annotation data includes type annotation text and annotation action data;

[0008] Extracting a region image from the original image based on the annotation action data to obtain a target region image;

[0009] Performing image encoding on the original image to obtain original image features;

[0010] Performing image encoding on the target area image to obtain regional image features;

[0011] Based on a preset segmentation mask generation model, a segmentation mask is generated for the regional image features and the original image features to obtain a target segmentation mask for the unlabeled target;

[0012] The target segmentation mask and the type annotation text are combined to obtain target annotation data of the unlabeled target.

[0013] In some embodiments, the labeling action data includes positive attribute labeling data and negative attribute labeling data, the area indicated by the positive attribute labeling data in the original image belongs to the unlabeled target, and the area indicated by the negative attribute labeling data in the original image does not belong to the unlabeled target;

[0014] The extracting a region image from the original image based on the annotation action data to obtain a target region image includes:

[0015] Performing region image selection on the original image based on the positive attribute annotation data to determine a target positive attribute region image;

[0016] Performing region image selection on the original image based on the negative attribute annotation data to determine a target negative attribute region image;

[0017] The target region image is obtained by performing image region selection on the original image based on the target positive attribute region image and the target negative attribute region image.

[0018] In some embodiments, the positive attribute annotation data includes a positive attribute annotation action and positively annotated pixels of the positive attribute annotation action;

[0019] The performing regional image segmentation on the original image based on the positive attribute annotation data to determine a target positive attribute regional image includes:

[0020] Performing target edge detection on the original image to obtain target edge data, wherein the target edge data includes target edge pixels, and a difference between adjacent pixels of the target edge pixels in the original image is greater than a preset pixel difference;

[0021] Selecting associated edge pixels from the target edge pixels based on the positive attribute labeling action;

[0022] A regional image is selected for the original image based on the area surrounded by the associated edge pixels to determine the target positive attribute regional image, wherein the target positive attribute regional image includes all the positively annotated pixels.

[0023] In some embodiments, combining the target segmentation mask and the type annotation text to obtain target annotation data of the unlabeled target includes:

[0024] Obtaining mask feature points from the target segmentation mask;

[0025] Performing difference calculation on the mask feature point and the positively labeled pixel point to obtain a deviation value of the candidate pixel point;

[0026] Adding and calculating the deviation values ​​of the candidate pixel points corresponding to all the mask feature points to obtain the target mask deviation value;

[0027] If the target mask deviation value is less than or equal to the preset deviation value, the target segmentation mask and the type annotation text are combined to obtain the target annotation data of the unlabeled target.

[0028] In some embodiments, the performing image region selection on the original image based on the target positive attribute region image and the target negative attribute region image to obtain the target region image includes:

[0029] Selecting an associated negative attribute region image from the target negative attribute region image based on the target positive attribute region image, wherein an overlap area between the associated negative attribute region image and the target positive attribute region image is greater than a preset region overlap threshold;

[0030] The overlapping area between the associated negative attribute area image and the target positive attribute area image is deleted from the target positive attribute area image to obtain the target area image.

[0031] In some embodiments, acquiring the original image and the unlabeled object in the original image includes:

[0032] Acquire a labeled image, wherein the labeled image includes candidate objects and candidate object types of the candidate objects;

[0033] Performing target screening on the candidate targets based on the pre-stored preset target type and the candidate target type to determine the unlabeled target, wherein the candidate target type of the unlabeled target is different from the preset target type;

[0034] The labeled image is segmented based on the unlabeled object to obtain the original image.

[0035] In some embodiments, the method further comprises:

[0036] Obtaining the number of unlabeled targets to obtain the number of newly added targets;

[0037] When the number of newly added targets is equal to a preset number threshold, constructing a newly added annotation sample based on the original image and the target annotation data;

[0038] The initial segmentation model is trained based on the newly added labeled samples and the preset sample set to obtain a target segmentation model.

[0039] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes an image-based object labeling device, the device comprising:

[0040] A first acquisition module, used to acquire an original image and an unlabeled object in the original image;

[0041] A second acquisition module is used to acquire initial annotation data of the unlabeled target, wherein the initial annotation data includes type annotation text and annotation action data;

[0042] An image extraction module, used for performing regional image extraction on the original image based on the annotation action data to obtain a target regional image;

[0043] A first encoding module, used to perform image encoding on the original image to obtain original image features;

[0044] A second encoding module is used to perform image encoding on the target area image to obtain area image features;

[0045] A mask generation module, used to generate a segmentation mask for the regional image features and the original image features based on a preset segmentation mask generation model to obtain a target segmentation mask of the unlabeled target;

[0046] The data combination module is used to combine the target segmentation mask and the type annotation text to obtain the target annotation data of the unlabeled target.

[0047] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0048] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0049] The image-based target labeling method, apparatus, device, and storage medium proposed in the embodiments of the present application generate a segmentation mask based on the encoded features of the target region image determined based on the labeling action data and the original image features corresponding to the original image, thereby obtaining the target segmentation mask of the unlabeled target, thereby determining the complete target labeling data of the unlabeled target. Specifically, the original image and the unlabeled target of the original image are obtained; further, the initial labeling data of the unlabeled target is obtained, the initial labeling data includes type labeling text and labeling action data, and the original image is subjected to regional image extraction based on the labeling action data to obtain the target region image; further, the original image is subjected to image encoding to obtain the original image features; the target region image is subjected to image encoding to obtain the regional image features; further, the regional image features and the original image features are subjected to segmentation mask generation based on a preset segmentation mask generation model to obtain the target segmentation mask of the unlabeled target; the target segmentation mask and the type labeling text are subjected to data combination to obtain the target labeling data of the unlabeled target. Compared with the related art that only uses manual annotation methods, this application can first determine the target area image in combination with the annotation action data, and then generate a target segmentation mask that can better annotate the unannotated targets based on the original image and the target area image. Therefore, this application can more effectively improve the annotation efficiency and accuracy of newly added unannotated targets in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a first flow chart of the image-based target labeling method provided by an embodiment of the present application;

[0051] Figure 2 yes Figure 1 A flowchart of step S110 in FIG.

[0052] Figure 3A is a schematic diagram of annotating an original image based on annotation points provided in an embodiment of the present application;

[0053] Figure 3B This is a schematic diagram of annotating an original image based on strokes provided in an embodiment of the present application;

[0054] Figure 4 yes Figure 1 A flowchart of step S130 in FIG.

[0055] Figure 5 is a schematic diagram of a target area image for determining an unlabeled target provided by an embodiment of the present application;

[0056] Figure 6 yes Figure 4 A flowchart of step S410 in FIG.

[0057] Figure 7yes Figure 4 A flowchart of step S430 in FIG.

[0058] Figure 8 is a schematic diagram of a flow chart of determining a target segmentation mask provided in an embodiment of the present application;

[0059] Fig. 9 yes Figure 1 A flowchart of step S170 in FIG.

[0060] Fig.10 is a second flow chart of the image-based target labeling method provided in an embodiment of the present application;

[0061] Fig.11 is a structural schematic diagram of an image-based target labeling device provided in an embodiment of the present application;

[0062] Fig.12 It is a hardware structure diagram of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] First, some nouns involved in this application are analyzed:

[0067] Artificial Intelligence (AI): is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0068] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further perform image processing to make computer processing into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image segmentation, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0069] Image-based target annotation refers to the process of adding labels to specific targets in an image. These labels can be text, borders, or other forms of marks. Model training based on target-annotated images enables the model to quickly identify targets in the image that match the annotation type. In the self-service car damage claims scenario of financial technology, the claimant is required to take a photo, and AI automatically identifies the parts involved in the photo. At this time, the segmentation model trained based on the target-annotated image can quickly segment the target in the image, such as auto parts, and further detect whether the target is damaged and the degree of damage, which can serve as the basis for insurance claims. The core modules of this system usually include auto parts detection (or segmentation) models and damage detection (or segmentation) models, and training these models requires a large number of labeled samples.

[0070] However, real-world car damage images are very complex. For example, almost all factors such as damage location, damage form, surrounding environment, lighting, occlusion, shooting equipment, shooting angle, etc. are difficult to control. Such scenarios have high requirements for the scale of training samples and the types of targets to be identified. Therefore, the types of targets supported by the system and the recognition performance of various targets are a continuous iterative process. The important influencing factor of this iteration is the labeling of new targets.

[0071] At present, the related art object annotation methods based on images usually annotate rectangular boxes or arbitrary polygons in the image. The rectangular boxes are generally used for detection, and the area surrounded by arbitrary polygons is used for segmentation. However, the types of annotations in the existing annotation sample set are limited, and because the manual actions required for rectangular and arbitrary polygon annotation in the image are relatively complicated, when the annotator annotates the newly added refined targets in the original image, it takes a long time and is prone to problems such as missing labels, wrong labels, and inaccurate annotation areas. Therefore, how to provide a method that can improve the efficiency and accuracy of annotating newly added unlabeled targets in an image has become a technical problem that needs to be solved urgently.

[0072] Based on this, the embodiments of the present application provide an image-based target labeling method and apparatus, device, and storage medium, which can improve the labeling efficiency and accuracy of newly added unlabeled targets in an image.

[0073] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0074] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0075] The image-based target labeling method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The image-based target labeling method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN) and cloud servers that provide basic cloud computing services such as big data and artificial intelligence platforms; the software can be an application that implements an image-based target labeling method, etc., but is not limited to the above forms.

[0076] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network personal computers (PC), minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0077] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to the identity or characteristics of the object, such as object identity information, object behavior data, object historical data, and object location information, the permission or consent of the object will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain sensitive personal information of the object, the separate permission or separate consent of the object will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the separate permission or separate consent of the object, the necessary object-related data for the normal operation of the embodiment of the present application will be obtained.

[0078] See also Figure 1 , Figure 1 is an optional flow chart of an image-based target annotation method provided in an embodiment of the present application. In some embodiments of the present application, Figure 1 The method may specifically include but is not limited to steps S110 to S170. Figure 1 These seven steps are introduced in detail.

[0079] Step S110, obtaining an original image and an unlabeled object in the original image;

[0080] Step S120, obtaining initial labeling data of the unlabeled target;

[0081] Step S130, extracting a region image from the original image based on the annotated action data to obtain a target region image;

[0082] Step S140, performing image encoding on the original image to obtain original image features;

[0083] Step S150, performing image encoding on the target region image to obtain region image features;

[0084] Step S160, generating a segmentation mask for the regional image features and the original image features based on a preset segmentation mask generation model to obtain a target segmentation mask for the unlabeled target;

[0085] Step S170, combining the target segmentation mask and the type annotation text to obtain target annotation data of the unlabeled target.

[0086] In step S110 of some embodiments, the original image refers to an image that contains at least one unlabeled target. An unlabeled target is equivalent to a newly added target that has not been fully labeled, that is, the category corresponding to the unlabeled target is a labeled category that is not included in the existing preset sample set. For example, in insurance claims, the category corresponding to the unlabeled target may be a component type or damage form that is not included. For example, the existing labeled categories include the left rearview mirror, but there is no further subdivision of components such as lenses and mirror rings. In the future, it is necessary to add the left rearview mirror lens, left rearview mirror ring, etc. for labeling. For another example, the existing damage types of sheet metal parts include scratches, dents, and cracks, and in the future, it is necessary to add the bends for labeling. In this way, by specifically and completely labeling the unlabeled targets, the newly added categories can be added to the existing preset sample set, so as to facilitate the subsequent detection (or segmentation) model to detect (or segment) the targets of this target type.

[0087] See also Figure 2 , Figure 2is an optional flowchart of step S110 provided in the embodiment of the present application. In some embodiments of the present application, step S110 may specifically include steps S210 to S230. Figure 2 These three steps are introduced in detail.

[0088] Step S210, obtaining a labeled image;

[0089] Step S220, screening candidate targets based on pre-stored preset target types and candidate target types to determine unlabeled targets;

[0090] Step S230 , performing image segmentation on the labeled image based on the unlabeled object to obtain an original image.

[0091] In step S210 of some embodiments, the annotated image refers to an image containing at least one target, and the annotated image includes at least one candidate target and a candidate target type corresponding to the candidate target. The candidate target can be a target that has been annotated, that is, the target type corresponding to the target has been stored in the preset sample set, and the candidate target can also be an unannotated target, that is, a newly added target that needs to be annotated in this application. The candidate target type refers to the target type to which the corresponding candidate target belongs, such as the target types that need to be newly annotated, such as the left rearview mirror lens, the left rearview mirror ring, etc.

[0092] In step S220 of some embodiments, the preset target type refers to a target type that has been stored in the preset sample set. After each candidate target type is obtained, the preset target type and the candidate target type can be type matched to determine whether the candidate target is a target of the preset target type that has been stored in the preset sample set.

[0093] It should be noted that the candidate target type can be in the form of annotated text, so that the preset target type and the candidate target type can be text-matched. Since the newly added target type is similar to the description of the pre-stored target type, such as the left rearview mirror and the left rearview mirror lens, only when the type text is completely matched can it be determined that the preset target type and the candidate target type are successfully matched, thereby determining that the candidate target is a target of the preset target type that has been stored in the preset sample set. In this way, if the type text does not completely match, it can be determined that the preset target type and the candidate target type are not successfully matched, thereby determining that the candidate target is a newly added unlabeled target.

[0094] It should be noted that, if it is found that there is no unlabeled object in the labeled image, step S210 may be re-executed until an unlabeled object is found.

[0095] In step S230 of some embodiments, after the unlabeled targets are determined, the labeled image is segmented based on the unlabeled targets to segment and eliminate the labeled targets that do not affect the labeling process of the unlabeled targets, so that the unlabeled targets can be more prominent in the original image, thereby improving the labeling efficiency of the unlabeled targets due to irrelevant information.

[0096] In the above steps S210 to S230, the present application matches the preset target type with the candidate target type of the candidate target in the annotated image to determine whether the annotated image contains a newly added unlabeled target, and further divides the annotated image based on the unlabeled target to obtain the original image. In this way, the efficiency of labeling the newly added unlabeled target in the image can be improved, and repeated labeling of the labeled target can be avoided.

[0097] In step S120 of some embodiments, after the unlabeled target is determined, initial labeling data of the unlabeled target can be further obtained, and the initial labeling data includes type labeling text and labeling action data. The type labeling text is used to indicate the target type of the newly added unlabeled target, and the labeling action data is used to locate the edge area of ​​the unlabeled target. If the newly added unlabeled target is the left rearview mirror and the left rearview mirror lens, the type labeling text can be "car left rearview mirror" and "car left rearview mirror lens".

[0098] It should be noted that the annotation action data can be data generated based on the annotation action of the annotation object on the unannotated target, and the annotation action can be a point, a rectangular box, a stroke, etc., and because the point, rectangular box, stroke, etc. have a certain amount of leeway, they do not require the precise position of the traditional annotation rectangular box or edge connection polygon, which can significantly reduce the annotation time of each image. Figure 3A As shown in the figure, the type annotation text corresponding to the unlabeled target in the original image is "car left rearview mirror frame". If the action of labeling points is used to generate labeling action data, then the original image can be labeled in the form of points (such as Figure 3A The solid circle and hollow circle in the figure mark the area where the left rearview mirror frame of the car is located. Figure 3B As shown in FIG. 1 , if the stroke action is used to generate the annotation action data, then the original image can be marked with irregular lines (such as Figure 3B The area where the left rearview mirror frame of the car is located is marked in the form of irregular solid lines and irregular dotted lines.

[0099] It should be noted that the annotation action data may include at least one positive attribute annotation data and at least one negative attribute annotation data. The area indicated by the positive attribute annotation data in the original image belongs to an unlabeled target, and the area indicated by the negative attribute annotation data in the original image does not belong to an unlabeled target. The embodiment of the present application may use different annotation actions to represent annotation data of different attribute types, that is, each type of annotation action may have both positive and negative attributes. Figure 3A As shown in the figure, the data corresponding to the annotation points formed by the solid circles can represent the positive attribute annotation data, and the data corresponding to the annotation points formed by the hollow circles can be used to represent the negative attribute annotation data. In addition, the negative attribute annotation data can also be represented in the form of strokes, such as Figure 3B As shown, the data corresponding to the irregular solid line can represent positive attribute labeling data, and the data corresponding to the irregular dotted line can represent negative attribute labeling data. The present application does not specifically limit the specific implementation form of the positive and negative attribute labeling action.

[0100] It should be noted that, in actual applications, the marking actions corresponding to the positive and negative attributes of the present application can be implemented independently by the mouse, or by a combination of the mouse and shortcut keys, which is not specifically limited here.

[0101] It should be noted that the positive attribute annotated data in this application is not strictly required to be within the area corresponding to the unlabeled target. A small amount of cross-border data is allowed without affecting the segmentation effect, thus relaxing the restrictions on manual operations.

[0102] It should be noted that if the unlabeled target is not further subdivided into parts, the newly added unlabeled target is usually near the labeled target. In this case, the newly added unlabeled target can be located based on the existing associated annotations. For example, if the unlabeled target is the left rearview mirror frame of the car, the position of the left rearview mirror annotated in the preset sample set can be found first, which is the position of the newly added left rearview mirror lens and left rearview mirror ring. If the newly added target is a new damage, that is, there is no associated labeled target, the damage position can also be located based on the existing damage annotations, because damage usually occurs in a certain position.

[0103] In step S130 of some embodiments, after the labeling action data corresponding to each unlabeled target is determined, a region image may be extracted from the original image based on the labeling action data to obtain a target region image.

[0104] See also Figure 4 , Figure 4 is an optional flowchart of step S130 provided in an embodiment of the present application. In some embodiments of the present application, step S130 may specifically include steps S410 to S430. Figure 4 These three steps are introduced in detail.

[0105] Step S410, performing region image selection on the original image based on the positive attribute annotation data to determine a target positive attribute region image;

[0106] Step S420, performing region image selection on the original image based on the negative attribute annotation data to determine a target negative attribute region image;

[0107] Step S430 , performing image region selection on the original image based on the target positive attribute region image and the target negative attribute region image to obtain a target region image.

[0108] In step S410 of some embodiments, the positive attribute annotation data includes a positive attribute annotation action and positively annotated pixels of the positive attribute annotation action, that is, at least one positively annotated pixel constitutes a positive attribute annotation action. For example, if a positive attribute annotation action is performed in a rectangular frame, all positively annotated pixels contained in a positive attribute rectangular frame can determine a corresponding positive attribute annotation action. Further, a target positive attribute region image can be determined from the original image based on multiple positive attribute annotation data, wherein the target positive attribute region image almost contains all positively annotated pixels corresponding to the positive attribute annotation data.

[0109] In step S420 of some embodiments, similarly, the negative attribute annotation data includes the negative attribute annotation action and the negatively annotated pixel points of the negative attribute annotation action, that is, at least one negatively annotated pixel point constitutes the negative attribute annotation action. For example, if the negative attribute annotation action is performed in the form of a rectangular frame, all the negatively annotated pixel points contained in a rectangular frame of a negative attribute can determine a corresponding negative attribute annotation action. Further, a target negative attribute area image can be determined from the original image based on multiple negative attribute annotation data, wherein the target positive attribute area image almost contains all the negatively annotated pixel points corresponding to the negative attribute annotation data.

[0110] It should be noted that each time a positive attribute annotation data is obtained, the current target positive attribute area image that meets the requirements can be determined in time, and after obtaining the next positive attribute annotation data, the current positive attribute annotation data that meets the requirements can be re-determined based on the current two positive attribute annotation data. In other words, after obtaining each annotation action of the annotation object, the impact on the annotation mask can be seen immediately, and after obtaining the next annotation action, a more accurate annotation mask can be modified.

[0111] For example, Figure 5As shown, the original image is the left side rearview mirror of a vehicle, and the current unlabeled target is the "left side rearview mirror frame". If the positive attribute labeling action and the negative attribute labeling action are labeled in the form of labeling points, and the solid labeling points represent the positive attribute labeling action, and the hollow labeling points represent the negative attribute labeling action, then, based on the labeling points corresponding to the multiple positive attribute labeling actions, the target positive attribute area image 510 (i.e., the area surrounded by the labeling points corresponding to the multiple positive attribute labeling actions) can be determined, and based on the labeling points corresponding to the multiple negative attribute labeling actions, the target negative attribute area image 520 (i.e., the area surrounded by the labeling points corresponding to the multiple negative attribute labeling actions) can be determined.

[0112] See also Figure 6 , Figure 6 is an optional flowchart of step S410 provided in an embodiment of the present application. In some embodiments of the present application, step S410 may specifically include steps S610 to S630. Figure 6 These three steps are introduced in detail.

[0113] Step S610, performing target edge detection on the original image to obtain target edge data;

[0114] Step S620, selecting associated edge pixels from target edge pixels based on the positive attribute labeling action;

[0115] Step S630 , performing regional image selection on the original image based on the area surrounded by the associated edge pixels to determine a target positive attribute regional image.

[0116] In step S610 of some embodiments, before obtaining the positive attribute annotation data and the negative attribute annotation data, the present application may first perform target edge detection on the original image to obtain target edge data. Target edge data refers to edge data that divides the target range in the original image, and the target edge data includes target edge pixels, and the target edge pixel difference in the original image is greater than the preset pixel difference. In other words, a target range can be determined between multiple similar target edge pixels, that is, the edge range of the target can be determined based on the lines formed by the pixels with obvious pixel changes.

[0117] In step S620 of some embodiments, after the positive attribute annotation data is obtained, multiple associated edge pixel points that are closer to the positively annotated pixel point can be selected from the target edge pixel points based on the positive attribute annotation action, and the area surrounded by the multiple associated edge pixel points at this time can include the positive attribute annotation action. In this way, by selecting associated edge pixel points from the target edge pixel points, the range of the positive attribute annotation action in the original image can be further determined, thereby improving the accuracy of the annotation.

[0118] In step S630 of some embodiments, further, regional image selection is performed on the original image based on the area surrounded by the associated edge pixels to determine the target positive attribute regional image, that is, the edge pixels of the target positive attribute regional image are the associated edge pixels, and the target positive attribute regional image can contain all or more than a preset proportion (such as 80%, 90%, etc.) of the positively annotated pixels.

[0119] It should be noted that after obtaining the negative attribute annotation data, similar to the above steps S620 and S630, the associated edge pixel points that are closer to the negatively annotated pixel points are first determined, so as to determine the target negative attribute area image from the original image based on the area enclosed by the selected associated edge pixel points, and the target negative attribute area image can contain all or more than a preset proportion (such as 80%, 90%, etc.) of the negatively annotated pixel points.

[0120] In the above steps S610 to S630, based on the association between the corresponding labeled pixel points and the target edge pixel points in the labeling action data, the target positive attribute area image and the target negative attribute area image are divided from the original image, which can more accurately locate the area of ​​the unlabeled target, thereby improving the labeling accuracy of the unlabeled target.

[0121] In step S430 of some embodiments, after the target positive attribute area image and the target negative attribute area image are determined from the original image, image area selection can be performed on the original image to obtain a target area image. At this time, the target area image is the area image corresponding to the unlabeled target in the original image.

[0122] See also Figure 7 , Figure 7 is an optional flowchart of step S430 provided in the embodiment of the present application. In some embodiments of the present application, step S430 may specifically include steps S710 to S720. Figure 7 These two steps are introduced in detail.

[0123] Step S710, selecting an associated negative attribute region image from the target negative attribute region image based on the target positive attribute region image;

[0124] Step S720: deleting the overlapping area between the associated negative attribute area image and the target positive attribute area image from the target positive attribute area image to obtain the target area image.

[0125] In step S710 and step S720 of some embodiments, since there is only one target positive attribute region image determined based on the positive attribute labeling action, and the target negative attribute region image may include multiple target negative attribute region images, then an associated negative attribute region image may be selected from the target negative attribute region image based on the target positive attribute region image, and the overlap area of ​​the associated negative attribute region image and the target positive attribute region image is greater than a preset region overlap threshold. Figure 5 As shown, the original image is a left rearview mirror of a vehicle, the target positive attribute area image 510 determined based on the annotation points corresponding to the multiple positive attribute annotation actions is the maximum frame of the left rearview mirror, and the target negative attribute area image 520 determined based on the annotation points corresponding to the multiple negative attribute annotation actions is the frame of the lens in the left rearview mirror. Since the original image does not contain other negative attribute associated edge pixels, the target negative attribute area image 520 at this time is the associated negative attribute area image. At this time, the overlapping area of ​​the associated negative attribute area image and the target positive attribute area image is deleted from the target positive attribute area image 510, and the target area image 530 can be determined, that is, the area surrounded by the two lines in the figure is the mirror circle of the left rearview mirror.

[0126] In the above embodiment, the present application retains the pixel area corresponding to the positive attribute labeling action and masks the pixel area corresponding to the negative attribute labeling action, so that the target pixel area corresponding to the unlabeled target can be obtained, thereby improving the accuracy of target labeling.

[0127] In step S140 and step S150 of some embodiments, after determining the target area image, the present application can encode the original image and the target area image respectively, so as to obtain the original image features and the regional image features corresponding to the unlabeled target. The purpose of image encoding is to extract the key features of the image so that subsequent processing can be performed more effectively. The encoding process can use a deep learning model such as a convolutional neural network (CNN) to convert the image into a feature vector to represent the content and structure of the image. The regional image features and the original image features can include detail information such as edges, textures, and color distributions.

[0128] In step S160 of some embodiments, after obtaining the regional image features and the original image features, a preset segmentation mask generation model can be used to generate a target segmentation mask of the unlabeled target. This model can be a trained deep learning network that aims to identify different regions and assign masks to them, and the mask can identify which parts belong to the unlabeled target. The target segmentation mask can be a binary image, and the region corresponding to the unlabeled target can be assigned a value of 1, and the background region unrelated to the unlabeled target can be assigned a value of zero.

[0129] Example type, such as Figure 8 As shown, the present application can input the original image into the image coding model for image coding to obtain the original image features. At the same time, the initial annotation data of the unlabeled target in the original image can be input into the preset prompt word encoding model, and the prompt word encoding model can first perform regional image extraction on the original image based on the annotation action data to obtain the target regional image, and then perform image encoding on the target regional image to obtain regional image features. Afterwards, the regional image features and the original image features can be input together into the segmentation mask generation model for segmentation mask generation to obtain the target segmentation mask of each unlabeled target.

[0130] In step S170 of some embodiments, the target segmentation mask obtained is the mask image required for accurately segmenting the unlabeled target in the original image. In addition, after the annotation action data changes, the corresponding target segmentation mask will also change accordingly. Further, the target segmentation mask is combined with the relevant type annotation text to obtain the target annotation data of the unlabeled target. At this time, the target annotation data can be equivalent to a labeled sample based on the unlabeled target.

[0131] In some other embodiments, the present application may also perform a feature extraction on the area image corresponding to each annotation action in the annotation action data, and input the feature together with the original image features into the segmentation mask generation model for mask generation.

[0132] In the above embodiment, the present application accurately generates the target segmentation mask corresponding to the unlabeled target, and combines it with the type annotation text corresponding to the unlabeled target to form a data set for further analysis or training. This data combination is very important for the training and promotion of the model, and can help the model understand and identify new unlabeled targets.

[0133] See also Fig. 9 , Fig. 9 is an optional flowchart of step S170 provided in an embodiment of the present application. In some embodiments of the present application, step S170 may specifically include steps S910 to S940. Fig. 9 These four steps are introduced in detail.

[0134] Step S910, obtaining mask feature points from the target segmentation mask;

[0135] Step S920, performing difference calculation on the mask feature point and the positively labeled pixel point to obtain a deviation value of the candidate pixel point;

[0136] Step S930, summing up the candidate pixel deviation values ​​corresponding to all mask feature points to obtain a target mask deviation value;

[0137] Step S940: If the target mask deviation value is less than the preset deviation value, the target segmentation mask and the type annotation text are combined to obtain target annotation data of the unlabeled target.

[0138] In step S910 of some embodiments, since the acquired initial annotation data may have errors, resulting in the generated target segmentation mask being inaccurate, based on this, after obtaining the target segmentation mask, mask feature points may be extracted from the generated target segmentation mask. These feature points may be the boundaries or key positions of the region of interest in the segmentation mask, which can be used for subsequent processing, or they may be each feature point in the target segmentation mask. The process of obtaining mask feature points may involve image processing techniques, such as edge detection and contour extraction, to find the main structure in the segmentation mask.

[0139] In step S920 of some embodiments, further, a difference calculation is performed between each mask feature point and the positively annotated pixel point closest to the mask feature point to obtain a candidate pixel point deviation value. Because the positively annotated pixel point is related to the area where the unannotated object is located, this can help determine the accuracy and effectiveness of the mask feature point.

[0140] In step S930 of some embodiments, further, the deviation values ​​of the target pixels corresponding to all mask feature points are added up to obtain an overall target mask deviation value. That is, by summing up the deviation values ​​of each mask feature point, the quality of the entire target segmentation mask can be more comprehensively evaluated. This result provides a basis for determining whether the unlabeled target is accurately labeled.

[0141] In step S940 of some embodiments, after obtaining the target mask deviation corresponding to the current target segmentation mask, the target mask deviation value and the preset deviation value can be numerically compared. If the target mask deviation value is less than or equal to the preset deviation value, the target segmentation mask and the type annotation text are combined to obtain the target annotation data of the unlabeled target. If the target mask deviation value is greater than the preset deviation value, it means that the currently acquired target segmentation mask is not accurate enough, and the initial annotation data can be re-acquired to update the target segmentation mask. Among them, the preset deviation value refers to the preset deviation value required to obtain the target segmentation mask of the preset accuracy. For example, when the preset accuracy is 90%, the preset deviation value is 0.1.

[0142] In the above embodiment, after obtaining the target segmentation mask, the present application will also detect the target segmentation mask to determine whether the generated target segmentation mask can meet the mask requirements for unlabeled targets, thereby improving the subsequent training capabilities of the target detection model (or target segmentation model) based on the target segmentation mask, and can efficiently and accurately extract specific targets based on the target segmentation mask, ensuring that the generated labeled data is of high quality and reliability.

[0143] See also Fig.10 , Fig.10 is another optional flow chart of the method for object labeling based on an image provided by an embodiment of the present application. In some embodiments of the present application, after step S170, the method for object labeling based on an image provided by the present application may further include steps S1010 to S1030. Fig.10 These three steps are introduced in detail.

[0144] Step S1010, obtaining the number of unlabeled targets and obtaining the number of newly added targets;

[0145] Step S1020, when the number of newly added targets is equal to a preset number threshold, constructing a newly added annotation sample based on the original image and the target annotation data;

[0146] Step S1030, training the initial segmentation model based on the newly added labeled samples and the preset sample set to obtain a target segmentation model.

[0147] In steps S1010 to S1030 of some embodiments, after determining the target annotation data of multiple unlabeled targets, the present application can count the number of unlabeled targets in real time. When the number of newly added targets is equal to a preset number threshold, a preset number threshold of newly labeled samples can be constructed based on the original image and the target annotation data. Furthermore, the initial segmentation model can be trained based on the newly added annotation samples and the preset sample set to obtain a target segmentation model. That is to say, in practical applications, when the annotation amount of the newly added targets accumulates to the preset standard, the fine-tuning segmentation model can be trained on the basis of the initial segmentation model to improve its segmentation accuracy for the newly added targets. Furthermore, the segmentation model with improved accuracy can replace the initial segmentation model to assist manual annotation with better accuracy. In this way, a small number of rounds of iterations are performed to eventually obtain a model that meets the requirements for the support effect of the segmentation of the newly added targets for use as production deployment.

[0148] It should be noted that the non-Company software tools or components appearing in the embodiments of the present application are merely examples and do not represent actual use.

[0149] The target annotation method based on an image provided by the embodiment of the present application can improve the annotation speed of the newly added target with the help of the trained segmentation model of the supported target, such as removing the target area that is not associated with the unlabeled target from the original image. The annotation object can flexibly select the annotation action according to the specific situation, including the annotation forms such as points, rectangular boxes, and strokes, and there is a certain amount of leeway for the rectangular boxes and strokes, and the precise position of the traditional annotation rectangular boxes or edge-connected polygons is not required, so the annotation time for each image can be significantly reduced. In addition, the annotation object can see the impact on the annotation mask immediately for each annotation action, and can make the next annotation action immediately, modify to obtain a more accurate annotation mask, and the annotation action has no order, and the operation is more flexible, unlike rectangles or edge-connected polygons, which need to be in a predetermined order. Therefore, compared with the method of only using manual annotation in the related art, the present application can combine the annotation action data to first determine the target area image, and then generate a target segmentation mask that can better annotate the unlabeled target based on the original image and the target area image, so as to realize the automatic iteration of sample annotation and model training, and can more effectively improve the annotation efficiency and accuracy of the newly added unlabeled targets in the image.

[0150] See also Fig.11 , Fig.11 : is a schematic diagram of the module structure of the image-based target labeling device provided in an embodiment of the present application. The device can implement the image-based target labeling method of the above embodiment. The device may specifically include:

[0151] A first acquisition module 1110 is used to acquire an original image and an unlabeled object in the original image;

[0152] A second acquisition module 1120 is used to acquire initial annotation data of an unlabeled target, where the initial annotation data includes type annotation text and annotation action data;

[0153] An image extraction module 1130 is used to extract a region image from the original image based on the annotated action data to obtain a target region image;

[0154] A first encoding module 1140 is used to perform image encoding on the original image to obtain original image features;

[0155] The second encoding module 1150 is used to perform image encoding on the target region image to obtain region image features;

[0156] The mask generation module 1160 is used to generate a segmentation mask for the regional image features and the original image features based on a preset segmentation mask generation model to obtain a target segmentation mask for the unlabeled target;

[0157] The data combination module 1170 is used to combine the target segmentation mask and the type annotation text to obtain target annotation data of the unlabeled target.

[0158] It should be noted that the image-based target labeling device of the embodiment of the present application is used to implement the image-based target labeling method of the above-mentioned embodiment. The image-based target labeling device of the embodiment of the present application corresponds to the aforementioned image-based target labeling method. Please refer to the aforementioned image-based target labeling method for the specific processing process, which will not be repeated here.

[0159] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned image-based target labeling method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0160] See also Fig.12 , Fig.12 The hardware structure of a computer device according to another embodiment is illustrated. The computer device includes:

[0161] The processor 1210 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0162] The memory 1220 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1220 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1220, and the processor 1210 calls and executes the image-based target labeling method of the embodiment of this application;

[0163] Input / output interface 1230, used to implement information input and output;

[0164] The communication interface 1240 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WI FI, Bluetooth, etc.);

[0165] The bus 1250 transmits information between the various components of the device (e.g., the processor 1210, the memory 1220, the input / output interface 1230, and the communication interface 1240);

[0166] The processor 1210 , the memory 1220 , the input / output interface 1230 , and the communication interface 1240 are connected to each other in communication within the device via a bus 1250 .

[0167] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned image-based target labeling method is implemented.

[0168] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0169] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0170] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0171] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0172] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0173] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0174] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0175] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0176] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0177] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0178] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store programs.

[0179] The preferred embodiments of the present invention are described with reference to the accompanying drawings, but the scope of the present invention is not limited by the above. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. An image-based target labeling method, characterized in that: The method comprises: Acquire an original image and an unlabeled object in the original image; Acquire initial annotation data of the unlabeled target, wherein the initial annotation data includes type annotation text and annotation action data; Extracting a region image from the original image based on the annotation action data to obtain a target region image; Performing image encoding on the original image to obtain original image features; Performing image encoding on the target area image to obtain regional image features; Based on a preset segmentation mask generation model, a segmentation mask is generated for the regional image features and the original image features to obtain a target segmentation mask of the unlabeled target; The target segmentation mask and the type annotation text are combined to obtain target annotation data of the unlabeled target.

2. The method according to claim 1, characterized in that The labeling action data includes positive attribute labeling data and negative attribute labeling data, the area indicated by the positive attribute labeling data in the original image belongs to the unlabeled target, and the area indicated by the negative attribute labeling data in the original image does not belong to the unlabeled target; The extracting a region image from the original image based on the annotation action data to obtain a target region image includes: Performing region image selection on the original image based on the positive attribute annotation data to determine a target positive attribute region image; Performing region image selection on the original image based on the negative attribute annotation data to determine a target negative attribute region image; The target region image is obtained by performing image region selection on the original image based on the target positive attribute region image and the target negative attribute region image.

3. The method according to claim 2, characterized in that The positive attribute labeling data includes a positive attribute labeling action and positively labeled pixels of the positive attribute labeling action; The performing regional image segmentation on the original image based on the positive attribute annotation data to determine a target positive attribute regional image includes: Performing target edge detection on the original image to obtain target edge data, wherein the target edge data includes target edge pixels, and a difference between adjacent pixels of the target edge pixels in the original image is greater than a preset pixel difference; Selecting associated edge pixels from the target edge pixels based on the positive attribute labeling action; A regional image is selected for the original image based on the area surrounded by the associated edge pixels to determine the target positive attribute regional image, wherein the target positive attribute regional image includes all the positively annotated pixels.

4. The method according to claim 3, characterized in that The step of combining the target segmentation mask and the type annotation text to obtain target annotation data of the unlabeled target includes: Obtaining mask feature points from the target segmentation mask; Performing difference calculation on the mask feature point and the positively labeled pixel point to obtain a deviation value of the candidate pixel point; Adding and calculating the deviation values ​​of the candidate pixel points corresponding to all the mask feature points to obtain the target mask deviation value; If the target mask deviation value is less than or equal to the preset deviation value, the target segmentation mask and the type annotation text are combined to obtain the target annotation data of the unlabeled target.

5. The method according to claim 2, characterized in that: The step of selecting an image region of the original image based on the target positive attribute region image and the target negative attribute region image to obtain the target region image includes: Selecting an associated negative attribute region image from the target negative attribute region image based on the target positive attribute region image, wherein an overlap area between the associated negative attribute region image and the target positive attribute region image is greater than a preset region overlap threshold; The overlapping area between the associated negative attribute area image and the target positive attribute area image is deleted from the target positive attribute area image to obtain the target area image.

6. The method according to any one of claims 1 to 5, characterized in that: The obtaining of the original image and the unlabeled object of the original image comprises: Acquire a labeled image, wherein the labeled image includes candidate objects and candidate object types of the candidate objects; Performing target screening on the candidate targets based on the pre-stored preset target type and the candidate target type to determine the unlabeled target, wherein the candidate target type of the unlabeled target is different from the preset target type; The labeled image is segmented based on the unlabeled object to obtain the original image.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtaining the number of unlabeled targets to obtain the number of newly added targets; When the number of newly added targets is equal to a preset number threshold, constructing a newly added annotation sample based on the original image and the target annotation data; The initial segmentation model is trained based on the newly added labeled samples and the preset sample set to obtain a target segmentation model.

8. An image-based target labeling device, characterized in that: The device comprises: A first acquisition module, used to acquire an original image and an unlabeled object in the original image; A second acquisition module is used to acquire initial annotation data of the unlabeled target, wherein the initial annotation data includes type annotation text and annotation action data; An image extraction module, used for performing regional image extraction on the original image based on the annotation action data to obtain a target regional image; A first encoding module, used to perform image encoding on the original image to obtain original image features; A second encoding module is used to perform image encoding on the target area image to obtain area image features; A mask generation module, used to generate a segmentation mask for the regional image features and the original image features based on a preset segmentation mask generation model to obtain a target segmentation mask of the unlabeled target; The data combination module is used to combine the target segmentation mask and the type annotation text to obtain the target annotation data of the unlabeled target.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images