Image annotation method and electronic equipment
By performing image segmentation and positioning recognition on the image dataset, building an annotation database and training a general image recognition model, the problem of time-consuming and labor-intensive image labeling and low quality in the prior art is solved, and efficient and accurate image labeling is achieved.
Patent Information
- Application Number
- CN202411890648.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-20
- Publication Date
- 2025-04-11
AI Technical Summary
In the prior art, the image labeling process is time-consuming and labor-intensive and susceptible to personal subjective factors. The quality of automated labeling is low, and an improved image labeling method is urgently needed.
By performing image segmentation and positioning recognition on the acquired image data set, an annotation database is constructed, and the general image recognition model is trained based on this to obtain a model with the ability to output the annotation.
It realizes the learning of the accurate position and boundary feature information of different objects to be marked in the image data, and can accurately mark the specific positions and categories of different objects in the image, improving the efficiency and quality of image labeling.
Smart Images
Figure CN119810541A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to an image annotation method and an electronic device. Background Art
[0002] With the advancement of computer technology, image annotation is becoming increasingly important in the fields of computer vision and artificial intelligence.
[0003] In the related art, image annotation can be achieved through manual annotation or automatic annotation based on deep learning models. Among them, manual annotation has good annotation effect, but the annotation process is time-consuming and labor-intensive, and is easily affected by personal subjective factors. Although automatic annotation based on deep learning models greatly reduces manpower, the annotation quality is low. Therefore, a better image annotation method is urgently needed. Summary of the invention
[0004] In order to solve the above technical problems, the present disclosure provides an image annotation method and an electronic device, which can solve the problem of insufficient sample data.
[0005] The technical solution of the present disclosure is as follows:
[0006] In a first aspect, the present disclosure provides an image annotation method, which is applied to an electronic device, and the method includes: acquiring an image data set; the image data set includes at least one image data; performing image segmentation and positioning recognition on at least one image data in the image data set to obtain an annotation database, and the annotation database includes annotation data of at least one image data; using the annotation data of at least one image data in the annotation database and at least one image data to train a general image recognition model to obtain a trained general image recognition model; the trained general image recognition model has the ability to output annotated images.
[0007] In combination with the first aspect, in another possible implementation manner, the method further includes: obtaining the image to be labeled; inputting the image to be labeled into a trained general image recognition model and outputting the labeled image to be labeled; and obtaining a correction result of the labeled image to be labeled.
[0008] In combination with the first aspect, in another possible implementation manner, the method further includes: updating weight parameters of the trained general image recognition model using the image to be annotated and the correction results of the annotated image to be annotated.
[0009] In combination with the first aspect, in another possible implementation method, image segmentation and positioning recognition are performed on at least one image data in the image data set to obtain a labeling database, including: semantic segmentation and instance segmentation are performed on at least one image data to obtain an object to be identified in at least one image data; image segmentation includes semantic segmentation and / or instance segmentation; positioning recognition is performed on the object to be identified to obtain a category label of the object to be identified; and a correction result corresponding to the category label of the object to be identified is obtained, and the labeling data of at least one image data in the labeling database includes the correction result corresponding to the category label of the object to be identified.
[0010] In combination with the first aspect, in another possible implementation method, before performing image segmentation and positioning recognition on at least one image data in the image data set, the method also includes: preprocessing at least one image data in the image data set to obtain processed image data; the preprocessing includes at least one of denoising, cropping, and scaling; and storing the processed image data.
[0011] In combination with the first aspect, in another possible implementation manner, the annotation format of the annotation data is COCO or Pascal VOC.
[0012] In a second aspect, the present disclosure provides an image annotation device, which can be applied to an electronic device to implement the method in the first aspect. The functions of the image annotation device can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, such as an acquisition module, a processing module, and a training module.
[0013] The acquisition module is configured to acquire an image data set; the image data set includes at least one image data.
[0014] The processing module is configured to perform image segmentation and location recognition on at least one image data in the image data set to obtain a labeling database, wherein the labeling database includes labeling data of at least one image data.
[0015] The training module is configured to train the general image recognition model using the annotation data of at least one image data in the annotation database and at least one image data to obtain a trained general image recognition model; the trained general image recognition model has the ability to output annotated images.
[0016] In combination with the second aspect, in a possible implementation manner, the acquisition module is further configured to acquire the image to be annotated and acquire a correction result of the annotated image to be annotated.
[0017] The processing module is also configured to input the image to be annotated into the trained general image recognition model and output the annotated image to be annotated.
[0018] In conjunction with the second aspect, in a possible implementation, the training module is further configured to update the weight parameters of the trained general image recognition model using the image to be annotated and the correction results of the annotated image to be annotated.
[0019] In combination with the second aspect, in a possible implementation method, the processing module is also configured to perform semantic segmentation and instance segmentation on at least one image data to obtain an object to be identified in at least one image data; image segmentation includes semantic segmentation and / or instance segmentation; and locate and identify the object to be identified to obtain a category label of the object to be identified.
[0020] The acquisition module is further configured to acquire a correction result corresponding to the category label of the object to be identified, wherein the annotation data of at least one image data in the annotation database includes the correction result corresponding to the category label of the object to be identified.
[0021] In combination with the second aspect, in a possible implementation, the processing module is also configured to preprocess at least one image data in the image data set to obtain processed image data; the preprocessing includes at least one of denoising, cropping, and scaling; and the processed image data is stored.
[0022] In conjunction with the second aspect, in a possible implementation manner, the annotation format of the annotation data is COCO or Pascal VOC.
[0023] In a third aspect, the present disclosure provides an electronic device, comprising: a memory, a display screen, and one or more processors; the memory, the display screen, and the processor are coupled. The memory is used to store computer program codes, and the computer program codes include computer instructions; when the electronic device is running, the processor is used to execute one or more computer instructions stored in the memory, so that the electronic device executes the wind speed warning method as described in any one of the first aspects above.
[0024] In a fourth aspect, the present disclosure provides a computer storage medium, comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute any of the wind speed warning methods in the first aspect.
[0025] In a fifth aspect, the present disclosure provides a computer program product. When the computer program product is executed on an electronic device, the electronic device executes the wind speed warning method as described in any one of the first aspects.
[0026] In a sixth aspect, a device (for example, the device may be a chip system) is provided, the device including a processor for supporting a first device to implement the functions involved in the first aspect. In one possible design, the device also includes a memory for storing program instructions and data necessary for the first device. When the device is a chip system, it may be composed of a chip, or may include a chip and other discrete devices.
[0027] It should be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.
[0028] In the present disclosure, the above names do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of the present disclosure, they fall within the scope of the claims of the present disclosure and their equivalent technologies.
[0029] These and other aspects of the present disclosure will become more apparent from the following description.
[0030] Compared with the prior art, the technical solution provided by the present disclosure has the following advantages: by segmenting and locating the image data in the acquired image data set, building a labeling database, and then training the general image recognition model based on this, the trained general image recognition model can learn the accurate position, boundary and other feature information of different objects to be labeled in the image data. When processing new image data later, the trained general image recognition model can accurately output the labeled image, and the labeled image can accurately mark the specific position, category, etc. of different objects in the image, which helps to understand the content of the image data more carefully and accurately. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0033] Figure 1 This is one of the flowchart diagrams of an image annotation method provided in an embodiment of the present disclosure.
[0034] Figure 2 The second flowchart of an image annotation method provided in an embodiment of the present disclosure.
[0035] Figure 3 The third flowchart of an image annotation method provided in an embodiment of the present disclosure.
[0036] Figure 4 The fourth flowchart of an image annotation method provided in an embodiment of the present disclosure.
[0037] Figure 5 The fifth flowchart of an image annotation method provided in an embodiment of the present disclosure.
[0038] Figure 6 A schematic diagram of the structure of an image annotation device provided in an embodiment of the present disclosure.
[0039] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0040] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0042] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0043] In today's digital age, the massive growth of image data has triggered a huge demand for image processing technology. As one of the basic tasks in the field of computer vision, image annotation is crucial for the training and optimization of machine learning models. It involves identifying and classifying objects, scenes or activities in images, and is a key step in achieving intelligent image analysis.
[0044] At present, image annotation is mainly carried out in two ways: manual annotation and automatic annotation based on deep learning models. Among them, manual annotation is known for its high precision, but this method relies on a large amount of human resources, which is not only inefficient, but also difficult to completely avoid the influence of subjective bias. In contrast, automatic annotation uses intelligent means to achieve accurate classification, detailed description and in-depth analysis of image content, and is widely used in many key fields such as image retrieval, intelligent recommendation, intelligent monitoring and target detection. This method significantly reduces the manpower burden, but there is still room for improvement in the quality of annotation. Therefore, there is an urgent need for a better automatic image annotation method.
[0045] To this end, in response to the above problems, the disclosed embodiments provide an image annotation method, which segments and locates the image data in the acquired image data set, constructs an annotation database, and then trains the general image recognition model based on this, so that the trained general image recognition model can learn the accurate position, boundary and other feature information of different objects to be annotated in the image data. When processing new image data later, the trained general image recognition model can accurately output the annotated image, and the annotated image can accurately mark the specific position, category, etc. of different objects in the image, which helps to understand the content of the image data more carefully and accurately.
[0046] The following describes the image annotation method provided by the embodiment of the present disclosure. A person skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solution provided by the embodiment of the present disclosure is also applicable to similar technical problems.
[0047] The image annotation method provided by the present disclosure can be applied to electronic devices.
[0048] In some embodiments, the electronic device may be a server, a terminal, or other electronic devices for executing the image annotation method, which is not limited in the present disclosure.
[0049] The server may be a single server, or a server cluster consisting of multiple servers. In some implementations, the server cluster may also be a distributed cluster. The present disclosure does not limit the specific implementation of the server.
[0050] The terminal may be a mobile phone, tablet computer, desktop, laptop, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), augmented reality (AR), virtual reality (VR) device, etc., which can install and use content community applications (such as Kuaishou), and the present disclosure does not impose any special restrictions on the specific form of the electronic device. It can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device.
[0051] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0052] refer to Figure 1 , a flow chart of an image annotation method provided by an embodiment of the present disclosure. Figure 1 As shown, the method may include steps 101 to 103.
[0053] Step 101: Obtain an image dataset.
[0054] The image data set includes at least one image data.
[0055] In some instances, at least one image data in the image data set may come from multiple sources, such as public data sets, enterprise internal data, user uploaded data, etc. These diversified data provide rich materials and support for subsequent research and application.
[0056] Of course, the at least one image data may be images of various different categories, such as landscapes, portraits, virtual images, animals, and the like.
[0057] Optional, such as Figure 2 As shown, before performing image segmentation and location recognition on at least one image data in the image data set to obtain a labeling database in step 102, the method further includes:
[0058] Step 201: pre-process at least one image data in the image data set to obtain processed image data.
[0059] The preprocessing includes at least one of denoising, cropping, and scaling.
[0060] In some instances, after acquiring an image data set, at least one image data in the image data set can be preprocessed to improve the image quality in various aspects, so that each image data is more suitable for subsequent processing tasks in terms of clarity, feature significance, and size specifications, thereby providing guarantees for the entire image data processing process.
[0061] Exemplarily, the process of denoising at least one image data may include: denoising at least one image data by using median filtering, Gaussian filtering, wavelet denoising, etc., so as to effectively suppress noise, restore the original pure information of the image, improve the signal-to-noise ratio of the image, and make the edges of objects in the image clearer and the detailed features more obvious.
[0062] By cropping at least one image data, key areas in the image data can be extracted, and irrelevant edges and background parts can be removed, thereby focusing on the target object, reducing unnecessary data volume, improving processing efficiency, and highlighting key features of the image.
[0063] The process of cropping at least one image data may include: for at least one image data, using scaling methods such as bilinear interpolation, nearest neighbor interpolation or cubic convolution to adjust the image data to an appropriate size, which can not only meet the algorithm's requirements for the consistency of the input data size, but also reduce the amount of calculation to a certain extent, while ensuring that the key features of the image data are not obviously distorted or blurred, so that the image data can still accurately reflect the original information at the new size.
[0064] Step 202: Store the processed image data.
[0065] After the processed image data is obtained, the processed image data can be stored, thereby ensuring the security, accessibility and efficient management of the data.
[0066] Exemplarily, the processed image data may be stored in a database or a file system to facilitate subsequent processing.
[0067] Step 102: perform image segmentation and location recognition on at least one image data in the image data set to obtain a labeled database.
[0068] The annotation database includes annotation data of at least one image data.
[0069] In some examples, image segmentation and location recognition are performed on at least one image data in the image data set to generate detailed annotation data of the image data set. The annotation data of the image data set can then be integrated into the annotation database. In this way, through in-depth analysis and annotation, it can be ensured that each image data can be accurately identified, classified and recorded.
[0070] In some examples, the annotation format of the annotation data may be COCO or Pascal VOC. Of course, the annotation format of the annotation data may also be other formats, which is not limited in the present disclosure.
[0071] Optional, such as Figure 3 As shown, step 102 performs image segmentation and positioning recognition on at least one image data in the image data set to obtain a labeling database, including:
[0072] Step 301: Perform semantic segmentation and instance segmentation on at least one image data to obtain an object to be identified in the at least one image data.
[0073] Among them, image segmentation includes semantic segmentation and / or instance segmentation.
[0074] In some examples, the process of performing semantic segmentation and instance segmentation on at least one image data to obtain an object to be identified in the at least one image data may include: performing semantic segmentation and instance segmentation on the at least one image data using a preset network model to obtain an object to be identified in the at least one image data. Exemplarily, the preset network model may be a deep learning network model.
[0075] In some examples, the process of performing semantic segmentation on at least one image data may include: building a semantic segmentation model using a deep learning framework; for example, the semantic segmentation model is a fully convolutional network (FCN) and its variants, U-Net, etc., training the semantic segmentation model through sample annotated images to obtain a trained semantic segmentation model; inputting at least one image data into the trained semantic segmentation model for semantic segmentation to obtain a semantic category label corresponding to the at least one image data.
[0076] The process of performing instance segmentation on at least one image data may include: generating at least one candidate region in at least one image data using a region proposal network (RPN); performing classification and bounding box regression on at least one candidate region to determine the position and category of each target object in at least one candidate region, and at the same time, performing mask prediction on the pixels of each target object through a branch network to obtain an object to be identified in at least one image data.
[0077] Finally, the semantic category label corresponding to at least one image data and the object to be identified in at least one image data are combined to obtain the object to be identified in at least one image data and the semantic category label of the object to be identified. In this way, each object to be identified can be accurately separated from the image data, providing a basis for subsequent image processing, so that the electronic device can have a deeper understanding of the image content of the image data.
[0078] Exemplarily, the deep learning framework is TensorFlow, PyTorch, etc., and the semantic category labels may include people, cars, buildings, sky, and roads; the trained semantic segmentation model has the ability to predict the semantic category to which each pixel in the image data belongs; the candidate region may include the target object.
[0079] Step 302: locate and identify the object to be identified, and obtain a category label of the object to be identified.
[0080] After obtaining the object to be identified, the object to be identified is located and identified, and the process of obtaining the category label of the object to be identified may include: extracting feature vectors from the region of interest (ROI) of each object to be identified, inputting the extracted feature vectors into a pre-trained classifier (such as a Softmax layer), and outputting the category label of the object to be identified.
[0081] Step 303: Obtain the correction result corresponding to the category label of the object to be identified.
[0082] The annotation data of at least one image data in the annotation database includes a correction result corresponding to a category label of the object to be identified.
[0083] In some examples, after obtaining the category label of the object to be identified, the category label of the object to be identified can be sent to a review module, and a correction result corresponding to the category label of the object to be identified outputted from the review module can be received.
[0084] When the category label of the object to be identified is correct, the correction result corresponding to the category label of the object to be identified is consistent with the category label of the object to be identified; when the category label of the object to be identified is incorrect, the correction result corresponding to the category label of the object to be identified is different from the category label of the object to be identified, so as to ensure that the accuracy of the labeled data of at least one image data is higher.
[0085] Step 103: Train the general image recognition model using the annotation data of at least one image data in the annotation database and at least one image data to obtain a trained general image recognition model.
[0086] Among them, the trained general image recognition model has the ability to output labeled images.
[0087] After obtaining at least one image data and at least one image data annotation data, the universal image recognition model can be trained using the at least one image data annotation data and at least one image data in the annotation database to obtain a trained universal image recognition model.
[0088] In combination with the above, it can be known that the annotated data of at least one image data is data verified by the audit module. Therefore, using at least one image data and the annotated data of at least one image data to train the general image recognition model can make the recognition ability of the trained general image recognition model better. And because at least one image data has multiple sources, the trained general image recognition model has a strong generalization ability.
[0089] Optional, such as Figure 4 As shown, step 103 uses the annotation data of at least one image data in the annotation database and at least one image data to train the general image recognition model, and after obtaining the trained general image recognition model, the method further includes:
[0090] Step 401: Obtain an image to be annotated.
[0091] The image to be annotated is any image that needs to be annotated. The image to be annotated is an image obtained in any scene, such as medical scene, scientific scene, image retrieval scene, intelligent recommendation scene, intelligent monitoring scene, target detection scene, etc.
[0092] In some examples, the image to be annotated may also come from a video to be annotated. Exemplarily, key frames are extracted from the video to be annotated to obtain the image to be annotated.
[0093] Of course, the images to be annotated may also come from other sources, and the present disclosure does not limit this.
[0094] Step 402: input the image to be annotated into the trained general image recognition model, and output the annotated image to be annotated.
[0095] It is understandable that after obtaining the image to be annotated, it is necessary to preprocess the image to be annotated so that the preprocessed image to be annotated is consistent with the format of the input data required by the trained general image recognition model. Then the image to be annotated is input into the trained general image recognition model, and the trained general image recognition model outputs the annotated image to be annotated.
[0096] By using the trained general image recognition model to output the labeled images to be labeled, more accurate and reliable labeling results can be provided, with better scalability, and the labeling quality of the trained general image recognition model can be continuously improved, which can better adapt to image data in different fields and with different characteristics.
[0097] Step 403: Obtain the correction result of the annotated image.
[0098] After the trained general image recognition model outputs the annotated image to be annotated, the annotated image to be annotated can be sent to the review module, so that the review module can further review and correct the annotated image to be annotated, thereby obtaining the corrected result of the annotated image to be annotated. In this way, the annotation result of the image to be annotated can be guaranteed to be more accurate.
[0099] The trained general image recognition model is used to annotate the image to be annotated, which greatly improves the efficiency of annotation and reduces the labor and time costs. In addition, after obtaining the annotated image to be annotated, it can be further corrected through the review module (i.e., the manual correction capability is retained), further improving the quality of automated annotation.
[0100] Optional, such as Figure 5 As shown, after obtaining the correction result of the annotated image in step 403, the method further includes:
[0101] Step 501: update the weight parameters of the trained general image recognition model using the image to be annotated and the correction results of the annotated image to be annotated.
[0102] After obtaining the correction result of the annotated image to be annotated, the weight parameters of the trained general image recognition model can be further iterated by using the image to be annotated and the correction result of the annotated image to be annotated. In this way, the performance of the trained general image recognition model can be further improved, so that the trained general image recognition model can output more accurate and reliable annotation results when facing complex and diverse image annotation tasks.
[0103] The disclosed embodiment provides an image annotation method, which comprises acquiring an image data set; the image data set comprises at least one image data; performing image segmentation and location recognition on at least one image data in the image data set to obtain an annotation database, wherein the annotation database comprises the annotation data of at least one image data; training a general image recognition model using the annotation data of at least one image data in the annotation database and at least one image data to obtain a trained general image recognition model; the trained general image recognition model has the ability to output annotated images. By performing segmentation and location recognition on the image data in the acquired image data set, constructing an annotation database, and then training the general image recognition model based on the segmentation and location recognition, the trained general image recognition model can learn the accurate position, boundary and other feature information of different objects to be annotated in the image data. When processing new image data later, the trained general image recognition model can accurately output annotated images, and the annotated images can accurately mark the specific positions, categories, etc. of different objects in the image, which helps to understand the content of the image data more carefully and accurately.
[0104] In addition, the use of this image annotation method greatly improves the efficiency of annotation, reduces labor costs and time costs, reserves the intervention capabilities of manual annotation experts, and can make full use of manual intervention to improve the quality of automated annotation.
[0105] The above mainly introduces the solution provided by the embodiment of the present disclosure from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.
[0106] The disclosed embodiment can divide the electronic device into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the disclosed embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.
[0107] Corresponding to the method in the above-mentioned embodiment, the embodiment of the present disclosure further provides an image annotation device. The image annotation device is used to implement the above-mentioned image annotation method. The functions of the image annotation device can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions.
[0108] For example, Figure 6 A schematic diagram of the structure of a decision-making device is shown. Figure 6 As shown, the decision-making device may include: an acquisition module 601 , a processing module 602 and a training module 603 .
[0109] The acquisition module 601 is configured to acquire an image data set; the image data set includes at least one image data; for example, in combination with Figure 1 , the acquisition module 601 can be used to execute step 101.
[0110] The processing module 602 is configured to perform image segmentation and location recognition on at least one image data in the image data set to obtain a labeling database, wherein the labeling database includes labeling data of at least one image data; for example, in combination with Figure 1 , the processing module 602 can be used to execute step 102.
[0111] The training module 603 is configured to train the universal image recognition model using the annotation data of at least one image data in the annotation database and at least one image data to obtain a trained universal image recognition model; the trained universal image recognition model has the ability to output annotated images. Figure 1 , the training module 603 can be used to execute step 103.
[0112] Optionally, the acquisition module 601 is further configured to acquire the image to be annotated and acquire the correction result of the annotated image to be annotated. Figure 4 , the training module 603 can be used to execute step 401 and step 403.
[0113] The processing module 602 is further configured to input the image to be annotated into the trained general image recognition model and output the annotated image to be annotated; for example, Figure 4 , the training module 603 can be used to execute step 402.
[0114] Optionally, the training module 603 is further configured to update the weight parameters of the trained general image recognition model using the image to be annotated and the correction result of the annotated image to be annotated. Figure 5 , the training module 603 can be used to execute step 501.
[0115] Optionally, the processing module 602 is further configured to perform semantic segmentation and instance segmentation on at least one image data to obtain an object to be identified in at least one image data; image segmentation includes semantic segmentation and / or instance segmentation; positioning and identifying the object to be identified to obtain a category label of the object to be identified; for example, combining Figure 3 , the training module 603 can be used to execute step 301 and step 302.
[0116] The acquisition module 601 is further configured to acquire a correction result corresponding to the category label of the object to be identified, wherein the annotation data of at least one image data in the annotation database includes the correction result corresponding to the category label of the object to be identified. Figure 3 , the training module 603 can be used to execute step 303.
[0117] Optionally, the processing module 602 is further configured to preprocess at least one image data in the image data set to obtain processed image data; the preprocessing includes at least one of denoising, cropping, and scaling; and storing the processed image data. Figure 2 , the training module 603 can be used to execute step 201 and step 202.
[0118] Optionally, the annotation format of the annotation data is COCO or Pascal VOC.
[0119] Of course, the path selection device provided by the embodiment of the present disclosure includes but is not limited to the above modules, for example, the path selection device may also include a storage module 604. The storage module 604 may be used to store the program code of the write path selection device, and may also be used to store data generated by the write path selection device during operation, such as data in a write request, etc.
[0120] Figure 7 A structural diagram of an electronic device provided in an embodiment of the present disclosure, such as Figure 7 As shown, the electronic device 700 includes one or more processors 701 and a memory 702 .
[0121] Understandably, Figure 7 The electronic device 700 shown can implement all functions of the above-mentioned image annotation method. The functions of each module in the above-mentioned image annotation device can be implemented in the processor 701 of the electronic device 700. The storage module of the image annotation device is equivalent to the memory 702 of the electronic device 700.
[0122] The processor 701 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 700 to perform desired functions.
[0123] The memory 702 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 701 may execute the program instructions to implement the image annotation method, image annotation method, and / or other desired functions of the various embodiments of the present disclosure described above.
[0124] In one example, the electronic device 700 may further include: an input device 703 and an output device 704, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0125] Of course, to simplify, Figure 7 Only some of the components related to the present disclosure in the electronic device 700 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 700 may also include any other appropriate components.
[0126] In addition to the above methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes the steps of the image annotation method in the above various embodiments of the present specification.
[0127] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0128] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the image annotation method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.
[0129] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0130] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.
[0131] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and do not mean or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.
[0132] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.
[0133] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not meant to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
[0134] The above description has been given for the purpose of illustration and description. In addition, this description is not meant to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.
Claims
1. An image annotation method, applied to an electronic device, comprising: Get image dataset; The image data set includes at least one image data; Performing image segmentation and location recognition on at least one image data in the image data set to obtain a labeling database, wherein the labeling database includes labeling data of the at least one image data; The universal image recognition model is trained using the annotation data of the at least one image data in the annotation database and the at least one image data to obtain a trained universal image recognition model; the trained universal image recognition model has the ability to output annotated images.
2. The method according to claim 1, characterized in that The method further comprises: Get the image to be annotated; Inputting the image to be annotated into the trained general image recognition model, and outputting the annotated image to be annotated; Obtain a correction result of the labeled image.
3. The method according to claim 2, characterized in that The method further comprises: The weight parameters of the trained general image recognition model are updated by using the correction results of the image to be annotated and the annotated image to be annotated.
4. The method according to any one of claims 1 to 3, characterized in that The performing image segmentation and positioning recognition on at least one image data in the image data set to obtain a labeling database includes: Performing semantic segmentation and instance segmentation on the at least one image data to obtain an object to be identified in the at least one image data; the image segmentation includes the semantic segmentation and / or the instance segmentation; Positioning and identifying the object to be identified to obtain a category label of the object to be identified; A correction result corresponding to the category label of the object to be identified is obtained, wherein the annotation data of at least one image data in the annotation database includes the correction result corresponding to the category label of the object to be identified.
5. The method according to claim 1, characterized in that Before performing image segmentation and location recognition on at least one image data in the image data set, the method further includes: Preprocessing at least one image data in the image data set to obtain processed image data; the preprocessing includes at least one of denoising, cropping, and scaling; The processed image data is stored.
6. The method according to any one of claims 1 to 3, characterized in that The annotation format of the annotation data is COCO or Pascal VOC.
7. An image annotation device, characterized in that: The device comprises: An acquisition module is configured to acquire an image data set; the image data set includes at least one image data; a processing module, configured to perform image segmentation and location recognition on at least one image data in the image data set to obtain a labeling database, wherein the labeling database includes labeling data of the at least one image data; The training module is configured to train the universal image recognition model using the annotation data of the at least one image data in the annotation database and the at least one image data to obtain a trained universal image recognition model; the trained universal image recognition model has the ability to output annotated images.
8. The device according to claim 7, characterized in that The method further comprises: The acquisition module is further configured to acquire the image to be annotated; and acquire the correction result of the annotated image to be annotated; The processing module is further configured to input the image to be annotated into the trained general image recognition model and output the annotated image to be annotated.
9. An electronic device, characterized in that: include: A touch screen, comprising a touch sensor and a display screen; one or more processors; Memory; One or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device executes the image annotation method as described in any one of claims 1 to 7.
10. A computer-readable storage medium having computer program instructions stored thereon; characterized in that: When the computer program instructions are executed by an electronic device, the electronic device implements the image annotation method according to any one of claims 1 to 7.
11. A computer program product, characterized in that include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to enable the electronic device to implement the image annotation method according to any one of claims 1 to 7 when executing the computer program.