Image annotation method, electronic equipment, system and storage medium
By constructing and iteratively training two parallel network models, using labeled and unlabeled data to update parameters with each other, the problems of high cost and poor accuracy in medical image processing are solved, and the generalization ability and labeling accuracy of the model are improved.
Patent Information
- Application Number
- CN202411411938.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2025-01-17
AI Technical Summary
In existing medical image processing, the cost is high and the accuracy is poor, and the overall generalization performance of the model is affected, mainly due to the inconsistent distribution of labeled data and unlabeled data.
Two parallel network models are built, through iterative training and data augmentation, the network parameters are updated with labeled data and unlabeled data until the loss function converges, and an image detection model is formed to improve the generalization ability of the model.
Effectively use unlabeled data for feature extraction, solve the problem of inconsistent distribution of labeled data and unlabeled data, and improve the labeling accuracy and generalization performance of the model.
Smart Images
Figure CN119323675A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of image data processing, and more specifically, to an image annotation method, electronic device, system and storage medium. Background Art
[0002] With the development of artificial intelligence technology, more and more AI technologies have been widely used in various fields. Among them, deep learning, as a kind of machine learning, is to learn the inherent laws and representation levels of sample data. The information obtained in the learning process is of great help in the interpretation of data such as text, images and sounds. For example, the development of deep learning has promoted the application of computer vision technology in the medical field, especially medical image-assisted reading, that is, using deep learning technology to analyze and learn large-scale data sets to achieve auxiliary diagnosis and treatment based on medical images.
[0003] At present, as a tumor originating from glial cells in the brain, the symptoms of glioma are closely related to factors such as the malignancy of the tumor, the patient's age, the location of the tumor, and the growth rate. Its most common symptoms include headache, nausea, vomiting, and epileptic seizures. During the consultation process for such diseases, doctors will assist in the diagnosis based on medical records, neurological examinations, and head CT or MRI examinations.
[0004] However, most of the medical image segmentation algorithms that have been proposed require a large amount of labeled data to meet performance requirements. The labeling of medical images requires a high level of professionalism from the labelers, and the labeling process is time-consuming, labor-intensive, and costly, making it difficult to obtain a large amount of high-quality labeled data for medical images. In addition, although semi-supervised learning can use a small amount of labeled data and a large amount of unlabeled data to improve the performance of image segmentation through deep learning technology, there is usually a distribution inconsistency between a small amount of labeled images and a large amount of unlabeled data, resulting in poor accuracy when using a consistent method to process all images, and the overall generalization performance of the model is affected.
[0005] Therefore, there is an urgent need for an image processing solution to improve the reliability and accuracy of medical image processing. Summary of the invention
[0006] In view of the above problems, the purpose of this application is to provide an image annotation method to solve the problems of high cost, poor accuracy, and affected overall generalization performance of the model in existing medical image processing.
[0007] In a first aspect, an embodiment of the present application provides an image annotation method, comprising: obtaining original image training data, and simultaneously constructing a first network model and a second network model; wherein the original image training data includes unlabeled data and labeled data;
[0008] Iteratively training the first network model based on the labeled data; and iteratively training the second network model based on the original image training data;
[0009] After each iteration is completed, the network parameters of the first network model are updated by the network parameters of the second network model until the value of the loss function of the second network model converges within a preset range, thereby forming an image detection model;
[0010] The image to be processed is annotated based on the image detection model.
[0011] Furthermore, the first network model and the second network model are arranged in parallel; and the first network model and the second network model have the same structure.
[0012] Further, iteratively training the first network model based on the labeled data includes: obtaining the labeled data in the original image training data;
[0013] Performing weak augmentation and / or strong augmentation processing on the labeled data respectively to obtain corresponding weak augmented labeled data and strong augmented labeled data;
[0014] The first network model is iteratively trained based on the labeled data and / or the weakly augmented labeled data and / or the strongly augmented labeled data.
[0015] Furthermore, the iterative training of the second network model based on the original image training data includes:
[0016] Performing weak augmentation and / or strong augmentation processing on the original image training data to obtain corresponding weak augmentation data and strong augmentation data;
[0017] The second network model is iteratively trained based on the original image training data and / or the weak augmented data and / or the strong augmented data.
[0018] Furthermore, the loss function of the second network model is expressed as:
[0019]
[0020] in, represents the loss of the overall labeled data in the second network model, represents the loss of the overall unlabeled data in the second network model, and α represents the adjustment weight.
[0021] Furthermore, the loss expression of the overall labeled data in the second network model is:
[0022]
[0023]
[0024] Among them, N b represents the number of labeled data, represents the loss of the e-th labeled data, W represents the width of the labeled data, H represents the height of the labeled data, and y i represents the label of the i-th point, S w,i represents the predicted value of the i-th point on the weakly augmented labeled data, S S,i represents the predicted value of the i-th point on the strongly augmented labeled data, l ce (y i ,S w,i ) represents the loss between the i-th point on the weakly augmented labeled data and the labeled data, l ce (y i ,S S,i ) represents the loss between the i-th point on the strongly augmented labeled data and the labeled data, l ce (S w,i ,S S,i ) represents the loss between the weakly augmented labeled data and the strongly augmented labeled data at the i-th point.
[0025] Furthermore, the loss expression of the overall unlabeled data in the second network model is:
[0026]
[0027]
[0028] Among them, N u represents the number of unlabeled data, represents the loss of the f-th unlabeled data, S o,i represents the predicted value of the i-th point of the unlabeled data generated by the first network model, S w,i represents the predicted value of the i-th point on the weakly augmented data generated by the second network model, S S,i represents the predicted value of the i-th point on the strongly augmented data generated by the second network model, l ce (S o,i ,S w,i ) represents the loss between the i-th point on the unlabeled data and the weakly augmented data, l ce (S o,i ,S S,i ) represents the loss between the i-th point on the unlabeled data and the strongly augmented data, l ce (S w,i ,S S,i ) represents the loss between the weak augmented data and the strong augmented data at the i-th point.
[0029] Furthermore, the expression for updating the network parameters of the first network model by using the network parameters of the second network model is:
[0030]
[0031] in, represents the network parameters of the first network model at the tth iteration round, represents the network parameters of the second network model in the tth iteration round, and λ represents the update adjustment parameter;
[0032] The update adjustment parameter is a dynamically changing value, and the update adjustment parameter increases with the increase of iteration rounds.
[0033] In a second aspect, an embodiment of the present application provides an electronic device, including:
[0034] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to run the image annotation method in the first aspect.
[0035] In a third aspect, an embodiment of the present application provides an image annotation system, including:
[0036] A training data acquisition unit, used to acquire original image training data and simultaneously construct a first network model and a second network model; wherein the original image training data includes unlabeled data and labeled data;
[0037] A model training unit, configured to iteratively train the first network model based on the labeled data; and iteratively train the second network model based on the original image training data;
[0038] A parameter updating unit, configured to update the network parameters of the first network model by using the network parameters of the second network model after each iteration is completed, until the value of the loss function of the second network model converges within a preset range, thereby forming an image detection model;
[0039] A labeling unit is used to label the image to be processed based on the image detection model.
[0040] In a third aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the image annotation method in the first aspect.
[0041] The above-mentioned image annotation method, electronic device, system and storage medium are used to construct a first network model and a second network model, and the two network models are trained separately using different data. During the training process, the network parameters of the first network model are updated by the network parameters of the second network model until the loss function of the first network model converges within a preset range, thereby forming an image detection model that can effectively use unlabeled data for feature extraction, solve the problem of inconsistent distribution of labeled data and unlabeled data, and improve the generalization ability of the model.
[0042] In order to achieve the above and related purposes, one or more aspects of the present invention include the features that will be described in detail later. The following description and the accompanying drawings describe some exemplary aspects of the present invention in detail. However, these aspects indicate only some of the various ways in which the principles of the present invention can be used. In addition, the present invention is intended to include all these aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more apparent and easy to understand. In the accompanying drawings:
[0044] Figure 1 is a flowchart of an image annotation method according to an embodiment of the present invention;
[0045] Figure 2 is a principle diagram of an image annotation method according to an embodiment of the present invention;
[0046] Figure 3 is a block diagram of an image annotation system according to an embodiment of the present invention;
[0047] Figure 4 is a schematic block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0048] In the following description, for the purpose of illustration, in order to provide a comprehensive understanding of one or more embodiments, many specific details are set forth. However, it is apparent that these embodiments may also be implemented without these specific details. In other examples, for ease of describing one or more embodiments, known structures and devices are shown in the form of block diagrams.
[0049] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.
[0050] In order to solve the current problems of high cost, poor accuracy, and affected overall generalization performance of the model in medical image processing, the embodiment of the present application designs an image annotation solution, constructs a first network model and a second network model, and uses different data to train the two network models separately. During the training process, the network parameters of the first network model are updated by the network parameters of the second network model until the loss function of the first network model converges within a preset range, thereby forming an image detection model. Finally, the image detection model is used to annotate and segment medical images of areas such as the trachea and lesions, which can effectively extract features from unlabeled data, solve the problem of inconsistent distribution of labeled data and unlabeled data, and improve the generalization ability of the model.
[0051] To describe the image annotation method of the present invention in detail, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0052] Figure 1 The process of the image annotation method according to an embodiment of the present invention is shown. Figure 2 The schematic principle of the image annotation method according to an embodiment of the present invention is shown.
[0053] like Figure 1 and Figure 2 As shown in the figure, the image annotation method of the embodiment of the present invention at least includes the following steps:
[0054] S100: Acquire original image training data, and simultaneously construct a first network model and a second network model; wherein the original image training data includes unlabeled data and labeled data.
[0055] Among them, the original image training data can be image information related to medicine or other fields obtained through an external public database, and the original image training data includes unlabeled data and labeled data; and, a first network model and a second network model are constructed, and the two are set in parallel, and the same model structure can be used for simultaneous training.
[0056] S200: Iteratively train the first network model based on the labeled data; at the same time, iteratively train the second network model based on the original image training data.
[0057] Among them, the first neural network model can adopt UNet, Vnet and its variant networks. In order to expand the amount of labeled data, the labeled data (that is, the training data with labeled data in the original image training data) can be randomly processed by original image, weak augmentation and strong augmentation. During the processing, the image information and the corresponding label data are processed at the same time, that is, the labeled image data is flipped, and the corresponding label data is also flipped accordingly.
[0058] Specifically, iteratively training the first network model based on the labeled data includes:
[0059] S110: Obtaining labeled data in the original image training data; performing weak augmentation and / or strong augmentation processing on the labeled data to obtain corresponding weak augmented labeled data and strong augmented labeled data;
[0060] S120: Iteratively train the first network model based on the labeled data and / or the weakly augmented labeled data and / or the strongly augmented labeled data.
[0061] The augmentation processing of the labeled data may be random, that is, a part of the data may be randomly subjected to weak augmentation or strong augmentation processing, and then the first network model is trained by the labeled data and the processed data.
[0062] Furthermore, the iterative training of the second network model based on the original image training data includes:
[0063] S210: Performing weak augmentation and / or strong augmentation processing on the original image training data to obtain corresponding weak augmentation data and strong augmentation data;
[0064] S220: Iteratively train the second network model based on the original image training data and / or the weak augmented data and / or the strong augmented data.
[0065] Similarly, the labeled data and unlabeled data in the original image data are randomly augmented respectively, and then the second network model is iteratively trained using the original image training data and the weakly augmented data and / or the strongly augmented data.
[0066] Specifically, the weak augmentation processing includes flipping, translation, rotation, scaling and cropping; the strong augmentation processing includes brightness adjustment, contrast adjustment, histogram equalization, random area occlusion, sharpening and blurring.
[0067] As an example, blur processing can be performed in a variety of ways such as mean blur or Gaussian blur. Among them, the blur algorithm can be used to obtain the cumulative sum of a certain characteristic value of related pixels in a certain area around a pixel in the original image data and the corresponding weight, and then the result value is obtained. Since it is impossible to distinguish information such as the edge of the image, the details of the blurred image may be lost. Therefore, a threshold can also be set. When the difference between the pixel in a certain area and the center point pixel is greater than the threshold, its weight is set to be smaller. In this way, for the relatively smooth areas, it is not much different from the original algorithm. For the edge areas where the pixel value changes are more obvious, the original information can be effectively retained, and the edge information can be retained while reducing noise.
[0068] S300: After each iteration is completed, the network parameters of the first network model are updated by the network parameters of the second network model until the value of the loss function of the second network model converges within a preset range, thereby forming an image detection model.
[0069] In this step, the completion of each iteration can be understood as the completion of a round of data training, that is, a Batch. All training data are used once, which can be understood as one iteration. Therefore, in the following training expression formulas, the loss of the overall data (for example, the overall labeled data) can be understood as the loss of related data in a batch process; after multiple rounds of training, the second network model can be verified by the verification data set so that the value of the loss function on the verification data set converges within a preset range; in addition, the training of the second network model can also be completed by setting the number of training times or until the value of the loss function does not change significantly in multiple iterations.
[0070] Specifically, the loss function of the second network model is expressed as:
[0071]
[0072] in, represents the loss of labeled data in the second network model, represents the loss of the overall unlabeled data in the second network model, and α represents the adjustment weight.
[0073] Furthermore, the loss expression of the overall labeled data in the second network model is:
[0074]
[0075]
[0076] Among them, N b Indicates the number of labeled data in a Batch. represents the loss of the e-th labeled data, W represents the width of the labeled data, H represents the height of the labeled data, and y i represents the label of the i-th point, S w,i represents the predicted value of the i-th point on the weakly augmented labeled data, S S,i represents the predicted value of the i-th point on the strongly augmented labeled data, l ce (y i ,S w,i ) represents the loss between the i-th point on the weakly augmented labeled data and the labeled data, l ce (y i ,S S,i ) represents the loss between the i-th point on the strongly augmented labeled data and the labeled data, l ce (S w,i ,S S,i ) represents the loss between the weakly augmented labeled data and the strongly augmented labeled data at the i-th point.
[0077] Furthermore, the loss expression of the overall unlabeled data in the second network model is:
[0078]
[0079]
[0080] Among them, N u Indicates the number of unlabeled data in a Batch. represents the loss of the f-th unlabeled data, S o,i represents the predicted value of the i-th point of the unlabeled data generated by the first network model, S w,i represents the predicted value of the i-th point on the weakly augmented data generated by the second network model, S S,i represents the predicted value of the i-th point on the strongly augmented data generated by the second network model, l ce (S o,i ,S w,i ) represents the loss between the i-th point on the unlabeled data and the weakly augmented data, l ce (S o,i ,S S,i ) represents the loss between the i-th point on the unlabeled data and the strongly augmented data, l ce (S w,i ,S S,i ) represents the loss between the weak augmented data and the strong augmented data point i, S o,irepresents the predicted value of the i-th point of the unlabeled data, which can be generated by the first network model.
[0081] Furthermore, the expression for updating the network parameters of the first network model by using the network parameters of the second network model is:
[0082]
[0083] in, represents the network parameters of the first network model at the tth iteration round, represents the network parameters of the second network model in the tth iteration round, λ represents the update adjustment parameter; the update adjustment parameter can be set to a dynamic value, the update adjustment parameter gradually increases with the increase of iteration rounds, and its maximum value is limited to 0.5. The specific value of λ can refer to the formula
[0084] S400: Annotating the image to be processed based on the image detection model.
[0085] In this step, after the image detection model is trained, it can be used to label and segment target areas such as organs and lesions in medical images, so that it can be applied to more complex medical scenarios, providing more options for the subsequent expansion of application areas.
[0086] It can be seen that in the model training process of the embodiment of the present invention, there are differences in the data input into the first network model T and the second network model S. Weak image augmentation and strong image augmentation operations are used to perform consistency regularization constraints on the same image, and pseudo labels (predictions) are generated through the T network to supervise the S network for feature learning. The S network is used to iteratively update the parameters of the T network. The two networks assist each other and can effectively use unlabeled data for feature extraction, solve the problem of inconsistent distribution of labeled data and unlabeled data, and improve the accuracy of model annotation. It can be applicable to CT image processing in the medical field.
[0087] Corresponding to the above-mentioned image annotation method, the present invention also provides an image annotation system.
[0088] Specifically, Figure 3 FIG. 2 shows a block diagram of the principle of the image annotation system according to an embodiment of the present invention. Figure 3The image annotation system 100 of the embodiment of the present invention can be installed in an electronic device. According to the functions to be implemented, the image annotation system 100 may include: a training data acquisition unit 101, a model training unit 102, a parameter updating unit 103, and an annotation unit 104. The unit described in the present invention may also be referred to as a module, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.
[0089] In this embodiment, the functions of each module / unit are as follows:
[0090] The training data acquisition unit 101 is used to acquire original image training data, the original image training data includes unlabeled data and labeled data, and at the same time, construct a first network model and a second network model;
[0091] A model training unit 102, configured to iteratively train the first network model based on the labeled data; and iteratively train the second network model based on the original image training data;
[0092] A parameter updating unit 103, configured to update the network parameters of the first network model by using the network parameters of the second network model after each iteration is completed, until the value of the loss function of the second network model converges within a preset range, thereby forming an image detection model;
[0093] The labeling unit 104 is used to label the image to be processed based on the image detection model.
[0094] Specifically, for the specific implementation of each unit, reference may be made to the description in the above method implementation, which will not be described one by one here.
[0095] Figure 4 An electronic device provided according to an embodiment of the present application is shown, comprising: a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to run the image annotation method in the first aspect.
[0096] like Figure 4 As shown, the electronic device 1 may include a processor 10 , a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10 , such as an image annotation program 12 .
[0097] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the image annotation program, etc., but also be used to temporarily store data that has been output or is to be output.
[0098] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules (such as image annotation programs, etc.) stored in the memory 11, and calls data stored in the memory 11 to execute various functions of the electronic device 1 and process data.
[0099] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.
[0100] It can be seen that Figure 4Only an electronic device with components is shown. It can be understood by those skilled in the art that the structure shown does not constitute a limitation on the electronic device 1, and it can include fewer or more components than shown, or combine certain components, or arrange components differently. For example, although not shown, the electronic device 1 can also include a power supply (such as a battery) to power each component. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, so that the functions of charging management, discharging management, and power consumption management are realized through the power management device. The power supply can also include any components such as one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 can also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
[0101] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.
[0102] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.
[0103] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited by this structure. The image annotation program 12 stored in the memory 11 of the electronic device 1 is a combination of multiple instructions, which can be implemented when running in the processor 10:
[0104] Acquire original image training data, wherein the original image training data includes unlabeled data and labeled data, and construct a first network model and a second network model;
[0105] Iteratively training the first network model based on the labeled data; and iteratively training the second network model based on the original image training data;
[0106] After each iteration is completed, the network parameters of the first network model are updated by the network parameters of the second network model until the value of the loss function of the second network model converges within a preset range, thereby forming an image detection model;
[0107] The image to be processed is annotated based on the image detection model.
[0108] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).
[0109] It should be understood that the method embodiment, the electronic device embodiment and the system embodiment may correspond to each other, and similar descriptions may refer to the system embodiment. To avoid repetition, they will not be described here.
[0110] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
[0111] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0112] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0113] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0114] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.
[0115] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.
[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.
[0117] The image annotation method, electronic device, system and storage medium according to the present invention are described as examples with reference to the accompanying drawings. However, those skilled in the art should understand that various improvements can be made to the image annotation method, electronic device, system and storage medium proposed by the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the attached claims.
Claims
1. An image annotation method, characterized in that: include: Acquire original image training data, and simultaneously construct a first network model and a second network model; wherein the original image training data includes unlabeled data and labeled data; Iteratively training the first network model based on the labeled data; and iteratively training the second network model based on the original image training data; After each iteration is completed, the network parameters of the first network model are updated by the network parameters of the second network model until the value of the loss function of the second network model converges within a preset range, thereby forming an image detection model; The image to be processed is annotated based on the image detection model.
2. The image annotation method according to claim 1, characterized in that: The first network model and the second network model have the same structure.
3. The image annotation method according to claim 1, characterized in that: Iteratively training the first network model based on the labeled data includes: Obtaining labeled data from the original image training data; Performing weak augmentation and / or strong augmentation processing on the labeled data respectively to obtain corresponding weak augmented labeled data and strong augmented labeled data; The first network model is iteratively trained based on the labeled data and / or the weakly augmented labeled data and / or the strongly augmented labeled data.
4. The image annotation method according to claim 1, characterized in that: The iterative training of the second network model based on the original image training data comprises: Performing weak augmentation and / or strong augmentation processing on the original image training data to obtain corresponding weak augmentation data and strong augmentation data; The second network model is iteratively trained based on the original image training data and / or the weak augmented data and / or the strong augmented data.
5. The image annotation method according to claim 3, characterized in that: The expression of the loss function of the second network model is: in, represents the loss of the overall labeled data in the second network model, represents the loss of the overall unlabeled data in the second network model, and α represents the adjustment weight.
6. The image annotation method according to claim 5, characterized in that: The loss expression of the overall labeled data in the second network model is: Among them, N b represents the number of labeled data, represents the loss of the e-th labeled data, W represents the width of the labeled data, H represents the height of the labeled data, and y i represents the label of the i-th point, S w,i represents the predicted value of the i-th point on the weakly augmented labeled data, S S,i represents the predicted value of the i-th point on the strongly augmented labeled data, l ce (y i ,S w,i ) represents the loss between the i-th point on the weakly augmented labeled data and the labeled data, l ce (y i ,S S,i ) represents the loss between the i-th point on the strongly augmented labeled data and the labeled data, l ce (S w,i ,S S,i ) represents the loss between the weakly augmented labeled data and the strongly augmented labeled data at the i-th point.
7. The image annotation method according to claim 6, characterized in that: The loss expression of the overall unlabeled data in the second network model is: Among them, N u represents the number of unlabeled data, represents the loss of the f-th unlabeled data, S o,i represents the predicted value of the i-th point of the unlabeled data generated by the first network model, S w,i represents the predicted value of the i-th point on the weakly augmented data generated by the second network model, S S,i represents the predicted value of the i-th point on the strongly augmented data generated by the second network model, l cw (S o,i ,S w,i ) represents the loss between the i-th point on the unlabeled data and the weakly augmented data, l ce (S o,i ,S S,i ) represents the loss between the i-th point on the unlabeled data and the strongly augmented data, l ce (S w,i ,S S,i ) represents the loss between the weak augmented data and the strong augmented data at the i-th point.
8. The image annotation method according to claim 1, characterized in that: The expression for updating the network parameters of the first network model by using the network parameters of the second network model is: in, represents the network parameters of the first network model at the tth iteration round, represents the network parameters of the second network model in the tth iteration round, and λ represents the update adjustment parameter; The update adjustment parameter is a dynamically changing value, and the update adjustment parameter increases with the increase of iteration rounds.
9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to run the image annotation method according to any one of claims 1 to 8.
10. An image annotation system, characterized in that: include: A training data acquisition unit, used to acquire original image training data and simultaneously construct a first network model and a second network model; wherein the original image training data includes unlabeled data and labeled data; A model training unit, configured to iteratively train the first network model based on the labeled data; and iteratively train the second network model based on the original image training data; A parameter updating unit, configured to update the network parameters of the first network model by using the network parameters of the second network model after each iteration is completed, until the value of the loss function of the second network model converges within a preset range, thereby forming an image detection model; A labeling unit is used to label the image to be processed based on the image detection model.
11. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the image annotation method according to any one of claims 1 to 8.