Image classification machine learning method and system based on region of interest and smoothed annotation

Abstract

The application discloses an image classification machine learning method and system based on an attention region and smooth labeling, and the method comprises the following steps: S10, acquiring a plurality of image data, and generating an image data set according to the plurality of image data; S20, dividing each image in the image data set into one or a plurality of attention regions, and setting one or a plurality of classification labels for each attention region; S30, labeling each image in the image data set according to the attention region and the classification label, obtaining a labeled data set, and the labeling comprises confidence and image classification; and S40, training an image classification model according to the image data set and the labeled data set by using a machine learning algorithm. By dividing the image into a plurality of attention regions and labeling the image according to the attention region and the classification label of the attention region, smooth labeled data can be efficiently output, so that the training effect of the image classification model is improved, and the classification accuracy of the image classification model is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, and in particular to a machine learning method and system for image classification based on regions of interest and smooth annotation. Background Technology

[0002] Existing technologies generally include the following steps: (1) labeling the image dataset to indicate the classification of each image; each image may belong to one or more classifications. (2) training an image classification model on the aforementioned dataset using machine learning methods. (3) using the aforementioned model to classify images that are not in the aforementioned dataset.

[0003] Existing technologies have some drawbacks: In traditional image classification dataset annotation methods, for any image in the dataset, it either belongs to a certain category or does not, with no intermediate state. This is often too rigid and cannot accurately reflect the actual situation, i.e., some images may only partially belong to a certain category.

[0004] Image classification models trained on rigidly labeled datasets will have limited accuracy. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide an image classification machine learning method and system based on regions of interest and smooth annotations, in order to solve the technical problem that the lack of intermediate states in the annotations of existing image classification datasets leads to low accuracy of image classification models trained on existing image classification datasets.

[0006] To achieve the above objectives, firstly, an image classification machine learning method based on regions of interest and smooth annotation is provided, comprising the following steps:

[0007] S10: Acquire several image data and generate an image dataset based on the several image data;

[0008] S20: Divide each image in the image dataset into one or more regions of interest, and assign one or more classification labels to each region of interest;

[0009] S30: Based on the region of interest and the classification label, each image in the image dataset is labeled to obtain a labeled dataset, wherein the labeling includes confidence and image classification;

[0010] S40: Using machine learning algorithms, an image classification model is trained based on the image dataset and the labeled dataset.

[0011] A further improvement of the present invention is that step S10 specifically includes:

[0012] S11: Perform image enhancement on the aforementioned image data to obtain several enhanced image data;

[0013] S12: The size of several enhanced image data is unified by interpolation to obtain several unified image data;

[0014] S13: Normalize the unified image data to obtain normalized image data.

[0015] S14: Generate an image dataset based on several of the normalized image data.

[0016] A further improvement of the present invention is that step S20 specifically includes:

[0017] S21: Perform feature extraction based on the image dataset to obtain several basic features;

[0018] S22: Generate several classification labels based on several of the aforementioned basic features;

[0019] S23: Used to divide each of the images into one or more regions of interest based on the image dataset and a preset image segmentation model;

[0020] S24: Based on the aforementioned classification labels, assign one or more of the aforementioned labels to each region of interest.

[0021] A further improvement of the present invention is that step S30 specifically includes:

[0022] S31A: Based on one or more regions of interest in each of the images, select several images in which no regions of interest overlap, and use them as several first images;

[0023] S32A: Calculate the area of ​​several first images and the area of ​​each region of interest in each first image;

[0024] S33A: In each of the first images, the relative area of ​​each region of interest in the first image is calculated based on the area of ​​the first image and the area of ​​each region of interest in the first image.

[0025] S34A: In each of the first images, the relative areas of several regions of interest corresponding to the same category label are added together to obtain the first weight of the same category label. All category labels are traversed to obtain the first weight of each category label in the first image.

[0026] S35A: Calculate the first confidence level and the first image classification based on the first weight of each classification label in the first image;

[0027] S36A: The first image is labeled according to the first confidence level and the first image classification to obtain a labeled dataset.

[0028] A further improvement of the present invention is that step S30 specifically includes:

[0029] S31B: Based on several regions of interest in each of the images, select several images in which the regions of interest overlap, and use them as several second images;

[0030] S32B: Based on several regions of interest in each of the second images and one or more classification labels corresponding to each region of interest, several classification labels existing in each of the second images are used as several classification labels to be processed in each of the second images.

[0031] S33B: In each of the second images, one or more of the several classification labels to be processed are selected as feature classifications, and the rest are selected as target classifications. The region of interest corresponding to the target classification is the target region, and the region of interest corresponding to the feature classification is the feature region.

[0032] S34B: In each of the second images, for each target category, the number of overlaps between the target region and the feature region is selected as the second weight;

[0033] S35B: In each of the second images, the second confidence level of the target classification is calculated based on the second weight and the preset correlation coefficient;

[0034] S36B: The second image is labeled according to the target classification and the second confidence level of the target classification to obtain the labeled dataset.

[0035] A further improvement of the present invention is that step S40 specifically includes:

[0036] S41: Establish an initial image classification model and set the loss function;

[0037] S42: Input the image dataset into the initial image classification model to generate predicted values;

[0038] S43: Calculate the difference between the labeled dataset and the predicted value based on the loss function to obtain the numerical residual;

[0039] S44: Update the initial image classification model based on the numerical residual and the predicted value, and repeat steps S42 to S43 until the loss function meets the preset cutoff condition. Output the initial image classification model at this time as the image classification model.

[0040] Secondly, the present invention provides an image classification machine learning system based on regions of interest and smooth annotations, comprising:

[0041] The data acquisition module is used to acquire several image data and generate an image dataset based on the several image data;

[0042] The segmentation module is used to divide each image in the image dataset into one or more regions of interest, and to assign one or more classification labels to each region of interest;

[0043] The annotation module is used to annotate each image in the image dataset according to the region of interest and the classification label to obtain an annotated dataset, wherein the annotation includes confidence and image classification;

[0044] The model training module is used to train an image classification model using machine learning algorithms based on the image dataset and the labeled dataset.

[0045] Thirdly, an electronic device is provided, comprising:

[0046] One or more processors;

[0047] Storage device for storing one or more programs.

[0048] When the one or more programs are executed by the one or more processors, the one or more processors implement the image classification machine learning method and system based on region of interest and smooth annotation as described in the first aspect.

[0049] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the image classification machine learning method and system based on region of interest and smooth annotation as described in the first aspect.

[0050] The above technical solution has the following beneficial technical effects:

[0051] This invention divides an image into several regions of interest and labels the image according to the regions of interest and their classification labels, which can efficiently produce smooth labeled data, thereby improving the training effect of the image classification model and effectively improving the classification accuracy of the image classification model. Attached Figure Description

[0052] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0053] Figure 1 This is a flowchart of the image classification machine learning method based on region of interest and smooth annotation of the present invention;

[0054] Figure 2 This is a flowchart of step S10 in this invention;

[0055] Figure 3 This is a flowchart of step S20 in this invention;

[0056] Figure 4 This is a flowchart of step S30A in this invention;

[0057] Figure 5 This is a flowchart of step S30B in this invention;

[0058] Figure 6 This is a flowchart of step S40 in this invention;

[0059] Figure 7 This is a flowchart of the image classification machine learning system based on region of interest and smooth annotation of the present invention;

[0060] Figure 8 This is a schematic diagram of the structure of a computer system according to an embodiment of the present invention. Detailed Implementation

[0061] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0062] Example 1

[0063] like Figure 1 As shown, this embodiment of the invention provides an image classification machine learning method based on regions of interest and smooth annotations, which includes the following steps:

[0064] S10: Acquire several image data and generate an image dataset based on the several image data;

[0065] S20: Divide each image in the image dataset into one or more regions of interest, and assign one or more classification labels to each region of interest;

[0066] S30: Based on the region of interest and the classification label, each image in the image dataset is labeled to obtain a labeled dataset, wherein the labeling includes confidence and image classification;

[0067] S40: Using machine learning algorithms, an image classification model is trained based on the image dataset and the labeled dataset.

[0068] Specifically, such as Figure 2 As shown, step S10 includes:

[0069] S11: Perform image enhancement on the aforementioned image data to obtain several enhanced image data;

[0070] S12: The size of several enhanced image data is unified by interpolation to obtain several unified image data;

[0071] S13: Normalize the unified image data to obtain normalized image data.

[0072] S14: Generate an image dataset based on several of the normalized image data.

[0073] Specifically, in step S11, the image enhancement includes contrast enhancement, sharpening, denoising, and color enhancement. Contrast enhancement adjusts the image's histogram to make bright areas brighter and dark areas darker. Sharpening enhances the image's edges and details, making it clearer. Denoising removes noise from the image, such as Gaussian noise or salt-and-pepper noise. Color enhancement improves the visual effect by adjusting the image's color balance or saturation. Step S11 improves the image's visual effect and enhances certain features, making it easier for machines to recognize.

[0074] Specifically, in step S12, the interpolation method is used to adjust all images to the same resolution so that they can be processed uniformly. The interpolation method includes Nearest Neighbor Interpolation, Bilinear Interpolation, or Bicubic Interpolation. Taking Nearest Neighbor Interpolation as an example, when adjusting the image size, the value of the target pixel is directly taken from the value of the nearest source image pixel. This method does not introduce new pixel values, thus its calculation speed is fast. Nearest Neighbor Interpolation adjusts the image to the specified resolution using Python and the OpenCV library. Bilinear Interpolation is suitable for image scaling, has a relatively fast calculation speed, and produces good image quality. Bicubic Interpolation (cv2.INTER_CUBIC) produces even higher image quality but has a slower calculation speed, making it suitable for scenarios with high image quality requirements.

[0075] Specifically, in step S13, the normalization is used to adjust the pixel values ​​of the image to a specific range (e.g., [0, 1] or [-1, 1]) to eliminate differences in pixel value ranges between different images. Normalization is used for the input of deep learning models because models typically have certain requirements on the range of input data. The normalization methods include linear normalization and mean normalization. Linear normalization is used to scale pixel values ​​from the original range to [0, 1]. Mean normalization is used to subtract the mean from the pixel value and divide by the standard deviation. Linear normalization is suitable for data with strict requirements on the data range and relatively uniform data distribution, but it is sensitive to outliers. Mean normalization is suitable for scenarios with high requirements on data centering and relatively stable data distribution (close to a normal distribution), and it is more robust to outliers. The choice is based on the quality of the acquired image data.

[0076] Specifically, such as Figure 3 As shown, step S20 includes:

[0077] S21: Perform feature extraction based on the image dataset to obtain several basic features;

[0078] S22: Generate several classification labels based on several of the aforementioned basic features;

[0079] S23: Used to divide each of the images into one or more regions of interest based on the image dataset and a preset image segmentation model;

[0080] S24: Based on the aforementioned classification labels, assign one or more of the aforementioned labels to each region of interest.

[0081] Specifically, in steps S21 and S22, the feature extraction includes color feature extraction, texture feature extraction, shape feature extraction, or transform-based feature extraction. Color feature extraction is used to extract the image's histogram or color moments, etc. Texture feature extraction uses methods such as Gray-level Co-occurrence Matrix (GLCM) and Local Binary Patterns (LBP). Shape feature extraction is used to extract contour or boundary information, etc. Transform-based feature extraction uses wavelet transform or Fourier transform, etc. In step S21, deep learning methods can also be used to establish a neural network to automatically learn the image's feature representation, thereby achieving feature extraction.

[0082] Specifically, in steps S23 and S24, the preset image segmentation model is based on Labelme or other segmentation tools, and a classification label is added to each region. Labelme generates a JSON file containing the coordinates and classification label of each region of interest. For example, to obtain an image classification model that can classify "eating" and "reading" scenes, when segmenting an image, if one image contains 10 people, 7 eating and 3 reading, a rectangle is drawn in Labelme to enclose the 7 people eating (this is a "region of interest"), and the classification label of this rectangle is set to "eating". Another rectangle is drawn to enclose the 3 people reading (this is also a "region of interest"), and the classification label of this rectangle is set to "reading". If these two regions of interest overlap, they can be split into more rectangular regions, or polygonal regions of interest can be used to eliminate the overlap; or overlap can be allowed.

[0083] Specifically, such as Figure 4 As shown, step S30 includes:

[0084] S31A: Based on one or more regions of interest in each of the images, select several images in which no regions of interest overlap, and use them as several first images;

[0085] S32A: Calculate the area of ​​several first images and the area of ​​each region of interest in each first image;

[0086] S33A: In each of the first images, the relative area of ​​each region of interest in the first image is calculated based on the area of ​​the first image and the area of ​​each region of interest in the first image.

[0087] S34A. In each of the first images, the relative areas of several regions of interest corresponding to the same category label are added together to obtain the first weight of the same category label. All category labels are traversed to obtain the first weight of each category label in the first image.

[0088] S35A. Calculate the first confidence level and the first image classification based on the first weight of each classification label in the first image;

[0089] S36A. The first image is labeled according to the first confidence level and the first image classification to obtain the labeled dataset.

[0090] Specifically, in step S31A, the image with only one region of interest does not need to be filtered and is directly used as the first image in subsequent steps. During the filtering process, the geometric data (such as coordinates or boundaries) between two regions of interest are judged to determine whether there is overlap.

[0091] Specifically, in step S33A, the formula for calculating the relative area is as follows:

[0092] Relative area = Area of ​​region of interest / Area of ​​the image containing the region of interest;

[0093] Specifically, in step S35A, the confidence level is calculated using the following formula:

[0094] First confidence level = First weight α ;

[0095] In the formula, α is the first confidence constant, preferably 0.5, and this first confidence formula is used for the same classification label. Assuming the first weight of the "eating" region is 0.4, then the first confidence score for this image belonging to the "eating" category is 0.4. 0.5 =0.63; the first weight of the "reading" region is 0.2, therefore the first confidence level for this image to belong to the "reading" category is 0.2. 0.5 =0.45. The annotation includes several first confidence levels and several first image classifications. The several image classifications are several classification labels corresponding to the first confidence levels. For example, in image A, the first confidence level of the classification label "reading" is 0.45, so the image label of image A contains "reading".

[0096] Specifically, such as Figure 5 As shown, step S30 includes:

[0097] S31B: Based on several regions of interest in each of the images, select several images in which the regions of interest overlap, and use them as several second images;

[0098] S32B: Based on several regions of interest in each of the second images and one or more classification labels corresponding to each region of interest, several classification labels existing in each of the second images are used as several classification labels to be processed in each of the second images.

[0099] S33B: In each of the second images, one or more of the several classification labels to be processed are selected as feature classifications, and the rest are selected as target classifications. The region of interest corresponding to the target classification is the target region, and the region of interest corresponding to the feature classification is the feature region.

[0100] S34B: In each of the second images, for each target category, the number of overlaps between the target region and the feature region is selected as the second weight;

[0101] S35B: In each of the second images, the second confidence level of the target classification is calculated based on the second weight and the preset correlation coefficient;

[0102] S36B: The second image is labeled according to the target classification and the second confidence level of the target classification to obtain the labeled dataset.

[0103] Specifically, the formula for calculating the second confidence level is as follows:

[0104] ;

[0105] In the formula, β is the second confidence constant. Assume the feature classification is "person," and the target classifications are "eating" and "reading." The correlation coefficient between "person" and "eating" is 1, and the correlation coefficient between "person" and "reading" is 1. The number of "person" regions intersecting with the "eating" region is 7, and the number of "person" regions intersecting with the "reading" region is 3. Therefore, the confidence level for this image to belong to the "eating" category is: The confidence level for the category "reading" is: The confidence level mentioned in step S30 includes a first confidence level and a second confidence level.

[0106] Specifically, such as Figure 6 As shown, step S40 includes:

[0107] S41: Establish an initial image classification model and set the loss function;

[0108] S42: Input the image dataset into the initial image classification model to generate predicted values;

[0109] S43: Calculate the difference between the labeled dataset and the predicted value based on the loss function to obtain the numerical residual;

[0110] S44: Update the initial image classification model based on the numerical residual and the predicted value, and repeat steps S42 to S43 until the loss function meets the preset cutoff condition. Output the initial image classification model at this time as the image classification model.

[0111] Specifically, in step S41, the establishment of the initial image classification model includes optimizer selection, learning rate and learning rate decay strategy settings, batch size settings, and iteration count settings. The optimizer includes Stochastic Gradient Descent (SGD), Adaptive Moment Estimation (Adam), or Root Mean Square Propagation (RMSprop). Stochastic Gradient Descent calculates the gradient by randomly selecting one or more samples (mini-batch), thereby updating the model parameters. SGD's advantages include high computational efficiency and suitability for large-scale datasets, but its convergence speed may be slow, and it is prone to getting trapped in local optima. Adaptive Moment Estimation dynamically adjusts the learning rate of each parameter by calculating the first moment (mean) and second moment (uncentered variance) of the gradient. Adam's advantages include fast convergence speed, suitability for non-convex optimization problems, and robustness to hyperparameter selection. Root Mean Square Propagation dynamically changes the learning rate by adjusting the moving average of the squared gradient. It is particularly suitable for handling problems with large gradient changes and can effectively prevent gradient explosion or gradient vanishing. The core idea of ​​RMSprop is to normalize the gradient by dividing by the mean of the squared gradients, thereby stabilizing the optimization process. The batch size determines the number of samples for each training iteration, and the number of iterations determines the total training iterations. The loss function used is the cross-entropy loss function.

[0112] Specifically, in step S42, the image dataset is divided into a training set, a test set, and a validation set according to a preset ratio (e.g., 7:1.5:1.5). The image data input to the initial image classification model in step S42 is all training set data, and subsequent iterative updates also only involve the training set data. The validation set is used to evaluate the model's performance during training. Through the validation set, the performance of different hyperparameters can be compared, and the optimal model and hyperparameters can be selected. The validation set helps detect whether the model is overfitting (i.e., performing well on the training set but poorly on new data). If the model performs well on the training set but poorly on the validation set, it indicates that the model may be overfitting. The validation set is also used to adjust the model's hyperparameters (e.g., learning rate, regularization parameters, number of network layers, etc.). By evaluating the performance of different hyperparameters on the validation set, the optimal combination of hyperparameters is selected. The test set is used to evaluate the model's performance on completely unseen data. The test set is used only after model training and validation are completed, and is used to evaluate the model's generalization ability. The purpose of the test set is to simulate the model's performance in real-world applications. It provides an independent evaluation environment to ensure the model's performance on real data.

[0113] Specifically, the numerical residual is the difference between the predicted value and the data in the labeled dataset. Based on this numerical residual, the gradient is calculated, and the network weights are updated through backpropagation.

[0114] Specifically, in step S44, data is saved periodically each time the initial image classification model is updated to avoid data loss.

[0115] Example 2

[0116] like Figure 7 As shown, this embodiment of the invention provides an image classification machine learning system based on regions of interest and smooth annotation, which includes the following modules:

[0117] The data acquisition module is used to acquire several image data and generate an image dataset based on the several image data;

[0118] The segmentation module is used to divide each image in the image dataset into one or more regions of interest, and to assign one or more classification labels to each region of interest;

[0119] The annotation module is used to annotate each image in the image dataset according to the region of interest and the classification label to obtain an annotated dataset, wherein the annotation includes confidence and image classification;

[0120] The model training module is used to train an image classification model using machine learning algorithms based on the image dataset and the labeled dataset.

[0121] Specifically, the data acquisition module includes:

[0122] An enhancement unit is used to perform image enhancement on several of the image data to obtain several enhanced image data.

[0123] A size unification unit is used to unify the size of several enhanced image data using an interpolation method to obtain several unified image data.

[0124] A normalization unit is used to normalize several of the unified image data to obtain several normalized image data.

[0125] The generation unit is used to generate an image dataset based on several normalized image data.

[0126] Specifically, the partitioning module includes:

[0127] The feature extraction unit is used to extract features from the image dataset to obtain several basic features;

[0128] A label generation unit is used to generate several classification labels based on several of the aforementioned basic features;

[0129] The segmentation unit is used to divide each image into one or more regions of interest based on the image dataset and a preset image segmentation model;

[0130] The corresponding unit is used to set one or more labels for each region of interest based on the aforementioned classification labels.

[0131] Specifically, the annotation module includes:

[0132] The first filtering unit is used to filter out several images that do not have overlapping regions of interest based on one or more regions of interest in each image, and to select several first images.

[0133] An area calculation unit is used to calculate the area of ​​several first images and the area of ​​each region of interest in each of the first images;

[0134] The relative area calculation unit is used to calculate the relative area of ​​each region of interest in the first image based on the area of ​​the first image and the area of ​​each region of interest in the first image.

[0135] The first weight calculation unit is used to add the relative areas of several regions of interest corresponding to the same classification label in each of the first images to obtain the first weight of the same classification label, and to traverse all classification labels to obtain the first weight of each classification label in the first image.

[0136] The first confidence calculation unit is used to calculate the first confidence and the first image classification based on the first weight of each classification label in the first image;

[0137] The first annotation unit is used to annotate the first image based on the first confidence level and the first image classification to obtain an annotated dataset.

[0138] Specifically, the annotation module also includes:

[0139] The second filtering unit is used to filter out several images in which the regions of interest overlap, based on several regions of interest in each image, and to select several second images.

[0140] The label filtering unit is used to select several classification labels present in each second image as several classification labels to be processed for each second image based on several interest regions in each second image and one or more classification labels corresponding to each interest region.

[0141] The label segmentation unit is used to select one or more of the several classification labels to be processed as feature classifications in each of the second images, and the rest as target classifications. The region of interest corresponding to the target classification is the target region, and the region of interest corresponding to the feature classification is the feature region.

[0142] The second weight calculation unit is used to select the number of overlaps between the target region and the feature region as the second weight for each target category in each of the second images.

[0143] The second confidence calculation unit is used to calculate the second confidence of the target classification in each of the second images based on the second weight and a preset correlation coefficient.

[0144] The second annotation unit is used to annotate the second image according to the target classification and the second confidence level of the target classification to obtain an annotated dataset.

[0145] Specifically, the training module includes:

[0146] The initial modeling unit is used to build the initial image classification model and set the loss function;

[0147] The prediction unit is used to input the image dataset into the initial image classification model to generate predicted values;

[0148] The residual calculation unit is used to calculate the difference between the labeled dataset and the predicted value according to the loss function, and obtain the numerical residual.

[0149] The update unit is used to update the initial image classification model based on the numerical residual and the predicted value, and to repeatedly execute the prediction unit to the residual calculation unit until the loss function meets the preset cutoff condition, and output the initial image classification model at this time as the image classification model.

[0150] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0151] This invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements any of the methods described above.

[0152] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. Of course, there are other types of readable storage media, such as quantum memories, graphene memories, etc.

[0153] The present invention also provides an electronic device. The electronic device of an embodiment of the present invention includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the image classification machine learning method based on region of interest and smooth annotation provided by the present invention.

[0154] The following is for reference. Figure 8 It shows a schematic diagram of the structure of a computer system 800 suitable for implementing an electronic device according to embodiments of the present invention. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0155] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 802 or programs loaded from storage section 808 into random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the computer system 800. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0156] The following components are connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 810 as needed so that computer programs read from it can be installed into storage section 808 as needed.

[0157] In particular, according to the embodiments disclosed in this invention, the processes described in the above main step diagrams can be implemented as computer software programs. For example, embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the main step diagrams. In the above embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by central processing unit 801, it performs the functions defined in the system of this invention.

[0158] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0160] The units described in the embodiments of the present invention can be implemented in software or hardware. The described units can also be located in a processor; for example, a processor can be described as including a front-end response unit, a receiving unit, and a request unit. The names of these units do not necessarily limit the specific unit itself.

[0161] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image classification machine learning method based on a region of interest and a smooth annotation, characterized in that, The method comprises the following steps: S10: obtaining a plurality of image data and generating an image data set according to the plurality of image data; S20: dividing each image in the image data set into one or a plurality of attention regions and setting one or a plurality of classification labels for each attention region; S30: labeling each image in the image data set according to the attention region and the classification label to obtain a labeled data set, wherein the labeling comprises a confidence and an image classification; S40: training an image classification model according to the image data set and the labeled data set by using a machine learning algorithm; In the step S30, the method specifically comprises: S31A: screening a plurality of first images from the plurality of images according to one or a plurality of attention regions in each image, wherein the first images do not have overlapping attention regions; S32A: calculating the area of the plurality of first images and the area of each attention region in each first image; S33A: in each first image, calculating the relative area of each attention region in the first image according to the area of the first image and the area of each attention region in the first image; S34A: in each first image, adding the relative areas of a plurality of attention regions corresponding to the same classification label to obtain a first weight of the same classification label, and traversing all classification labels to obtain the first weight of each classification label in the first image; S35A: calculating a first confidence and a first image classification according to the first weight of each classification label in the first image; S36A: labeling the first image according to the first confidence and the first image classification to obtain a labeled data set; S31B: screening a plurality of second images from the plurality of images according to a plurality of attention regions in each image, wherein the second images have overlapping attention regions; S32B: according to a plurality of attention regions in each second image and one or a plurality of classification labels corresponding to each attention region, taking a plurality of classification labels existing in each second image as a plurality of to-be-processed classification labels of each second image; S33B: in each second image, selecting one or a plurality of to-be-processed classification labels as feature classifications and the rest as target classifications from the plurality of to-be-processed classification labels, wherein the attention region corresponding to the target classification is a target region and the attention region corresponding to the feature classification is a feature region; S34B: in each second image, screening the number of overlapping target regions and feature regions for each target classification as a second weight; S35B: in each second image, calculating a second confidence of the target classification according to the second weight and a preset correlation coefficient; S36B: labeling the second image according to the target classification and the second confidence of the target classification to obtain a labeled data set. 2.The image classification machine learning method based on the region of interest and the smooth annotation according to claim 1, wherein, In the step S10, the method specifically comprises: S11: performing image enhancement on the plurality of image data to obtain a plurality of enhanced image data; S12: performing size unification on the plurality of enhanced image data by using an interpolation method to obtain a plurality of unified image data; S13: normalizing the plurality of unified image data to obtain a plurality of normalized image data; S14: generating an image data set according to the plurality of normalized image data. 3.The image classification machine learning method based on region of interest and smoothed annotation of claim 1, wherein, The step S20 specifically comprises: S21: extracting features according to the image data set to obtain a plurality of basic features; S22: generating a plurality of classification labels according to the plurality of basic features; S23: dividing each of the images into one or a plurality of attention regions according to the image data set and a preset image division model; S24: setting one or a plurality of labels for each attention region according to the plurality of classification labels. 4.The image classification machine learning method based on region of interest and smoothed annotation of claim 1, wherein, In step S40, specifically comprising: S41: establishing an initial image classification model and setting a loss function; S42: inputting the image data set into the initial image classification model to generate a predicted value; S43: calculating the difference between the labeled data set and the predicted value according to the loss function to obtain a numerical residual; S44: updating the initial image classification model according to the numerical residual and the predicted value, repeating steps S42 to S43 until the loss function meets the preset stop condition, and outputting the initial image classification model at this time as the image classification model. 5.The image classification machine learning method based on region of interest and smoothed annotation of claim 1, wherein, Further comprising step S50: obtaining image data to be classified, inputting the image data to be classified into the image classification model to obtain an image classification result.

6. An image classification machine learning system based on region of interest and smooth annotation, characterized by, Comprise: A data acquisition module for acquiring a plurality of image data and generating an image data set according to the plurality of image data; A division module for dividing each image in the image data set into one or a plurality of attention regions, and setting one or a plurality of classification labels for each attention region; A labeling module for labeling each image in the image data set according to the attention region and the classification label to obtain a labeled data set, the labeling including confidence and image classification; A model training module for training an image classification model according to the image data set and the labeled data set using a machine learning algorithm; In the labeling module, comprising: A first screening unit for screening a plurality of images without overlapping attention regions according to one or a plurality of attention regions in each of the images as a plurality of first images; An area calculation unit for calculating the area of a plurality of first images and the area of each attention region in each first image; A relative area calculation unit for calculating the relative area of each attention region in a first image according to the area of the first image and the area of each attention region in the first image in each first image; A first weight calculation unit for adding the relative areas of a plurality of attention regions corresponding to the same classification label in each first image to obtain the first weight of the same classification label, and traversing all classification labels to obtain the first weight of each classification label in the first image; A first confidence calculation unit for calculating the first confidence and the first image classification according to the first weight of each classification label in the first image; The first labeling unit is configured to label the first image according to the first confidence and the first image classification, to obtain a labeled data set; The second screening unit is configured to screen, according to the plurality of attention regions in each of the images, a plurality of images in which the attention regions overlap, as a plurality of second images; The label screening unit is configured to, according to the plurality of attention regions in each of the second images and one or more classification labels corresponding to each of the attention regions, take the plurality of classification labels existing in each of the second images as a plurality of to-be-processed classification labels of each of the second images; The label division unit is configured to, in each of the second images, select one or more of the plurality of to-be-processed classification labels as a feature classification, and the rest as a target classification, wherein the attention region corresponding to the target classification is a target region, and the attention region corresponding to the feature classification is a feature region; The second weight calculation unit is configured to, in each of the second images, for each target classification, screen out the number of overlaps between the target region and the feature region as a second weight; The second confidence calculation unit is configured to, in each of the second images, calculate a second confidence of the target classification according to the second weight and a preset correlation coefficient; The second labeling unit is configured to label the second image according to the target classification and the second confidence of the target classification, to obtain a labeled data set.

7. An electronic device, comprising: comprise: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the image classification machine learning method based on the attention region and the smooth labeling according to any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the image classification machine learning method based on the attention region and the smooth labeling according to any one of claims 1-5.

Images