A method and system for optimizing computing resources of an image processing algorithm

Abstract

The application discloses a kind of image processing algorithm's computing resource optimization processing method and system, the method includes: the training data is divided into first part and second part;Supervised machine learning training is carried out using all groups of training data in first part to obtain first machine learning model;The picture in each group in first part is split into multiple blocks, each block is input into first machine learning model, and the label of output is obtained;A set of second training data is generated;All groups of training data in second part are input into second machine learning model, and the picture and the block corresponding to the picture of each group in second part are obtained, third training data is generated;Third training data is used to carry out incremental training to first machine learning model.The accuracy of machine learning model is reduced more by the application, so as to improve the training efficiency of machine learning model while ensuring a certain degree of accuracy of machine learning model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image, in particular, to a method and system for optimizing computing resources of an image processing algorithm. BACKGROUND

[0002] Supervised learning, also known as supervised machine learning, is a subcategory of machine learning and artificial intelligence. It is defined as the use of a labeled dataset to train an algorithm to classify or accurately predict outcomes from data. When input data is fed into the model, it adjusts its weights until the model is properly fitted, which is part of the cross-validation process. Supervised learning can help organizations solve a variety of real-world problems at scale, such as classifying spam in separate folders in an inbox.

[0003] Supervised learning uses a training set to teach a model to produce the desired output. This training dataset includes inputs and correct outputs, which allows the model to learn over time. The algorithm measures its accuracy through a loss function, adjusting until the error is sufficiently minimized. Supervised learning can be divided into two categories of problems when data mining - classification and regression: Classification uses an algorithm to accurately assign test data to a specific category. It identifies specific entities in a dataset and tries to draw some conclusions about how to label or define those entities. Common classification algorithms are linear classifiers, support vector machines (SVM), decision trees, k-nearest neighbors, and random forests. Regression is used to understand the relationship between dependent and independent variables. It is commonly used for prediction, such as predicting sales revenue for a given business. Linear regression, logistic regression, and polynomial regression are commonly used regression algorithms.

[0004] Supervised learning generally uses training data with expert-labeled labels to learn a function mapping from input variables X to input variables Y. Y = f(X), the training data is usually in the form of (n x x, y), where n represents the size of the training sample, and x and y are sample values of variables X and Y, respectively.

[0005] In image recognition, a supervised training method is generally used. The training data used in image recognition basically includes a picture and a label identifying the name or nature of the subject in the picture, which is usually labeled manually. For example, in the training of a machine learning model for identifying good and bad tomatoes, each set of training data includes a picture of a good tomato or a bad tomato, and a label identifying whether the tomato is good or bad. For example, in the training of a machine learning model for distinguishing cats and dogs, each set of training data includes a picture of a cat or a dog, and a label identifying whether the animal is a cat or a dog.

[0006] For supervised machine learning training, the more training data it has, the more accurate the model it trains. However, the more training data it has, the longer it takes to train, especially when the computing resources used for training are insufficient. In order to reduce training time and improve training efficiency, the training data is usually reduced, but this will affect the accuracy of the machine learning model trained. SUMMARY

[0007] Embodiments of the present application provide a method and system for optimizing the computing resources of an image processing algorithm to at least solve the problem of reduced accuracy of a machine learning model caused by reducing the amount of training data to improve the training efficiency of the machine learning model.

[0008] According to an aspect of the present application, a method for optimizing the computing resources of an image processing algorithm is provided, comprising: obtaining a plurality of sets of training data and dividing the training data into a first part and a second part, wherein the first part and the second part each include at least two sets of training data; each set of training data includes an image and a label indicating the name of an entity present in the image, and the plurality of sets of training data are used to train a supervised machine learning model; using all sets of training data in the first part to perform supervised machine learning training to obtain a first machine learning model; splitting the image in each set in the first part into multiple blocks, inputting each block into the first machine learning model to obtain the output label; if the output label of a block in the first machine learning model is the same as the corresponding label of the block, a second set of training data is generated, wherein the second set of training data includes an image and one or two blocks in the image, and the output label of the one or two blocks in the image in the first machine learning model is the same as the corresponding label of the image; using the second training data to train a second machine learning model, inputting all sets of training data in the second part into the second machine learning model to obtain the image and the corresponding block of the image in each set in the second part; associating the corresponding block of each image in the second part with the corresponding label of the block to generate a third training data; and using the third training data to perform incremental training on the first machine learning model.

[0009] Further, using all sets of training data in the first part to perform supervised machine learning training to obtain a first machine learning model comprises: performing clarity processing on the image in each set of training data, wherein the clarity processing is used to reduce the clarity of the image; establishing a corresponding association between the image with reduced clarity and the original image and saving it; and using the image with reduced clarity and its corresponding label as training data to perform supervised machine learning training.

[0010] Further, the incrementally training the first machine learning model using the third training data comprises: replacing the blocks in the third training data with blocks in original pictures without reduced clarity according to the association relationship; and incrementally training the first machine learning model using the third training data in which the blocks are replaced with the blocks in the original pictures.

[0011] Further, the obtaining the multiple groups of training data comprises: obtaining all training data, reserving a part of the training data, and using the training data other than the reserved part of the training data as the multiple groups of training data; and after the incrementally training the first machine learning model using the third training data, the method further comprises: verifying the first machine learning model after the incrementally training using the reserved part of the training data.

[0012] According to another aspect of the present application, a computing resource optimization processing system of an image processing algorithm is also provided, comprising: a data segmentation module configured to obtain multiple groups of training data and divide the training data into a first part and a second part, wherein the first part and the second part each include at least two groups of training data; each group of training data includes a picture and a label indicating a name of an entity existing in the picture, and the multiple groups of training data are used for training a supervised machine learning model; a first training module configured to use all groups of training data in the first part to perform supervised machine learning training to obtain a first machine learning model; a splitting module configured to split the picture in each group in the first part into multiple blocks, input each block into the first machine learning model, and obtain an output label; a first generation module configured to generate a group of second training data if the output label of a block in the first machine learning model is the same as the label corresponding to the block, wherein the second training data includes a picture and one or two blocks in the picture, and the output label of the one or two blocks in the picture in the first machine learning model is the same as the label corresponding to the picture; a second generation module configured to use the second training data to train a second machine learning model, input all groups of training data in the second part into the second machine learning model, and obtain the picture in each group in the second part and the block corresponding to the picture; establish an association between the block corresponding to each picture in the second part and the label corresponding to the block, and generate third training data; and a second training module configured to incrementally train the first machine learning model using the third training data.

[0013] Further, the first training module is configured to: perform sharpness processing on the pictures in the training data of each group, wherein the sharpness processing is configured to reduce the sharpness of the pictures; establish a corresponding association between the pictures with reduced sharpness and the original pictures and save the association; and use the pictures with reduced sharpness and the corresponding labels as training data to perform supervised machine learning training.

[0014] Further, the first training module is configured to: replace the blocks in the third training data with the blocks in the original pictures without reduced sharpness according to the association; and use the third training data with the blocks replaced with the blocks in the original pictures to perform incremental training on the first machine learning model.

[0015] Further, the data segmentation module is configured to: obtain all the training data, reserve a part of the training data from the all the training data, and use the training data other than the reserved part of the training data as the multiple groups of training data; and the system further comprises a verification module configured to: after the incremental training on the first machine learning model using the third training data, use the first machine learning model after the incremental training using the reserved part of the training data to perform verification.

[0016] In the embodiments of the present application, a plurality of groups of training data are obtained, and the training data are divided into a first part and a second part, wherein the first part and the second part each include at least two groups of training data; each group of training data includes an image and a label indicating the name of an entity existing in the image, and the plurality of groups of training data are used for training of a supervised machine learning model; a first machine learning model is obtained by using all groups of training data in the first part for supervised machine learning training; the image in each group in the first part is split into a plurality of blocks, each block is input into the first machine learning model, and output labels are obtained; if the label output by a block in the first machine learning model is the same as the label corresponding to the block, a second group of training data is generated, wherein the second training data includes an image and one or two blocks in the image, and the label output by the one or two blocks in the image in the first machine learning model is the same as the label corresponding to the image; a second machine learning model is obtained by using the second training data for training, all groups of training data in the second part are input into the second machine learning model, and the image and the block corresponding to the image in each group in the second part are obtained; the block corresponding to each image in the second part is associated with the label corresponding to the block, and third training data are generated; the first machine learning model is incrementally trained by using the third training data. The present application solves the problem of a large decrease in the accuracy of a machine learning model caused by reducing the amount of training data to improve the training efficiency of the machine learning model in the related art, thereby improving the training efficiency of the machine learning model while ensuring a certain degree of accuracy of the machine learning model. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are used to interpret the application together with the specification. The illustrative embodiments of the present application and their description serve to explain the application. In the drawings:

[0018] Figure 1 is a flowchart of a computing resource optimization processing method of an image processing algorithm according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] It should be noted that the embodiments and features in the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0021] In the following embodiments, the main idea is to combine multiple different relatively weak machine learning models to process training data, appropriately reduce the data volume of the training data, and thus appropriately improve the training efficiency without reducing the training data volume.

[0022] In the following embodiments, a picture is trained to finally obtain a machine learning model (also referred to as a neural network model), and the picture can be various pictures, that is, the following embodiments can be applied to multiple scenes, and the following exemplary descriptions are made for these scenes.

[0023] Scenario one

[0024] In this scenario, the appearance of the battery module is detected, which can include the following steps: obtaining the appearance picture of the battery module; inputting the appearance picture into the encoding layer, extracting the features of the appearance picture through the encoding layer; analyzing the features extracted by the encoding layer through the decoding layer, and outputting the result. The encoding layer includes a data preprocessing module, a multi-dimensional feature extraction module, and at least one three-dimensional feature extraction module arranged in sequence; the main function of the data preprocessing module is to read the image input and standardize it to a vector input; the multi-dimensional feature extraction module is used to extract a feature map from the input vector, and the multi-dimensional spatial feature extraction module is designed as a depth separable convolution layer, and uses a standardization function and a ReLU function to obtain two types of feature maps from the input image. The feature maps are low-level feature maps, which contain low-level spatial features such as texture, grayscale, etc.; high-level feature maps, which contain deep semantic information; the three-dimensional feature extraction module is used to convert the appearance picture into a three-dimensional image, and extract a three-dimensional feature map from the three-dimensional image using three-dimensional convolution, the three-dimensional feature map includes three-dimensional spectral features and / or three-dimensional spatial features; the purpose of the three-dimensional feature extraction module is to obtain effective spectral and spatial features by reshaping the input image and associating far and near pixels. The three-dimensional feature extraction module converts the image into a three-dimensional image by reconstruction, wherein the input image is divided into parts, then a third dimension is added, and finally a three-dimensional image is obtained. In this way, unlike standard two-dimensional convolution, spectral and spatial features can also be obtained from the image without destroying the original image structure. The reconstructed three-dimensional image is subjected to three-dimensional convolution to obtain a four-dimensional feature matrix. The third and fourth dimensions of the four-dimensional feature matrix are combined to reconstruct the feature map.

[0025] Scenario two

[0026] In this scenario, the image method is used to detect the defects of insulators. First, the insulator string is identified in the photographed image, and then the defect type is judged according to the computer image algorithm. Specifically, the Top hat algorithm can identify the contour information of insulators in complex background images; the multimodal fusion detection technology is used to realize the identification of various defects of insulators and to remind the defects in real time; the YOLOv5 algorithm is improved to enable the model to obtain more object edge information and adapt to pictures of different resolutions; through transfer learning, the related instances or features in the continuously increasing defect dataset can be migrated to the micro data set, thereby improving the generalization ability of the model. The above-mentioned Top hat algorithm can identify the contour information of insulators in complex background images, which specifically includes the following contents: for insulator defect image recognition in substations, the insulator equipment must be accurately locked first, and then the defect situation can be further analyzed; the local contour information recognition in complex background pictures is a smoothing filtering operation on the image, which removes the noise interference and clutter interference in the complex background image, thereby improving the accuracy of the local contour information recognition in the complex background image; the Top hat algorithm is the difference between the original image and the "opening operation" result image; when opening operation, the area of the crack or local low brightness is enlarged, so the effect image obtained by the Top hat algorithm highlights the area brighter than the surrounding area of the original image contour; when the insulator is in the complex background of multiple devices in the substation, its own shape presents a relatively regular shape, and the Top hat algorithm is used to extract its contour information. The above-mentioned multimodal fusion detection technology is specifically a multimodal fusion detection technology (Multimodal-Fusion Detection) constructed by using deep learning computer vision technology, combining machine learning, classical CV algorithm and other signal processing methods according to the special scene of the specific substation and the shape characteristics of the insulator equipment, and using the "add" joint method; in different hidden layers, the semantic subspace is shared, and the converted single modal feature vectors are combined together, thereby realizing multimodal fusion. The above-mentioned improved YOLOv5 algorithm specifically includes: considering the complexity of the insulator video or image data, the corresponding data processing is performed, and the insulator detection dataset is made; based on the YOLOv5 model as the main body, first, the image pyramid structure is used to fuse the features of different levels to obtain feature maps of different scales for position and category prediction; then, the k-means unsupervised clustering algorithm is used, the image is preprocessed appropriately, the data can be analyzed preliminarily, the valuable information is mined, and the target frame dimension is clustered to increase the number of anchor boxes (Anchorbox), so that the model can obtain more object edge information; finally, in the training process, multiple size pictures are used for training, so that the model can adapt to pictures of different resolutions; thereby identifying the feature position of the insulator defect image and judging the defect category.Through transfer learning, relevant instances or features in an increasing defect dataset can be transferred to a micro-data set, thereby improving the generalization ability of the model. Specifically, in the Faster R-CNN model, the feature layer learned by the VGG-16 convolutional neural network in the ImageNet is transferred to the defect recognition problem with less data. The transferred feature layer includes complex and diverse feature extraction capabilities (edge features, texture features, and local abstract features, etc.).

[0027] Scenario three

[0028] In this scenario, the escalator action is identified through images. The scenario can include the following steps: obtaining an original image captured by a terminal device; pre-processing the original image, including randomly rotating the original image; randomly changing the contrast of the image; adding noise to the original image; randomly cropping the image, filling other pixels with the mean value of the image pixels, expanding the picture to 600x600, and finally performing a mirror flip operation on the image; using an improved MaskRCNN to detect the single escalator recognition area and the human key point area in the original image. The improved MaskRCNN includes adding a human key point detection branch to the original MaskRCNN. The improved MaskRCNN model can realize semantic segmentation of the single escalator recognition in the original image and detect human key points. Jointly judging the single escalator recognition area and the human key point area and outputting the escalator posture recognition result. The joint judgment includes taking the pixel coordinates of the center of the key point area in the human key point detection output by the model as the center of a circular area and taking a plurality of pixel units as the radius. If the pixel value of the single escalator area after semantic segmentation falls within the circular area, it can be determined that the staff has an escalator action. If the pixel value of the single escalator after semantic segmentation does not fall within the circular area, it can be determined that there is no escalator action.

[0029] The preprocessing of the original images specifically includes: random rotation of 0-360 degrees and changing contrast of all original images, adding different degrees of Gaussian noise and salt and pepper noise to the images. The improved MaskRCNN specifically includes: adding a branch of human key point detection regression after the RPN network, which is parallel to the category classification branch, the bounding box regression branch and the mask branch. The improved MaskRCNN can realize the functions of semantic segmentation and key point detection in one neural network. The human key points are hand parts. The joint judgment includes taking the pixel coordinates of the center of the key point region in the human key point detection output by the model as the center of a circle and taking 10 pixel units as the radius to establish a circular region. If the pixel value of the escalator region after semantic segmentation falls within the region of the circle, it can be determined that the staff exists escalator action; if the pixel value of the escalator after semantic segmentation does not fall within the region of the circle, it can be determined that there is no escalator action.

[0030] The algorithms involved in the above scenario will be described below.

[0031] Mask R-CNN

[0032] Mask R-CNN is an instance segmentation algorithm, which can be used for "target detection", "target instance segmentation" and "target key point detection".

[0033] The difference and connection between instance segmentation and semantic segmentation: semantic segmentation and instance segmentation are both small fields in target segmentation, and are used for image segmentation processing. The difference is that the target segmentation in the general sense refers to semantic segmentation, which has a long development history and has made good progress. At present, many scholars are engaged in research in this field. However, instance segmentation is a small field that has developed in recent years, which is more complex than the former, and there are fewer current researchers. It is a hot field with research space, which is an exploring field. In semantic segmentation, they are given the same color, while in instance segmentation, they are given different colors. That is, instance segmentation needs to make more detailed segmentation of the same object on the basis of semantic segmentation.

[0034] Mask R-CNN is a very flexible framework that can add different branches to complete different tasks, including target classification, target detection, semantic segmentation, instance segmentation, and human pose recognition. Mask R-CNN aims to achieve high speed, high accuracy (high classification accuracy, high detection accuracy, high instance segmentation accuracy, etc.), simplicity, and ease of use.

[0035] High speed and high accuracy: Mask R-CNN can be compared with the classic target detection algorithm Faster-rcnn and the classic semantic segmentation algorithm FCN. Faster-rcnn can achieve fast and accurate target detection, and FCN can achieve precise semantic segmentation. Mask R-CNN is more complex than Faster-rcnn, but it can still achieve a speed of 5fps, which is comparable to the original Faster-rcnn. Due to the discovery of the pixel bias problem in ROIPooling, the corresponding ROIAlign strategy is proposed, plus the precise pixel MASK of FCN, which can achieve high accuracy.

[0036] Simple and intuitive: The whole idea of Mask R-CNN algorithm is very simple, which is to add FCN to the original Faster-rcnn algorithm to generate the corresponding MASK branch. That is, Faster-rcnn+FCN, more detailed RPN+ROIAlign+Fast-rcnn+FCN.

[0037] Easy to use: The whole Mask R-CNN algorithm is very flexible and can be used to complete multiple tasks, including target classification, target detection, semantic segmentation, instance segmentation, and human pose recognition. This fully demonstrates the easy-to-use feature of Mask R-CNN.

[0038] FasterR-C N N

[0039] With the rapid development of artificial intelligence and computer vision technology, target detection as one of the core technologies has been widely used in various fields such as autonomous driving, security monitoring, medical image analysis, etc. Faster R-CNN algorithm as an important technology in target detection field, its high efficiency and accuracy has been widely recognized. This paper will analyze the principle, process, advantages and disadvantages of Faster R-CNN algorithm and its practical application, hoping to provide useful reference for readers.

[0040] In the field of object detection, traditional methods usually consist of two stages: generating candidate regions and classifying these regions. However, these methods are slow in processing, limiting the possibility of real-time applications. R-CNN achieves object detection by extracting fixed-size candidate regions in an image and then classifying each candidate region. Although R-CNN performs well in accuracy, its processing speed is very slow. To solve this problem, Faster R-CNN algorithm emerges as the times require.

[0041] Faster R-CNN algorithm mainly includes the following four steps:

[0042] Feature extraction conv layers: First, the input image is passed through a series of convolutional layers, activation functions, and pooling layers for feature extraction, resulting in corresponding feature maps. The purpose of this step is to extract key information from the image, providing a foundation for subsequent candidate region generation and classification. Region Proposal Networks (RPN) is the core part of Faster R-CNN algorithm, which is responsible for generating candidate regions. RPN generates a series of candidate boxes by sliding a small window on the last layer of the fully convolutional network feature map, and performs binary classification (foreground or background) and boundary box regression on each candidate box, thus obtaining accurate candidate regions. This step greatly reduces the number of candidate regions and improves the running speed of the algorithm. The role of the ROIPooling layer is to map candidate regions of different sizes to fixed-size feature maps for subsequent classification and regression operations. Through ROIPooling, the algorithm can handle input images and candidate regions of any size, enhancing the robustness of the algorithm. Classification: Finally, the fixed-size feature maps output by the ROIPooling layer are input to the fully connected layer for classification, obtaining the class of each candidate region. At the same time, boundary box regression can be used to fine-tune the position of the candidate region, further improving the accuracy of object detection.

[0043] Pros and cons of Faster R-CNN: Efficiency: By introducing RPN, the algorithm can quickly generate accurate candidate regions, reducing the amount of calculation and improving the running speed. Accuracy: Through ROIPooling and the classification and regression of the fully connected layer, the algorithm can accurately identify and locate target objects. However, Faster R-CNN algorithm also has some disadvantages: Poor detection effect on small objects: Since the size of the candidate frame generated by RPN is fixed, for small objects, it may not be able to generate enough candidate frames, resulting in poor detection effect. Limited processing capacity for large objects: When the size of the target object is too large, RPN may not be able to generate enough candidate frames to cover the entire object, affecting the detection effect. Faster R-CNN algorithm performs well in practical application and is widely used in automatic driving, security monitoring, medical image analysis and other fields. For example, in the field of automatic driving, Faster R-CNN can be used to identify pedestrians, vehicles and other obstacles on the road, providing important perception information for the automatic driving system. In the field of security monitoring, Faster R-CNN can be used to detect abnormal events such as intruders, fires, etc., improving the intelligent level of the monitoring system. In the field of medical image analysis, Faster R-CNN can be used to assist doctors in diagnosing diseases such as lung nodules, tumors, etc.

[0044] When performing machine learning training, the following server architecture can be used, of course other servers can also be used, or a single service can also be used.

[0045] The server architecture can be constructed by the following steps: S1, dividing the working nodes geographically dispersed in the wide area network topology into different clusters according to the link physical length and the link bandwidth; step S1 includes the following steps: S11, calculating the weight of each link geographically dispersed in the wide area network topology according to the link physical length and the link bandwidth; S12, calculating the shortest path between any two working nodes according to the weight set constructed by the weight of each link; S13, initializing each working node as a cluster; S14, merging the two closest clusters into one cluster according to the shortest path between any two working nodes; S15, repeating step S14 until the distance between the two closest clusters is greater than a set threshold, and the cluster division is completed; S2, judging whether the number of the current cluster is 1, if yes, jumping to step S3, if not, jumping to step S4; S3, placing the local parameter server and the global parameter server for the unique cluster; step S3 includes the following steps: S31, traversing each working node in the unique cluster, calculating the average distance between each working node and the remaining working nodes; S32, selecting the working node with the minimum average distance in the unique cluster as the local parameter server and the global parameter server of the unique cluster, to realize the placement of the local parameter server and the global parameter server for the unique cluster; S4, placing the local parameter server and the global parameter server for all clusters; step S4 includes the following steps: S41, traversing each working node in all clusters, calculating the average distance between each working node in each cluster and the remaining working nodes, and selecting the working node with the minimum average distance in each cluster as the local parameter server of the cluster; S42, traversing each working node in the wide area network topology, calculating the average distance between each working node and all local parameter servers; S43, selecting the working node with the minimum average distance in the wide area network topology as the global parameter server of the wide area network topology, and removing the working node from the cluster, to realize the placement of the local parameter server and the global parameter server for all clusters.

[0046] To solve the problems in the related art, in the following embodiments, a computing resource optimization processing method of an image processing algorithm is provided, Figure 1 is a flowchart of the computing resource optimization processing method of the image processing algorithm according to the embodiments of the present application, and the steps involved in the method in Figure 1 will be described below.

[0047] In step S102, a plurality of sets of training data are obtained, and the training data are divided into a first part and a second part, wherein the first part and the second part each include at least two sets of training data; each set of training data includes an image and a label indicating the name of an entity present in the image, and the plurality of sets of training data are used for training a supervised machine learning model.

[0048] As an embodiment that can be added, N pictures are randomly selected from the plurality of groups of training data, wherein the number of N is 5% to 10% of the number of groups of training data; an average size of the N pictures is obtained, wherein the average size is represented by a pixel value; training data corresponding to pictures with an average pixel value are taken as the second part, and training data corresponding to the remaining pictures are taken as the first part.

[0049] In another embodiment, a first number of training data of the first part is obtained; a second number of training data of the second part is obtained, and it is determined whether the first number and the second number satisfy a predetermined condition, wherein the predetermined condition is pre-set; if the predetermined condition is not satisfied, N pictures are reselected and the first part and the second part are re-divided according to the average pixel value of the N pictures until the first number and the second number satisfy the predetermined condition.

[0050] In this step, obtaining the plurality of groups of training data includes: obtaining all training data, reserving a part of the training data, and taking other training data in the all training data except the part of the training data as the plurality of groups of training data; after the first machine learning model is incrementally trained using the third training data, the method further includes: verifying the first machine learning model after incrementally training using the part of the training data.

[0051] In step S104, supervised machine learning training is performed using all groups of training data in the first part to obtain a first machine learning model.

[0052] In step S106, pictures in each group in the first part are split into multiple blocks, and each block is input into the first machine learning model to obtain an output label.

[0053] As an embodiment that can be added, a first time consumed by supervised machine learning training using all groups of training data in the first part to obtain a first machine learning model is obtained; a second time required for continuing incrementally training using training data in the second part is estimated according to the first time, the number of training data in the first part, and the number of training data in the second part; a gap between the second time and an expected training time is determined, and the number of blocks into which the pictures are split is determined according to the gap, wherein the larger the gap is, the more the number of split blocks is.

[0054] If the label output by the first machine learning model for a block is the same as the label corresponding to the block, a second set of training data is generated, wherein the second set of training data includes a picture and one or two blocks in the picture, and the label output by the first machine learning model for the one or two blocks in the picture is the same as the label corresponding to the picture.

[0055] In step S110, the second machine learning model is obtained by training using the second set of training data. The training data of all groups in the second part are input into the second machine learning model, and the picture and the block corresponding to the picture in each group in the second part are obtained. The block corresponding to each picture in the second part is associated with the label corresponding to the block, and the third set of training data is generated.

[0056] In step S112, the first machine learning model is incrementally trained using the third set of training data.

[0057] As an optional implementation, for example, the step of obtaining the first machine learning model by supervised machine learning training using the training data of all groups in the first part can include: performing sharpness processing on the picture in each group of training data, wherein the sharpness processing is used to reduce the sharpness of the picture; establishing a corresponding association between the picture with reduced sharpness and the original picture and saving the association; and using the picture with reduced sharpness and the label corresponding to the picture as training data for supervised machine learning training.

[0058] As an additional implementation, reducing the sharpness of the picture includes: obtaining the resolution of the picture, wherein the resolution is the product of the long-side pixel value and the wide-side pixel value; finding a plurality of common divisors of the long-side pixel value and the wide-side pixel value, and taking the middle value of the plurality of common divisors; obtaining the ratio M1:M2 of the long-side pixel value and the wide-side pixel value, multiplying M1 by the middle value to obtain the length, multiplying M2 by the middle value to obtain the width, and averaging the pixel values involved in the area of the length multiplied by the width to obtain one pixel point, thereby completing the reduction of the sharpness of the picture.

[0059] Optionally, the sharpness of the picture is evaluated, and the sharpness of the picture is reduced if the sharpness of the picture is greater than a threshold. Reducing the sharpness of the picture can further include the following steps: obtaining an original picture; performing bilateral filtering on the original picture to filter out impulse noise, to keep edges, to reduce fluctuations in the gray scale values of adjacent pixels, and to obtain a picture gray scale graph; performing gradient-based edge detection on the gray scale graph to obtain a gradient graph; obtaining a hidden gradient function according to a metric pixel surrounding gray scale change matrix, and analyzing the continuous change of the surrounding gray scale of each pixel in the gradient graph based on the hidden gradient function; taking a global average of the analysis result as an evaluation value of the sharpness of the picture; and normalizing the evaluation value to obtain a picture sharpness score. Optionally, the original identification document picture is processed by a bilateral filter to keep edges and reduce noise and smoothness, and the edge-keeping filtering can filter out impulse noise, thereby reducing large fluctuations in the gray scale values of adjacent pixels. The bilateral filtering uses a weighted average method to process the brightness values of the surrounding pixels of a pixel in the original picture based on a Gaussian distribution, and uses the weighted average value as the intensity of the pixel. The calculation weight of the weighted average processing includes the Euclidean distance and the radiation difference in the pixel range.

[0060] In another optional manner, the incrementally training the first machine learning model using the third training data includes: replacing the blocks in the third training data with blocks in the original picture without reduced sharpness according to the association relationship; and incrementally training the first machine learning model using the third training data in which the blocks are replaced with the blocks in the original picture.

[0061] The above steps solve the problem of a large reduction in the accuracy of a machine learning model caused by reducing the amount of training data to improve the training efficiency of the machine learning model in the related art, thereby improving the training efficiency of the machine learning model while ensuring a certain degree of accuracy of the machine learning model.

[0062] In this embodiment, an electronic device is provided, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the method in the above embodiments.

[0063] The aforementioned program can run on a processor or be stored in memory (or a computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0064] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented using different modules, and different steps can be implemented using different modules.

[0065] The embodiment provides the device or system. The system is a computing resource optimization processing system of an image processing algorithm, and the system comprises a data segmentation module, a first training module, a splitting module, a first generation module, a second generation module and a second training module.

[0066] The system or the device is used for realizing the functions of the method in the above-mentioned embodiment, each module in the system or the device corresponds to each step in the method, and the description has been made in the method, and thus no further description is made here.

[0067] Optionally, the first training module is used for performing definition processing on the picture in each group of training data, wherein the definition processing is used for reducing the definition of the picture; the picture with reduced definition and the original picture are associated and saved; and the picture with reduced definition and the corresponding label thereof are used as training data for supervised machine learning training.

[0068] Optionally, the first training module is used for replacing the block in the third training data with a block in the original picture without reduced definition according to the association relationship; and the first machine learning model is incrementally trained by using the third training data with the block replaced with the block in the original picture.

[0069] Optionally, the data segmentation module is configured to: obtain all training data, reserve a part of the training data, and use the training data other than the reserved part of the training data as the plurality of groups of training data; and the system further comprises a verification module configured to verify the first machine learning model after the incremental training of the first machine learning model using the third training data and the reserved part of the training data.

[0070] The system or device is used to implement the functions of the method in the above-mentioned embodiments, each module in the system or device corresponds to each step in the method, and has been described in the method and will not be repeated here.

[0071] The above embodiments solve the problem of the reduction of the accuracy of the machine learning model caused by the reduction of the amount of training data to improve the training efficiency of the machine learning model in the related art, thereby improving the training efficiency of the machine learning model while ensuring a certain degree of accuracy of the machine learning model.

[0072] The above is only an embodiment of the present application and is not intended to limit the present application. Those skilled in the art can make various changes and modifications to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for optimizing the computing resources of an image processing algorithm, characterized in that, The method comprises the following steps: obtaining a plurality of sets of training data, and dividing the training data into a first part and a second part, wherein the first part and the second part each include at least two sets of training data; each set of training data includes an image and a label indicating the name of an entity existing in the image, and the plurality of sets of training data are used for supervised machine learning model training; using all sets of training data in the first part to perform supervised machine learning training to obtain a first machine learning model; splitting the image in each set in the first part into a plurality of blocks, and inputting each block into the first machine learning model to obtain an output label; if a block outputs the same label in the first machine learning model as the label corresponding to the block, a set of second training data is generated, wherein the second training data includes an image and one or two blocks in the image, and the label output by the one or two blocks in the image in the first machine learning model is the same as the label corresponding to the image; using the second training data to obtain a second machine learning model, inputting all sets of training data in the second part into the second machine learning model to obtain the image in each set in the second part and the block corresponding to the image, and establishing a correspondence between the block corresponding to each image in the second part and the label corresponding to the block to generate third training data; using the third training data to perform incremental training on the first machine learning model.

2. The method of claim 1, wherein, The method comprises the following steps: performing clarity processing on the image in each set of training data, wherein the clarity processing is used to reduce the clarity of the image; establishing a corresponding association between the image with reduced clarity and the original image and saving the association; using the image with reduced clarity and the label corresponding to the image as training data to perform supervised machine learning training.

3. The method of claim 2, wherein: obtaining the plurality of sets of training data comprises: obtaining all training data, reserving a part of the training data, and using the training data other than the reserved part of the training data as the plurality of sets of training data; after using the third training data to perform incremental training on the first machine learning model, the method further comprises: using the first machine learning model after incremental training on the reserved part of the training data to perform verification.

4. A system for optimizing computing resources of an image processing algorithm, characterized in that, The method comprises the following steps: a data segmentation module is configured to obtain a plurality of sets of training data, and divide the training data into a first part and a second part, wherein the first part and the second part each include at least two sets of training data; each set of training data includes an image and a label indicating the name of an entity existing in the image, and the plurality of sets of training data are used for supervised machine learning model training; a first training module is configured to use all sets of training data in the first part to perform supervised machine learning training to obtain a first machine learning model; a second training module is configured to split the image in each set in the first part into a plurality of blocks, input each block into the first machine learning model, and obtain an output label; The splitting module is configured to split the pictures in each group in the first part into multiple blocks, and input each block into the first machine learning model to obtain an output label; The first generating module is configured to generate a second training data if a label output by the first machine learning model for a block is the same as a label corresponding to the block, wherein the second training data comprises a picture and one or two blocks in the picture, and the label output by the first machine learning model for the one or two blocks in the picture is the same as the label corresponding to the picture; The second generating module is configured to train a second machine learning model using the second training data, input training data of all groups in the second part into the second machine learning model, and obtain the picture in each group in the second part and a block corresponding to the picture; and establish a correspondence between the block corresponding to the picture in the second part and the label corresponding to the block, and generate third training data; The second training module is configured to perform incremental training on the first machine learning model using the third training data.

5. The system of claim 4, wherein, The first training module is configured to: perform sharpness processing on the picture in the training data of each group, wherein the sharpness processing is configured to reduce the sharpness of the picture; establish a corresponding association between the picture with reduced sharpness and the original picture and save the association; use the picture with reduced sharpness and the label corresponding to the picture as training data to perform supervised machine learning training.

6. The system of claim 5, wherein: The data segmentation module is configured to: obtain all training data, reserve a part of the training data, and use the training data other than the reserved part of the training data as the multiple groups of training data. The system further comprises a verification module configured to verify the first machine learning model after incremental training using the third training data and the reserved part of the training data.

7. An electronic device comprising a memory and a processor; wherein, The memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps of any one of claims 1 to 3.

8. A readable storage medium having stored thereon computer instructions, wherein, The computer instructions are executed by the processor to implement the method steps of any one of claims 1 to 3.

Images