Computing resource optimization processing method and system of image processing algorithm

Abstract

The invention discloses a computing resource optimization processing method and system for an image processing algorithm. The method comprises the following steps: dividing training data into a first part and a second part; performing supervised machine learning training by using all groups of training data in the first part to obtain a first machine learning model; splitting the pictures in each group in the first part into a plurality of blocks, and inputting each block into a first machine learning model to obtain an output label; generating a group of second training data; inputting all groups of training data of the second part into a second machine learning model to obtain pictures of each group in the second part and blocks corresponding to the pictures, and generating third training data; and performing incremental training on the first machine learning model by using the third training data. According to the invention, the problem that the accuracy of the machine learning model is greatly reduced is solved, so that the training efficiency of the machine learning model is improved, and the accuracy of the machine learning model is ensured to a certain extent.

Description

Technical Field

[0001] The present application relates to the field of images, and more specifically, to a method and system for optimizing computing resources of an image processing algorithm. Background Art

[0002] Supervised learning, also known as supervised machine learning, is a subcategory of machine learning and artificial intelligence. It is defined as the use of labeled datasets to train an algorithm in order to classify data or accurately predict outcomes. As input data is fed into the model, it adjusts its weights until the model is appropriately fit as part of the cross-validation process. Supervised learning can help organizations solve a variety of real-world problems at scale, such as sorting spam emails into separate folders in an inbox.

[0003] Supervised learning uses a training set to teach the model to produce the desired output. This training data set includes the input and the correct output, which allows the model to learn over time. The algorithm measures its accuracy through a loss function and adjusts until the error is sufficiently minimized. Supervised learning can be divided into two types of problems when it comes to data mining - classification and regression: Classification uses an algorithm to accurately assign test data to specific categories. It identifies specific entities in the data set and tries to draw some conclusions about how to label or define these entities. Common classification algorithms are linear classifiers, support vector machines (SVM), decision trees, k-nearest neighbors, and random forests. Regression is used to understand the relationship between dependent and independent variables. It is often used for prediction, such as predicting the sales revenue of a given business. Linear regression, logistic regression, and polynomial regression are commonly used regression algorithms.

[0004] Supervised learning usually uses training data with labels annotated by experts to learn a function mapping from input variable X to input variable Y. Y = f(X), and the training data is usually in the form of (n×x, y), where n represents the size of the training sample, and x and y are the sample values ​​of variables X and Y respectively.

[0005] In image recognition, supervised training methods are generally used. The training data used for image recognition basically includes a picture and a label that identifies the name or nature of the subject in the picture. These labels are usually annotated manually. For example, in the training of a machine learning model to identify good and bad tomatoes, each set of training data includes a picture of a good tomato or a bad tomato, and a label to identify whether the tomato is good or bad. For another example, in the training of a machine learning model to distinguish between cats and dogs, each set of training labels includes a picture of a cat or a dog, and a label to identify whether the animal is a cat or a dog.

[0006] For supervised machine learning training, the larger the amount of training data, the higher the accuracy of the trained model. However, the larger the amount of training data, the longer the training time, especially when the computing resources used for training are insufficient. In order to reduce training time and improve training efficiency, it is usually adopted to reduce the training data, but this will affect the accuracy of the machine learning model at the end of training. Summary of the invention

[0007] The embodiments of the present application provide a method and system for optimizing computing resources of an image processing algorithm, so as to at least solve the problem in the related art that the accuracy of the machine learning model is greatly reduced due to the reduction of the amount of training data to improve the training efficiency of the machine learning model.

[0008] According to one aspect of the present application, a method for optimizing computing resources of an image processing algorithm is provided, comprising: obtaining multiple sets of training data, and dividing the training data into a first part and a second part, wherein the first part and the second part each include at least two sets of training data; each set of training data includes a picture and a label indicating the name of an entity object present in the picture, and the multiple sets of training data are used to train a supervised machine learning model; using the training data of all groups in the first part to perform supervised machine learning training to obtain a first machine learning model; splitting the pictures in each group in the first part into multiple blocks, inputting each block into the first machine learning model to obtain an output label; if a block If the label output in the first machine learning model is the same as the label corresponding to the block, a set of second training data is generated, wherein the second training data includes a picture and one or two blocks in the picture, and the label output in the first machine learning model for the one or two blocks in the picture is the same as the label corresponding to the picture; the second machine learning model is obtained by training using the second training data, and the training data of all groups in the second part are input into the second machine learning model to obtain the picture of each group in the second part and the block corresponding to the picture; the block corresponding to each picture in the second part is linked to the label corresponding to the block to generate third training data; and the first machine learning model is incrementally trained using the third training data.

[0009] Furthermore, using the training data of all groups in the first part to perform supervised machine learning training to obtain a first machine learning model includes: performing clarity processing on the images in each group of training data, wherein the clarity processing is used to reduce the clarity of the images; establishing a corresponding association relationship between the images with reduced clarity and the original images and saving them; and using the images with reduced clarity and their corresponding labels as training data for supervised machine learning training.

[0010] Furthermore, using the third training data to perform incremental training on the first machine learning model includes: replacing blocks in the third training data with blocks in the original image without reducing clarity according to the association relationship; and using the third training data replaced with blocks in the original image to perform incremental training on the first machine learning model.

[0011] Furthermore, obtaining the multiple groups of training data includes: obtaining all the training data, reserving a part of the training data from all the training data, and using the other training data from all the training data except the reserved part of the training data as the multiple groups of training data; after using the third training data to perform incremental training on the first machine learning model, the method also includes: verifying the first machine learning model after the incremental training using the reserved part of the training data.

[0012] According to another aspect of the present application, a computing resource optimization processing system for an image processing algorithm is also provided, including: a data segmentation module, used to obtain multiple groups of training data, and divide the training data into a first part and a second part, wherein the first part and the second part each include at least two groups of training data; each group of training data includes a picture and a label for indicating the name of an entity object present in the picture, and the multiple groups of training data are used to train a supervised machine learning model; a first training module, used to use the training data of all groups in the first part to perform supervised machine learning training to obtain a first machine learning model; a splitting module, used to split the pictures in each group in the first part into multiple blocks, and input each block into the first machine learning model to obtain an output label; a first generation module A block is used to generate a set of second training data if the label output by a block in the first machine learning model is the same as the label corresponding to the block, wherein the second training data includes a picture and one or two blocks in the picture, and the label output by the one or two blocks in the picture in the first machine learning model is the same as the label corresponding to the picture; a second generation module is used to use the second training data to train to obtain a second machine learning model, input the training data of all groups of the second part into the second machine learning model, obtain the picture of each group in the second part and the block corresponding to the picture; establish a connection between the block corresponding to each picture in the second part and the label corresponding to the block to generate third training data; a second training module is used to use the third training data to perform incremental training on the first machine learning model.

[0013] Furthermore, the first training module is used to: perform clarity processing on the images in each group of training data, wherein the clarity processing is used to reduce the clarity of the images; establish a corresponding association relationship between the images with reduced clarity and the original images and save them; and use the images with reduced clarity and their corresponding labels as training data for supervised machine learning training.

[0014] Furthermore, the first training module is used to: replace the blocks in the third training data with blocks in the original picture without reducing the clarity according to the association relationship; and use the third training data replaced with blocks in the original picture to perform incremental training on the first machine learning model.

[0015] Furthermore, the data segmentation module is used to: obtain all training data, reserve a part of training data from all the training data, and use other training data from all the training data except the reserved part of training data as the multiple groups of training data; the system also includes: a verification module, which is used to verify the first machine learning model after incremental training using the reserved part of training data after incremental training of the first machine learning model using the third training data.

[0016] In an embodiment of the present application, a plurality of sets of training data are obtained and the training data are divided into a first part and a second part, wherein the first part and the second part each include at least two sets of training data; each set of training data includes a picture and a label indicating the name of a physical object present in the picture, and the plurality of sets of training data are used to train a supervised machine learning model; supervised machine learning training is performed using the training data of all the groups in the first part to obtain a first machine learning model; the pictures in each group in the first part are split into multiple blocks, and each block is input into the first machine learning model to obtain an output label; if a block is in the first machine learning model If the output label is the same as the label corresponding to the block, a set of second training data is generated, wherein the second training data includes a picture and one or two blocks in the picture, and the label outputted by the one or two blocks in the picture in the first machine learning model is the same as the label corresponding to the picture; the second machine learning model is trained using the second training data, and the training data of all groups in the second part are input into the second machine learning model to obtain the picture of each group in the second part and the block corresponding to the picture; the block corresponding to each picture in the second part is connected with the label corresponding to the block to generate third training data; the first machine learning model is incrementally trained using the third training data. This application solves the problem of a large reduction in the accuracy of the machine learning model caused by reducing the amount of training data to improve the training efficiency of the machine learning model in the related art, thereby improving the training efficiency of the machine learning model while ensuring a certain degree of accuracy of the machine learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 It is a flowchart of a method for optimizing computing resources of an image processing algorithm according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0021] In the following implementation, the main idea is to combine multiple different relatively weak machine learning models to process training data, appropriately reduce the amount of training data, and thus appropriately improve the training efficiency without reducing the amount of training data.

[0022] In the following implementation, pictures are trained to finally obtain a machine learning model (also called a neural network model). The pictures can be various pictures, that is, the following implementation can be applied to multiple scenarios. These scenarios are exemplarily described below.

[0023] Scene 1

[0024] In this scenario, the appearance inspection of the battery module can include the following steps: obtaining the appearance picture of the battery module; inputting the appearance picture into the coding layer, extracting the features of the appearance picture through the coding layer; parsing the features extracted by the coding layer through the decoding layer, and outputting the results. The coding layer includes a data preprocessing module, a multidimensional feature extraction module and at least one three-dimensional feature extraction module arranged in sequence; the main function of the data preprocessing module is to read the image input and normalize it into a vector input; the multidimensional feature extraction module is used to extract feature maps from the input vector, and the multidimensional space feature extraction module is designed as a depth-separable convolutional layer, and uses the normalization function and the ReLU function to obtain two types of feature maps from the input image. The above-mentioned feature maps are low-level feature maps, including low-level spatial features such as texture, grayscale, etc.; high-level feature maps, including deep semantic information; the above-mentioned three-dimensional feature extraction module is used to convert the above-mentioned appearance picture into a three-dimensional image, and use three-dimensional convolution to extract a three-dimensional feature map from the above-mentioned three-dimensional image, and the above-mentioned three-dimensional feature map includes three-dimensional spectral features and / or three-dimensional spatial features; the purpose of the three-dimensional feature extraction module is to obtain effective spectral and spatial features by reshaping the input image and associating far and near pixels. The three-dimensional feature extraction module converts the image into a three-dimensional image by reconstruction, wherein the input image is divided into parts, and then the third dimension is added to finally obtain a three-dimensional image. In this way, unlike the standard two-dimensional convolution, spectral and spatial features can also be obtained from the image without destroying the original image structure. The reconstructed three-dimensional image is subjected to three-dimensional convolution to obtain a four-dimensional feature matrix. The third and fourth dimensions of the four-dimensional feature matrix are combined to reconstruct the feature map.

[0025] Scene 2

[0026] In this scenario, the image method is used to detect insulator defects. First, the insulator string is identified in the captured image, and then the defect type is determined based on the computer image algorithm. Specifically, the Tophat algorithm can identify the contour information of the insulator in the complex background image; the multimodal fusion detection technology is used to realize the identification of various defects of the insulator, and the defects are reminded in real time; the YOLOv5 algorithm is improved and enhanced so that the model can obtain more object edge information and adapt to pictures of different resolutions; through transfer learning, the relevant instances or features in the increasing defect data set can be migrated to the trace data set, thereby improving the generalization ability of the model. The above-mentioned Tophat algorithm can identify the contour information of insulators in complex background images, which specifically includes the following contents: to identify defective images of insulators in substations, the insulator equipment must be accurately locked first, and then the defect situation can be further analyzed. The recognition of local contour information in complex background images is to perform some smoothing filtering operations on the image to remove noise interference and clutter interference in the complex background image, thereby improving the accurate recognition of local contour information of the complex background image; the Tophat algorithm is the difference between the original image and the "open operation" result image. The open operation magnifies the cracks or local low-brightness areas. Therefore, the effect image obtained by the Tophat algorithm highlights the areas that are brighter than the areas around the original image contour; when the insulator is in the complex background of multiple devices in the substation, its own appearance is relatively regular, and the Tophat algorithm is used to extract its contour information. The above-mentioned multimodal fusion detection technology is specifically based on the special scenes of the specific substation and the appearance characteristics of the insulator equipment. It uses deep learning computer vision technology, integrates machine learning, classic CV algorithms and other signal processing methods, and constructs a multimodal fusion detection technology (Multimodal-FusionD etection). It adopts the "addition" joint method; shared semantic subspaces are realized in different hidden layers, and the converted single-modal feature vector semantics are combined together to achieve multimodal fusion. The above improvements enhance the YOLOv5 algorithm specifically including: considering the complexity of insulator video or image data, corresponding data processing is performed on it to produce an insulator detection data set; based on the YOLOv5 model as the main body, the image pyramid structure is firstly adopted to fuse features at different levels to obtain feature maps of different scales for position and category prediction; then the kmeans unsupervised clustering algorithm is used to preprocess the image appropriately, and the data can be preliminarily analyzed to mine implicit value information, and then the target box dimension is clustered to increase the number of prior boxes (Anchorbox) so that the model can obtain more object edge information; finally, in the training process, multi-size pictures are used for training so that the model can adapt to pictures of different resolutions; thereby identifying the feature position of the insulator defect image and judging the defect category.Through transfer learning, relevant instances or features in the ever-increasing defect data set can be migrated to a small data set, thereby improving the generalization ability of the model. Specifically, in the FasterR-C NN (an algorithm) model, the feature layer learned by the VGG-16 convolutional neural network in ImageNet (a computer vision system recognition project) is migrated to the defect recognition problem with less data. The migrated feature layer covers complex and diverse feature extraction capabilities (edge ​​features, texture features, local abstract features, etc.).

[0027] Scene 3

[0028] In this scenario, escalator actions are recognized through images, and the scenario may include the following steps: obtaining the original image captured by the terminal device; preprocessing the original image, including randomly rotating the original image; randomly changing the image contrast; adding noise to the original image; randomly cutting the image, using the image pixel mean to fill other pixels, so that the image is expanded to 600x600, and finally performing a mirror flip operation on the image; using the improved MaskRCNN to detect and output the single escalator recognition area and the human key point area in the original image. The above-mentioned improved MaskRCNN includes adding the human key point detection branch to the original MaskRCN N, such an improved MaskRCNN model can not only realize the semantic segmentation of single-ladder recognition in the original image, but also detect the key points of the human body; make a joint judgment on the single-ladder recognition area and the key point area of ​​the human body, and output the escalator posture recognition result. The above joint judgment includes taking the pixel coordinates of the center of the key point area in the key point detection of the human body output by the model as the center of the circle and multiple pixel units as the radius to establish a circular area. If the pixel value of the single-ladder area after semantic segmentation falls within the area of ​​this circle, it can be determined that the staff has escalator action; if the pixel value of the single-ladder after semantic segmentation does not fall within the area of ​​this circle, it can be concluded that there is no escalator action.

[0029] The above-mentioned preprocessing of the original image is specifically as follows: randomly rotating all the original images by 0-360 degrees and changing the contrast, and adding different degrees of Gaussian noise and salt and pepper noise to the image. The above-mentioned improved MaskRCNN specifically includes: adding a branch for regressing human key point detection after the RPN network, parallel to the category classification branch, the bounding box regression branch and the mask branch. The improved MaskRCNN can realize the two functions of semantic segmentation and key point detection in one neural network. The above-mentioned human key point is the human hand. The above-mentioned joint judgment includes taking the pixel coordinates of the center of the key point area in the human key point detection output by the model as the center of the circle and 10 pixel units as the radius to establish a circular area. If the pixel value of the single ladder area after semantic segmentation falls within the area of ​​this circle, it can be determined that the staff has escalator action; if the pixel value of the single ladder after semantic segmentation does not fall within the area of ​​this circle, it can be concluded that there is no escalator action.

[0030] The algorithms involved in the above scenarios are described below.

[0031] Mask R-CNN

[0032] Mask R-CNN is an instance segmentation algorithm that can be used for "target detection", "target instance segmentation", and "target key point detection".

[0033] The difference and connection between instance segmentation and semantic segmentation: Semantic segmentation and instance segmentation are two small fields in target segmentation, both of which are used to segment the input images; Difference: Target segmentation in the usual sense refers to semantic segmentation, which has a long history of development and has made great progress. Currently, many scholars are doing research in this area; however, instance segmentation is a small field that has been separated from the field of target segmentation and has only developed in recent years. Compared with the former, the latter is more complex and there are relatively few scholars currently studying it. It is a hot field with room for research and a field that is being explored. They are given the same color in semantic segmentation, but different colors in instance segmentation. That is, instance segmentation requires a more refined segmentation of similar objects based on semantic segmentation.

[0034] Mask R-CNN is a very flexible framework that can add different branches to complete different tasks, such as object classification, object detection, semantic segmentation, instance segmentation, human posture recognition, etc. Mask R-CNN is expected to achieve the following goals: high speed, high accuracy (high classification accuracy, high detection accuracy, high instance segmentation accuracy, etc.), simple and intuitive, and easy to use.

[0035] High speed and high accuracy: The classic object detection algorithm Faster-rcnn and the classic semantic segmentation algorithm FCN can be compared. Faster-rcnn can complete the object detection function quickly and accurately; FCN can complete the semantic segmentation function accurately. Mask R-CNN is more complex than Faster-rcnn, but it can still reach a speed of 5fps, which is comparable to the speed of the original Faster-rcnn. Due to the discovery of the pixel deviation problem in ROIPooling, the corresponding ROIAlign strategy was proposed, and the precise pixel MASK of FCN enables it to achieve high accuracy.

[0036] Simple and intuitive: The idea of ​​the entire Mask R-CNN algorithm is very simple, which is to add FCN to the original Faster-rcnn algorithm to generate the corresponding MASK branch, that is, Faster-rcnn+FCN, and more detailed is RPN+ROIAlign+Fast-rcnn+FCN.

[0037] Easy to use: The entire Mask R-CNN algorithm is very flexible and can be used to complete a variety of tasks, including object classification, object detection, semantic segmentation, instance segmentation, human posture recognition, and many other tasks, which fully demonstrates its easy-to-use feature.

[0038] FasterR-C NN

[0039] With the rapid development of artificial intelligence and computer vision technology, object detection, as one of the core technologies, has been widely used in various fields, such as autonomous driving, security monitoring, medical image analysis, etc. As an important technology in the field of object detection, the Faster R-CNN algorithm has been widely recognized for its efficiency and accuracy. This article will deeply analyze the principles, processes, advantages and disadvantages, and practical applications of the Faster R-CNN algorithm, hoping to provide readers with useful references.

[0040] In the field of object detection, traditional methods are usually divided into two stages: generating candidate regions and classifying these regions. However, these methods have slow processing speeds, which limits the possibility of real-time applications. R-CNN achieves object detection by extracting candidate regions of fixed size in an image and then classifying each candidate region. Although R-CNN performs well in accuracy, its processing speed is very slow. To solve this problem, the Faster R-CNN algorithm came into being.

[0041] The Faster R-CNN algorithm mainly includes the following four steps:

[0042] Feature extraction conv layers: First, the input image is passed through a series of convolutional layers, activation functions, and pooling layers for feature extraction to obtain the corresponding feature map. The purpose of this step is to extract key information from the image and provide a basis for subsequent candidate region generation and classification. Region Proposal Networks (RPN) is the core part of the Faster R-CNN algorithm, which is responsible for generating candidate regions. RPN generates a series of candidate boxes by sliding a small window on the last layer of the feature map of the full convolutional network, and performs binary classification (foreground or background) and bounding box regression on each candidate box to obtain accurate candidate regions. This step greatly reduces the number of candidate regions and improves the running speed of the algorithm. The role of the ROIPooling layer is to map candidate regions of different sizes to feature maps of fixed size for subsequent classification and regression operations. Through ROIPooling, the algorithm can handle input images and candidate regions of any size, which enhances the robustness of the algorithm. Classification: Finally, the fixed-size feature map output by the ROIPooling layer is input into the fully connected layer for classification to obtain the category to which each candidate region belongs. At the same time, the position of the candidate area can be fine-tuned through bounding box regression to further improve the accuracy of object detection.

[0043] Advantages and disadvantages of Faster R-CNN: Efficiency: By introducing RPN, the algorithm can quickly generate accurate candidate regions, reduce the amount of calculation, and improve the running speed. Accuracy: Through ROIPooling and classification and regression of the fully connected layer, the algorithm can accurately identify and locate the target object. However, the Faster R-CNN algorithm also has some disadvantages: Poor detection effect on small objects: Since the size of the candidate box generated by RPN is fixed, it may not be able to generate enough candidate boxes for objects of smaller size, resulting in poor detection effect. Limited processing ability for oversized objects: When the size of the target object is too large, RPN may not be able to generate enough candidate boxes to cover the entire object, thus affecting the detection effect. The Faster R-CNN algorithm performs well in practical applications and is widely used in fields such as autonomous driving, security monitoring, and medical image analysis. For example, in the field of autonomous driving, Faster R-CNN can be used to identify obstacles such as pedestrians and vehicles on the road, providing important perception information for the autonomous driving system. In the field of security monitoring, Faster R-CNN can be used to detect abnormal events such as intruders and fires, and improve the intelligence level of the monitoring system. In the field of medical image analysis, Faster R-CNN can be used to assist doctors in diagnosing diseases such as lung nodules and tumors.

[0044] When conducting machine learning training, you can use the following server architecture. Of course, you can also use other servers, or use a single service.

[0045] The server architecture can be constructed using the following steps: S1. Based on the physical length of the link and the link bandwidth, the working nodes geographically dispersed in the wide area network topology are divided into different clusters; step S1 includes the following sub-steps: S11. Based on the physical length of the link and the link bandwidth, the weight of each link geographically dispersed in the wide area network topology is calculated; S12. Based on the weight set constructed based on the weight of each link, the shortest path between any two working nodes is calculated; S13. Each working node is initialized as a cluster; S14. Based on the shortest path between any two working nodes, the two closest clusters are merged into one cluster; S15. Repeat step S14 until the distance between the two closest clusters is greater than the set threshold, and the cluster division is completed; S2. Determine whether the current number of clusters is 1. If so, jump to step S3, if not, jump to step S4; S3. Place local parameter servers and global parameter servers for the unique cluster; step S3 includes the following sub-steps: S31. Traverse the unique cluster. For each working node in a cluster, calculate the average distance between each working node and the remaining working nodes; S32, select the working node with the smallest average distance in the unique cluster as the local parameter server and the global parameter server of the unique cluster, and realize the placement of the local parameter server and the global parameter server of the unique cluster; S4, place the local parameter server and the global parameter server for all clusters; step S4 includes the following sub-steps: S41, traverse each working node in all clusters, calculate the average distance between each working node and the remaining working nodes in each cluster, and select the working node with the smallest average distance in each cluster as the local parameter server of the cluster; S42, traverse each working node in the wide area network topology, calculate the average distance between each working node and all local parameter servers; S43, select the working node with the smallest average distance in the wide area network topology as the global parameter server of the wide area network topology, and remove the working node from the cluster where it is located, so as to realize the placement of the local parameter server and the global parameter server of all clusters.

[0046] In order to solve the problems in the related art, in the following implementation, a method for optimizing computing resources of an image processing algorithm is provided. Figure 1 is a flowchart of a method for optimizing computing resources of an image processing algorithm according to an embodiment of the present application. Figure 1 The steps involved in the method are described.

[0047] Step S102: Acquire multiple groups of training data, and divide the training data into a first part and a second part, wherein the first part and the second part each include at least two groups of training data; each group of training data includes a picture and a label for indicating the name of the physical object existing in the picture, and the multiple groups of training data are used to train a supervised machine learning model.

[0048] As an implementation method that can be added, N pictures are randomly selected from the multiple groups of training data, wherein the number N is 5% to 10% of the number of groups of the training data; the average size of the N pictures is obtained, wherein the average size is represented by a pixel value; the training data corresponding to the pictures whose pixel values ​​are greater than the average are taken as the second part, and the training data corresponding to the remaining pictures are taken as the first part.

[0049] In another embodiment, a first amount of training data of the first part is obtained; a second amount of training data of the second part is obtained, and it is determined whether the first amount and the second amount satisfy a predetermined condition, wherein the predetermined condition is pre-set; if the predetermined condition is not satisfied, N pictures are reselected and the first part and the second part are re-divided according to the average pixel value of the N pictures until the first amount and the second amount satisfy the predetermined condition.

[0050] In this step, obtaining the multiple groups of training data includes: obtaining all the training data, reserving a part of the training data from all the training data, and using the other training data from all the training data except the reserved part of the training data as the multiple groups of training data; after using the third training data to perform incremental training on the first machine learning model, the method also includes: verifying the first machine learning model after the incremental training using the reserved part of the training data.

[0051] Step S104: Use the training data of all groups in the first part to perform supervised machine learning training to obtain a first machine learning model.

[0052] Step S106: split the pictures in each group in the first part into multiple blocks, and input each block into the first machine learning model to obtain output labels.

[0053] As an implementation method that can be added, a first time consumed by using the training data of all groups in the first part to perform supervised machine learning training to obtain a first machine learning model is obtained; a second time required to continue using the training data of the second part for incremental training is estimated based on the first time, the amount of training data in the first part, and the amount of training data in the second part; a difference between the second time and the expected training time is determined, and the number of blocks into which the image is split is determined based on the difference, wherein the larger the difference, the greater the number of blocks into which the image is split.

[0054] Step S108: If the label output by a block in the first machine learning model is the same as the label corresponding to the block, a set of second training data is generated, wherein the second training data includes a picture and one or two blocks in the picture, and the label output by the one or two blocks in the picture in the first machine learning model is the same as the label corresponding to the picture.

[0055] Step S110: Use the second training data to train to obtain a second machine learning model, input the training data of all groups in the second part into the second machine learning model, and obtain the picture of each group in the second part and the block corresponding to the picture; establish a connection between the block corresponding to each picture in the second part and the label corresponding to the block to generate third training data.

[0056] Step S112: Use the third training data to perform incremental training on the first machine learning model.

[0057] As an optional implementation, for example, the step of using the training data of all groups in the first part to perform supervised machine learning training to obtain a first machine learning model may include: performing clarity processing on the images in each group of training data, wherein the clarity processing is used to reduce the clarity of the images; establishing a corresponding association relationship between the images with reduced clarity and the original images and saving them; and using the images with reduced clarity and their corresponding labels as training data for supervised machine learning training.

[0058] As an implementation method that can be added, reducing the clarity of the image includes: obtaining the resolution of the image, wherein the resolution is the product of the long side pixel value and the wide side pixel value; finding multiple common divisors of the long side pixel value and the wide side pixel value, and taking the middle value of the multiple common divisors; obtaining the ratio M1:M2 of the long side pixel value and the wide side pixel value, multiplying M1 by the middle value as the length, multiplying M2 by the middle value as the width, taking the average of the pixel values ​​involved in the area of ​​the length multiplied by the width to convert it into a pixel point, thereby completing the reduction of the clarity of the image.

[0059] Optionally, the clarity of the image is evaluated, and when the clarity of the image is greater than a threshold, the clarity of the image is reduced. Reducing the clarity of the image may also include the following steps: obtaining the original image; performing bilateral filtering on the original image to filter out impulse noise, achieve edge preservation, reduce the fluctuation of the grayscale values ​​of adjacent pixels, and obtain a grayscale image of the image; performing gradient-based edge detection on the grayscale image to obtain a gradient image; obtaining an implicit gradient function based on a matrix of grayscale changes around a measured pixel, and analyzing the continuous changes of grayscale around each pixel in the gradient image based on the implicit gradient function; taking the global average of the analysis results as an evaluation value of the image clarity; and normalizing the evaluation value to obtain an image clarity score. Optionally, a bilateral filter is used to perform bilateral filtering on the original certificate image. The edge-preserving filter can filter out impulse noise, thereby reducing large fluctuations in the grayscale values ​​of adjacent pixels, and achieving the effect of maintaining edges and reducing noise and smoothing. The bilateral filtering process uses a weighted averaging method, and uses a weighted averaging method based on Gaussian distribution to process the brightness values ​​of pixels surrounding the pixel point in the original image, and uses the weighted average value as the intensity of the pixel point. The calculation weights of the weighted averaging process include the Euclidean distance and the radiation difference in the pixel range domain.

[0060] In another optional embodiment, using the third training data to perform incremental training on the first machine learning model includes: replacing blocks in the third training data with blocks in the original image without reducing clarity according to the association relationship; and using the third training data replaced with blocks in the original image to perform incremental training on the first machine learning model.

[0061] The above steps solve the problem in related technologies that the accuracy of the machine learning model is greatly reduced due to reducing the amount of training data to improve the training efficiency of the machine learning model, thereby improving the training efficiency of the machine learning model while ensuring a certain degree of accuracy of the machine learning model.

[0062] In this embodiment, an electronic device is provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to execute the method in the above embodiment.

[0063] The above program can be run in the processor, or it can also be stored in the memory (or computer-readable medium), which includes permanent and non-permanent, removable and non-removable media. Information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0064] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more blocks can be implemented by different modules corresponding to different steps.

[0065] Such a device or system is provided in this embodiment. The system is called a computing resource optimization processing system for an image processing algorithm, and the system includes: a data segmentation module, used to obtain multiple groups of training data, and divide the training data into a first part and a second part, wherein the first part and the second part each include at least two groups of training data; each group of training data includes a picture and a label indicating the name of the physical object present in the picture, and the multiple groups of training data are used to train a supervised machine learning model; a first training module, used to use the training data of all groups in the first part to perform supervised machine learning training to obtain a first machine learning model; a splitting module, used to split the pictures in each group in the first part into multiple blocks, and input each block into the first machine learning model to obtain an output label; a first generation module, used to If the label output by a block in the first machine learning model is the same as the label corresponding to the block, a set of second training data is generated, wherein the second training data includes a picture and one or two blocks in the picture, and the label output by the one or two blocks in the picture in the first machine learning model is the same as the label corresponding to the picture; a second generation module is used to use the second training data for training to obtain a second machine learning model, input the training data of all groups in the second part into the second machine learning model, and obtain the picture of each group in the second part and the block corresponding to the picture; establish a connection between the block corresponding to each picture in the second part and the label corresponding to the block to generate third training data; a second training module is used to use the third training data to perform incremental training on the first machine learning model.

[0066] The system or device is used to implement the functions of the method in the above-mentioned embodiment. Each module in the system or device corresponds to each step in the method, which has been explained in the method and will not be repeated here.

[0067] Optionally, the first training module is used to: perform clarity processing on the images in each group of training data, wherein the clarity processing is used to reduce the clarity of the images; establish a corresponding association relationship between the images with reduced clarity and the original images and save them; and use the images with reduced clarity and their corresponding labels as training data for supervised machine learning training.

[0068] Optionally, the first training module is used to: replace the blocks in the third training data with blocks in the original image without reducing clarity according to the association relationship; and perform incremental training on the first machine learning model using the third training data replaced with blocks in the original image.

[0069] Optionally, the data segmentation module is used to: obtain all training data, reserve a part of training data from all the training data, and use other training data from all the training data except the reserved part of training data as the multiple groups of training data; the system also includes: a verification module, used to verify the first machine learning model after incremental training using the reserved part of training data after incremental training of the first machine learning model using the third training data.

[0070] The system or device is used to implement the functions of the method in the above-mentioned embodiment. Each module in the system or device corresponds to each step in the method, which has been explained in the method and will not be repeated here.

[0071] The above implementation manner solves the problem in the related art that the accuracy of the machine learning model is greatly reduced due to the reduction in the amount of training data to improve the training efficiency of the machine learning model, thereby improving the training efficiency of the machine learning model while ensuring a certain degree of accuracy of the machine learning model.

[0072] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for optimizing computing resources of an image processing algorithm, characterized in that: include: Acquire multiple sets of training data, and divide the training data into a first part and a second part, wherein the first part and the second part each include at least two sets of training data; each set of training data includes a picture and a label indicating the name of a physical object in the picture, and the multiple sets of training data are used to train a supervised machine learning model; Perform supervised machine learning training using the training data of all groups in the first part to obtain a first machine learning model; Splitting the images in each group in the first part into multiple blocks, and inputting each block into the first machine learning model to obtain output labels; If a label output by a block in the first machine learning model is the same as a label corresponding to the block, generating a set of second training data, wherein the second training data includes a picture and one or two blocks in the picture, and the label output by the one or two blocks in the picture in the first machine learning model is the same as the label corresponding to the picture; Using the second training data to perform training to obtain a second machine learning model, inputting the training data of all groups in the second part into the second machine learning model to obtain a picture of each group in the second part and a block corresponding to the picture; establishing a connection between a block corresponding to each picture in the second part and a label corresponding to the block to generate third training data; Incrementally train the first machine learning model using the third training data.

2. The method according to claim 1, characterized in that: The first machine learning model obtained by performing supervised machine learning training using the training data of all the groups in the first part includes: Performing definition processing on the pictures in each group of training data, wherein the definition processing is used to reduce the definition of the pictures; Establish a corresponding association relationship between the image with reduced definition and the original image and save them; The images with reduced definition and their corresponding labels are used as training data for supervised machine learning training.

3. The method according to claim 2, characterized in that Incrementally training the first machine learning model using the third training data includes: According to the association relationship, the blocks in the third training data are replaced with blocks in the original image whose clarity is not reduced; and the first machine learning model is incrementally trained using the third training data replaced with blocks in the original image.

4. The method according to any one of claims 1 to 3, characterized in that Acquiring the multiple sets of training data includes: acquiring all the training data, reserving a portion of training data from all the training data, and using the other training data from all the training data except the reserved portion of training data as the multiple sets of training data; After incrementally training the first machine learning model using the third training data, the method further includes: verifying the first machine learning model after incremental training using the reserved part of the training data.

5. A computing resource optimization processing system for an image processing algorithm, characterized in that: include: A data segmentation module, used to obtain multiple sets of training data and divide the training data into a first part and a second part, wherein the first part and the second part each include at least two sets of training data; each set of training data includes a picture and a label indicating the name of an entity in the picture, and the multiple sets of training data are used to train a supervised machine learning model; A first training module, configured to perform supervised machine learning training using the training data of all the groups in the first part to obtain a first machine learning model; A splitting module, used for splitting the pictures in each group in the first part into multiple blocks, and inputting each block into the first machine learning model to obtain an output label; A first generating module is configured to generate a set of second training data if a label output by a block in the first machine learning model is the same as a label corresponding to the block, wherein the second training data includes a picture and one or two blocks in the picture, and a label output by the one or two blocks in the picture in the first machine learning model is the same as a label corresponding to the picture; a second generating module, configured to obtain a second machine learning model by training using the second training data, input the training data of all groups in the second part into the second machine learning model, obtain a picture of each group in the second part and a block corresponding to the picture; establish a connection between a block corresponding to each picture in the second part and a label corresponding to the block, and generate third training data; A second training module is used to perform incremental training on the first machine learning model using the third training data.

6. The system according to claim 5, characterized in that The first training module is used to: Performing definition processing on the pictures in each group of training data, wherein the definition processing is used to reduce the definition of the pictures; Establish a corresponding association relationship between the image with reduced definition and the original image and save them; The images with reduced definition and their corresponding labels are used as training data for supervised machine learning training.

7. The system according to claim 6, characterized in that The first training module is used to: According to the association relationship, the blocks in the third training data are replaced with blocks in the original image whose clarity is not reduced; and the first machine learning model is incrementally trained using the third training data replaced with blocks in the original image.

8. The system according to any one of claims 5 to 7, characterized in that The data segmentation module is used to: obtain all training data, reserve part of the training data from all the training data, and use the other training data from all the training data except the reserved part of the training data as the multiple groups of training data; The system also includes: a verification module, which is used to verify the first machine learning model after incremental training using the reserved part of training data after incremental training of the first machine learning model using the third training data.

9. An electronic device comprising a memory and a processor; wherein: The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps described in any one of claims 1 to 4.

10. A readable storage medium having computer instructions stored thereon, wherein: When the computer instructions are executed by a processor, the method steps described in any one of claims 1 to 4 are implemented.

Images