Object size measurement method and system under complex background based on deep learning

Abstract

The invention provides a method and a system for measuring the size of an object under a complex background based on deep learning. The method comprises the following steps: training to obtain an RCF model under a current scene; shooting a to-be-detected object by using a camera, and collecting an image of the to-be-detected object; sending the to-be-detected object image to the RCF model in the current scene to obtain a coarse contour edge image of the to-be-detected object; processing the coarse contour edge image to obtain sub-pixel coordinate points of the to-be-detected object, and obtaining the image length and the image width of the to-be-detected object according to the sub-pixel coordinate points; and calculating the physical length and width of the to-be-detected object according to the calibration parameters of the camera, the image length and the image width. According to the method, the self-learning ability of the deep neural network is utilized, interference of a complex background is removed, only the edge image of the object to be detected is extracted, the object has sub-pixel precision through binary polynomial interpolation, the measurement precision is improved, and the adaptability of the measurement method is enhanced.

Description

Technical Field

[0001] The present application relates to the technical field of image recognition and measurement of object size under complex backgrounds, and in particular to a method and system for measuring object size under complex backgrounds based on deep learning. Background Art

[0002] During the object processing, it is necessary to know the corresponding object size in each process. For example, when producing thick plates, it is necessary to roll steel according to the billet size to ensure that the size of the produced object meets the standard. The traditional measurement method is to use measuring tools such as vernier calipers to manually measure the object to be inspected. However, this contact-based manual measurement method has low measurement efficiency and cannot contact the object in some scenarios.

[0003] With the development of artificial intelligence and machine vision technology, non-contact measurement methods have emerged. By taking a visual image of the object to be detected, the size of the object to be detected is measured using image processing and artificial intelligence technology. The image of the object to be detected is collected using an industrial camera. First, the edge contour of the object to be detected is extracted, then the coordinate parameters of the edge contour are calculated, and finally the physical size of the object to be detected is calculated in combination with the camera's calibration data.

[0004] The current method of measuring object size in complex backgrounds based on deep learning is highly dependent on the background information of the object. If the background of the object is too complex, it will affect the extraction of the object's edge information, and the background information will also be extracted. Therefore, a method is needed to automatically ignore the background information of non-target objects through autonomous training and only retain the edge contour of the target object to avoid the extraction of background information. Summary of the invention

[0005] The present application aims to solve the problem that when extracting the edge information of an object, the background information is also extracted in the existing method for measuring the size of an object under a complex background based on deep learning. On the one hand, a method for measuring the size of an object under a complex background based on deep learning is provided, comprising the following steps:

[0006] Training the RCF model to obtain the RCF model in the current scenario;

[0007] Setting camera calibration parameters, using the camera to photograph the object to be detected, and acquiring an image of the object to be detected;

[0008] Sending the image of the object to be detected into the RCF model in the current scene to obtain a rough contour edge map of the object to be detected;

[0009] Processing the rough contour edge map to obtain sub-pixel coordinate points of the object to be detected, and obtaining the image length and image width of the object to be detected according to the sub-pixel coordinate points;

[0010] The physical length and width of the object to be detected are calculated according to the calibration parameters of the camera and the image length and image width.

[0011] In a feasible implementation, the step of training the RCF model to obtain the RCF model in the current scenario includes:

[0012] Photographing the sample object and collecting the sample image;

[0013] Performing data enhancement processing on the sample image, and labeling the sample image after the data enhancement processing by using labeling software label to obtain a labeled sample;

[0014] The labeled samples are input into the RCF model for training to obtain the RCF model in the current scenario.

[0015] In a feasible implementation, the method of performing data enhancement processing on the sample image includes: a mosaic enhancement method, a hybrid enhancement method, and a random horizontal flipping method.

[0016] In a feasible implementation, the step of training the RCF model by annotating samples to obtain the RCF model in the current scenario includes:

[0017] The labeled samples are divided into a training set, a validation set, and a test set in a ratio of 3:1:1;

[0018] Sending the training set and the validation set into the RCF model for training to obtain the RCF model after training;

[0019] The generalization ability of the trained RCF model is evaluated through a test set, and the RCF model in the current scenario is obtained after the evaluation is qualified.

[0020] In a feasible implementation, when the training set and the validation set are fed into the RCF model for training, the RCF model training setting parameters are: 200 training rounds, learning rate: 1e -6 , learning rate step size: 3, learning rate decay: 0.6.

[0021] In a feasible implementation, the step of processing the coarse contour edge map to obtain corresponding sub-pixel coordinate points includes:

[0022] The rough outline edge Figure 2 valuing, and eliminating abnormal contours of the coarse contour edge map according to the area size of the contours in the coarse contour edge map;

[0023] Using non-maximum suppression and double threshold processing to refine the edge contour of the object to be detected in the coarse contour edge map to obtain a fine contour edge map;

[0024] A polynomial interpolation algorithm is used for edge points of the thin contour edge map to calculate sub-pixel coordinates of the object to be detected.

[0025] In a feasible implementation, in the dual threshold processing process, the maximum threshold is set to 0.25 and the minimum threshold is set to 0.2.

[0026] In a feasible implementation, the step of using a polynomial interpolation algorithm on the edge points of the thin contour edge map to calculate the sub-pixel coordinates of the object to be detected includes:

[0027] Find the contour of the object to be detected in the fine contour edge map by using the findcontours function of opencv;

[0028] Find four coordinate points of the object to be detected according to the outline;

[0029] According to the four coordinate points, the corresponding sub-pixel coordinate points are calculated using a quadratic Lagrange interpolation formula.

[0030] In a feasible implementation, the step of obtaining the image length and image width of the object to be detected according to the sub-pixel coordinate points includes:

[0031] The sub-pixel coordinate point of the object to be detected in the X direction is obtained as (X min , Y m ), (X max , Y n ), obtaining the image length L of the object to be detected;

[0032]

[0033] The sub-pixel coordinate point of the object to be detected in the Y direction is obtained as (X m , Y min ), (X n , Y max ), obtain the image width W of the object to be detected;

[0034]

[0035] In the formula, X min is the minimum horizontal coordinate of the four sub-pixel coordinate points, Y m X is the ordinate corresponding to the sub-pixel coordinate point with the minimum abscissa; max is the maximum horizontal coordinate of the four sub-pixel coordinate points, Y nY is the ordinate corresponding to the sub-pixel coordinate point with the maximum abscissa; min is the minimum ordinate of the four sub-pixel coordinate points, X m Y is the horizontal coordinate corresponding to the sub-pixel coordinate point with the smallest vertical coordinate; max is the maximum ordinate of the four sub-pixel coordinate points, X n is the horizontal coordinate corresponding to the sub-pixel coordinate point with the maximum vertical coordinate; L is the image length of the object to be detected, and W is the image width of the object to be detected.

[0036] On the other hand, the present application provides a system for measuring the size of an object in a complex background based on deep learning. The system is used to implement any one of the above-mentioned methods for measuring the size of an object in a complex background based on deep learning, including: an image acquisition module, an image preprocessing module, an object size detection module and an object size calculation module; the image acquisition module, the image preprocessing module, the object size detection module and the object size calculation module are all communication-connected.

[0037] The image acquisition module is used to take a picture of the object to be detected and acquire an image of the object to be detected;

[0038] The image preprocessing module is used to perform data enhancement and annotation on the image of the object to be detected;

[0039] The to-be-detected object size detection module is used to detect the rough contour edge map of the to-be-detected object image, process the rough contour edge map, and obtain the image length and image width of the to-be-detected object;

[0040] The module for calculating the size of the object to be detected is used to calculate the physical size of the object to be detected according to the image length and image width of the object to be detected.

[0041] The present application provides a method and system for measuring the size of an object under a complex background based on deep learning. For the problem of measuring objects under a complex background, an RCF model based on a specific scene can be trained by performing data enhancement on the image samples under the background. The method utilizes the self-learning ability of a deep neural network to remove the interference of the complex background, and only extracts the edge map of the object to be detected. After the binary polynomial interpolation is performed, the object has sub-pixel accuracy, so the measurement accuracy is high. After obtaining the edge contour containing only the target object, a series of processing is performed on the edge contour of the object to obtain the sub-pixel coordinates of the edge of the object, and the object has sub-pixel accuracy after the binary polynomial interpolation, which not only improves the object size measurement accuracy, but also enhances the adaptability of the measurement method. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the implementation of the present invention, and together with the specification are used to explain the principles of the embodiments of the present invention. Obviously, the accompanying drawings described below are only some embodiments of the implementation of the present invention, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0043] Figure 1 is a flowchart of a method for measuring the size of an object in a complex background based on deep learning, exemplarily shown in an embodiment of the present application;

[0044] Figure 2 It is a flowchart of training the RCF model by annotating samples to obtain the RCF model in the current scenario, as exemplified in an embodiment of the present application;

[0045] Figure 3 It is a structural diagram of an object size measurement system in a complex background based on deep learning, which is exemplified in an embodiment of the present application. DETAILED DESCRIPTION

[0046] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the embodiments of the present invention will be more comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the implementation of the embodiments of the present invention.

[0047] In the object processing process, it is necessary to know the corresponding object size in each process. For example, when producing thick plates, it is necessary to roll steel according to the billet size to ensure that the size of the produced object meets the standard. With the development of artificial intelligence and machine vision technology, non-contact measurement methods have emerged. By taking a visual image of the object to be detected, the size of the object to be detected is measured using image processing and artificial intelligence technology. The image of the object to be detected is collected by an industrial camera. First, the edge contour of the object to be detected is extracted, and then the coordinate parameters of the edge contour are calculated. Finally, the physical size of the object to be detected is calculated in combination with the calibration data of the camera. However, the current object size measurement method based on deep learning in a complex background is very dependent on the background information of the object. If the background of the object is too complex, the background information will also be extracted, which will affect the extraction of the edge information of the object.

[0048] In order to solve the above problems, the present application provides a method for measuring the size of an object under a complex background based on deep learning. Figure 1 As shown, the following steps are included:

[0049] S100: Train the RCF model to obtain the RCF model in the current scenario.

[0050] The core of the RCF (Richer Convolutional Features) model is its ability to extract and utilize rich convolutional features. Compared with traditional edge detection methods, RCF can more accurately capture edge information in images through multi-level convolutional feature fusion. By using deep network architectures such as VGG16, features are extracted from different levels of convolutional neural networks and fused to generate more refined edge maps.

[0051] In the present application embodiment, first, a large amount of image data containing the target object and its background are collected, and these data should cover different lighting conditions, angles, occlusion situations and background complexity. Then, the RCF model is trained using these data, and the contour edge of the target object can be accurately identified and extracted by adjusting the model parameters. The RCF model can learn the characteristic differences between the target object and the background by training with a large amount of image data containing the target object and its background. This enables the model to more accurately identify and extract the edge of the target object in subsequent processing, and reduce the interference of background information.

[0052] S200: Setting camera calibration parameters, photographing the object to be detected with the camera, and acquiring an image of the object to be detected.

[0053] Camera calibration is the process of solving the imaging model parameters of the camera through a series of experiments and calculations. These parameters include internal parameters (intrinsic parameters) such as the focal length and principal point coordinates of the camera, as well as external parameters (extrinsic parameters) such as the position and orientation of the camera in the world coordinate system. These parameters together constitute the geometric model of camera imaging, which is used to describe the mapping relationship between three-dimensional space points and two-dimensional image points.

[0054] S300: sending the image of the object to be detected to the RCF model in the current scene to obtain a rough contour edge map of the object to be detected.

[0055] The collected image of the object to be detected is input into the trained RCF model, and the RCF model will output a binary image containing the rough contour edge of the target object. In this image, the edge of the target object is highlighted, while the background is suppressed. The rough contour edge map is the basis for subsequent refinement processing, which helps to separate the target object from the complex background and prepare for subsequent accurate size measurement.

[0056] S400: Process the rough contour edge map to obtain sub-pixel coordinate points of the object to be detected, and obtain the image length and image width of the object to be detected according to the sub-pixel coordinate points. The sub-pixel level of accuracy can significantly improve the accuracy of size measurement, especially when measuring tiny sizes, which can effectively improve the measurement accuracy.

[0057] S500: Calculate the physical length and width of the object to be detected according to the calibration parameters of the camera and the image length and image width.

[0058] Specifically, before using the camera, it needs to be calibrated to obtain the camera's internal parameters (such as focal length, principal point coordinates, etc.) and external parameters (such as the relative position relationship between the camera and the world coordinate system). Then, using these calibration parameters and the size information of the object in the image, that is, the image length and image width, the actual physical size of the object is calculated through the geometric transformation formula.

[0059] The method of the present application successfully achieves accurate measurement of the size of the target object under complex backgrounds through the synergy of multiple steps such as RCF model training and application, image acquisition, acquisition and refinement of coarse contour edge map, camera calibration and calculation of physical size. Among them, the application of the RCF model and the acquisition of the coarse contour edge map are key steps to avoid the extraction of background information, which provides a solid foundation for the subsequent edge refinement and calculation of physical size.

[0060] In some embodiments of the present application, the steps of training the RCF model to obtain the RCF model in the current scenario include:

[0061] S110: Photograph the sample object and collect sample images; this step is the basis for model training, and diverse image data is collected by photographing the sample object. The sample images should cover different lighting conditions, angles, occlusions, and background complexities to simulate various scenarios that may be encountered in actual applications.

[0062] S120: Perform data enhancement processing on the sample image, and label the sample image after the data enhancement processing through labeling software label to obtain a labeled sample.

[0063] More training samples are generated by rotating, scaling, cropping, flipping, adding noise, and other operations on the original sample images. This helps the model learn more robust feature representations and improves its adaptability to different image transformations. Then, the enhanced images are accurately annotated using annotation software to mark the edges or contours of the target objects. These annotation information will serve as supervisory signals during model training to guide the model to learn how to accurately extract the edges of objects.

[0064] Specifically, in some embodiments of the present application, methods for performing data enhancement processing on sample images include: a mosaic enhancement method, a hybrid enhancement method, and a random horizontal flipping method.

[0065] Among them, the mosaic enhancement method is an image preprocessing technique that combines four pictures for cropping and splicing to generate new training samples. This method enriches the context of object detection and helps optimize the batch normalization (BN) calculation of the model. In target detection models such as YOLOv4, mosaic enhancement is widely used because it can simulate more complex scenes and improve the generalization ability of the model. By cropping and splicing different images, the mosaic enhancement method can generate a large number of new training samples, thereby increasing the diversity of the data. During the training process, mosaic enhancement helps the model better learn batch normalization parameters and improves the stability and performance of the model. Since the mosaic enhancement method can simulate more complex scenes, it helps improve the generalization ability of the model for unseen data.

[0066] Hybrid enhancement is a method that mixes two or more images and their labels in a certain ratio to generate new training samples. This method has been widely used in image recognition, object detection and other fields. Among them, Mixup is one of the most representative hybrid enhancement algorithms. Mixup generates new training samples by mixing two input images and their labels through linear interpolation. Hybrid enhancement helps reduce the overfitting of the model to the training data by generating more diverse training samples. Since the hybrid enhancement method introduces additional noise information, the model can learn to be robust to noise during the training process, thereby improving the robustness of the model.

[0067] Random horizontal flipping is a simple and effective data augmentation method. It increases the diversity of data by randomly flipping the image left and right. Compared with other complex data augmentation methods, random horizontal flipping is simple to implement, has low computational cost, and can generate new training samples, thereby increasing the diversity of data.

[0068] S130: Input the labeled samples into the RCF model for training to obtain the RCF model in the current scene. Input the labeled samples into the RCF model, calculate the error between the model output and the true annotation through forward propagation, and then use the back propagation algorithm to adjust the model parameters to minimize the error. This process will be iterated many times until the performance of the model on the validation set meets the predetermined requirements. During the training process, the RCF model will learn the feature differences between the target object and the background, and how to accurately extract the edges of objects in complex backgrounds. These feature representations are the basis for the model to accurately identify and extract the edges of objects. By adjusting hyperparameters such as learning rate, batch size, regularization parameters, and adopting appropriate optimization algorithms (such as Adam, SGD, etc.), the training process of the model can be further optimized to improve the performance and generalization ability of the model.

[0069] In some embodiments of the present application, reference Figure 2 As shown, the steps of training the RCF model by annotating samples to obtain the RCF model in the current scenario include:

[0070] S131: Divide the labeled samples into training set, validation set and test set in a ratio of 3:1:1.

[0071] Among them, the training set is used in the learning process of the model, that is, the model updates its weights and parameters through the training set data to minimize the loss function. The validation set is used to adjust the model's hyperparameters (such as learning rate, batch size, etc.) and monitor the performance of the model during the training process. The validation set does not participate in the training of the model, but is used to evaluate the performance of the model on unseen data for model selection and tuning. The test set is used to evaluate the generalization ability of the model after the model training is completed. The test set data is unknown during the model training process, so its evaluation results can more truly reflect the performance of the model in actual applications.

[0072] S132: Send the training set and the validation set to the RCF model for training to obtain a trained RCF model.

[0073] The training set data is fed into the RCF model, which calculates the output through forward propagation and adjusts the weights and parameters through the back-propagation algorithm to minimize the loss function. The validation set data is regularly used to evaluate the performance of the model during the training process in order to adjust the hyperparameters and tune the model. Through multiple iterations of training, the RCF model gradually learns how to extract the edge features of the target object from the input image and optimizes its weights and parameters to minimize the loss.

[0074] S133: The generalization ability of the trained RCF model is evaluated through the test set, and the RCF model for the current scenario is obtained if the evaluation is qualified.

[0075] Since the test set data is not seen by the model during the training process, its evaluation results can truly reflect the generalization ability of the model. Through the performance evaluation on the test set, it can be judged whether the model can accurately extract the edge of the object in the current scenario. If the evaluation results on the test set meet the preset performance indicators (such as accuracy, recall rate, etc.), the model is considered to be effective in the current scenario and can be deployed and applied as the final model. If the evaluation results on the test set are not ideal, it may be necessary to further adjust the model's hyperparameters, optimizer, data augmentation strategy, etc., and retrain until the model achieves satisfactory performance on the test set.

[0076] In some embodiments of the present application, when the training set and the validation set are fed into the RCF model for training, the RCF model training setting parameters are: Since the vgg16convs.mat pre-training model parameters are used, the number of training rounds is set to 200 times, and the learning rate is 1e -6 , learning rate step size: 3, learning rate decay: 0.6.

[0077] Specifically, the training round refers to the number of times the entire data set is traversed and learned. In some embodiments of the present application, setting it to 200 times means that the entire training set will be used completely to train the RCF model 200 times. Sufficient training rounds help the model to fully learn the features in the data, but too many rounds may also cause the model to overfit. Therefore, according to experimental training, it is concluded that selecting 200 training rounds can effectively balance the model learning ability and generalization ability.

[0078] The learning rate determines the step size of the model each time the weight is updated. A smaller learning rate means that the model will be more cautious when updating the weights, which helps the model remain stable during training, but may also lead to slower training and convergence to a local optimal solution. The learning rate is set to 1e -6 , is a relatively small value. Because the RCF model needs to be sensitive to the data, and there is more noise in the data set in complex background images, a smaller learning rate is required to ensure the stability of training.

[0079] The learning rate step size refers to the number of rounds in which the learning rate is adjusted in the learning rate adjustment strategy. In this case, setting it to 3 means that after every 3 training rounds, the learning rate will be adjusted according to the preset decay strategy. By adjusting the learning rate step size, you can control the speed at which the learning rate changes throughout the training process, thereby optimizing the training effect of the model.

[0080] Learning rate decay refers to gradually reducing the learning rate according to a preset strategy during the training process. In some embodiments of the present application, setting it to 0.6 means that each time the learning rate is adjusted, the current learning rate is multiplied by 0.6. Learning rate decay helps the model converge quickly in the early stages of training, and in the later stages of training, the weights of the model are fine-tuned by gradually reducing the learning rate, thereby improving the accuracy and generalization ability of the model.

[0081] In some embodiments of the present application, the step of processing the coarse contour edge map to obtain corresponding sub-pixel coordinate points includes:

[0082] S410: The rough outline edge Figure 2 The value is quantified, and according to the area size of the contour in the coarse contour edge map, the abnormal contour of the coarse contour edge map is eliminated.

[0083] Convert the rough contour edge map into an image containing only black and white colors, i.e. a binary image. This helps simplify subsequent processing steps because the edge contours in a binary image are clearer and easier to detect and identify. Based on the size of the contour area, abnormal contours that do not meet expectations or may be caused by noise can be identified and removed. This helps reduce errors and interference in subsequent processing and improves the accuracy of edge detection.

[0084] S420: Using non-maximum suppression and double threshold processing, the edge contour of the object to be detected in the coarse contour edge map is refined to obtain a fine contour edge map.

[0085] Non-maximum suppression is an edge refinement technique that retains local maximum points on the edge while removing other non-maximum points. This helps reduce the redundancy and blur of the edge, making the edge contour clearer and more accurate. At the same time, dual threshold processing can divide edge points into strong edge points and weak edge points by setting high and low thresholds. Strong edge points are usually located in the main part of the edge, while weak edge points may be located in the blurred or noisy area of ​​the edge. Through dual threshold processing, the edge contour can be further refined while reducing the influence of noise and pseudo edges.

[0086] Specifically, in some embodiments of the present application, the maximum threshold is set to 0.25 and the minimum threshold is set to 0.2.

[0087] The maximum threshold is a higher threshold used to distinguish strong edges in dual threshold processing. When the pixel value (or gradient value, depending on the specific algorithm) exceeds the maximum threshold, these pixels are considered strong edge points. Strong edge points are usually located in the main part of the edge in the image, with high contrast and clarity. By setting a higher maximum threshold, weak edge points caused by noise or slight changes in the image can be filtered out, thereby reducing false detection and noise interference.

[0088] The minimum threshold is the lower threshold used to distinguish weak edges in double threshold processing. When the pixel value (or gradient value) is between the maximum threshold and the minimum threshold, these pixels are considered weak edge points. Weak edge points may be located in the fuzzy area of ​​the edge or the transition part of the edge in the image. Although the contrast of weak edge points is low, they may contain important information about the shape and structure of objects in the image. By setting a reasonable minimum threshold, this important information can be retained while reducing the interference of noise.

[0089] In the embodiment of the present application, by distinguishing strong edges from weak edges, false detection and noise interference can be reduced, thereby improving the accuracy of edge detection. Double threshold processing has a certain robustness to illumination changes, noise and slight changes in the image, and can maintain a good edge detection effect under different conditions. By setting a reasonable minimum threshold, weak edge information in the image can be retained, thereby retaining more image details and structural information.

[0090] S430: using a polynomial interpolation algorithm for edge points of the thin contour edge map to calculate sub-pixel coordinates of the object to be detected.

[0091] The polynomial interpolation algorithm is used to estimate the value of unknown data points between known data points. In edge detection, the polynomial interpolation algorithm can be used to estimate more accurate sub-pixel coordinates at edge points. Through the polynomial interpolation algorithm, edge coordinates that are more accurate than the pixel level, namely sub-pixel coordinates, can be obtained. The accuracy of sub-pixel coordinates is usually higher than the pixel level, so it can provide more accurate edge position and shape information. This is especially important for applications that require high-precision edge detection (such as machine vision, image processing, etc.).

[0092] In some embodiments of the present application, the step of using a polynomial interpolation algorithm for edge points of a thin contour edge map to calculate sub-pixel coordinates of an object to be detected includes:

[0093] S431: Find the contour of the object to be detected in the thin contour edge map through the findcontours function of opencv. The function of findcontours is to find contours in a binary image. It will traverse each white pixel (edge) in the image and identify different contours based on their connectivity.

[0094] S432: Find four coordinate points of the object to be detected according to the contour.

[0095] Specifically, the four coordinate points are [x min ,y m ]、[x max ,y n ]、[x m ,y max ]、[x n,y min ].

[0096] Among them, x min is the minimum horizontal coordinate of the four coordinate points, y m is the ordinate corresponding to the coordinate point with the smallest abscissa; max is the maximum horizontal coordinate of the four coordinate points, y n is the ordinate corresponding to the coordinate point with the maximum abscissa; min is the minimum ordinate of the four coordinate points, x m is the horizontal coordinate corresponding to the coordinate point with the smallest vertical coordinate; max is the maximum ordinate of the four coordinate points, x n is the horizontal coordinate corresponding to the coordinate point with the maximum vertical coordinate.

[0097] S433: Calculate the corresponding sub-pixel coordinate points according to the four coordinate points using the quadratic Lagrange interpolation formula.

[0098] Specifically, the quadratic Lagrange interpolation formula L(x) is as follows:

[0099]

[0100] In the formula, y j is the function value, x i is the interpolation base point.

[0101] Specifically, take one of the coordinate points [x min ,y m ] as an example, take three points R(x min -1,y m ), R(x min ,y m ), R(x min +1,y m ) Take the gradient amplitude of these three points as the function value x min -1,x min , x min +1 is the interpolation base point, substitute the quadratic polynomial interpolation function; and let the derivative Similarly, take three points R(x min ,y m -1), R(x min ,y m ), R(x min ,y m +1) and perform the same operation, the sub-pixel edge coordinates (X min , Y m )for:

[0102]

[0103] In some embodiments of the present application, the step of obtaining the image length and image width of the object to be detected according to the sub-pixel coordinate points includes:

[0104] S401: Obtain the sub-pixel coordinate point of the object to be detected in the X direction (X min , Y m ), (X max , Y n ), and obtain the image length L of the object to be detected.

[0105]

[0106] S402: Obtain the sub-pixel coordinate point of the object to be detected in the Y direction (X m , Y mun ), (X n , Y max ), and obtain the image width W of the object to be detected.

[0107]

[0108] In the formula, X min is the minimum horizontal coordinate of the four sub-pixel coordinate points, Y m X is the ordinate corresponding to the sub-pixel coordinate point with the minimum abscissa; max is the maximum horizontal coordinate of the four sub-pixel coordinate points, Y n Y is the ordinate corresponding to the sub-pixel coordinate point with the maximum abscissa; min is the minimum ordinate of the four sub-pixel coordinate points, X m Y is the horizontal coordinate corresponding to the sub-pixel coordinate point with the smallest vertical coordinate; max is the maximum ordinate of the four sub-pixel coordinate points, X n is the horizontal coordinate corresponding to the sub-pixel coordinate point with the maximum vertical coordinate; L is the image length of the object to be detected, and W is the image width of the object to be detected.

[0109] From the above content, it can be seen that the present application provides a method and system for measuring the size of objects under complex backgrounds based on deep learning. For the problem of measuring objects under complex backgrounds, by performing data enhancement on the image samples under the background, an RCF model based on a specific scene can be trained. This method utilizes the self-learning ability of the deep neural network to remove the interference of the complex background, and only extracts the edge map of the object to be detected. After the binary polynomial interpolation is performed, the object has sub-pixel accuracy, so the measurement accuracy is high. After obtaining the edge contour containing only the target object, a series of processing is performed on the edge contour of the object to obtain the sub-pixel coordinates of the edge of the object. After the binary polynomial interpolation is performed, the object has sub-pixel accuracy, which not only improves the object size measurement accuracy, but also enhances the adaptability of the measurement method.

[0110] On the other hand, the present application provides a system for measuring the size of an object under a complex background based on deep learning, referring to Figure 3 As shown, it is used to implement the method for measuring the size of an object in a complex background based on deep learning in any of the above embodiments.

[0111] The system includes: an image acquisition module, an image preprocessing module, an object size detection module and an object size calculation module. The image acquisition module, the image preprocessing module, the object size detection module and the object size calculation module are all communication-connected.

[0112] The image acquisition module is used to take pictures of the object to be detected and collect images of the object to be detected, thereby ensuring that the system can obtain visual information of the object to be detected.

[0113] The image preprocessing module is used to perform data enhancement and annotation on the images of the objects to be detected; data enhancement may include adjusting image brightness, contrast, rotation, scaling, etc. to improve image quality and the robustness of the detection model. Annotation usually involves marking objects in the image so that subsequent detection and measurement processes can accurately identify them.

[0114] The object size detection module to be detected is used to detect the coarse contour edge map of the image of the object to be detected, process the coarse contour edge map, and obtain the image length and image width of the object to be detected; the object size detection module can accurately identify the size of the object in the image through edge detection, contour extraction, etc.

[0115] The module for calculating the size of the object to be detected is used to calculate the physical size of the object to be detected based on the image length and image width of the object to be detected. The physical size of the object to be detected is calculated by combining the known image resolution and the proportional relationship between the actual physical size of the object and the image size (such as the scaling factor obtained by camera calibration).

[0116] The object size measurement system based on deep learning under complex background provided by the embodiment of the present application can accurately measure the size of the object to be detected under complex backgrounds, such as lighting changes, occlusion, cluttered background, etc. Through deep learning technology, the system can automatically identify and extract the outline of the object, and then calculate its image size, and finally convert it into physical size. Compared with traditional manual measurement methods, the system can realize automated measurement, greatly improving the measurement efficiency. At the same time, due to the accurate extraction and recognition ability of the deep learning algorithm for image features, the measurement accuracy of the system has also been significantly improved. And by training the deep learning model, the system can learn and adapt to various complex backgrounds, so as to achieve accurate measurement of objects. The system has broad application prospects in the fields of intelligent manufacturing, industrial automation, etc.

[0117] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the disclosure of the specification and the examples. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

Claims

1. A method for measuring object size under complex background based on deep learning, characterized in that: The following steps are involved: Training the RCF model to obtain the RCF model in the current scenario; Setting camera calibration parameters, using the camera to photograph the object to be detected, and acquiring an image of the object to be detected; Sending the image of the object to be detected into the RCF model in the current scene to obtain a rough contour edge map of the object to be detected; Processing the rough contour edge map to obtain sub-pixel coordinate points of the object to be detected, and obtaining the image length and image width of the object to be detected according to the sub-pixel coordinate points; The physical length and width of the object to be detected are calculated according to the calibration parameters of the camera and the image length and image width.

2. The method for measuring object size under complex background based on deep learning according to claim 1, characterized in that: The step of training the RCF model to obtain the RCF model in the current scenario includes: Photographing the sample object and collecting the sample image; Performing data enhancement processing on the sample image, and labeling the sample image after the data enhancement processing by using labeling software label to obtain a labeled sample; The labeled samples are input into the RCF model for training to obtain the RCF model in the current scenario.

3. The method for measuring object size under complex background based on deep learning according to claim 2, characterized in that: The method for performing data enhancement processing on the sample image includes: a mosaic enhancement method, a hybrid enhancement method, and a random horizontal flipping method.

4. The method for measuring object size under complex background based on deep learning according to claim 2, characterized in that: The step of training the RCF model by annotating samples to obtain the RCF model in the current scenario includes: The labeled samples are divided into a training set, a validation set, and a test set in a ratio of 3:1:1; Sending the training set and the validation set into the RCF model for training to obtain the RCF model after training; The generalization ability of the trained RCF model is evaluated through a test set, and the RCF model in the current scenario is obtained after the evaluation is qualified.

5. The method for measuring object size under complex background based on deep learning according to claim 4, characterized in that: When the training set and the validation set are sent to the RCF model for training, the RCF model training setting parameters are: 200 training rounds, learning rate: 1e -6 , learning rate step size: 3, learning rate decay: 0.

6.

6. The method for measuring object size under complex background based on deep learning according to claim 1, characterized in that: The step of processing the rough contour edge map to obtain corresponding sub-pixel coordinate points comprises: Binarizing the rough contour edge map, and eliminating abnormal contours of the rough contour edge map according to the area of ​​the contours in the rough contour edge map; Using non-maximum suppression and double threshold processing to refine the edge contour of the object to be detected in the coarse contour edge map to obtain a fine contour edge map; A polynomial interpolation algorithm is used for edge points of the thin contour edge map to calculate sub-pixel coordinates of the object to be detected.

7. The method for measuring object size under complex background based on deep learning according to claim 6, characterized in that: In the dual threshold processing, the maximum threshold is set to 0.25 and the minimum threshold is set to 0.

2.

8. The method for measuring object size under complex background based on deep learning according to claim 6, characterized in that: The step of using a polynomial interpolation algorithm on the edge points of the thin contour edge map to calculate the sub-pixel coordinates of the object to be detected comprises: Find the contour of the object to be detected in the fine contour edge map by using the findcontours function of opencv; Find four coordinate points of the object to be detected according to the outline; According to the four coordinate points, the corresponding sub-pixel coordinate points are calculated using a quadratic Lagrange interpolation formula.

9. The method for measuring object size under complex background based on deep learning according to claim 6, characterized in that: The step of obtaining the image length and image width of the object to be detected according to the sub-pixel coordinate points comprises: The sub-pixel coordinate point of the object to be detected in the X direction is obtained as (X min , Y m ), (X max , Y n ), obtaining the image length L of the object to be detected; The sub-pixel coordinate point of the object to be detected in the Y direction is obtained as (X m , Y min ), (X n , Y max ), obtain the image width W of the object to be detected; Where, X min is the minimum horizontal coordinate of the four sub-pixel coordinate points, Y m X is the ordinate corresponding to the sub-pixel coordinate point with the minimum abscissa; max is the maximum horizontal coordinate of the four sub-pixel coordinate points, Y n Y is the ordinate corresponding to the sub-pixel coordinate point with the maximum abscissa; min is the minimum ordinate of the four sub-pixel coordinate points, X m Y is the horizontal coordinate corresponding to the sub-pixel coordinate point with the smallest vertical coordinate; max is the maximum ordinate of the four sub-pixel coordinate points, X n is the horizontal coordinate corresponding to the sub-pixel coordinate point with the maximum vertical coordinate; L is the image length of the object to be detected, and W is the image width of the object to be detected.

10. A system for measuring the size of an object under a complex background based on deep learning, the system being used to implement a method for measuring the size of an object under a complex background based on deep learning as claimed in any one of claims 1 to 9, characterized in that: include: Image acquisition module, image preprocessing module, object size detection module and object size calculation module; The image acquisition module, the image preprocessing module, the object size detection module and the object size calculation module are all connected in communication; The image acquisition module is used to take a picture of the object to be detected and acquire an image of the object to be detected; The image preprocessing module is used to perform data enhancement and annotation on the image of the object to be detected; The to-be-detected object size detection module is used to detect the rough contour edge map of the to-be-detected object image, process the rough contour edge map, and obtain the image length and image width of the to-be-detected object; The module for calculating the size of the object to be detected is used to calculate the physical size of the object to be detected according to the image length and image width of the object to be detected.

Images