Container loading and unloading positioning identification method based on machine vision

Abstract

The invention relates to the technical field of image processing, and discloses a container loading and unloading positioning identification method based on machine vision. The method comprises the following steps: firstly, obtaining a to-be-processed container image, and carrying out noise reduction processing, binarization processing and edge detection on the to-be-processed container image to obtain a processed container image; secondly, HOG features of the processed container image are extracted, and an HOG feature vector set is obtained; a final SVM classifier is obtained through training, the HOG feature vector set is input into the final SVM classifier for classification, and container recognition is completed. And after container identification is completed, obtaining a container top image, processing the container top image to obtain a container corner fitting lockhole image, and positioning the position of the container corner fitting lockhole by using a convolutional neural network to realize container loading and unloading identification and positioning. According to the method, the container image is processed, the purpose of container loading and unloading positioning identification is achieved, and the method is accurate and objective.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a container loading and unloading positioning and recognition method based on machine vision. Background Art

[0002] Chinese patent CN114219842A discloses a method for visual recognition, ranging and positioning in the automatic loading and unloading of port containers. The method specifically includes: forming a binocular camera system and deploying and installing it on both sides of the movable trolley on the crossbeam of the container bridge crane; using the binocular camera system to identify and planarly locate the container to obtain the planar position of the target container; then using the binocular camera system to achieve preliminary ranging of the container, and combining the structural size information of the container, accurately calculate the height of the container, and obtain the spatial position coordinates of the container through inverse perspective transformation; then by identifying the pre-painted mark on the spreader, the plane positioning of the spreader is achieved, and the distance between the spreader and the mobile trolley of the container bridge crane is measured, thereby obtaining the spatial position coordinates of the spreader, and realizing the recognition, ranging and positioning of the automatic loading and unloading of port containers. This invention does not use a method for quickly identifying container images, and the recognition speed is slow and the accuracy is low.

[0003] The traditional container loading and unloading positioning and identification method, when using equipment or manual measurement, does not pre-process the collected container images, nor does it use high-tech technologies such as neural networks and machine vision. As a result, there is no remedial measure when the measuring equipment fails. At the same time, the manual identification error rate is high and the port container loading and unloading speed is slow. Summary of the invention

[0004] In view of the problems in the related art, the present invention provides a container loading and unloading positioning and identification method based on machine vision to overcome the above-mentioned technical problems existing in the existing related art.

[0005] To solve the above technical problems, the present invention is achieved through the following technical solutions: The present invention is a container loading and unloading positioning and identification method based on machine vision, comprising the following steps: S1. Shoot a batch of container images to obtain container images to be processed, which constitute a set of container images to be processed. Perform noise reduction, binarization and edge detection on the container images to be processed to obtain processed container images, which constitute a set of processed container images. S2, extracting the pixel gradient magnitude and gradient direction of the processed container image in the processed container image set, and obtaining a HOG feature vector set using a HOG feature extraction method according to the gradient magnitude and gradient direction; S3, training the initial SVM classifier to obtain the final SVM classifier, inputting the HOG feature vector set into the final SVM classifier for classification, and completing container recognition; S4. After completing container recognition, a container top image set is obtained from the processed container image set, and after image distortion correction and background segmentation processing are performed on the container top images in the container top image set, a container corner fitting lock hole image set is obtained; a convolutional neural network is trained to obtain a convolutional neural network model, and the position of the container corner fitting lock hole is located to realize container loading and unloading recognition and positioning.

[0006] The invention obtains the container image to be processed, uses an adaptive median filtering algorithm to perform noise reduction on the container image to be processed, then uses the threshold method to calculate the binarization threshold, performs binarization on the denoised container image, and finally introduces an edge operator to extract the container edge; the image preprocessing method effectively solves the problem of balancing noise reduction and detail retention by assigning weights to pixel points, and the binarization process reduces the amount of subsequent calculations and facilitates image segmentation; secondly, according to the gradient size and gradient direction of the pixel points of the processed container image, a gradient histogram is constructed to merge and extract HOG features to obtain a HOG feature vector; this method can more accurately extract container shape information and contour features. The initial SVM classifier is trained to obtain the final SVM classifier to realize container recognition. The SVM classifier has significant advantages in solving nonlinear problems. Repeated training of unrecognized samples makes the SVM classifier have high recognition accuracy and fast recognition speed. After container recognition, the image of the top of the container is subjected to image distortion correction and background segmentation to obtain the image of the keyhole of the corner fitting of the container. Finally, the convolutional neural network is trained to obtain the convolutional neural network model, and the position of the keyhole of the corner fitting of the container in the keyhole image of the corner fitting of the container is located to realize the recognition and positioning of container loading and unloading. When training the convolutional neural network, the detection window of the detection area is introduced to speed up the processing speed while taking into account the accuracy.

[0007] Preferably, the S1 comprises the following steps: S11, take a batch of container images to obtain container images to be processed, form a container image set to be processed, select a container image to be processed in the container image set to be processed, Sample points are obtained and the sample points are quantized. The quantized sample points are the pixel points on the container image to be processed. The container image to be processed is converted into the container image matrix A to be processed as follows: ; in, Indicates the mth horizontal and nth vertical pixel on the container image to be processed; The red signal channel, the blue signal channel and the green signal channel on the container image to be processed are processed by using a weighted average method to obtain a grayscale container image; S12, using an adaptive median filter algorithm to perform noise reduction processing on the grayscale container image to obtain a noise-reduced container image, and forming a noise-reduced container image set. The specific steps are as follows: S121, set the size to A sliding window is placed on the container image matrix to be processed, and the central pixel point of the sliding window is found to obtain the gray value of the central pixel point of the sliding window, and the average gray value of all pixels in the sliding window is calculated, which is recorded as the gray value mean of the sliding window; S122, set the gray value threshold to , when the grayscale value of the central pixel of the sliding window is equal to the maximum grayscale value in the sliding window, or the grayscale value of the central pixel of the sliding window is equal to the minimum grayscale value in the sliding window, or the absolute value of the difference between the grayscale value of the central pixel of the sliding window and the grayscale value mean of the sliding window is greater than the grayscale value threshold, the central pixel of the sliding window is regarded as a noise pixel, otherwise the central pixel of the sliding window is not a noise pixel; S123, assigning weights to all pixels in the sliding window, calculating weighted gray values ​​of all pixels in the sliding window, recording them as a weighted gray value set, sorting the weighted gray value set in ascending order, finding an intermediate weighted gray value, and replacing the gray value of the noise pixel with the intermediate weighted gray value until all pixels in the container image matrix to be processed are denoised, obtaining a denoised container image matrix and a denoised container image, and forming a denoised container image set; S13, setting the gray level in the denoised container image to b, and calculating the probability of the occurrence of pixels in the denoised container image with gray levels from 0 to b, recorded as , calculate the average gray value of the pixels in the container image after denoising from gray level 0 to b, recorded as ; Divide the denoised container image into d grayscale value classes, and calculate the occurrence probability and average value of each grayscale value class. The calculation formula is as follows: , ; in, represents the occurrence probability of the d-th gray value class, represents the average value of the d-th gray value class, represents the probability of occurrence of a pixel with gray level d, Represents the probability of the occurrence of a pixel with gray level d-1, represents the average gray value of the pixel at gray level d-1, and ; According to the occurrence probability of the d-th gray value class and the average value of the d-th gray value class, the inter-class variance is obtained. When the inter-class variance is 2, the binarization threshold of the denoised container image is obtained, the pixel gray value of the pixel point in the denoised container image matrix whose gray value is less than the binarization threshold is set to 0, and the pixel gray value of the pixel point in the denoised container image matrix whose gray value is greater than the binarization threshold is set to 255, and the image binarization processing is completed to obtain the binarized container image, which constitutes a binarized container image set; S14. Set the Laplacian operator formula to f, record the horizontal direction of the binary container image as the x direction, and the vertical direction of the binary container image as the y direction, use the differential equation to perform second-order partial derivatives in the x direction and the y direction respectively, and bring the second-order partial derivative results into the Laplacian operator formula to obtain the Laplacian edge operator. The Laplacian edge operator performs a convolution operation on the binary container image through a Gaussian filter, outputs a processed container image, and forms a processed container image set.

[0008] The invention obtains the container image to be processed, uses an adaptive median filtering algorithm to perform noise reduction on the container image to be processed, and effectively solves the problem of balancing noise reduction and detail retention by assigning weights to pixel points; then uses the threshold method to calculate the binarization threshold, and binarizes the denoised container image to reduce the subsequent calculation amount; finally, introduces an edge operator to extract the container edge.

[0009] Preferably, S2 comprises the following steps: S21, selecting pixel points on the processed container images in the processed container image set, and the pixel point coordinates , pixel gray value , pixel The calculation formulas for the horizontal and vertical gradients at are as follows: , ; in, Represents pixel The horizontal gradient at Represents pixel The vertical gradient at Represents pixel coordinates The gray value of the pixel at Represents pixel coordinates The gray value of the pixel at Represents pixel coordinates The gray value of the pixel at Represents pixel coordinates Gray value of pixel point; Then the pixel The gradient size at , pixel The gradient direction at ; S22, converting the processed container image into a processed container image matrix, selecting a size of A pixel block is formed by cell units, the gradient direction of the cell unit in the direction of 0-360 degrees is calculated, and then the gradient direction of the cell unit in the direction of 0-360 degrees is evenly divided into 9 boxes, and the gradient information of the 9 boxes is calculated, the gradient information includes the gradient magnitude and the gradient direction, and the gradient information of the cell unit is counted by using a histogram to obtain the gradient information of the pixel block; the gradient information of other pixel blocks in the processed container image matrix is ​​calculated to obtain a processed container image gradient set, and the processed container image gradient set is recorded as a HOG (Histogram of Directed Gradients) feature vector set.

[0010] The invention constructs a gradient histogram through the gradient size and gradient direction of the pixel points of the processed container image to merge and extract HOG features and obtain a HOG feature vector; this method can more accurately extract the container shape information and contour features and improve the recognition rate.

[0011] Preferably, S3 comprises the following steps: S31, obtaining container images from the Internet to form a container sample image set, performing image preprocessing on the container sample images in the container sample image set, extracting HOG feature vectors, obtaining a HOG feature vector sample set, training an initial SVM classifier, and obtaining a final SVM classifier. The specific steps are as follows: S311. Set the kernel function of the SVM (support vector machine) classifier as the radial basis function, determine whether the input sample set is linearly separable through the classification surface equation, use the Lagrangian function to convert it into a classification function, and convert the sample set into a high-dimensional feature vector by introducing a penalty factor, a kernel function and a relaxation factor to obtain the optimal classification function to classify the sample set; S312, dividing the HOG feature vector sample set into a feature vector sample training set and a feature vector sample test set, inputting the feature vector sample training set into an initial SVM classifier, iterating continuously until the initial SVM classifier converges, and obtaining a trained SVM classifier; then inputting the feature vector sample training set and the feature vector sample test set that failed to be identified in the feature vector sample training set into the trained SVM classifier, and setting the error threshold to , when the output result error is less than the error threshold, the final SVM classifier is obtained; otherwise, the feature vector sample training set that failed to be recognized is repeatedly input into the trained SVM classifier for training until the output result error is less than the error threshold; S32, using the final SVM classifier to identify the container, inputting the HOG feature vector set into the final SVM classifier, outputting the identification result, and completing the container identification.

[0012] The invention obtains the final SVM classifier by training the initial SVM classifier. The SVM classifier has significant advantages in solving nonlinear problems. By repeatedly training unrecognized samples, container recognition is achieved, thereby improving recognition accuracy and recognition speed.

[0013] Preferably, S4 comprises the following steps: S41, after completing the container recognition, selecting the top-related images of the container from the processed container image set to obtain container top images, forming a container top image set, performing image distortion correction and background segmentation processing on the container top images in the container top image set, and obtaining a container corner fitting lock hole image set, the specific steps are as follows: S411, setting a cubic B-spline function, converting the container top image into a container top image matrix, selecting the coordinates of distorted pixel points in the container top image matrix, offsetting the distorted pixel points on the x-axis and the y-axis, and performing surface interpolation using the cubic B-spline function to obtain the offsets of the distorted pixel points on the x-axis and the y-axis; using the distorted pixel point coordinates plus the offsets of the distorted pixel points on the x-axis and the y-axis to obtain the actual pixel point coordinates; S412, using bilinear interpolation method to calculate the linear interpolation of the actual pixel point on the x-axis and y-axis, converting the actual pixel point into the interpolation pixel point, recorded as , select neighboring pixels around the interpolation pixel, respectively denoted as , , and , correcting the interpolated pixel grayscale value according to the grayscale value of the adjacent pixel point to obtain the distortion-corrected pixel point; sequentially performing distortion correction on all the distorted pixel points in the container top image matrix to obtain the distortion-corrected container top image, forming a set of distortion-corrected container top images; S413, converting the distortion-corrected container top image into a distortion-corrected container top image matrix, and assigning weights to the connected edges in the distortion-corrected container top image matrix. The calculation formula is as follows: ; in, Represents pixel and pixels The weight of the connected edges, Represents pixel Gray value, Represents pixel Gray value, Represents parameters; Calculate the pixel The weight of the edge connected to the end pixel is calculated to obtain the pixel difference corresponding to the distortion-corrected container top image and the background image, and then the background image in the distortion-corrected container top image is segmented according to the pixel difference to obtain the container corner fitting keyhole image, which constitutes the container corner fitting keyhole image set; S42, obtaining a sample image of the top of the container from the Internet, extracting the lock holes of the corner fittings of the container from the sample image of the top of the container, obtaining a set of images of the lock holes of the corner fittings of the container, and training a convolutional neural network using the set of images of the lock holes of the corner fittings of the container to obtain a convolutional neural network model, the specific steps are as follows: S421. Add a detection area detection window to the convolutional neural network and set the step size to , the convolutional neural network scans the target in the detection window of the detection area, and then moves to the right with a step size Slide the detection area detection window until the detection area detection window slides to the leftmost end of the image and the entire image is scanned; S422, dividing the container corner fitting lock hole images in the container corner fitting lock hole image set to obtain half container corner fitting lock hole images, complete container corner fitting lock hole images, and container corner fitting lock hole images in the up, down, left, and right directions, and mixing them to form a container corner fitting lock hole image training set and a container corner fitting lock hole image test set; setting the convolutional neural network activation function to be a Sigmoid function, inputting the container corner fitting lock hole image training set into the convolutional neural network for training, setting the maximum number of iterations to be g, and obtaining a trained convolutional neural network when the current number of iterations reaches the maximum number of iterations; then inputting the container corner fitting lock hole image test set into the trained convolutional neural network, setting the accuracy threshold to , when the accuracy of the output result is greater than the accuracy threshold, stop the iteration and obtain the convolutional neural network model; otherwise, adjust and update the weight parameters until the accuracy of the output result is greater than the accuracy threshold; S43, inputting the container corner fitting lock hole image set into a convolutional neural network model, outputting the container corner fitting lock hole positioning image, and forming a container loading corner fitting lock hole positioning image set; according to the positions of the container corner fitting lock hole positioning in the container corner fitting lock hole positioning image, the crane hoist loads and unloads the container to realize container loading and unloading identification and positioning.

[0014] After the container is identified, the invention performs image distortion correction and background segmentation processing on the image of the top of the container to obtain the image of the corner fitting lock hole of the container; finally, the convolutional neural network is trained to obtain the convolutional neural network model. By introducing the detection area detection window, the processing speed can be accelerated, and the position of the corner fitting lock hole of the container in the corner fitting lock hole image of the container can be accurately located, thereby realizing the identification and positioning of container loading and unloading.

[0015] The present invention also discloses a system for a container loading and unloading positioning and identification method based on machine vision, which specifically comprises: a container image preprocessing module, a HOG feature extraction module, a container identification module and a container corner piece lock hole position positioning module; The container image preprocessing module is used to perform noise reduction, binarization and edge detection on the container image; The HOG feature extraction is used to obtain a HOG feature vector set using a HOG feature extraction method on the processed container image; The container identification module is used to identify the container using an SVM classifier; The container corner fitting lock hole position positioning module is used to process the container top image and use a convolutional neural network model to locate the container corner fitting lock hole position.

[0016] The present invention has the following beneficial effects: 1. The invention obtains the container image to be processed, uses an adaptive median filtering algorithm to perform noise reduction on the container image to be processed, and effectively solves the problem of balancing noise reduction and detail retention by assigning weights to pixel points; then uses the threshold method to calculate the binarization threshold, and performs binarization on the denoised container image to reduce the amount of subsequent calculations.

[0017] 2. The invention constructs a gradient histogram to merge and extract HOG features to obtain a HOG feature vector, which can more accurately extract container shape information and contour features and improve the recognition rate.

[0018] 3. The invention obtains the final SVM classifier by training the initial SVM classifier. The SVM classifier has significant advantages in solving nonlinear problems. By repeatedly training unrecognized samples, container recognition is achieved, thereby improving recognition accuracy and recognition speed.

[0019] 4. The invention can obtain image information more accurately, reduce the amount of calculation and increase the processing speed by performing image distortion correction and background segmentation processing on the container top image.

[0020] 5. The invention obtains a convolutional neural network model by training a convolutional neural network, and introduces a detection area detection window to speed up the processing speed, accurately locate the position of the container corner fitting lock hole in the container corner fitting lock hole image, and realize container loading and unloading identification and positioning.

[0021] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the invention, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, they can also obtain drawings based on these drawings without paying creative work.

[0023] Figure 1 A schematic diagram of the process of container loading and unloading positioning and identification by the machine vision-based container loading and unloading positioning and identification system provided by the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] In the description of the present invention, it is necessary to understand that the terms "opening", "upper", "lower", "top", "middle", "inside" and the like indicating orientation or positional relationship are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the invention.

[0026] Example 1 Please refer to Figure 1 The present invention is a container loading and unloading positioning and identification method based on machine vision, comprising the following steps: S1. Shoot a batch of container images to obtain container images to be processed, which constitute a set of container images to be processed. Perform noise reduction, binarization and edge detection on the container images to be processed to obtain processed container images, which constitute a set of processed container images. The S1 comprises the following steps: S11, take a batch of container images to obtain container images to be processed, form a container image set to be processed, select a container image to be processed in the container image set to be processed, Sample points are obtained and the sample points are quantized. The quantized sample points are the pixel points on the container image to be processed. The container image to be processed is converted into the container image matrix A to be processed as follows: ; in, Indicates the mth horizontal and nth vertical pixel on the container image to be processed; The red signal channel, the blue signal channel and the green signal channel on the container image to be processed are processed by using a weighted average method to obtain a grayscale container image; S12, using an adaptive median filter algorithm to perform noise reduction processing on the grayscale container image to obtain a noise-reduced container image, and forming a noise-reduced container image set. The specific steps are as follows: S121, set the size to A sliding window is placed on the container image matrix to be processed, and the central pixel point of the sliding window is found to obtain the gray value of the central pixel point of the sliding window, and the average gray value of all pixels in the sliding window is calculated, which is recorded as the gray value mean of the sliding window; S122, set the gray value threshold to , when the grayscale value of the central pixel of the sliding window is equal to the maximum grayscale value in the sliding window, or the grayscale value of the central pixel of the sliding window is equal to the minimum grayscale value in the sliding window, or the absolute value of the difference between the grayscale value of the central pixel of the sliding window and the grayscale value mean of the sliding window is greater than the grayscale value threshold, the central pixel of the sliding window is regarded as a noise pixel, otherwise the central pixel of the sliding window is not a noise pixel; S123, assigning weights to all pixels in the sliding window, calculating weighted gray values ​​of all pixels in the sliding window, recording them as a weighted gray value set, sorting the weighted gray value set in ascending order, finding an intermediate weighted gray value, and replacing the gray value of the noise pixel with the intermediate weighted gray value until all pixels in the container image matrix to be processed are denoised, obtaining a denoised container image matrix and a denoised container image, and forming a denoised container image set; S13, setting the gray level in the denoised container image to b, and calculating the probability of the occurrence of pixels in the denoised container image with gray levels from 0 to b, recorded as , calculate the average gray value of the pixels in the container image after denoising from gray level 0 to b, recorded as ; Divide the denoised container image into d grayscale value classes, and calculate the occurrence probability and average value of each grayscale value class. The calculation formula is as follows: , ; in, represents the occurrence probability of the d-th gray value class, represents the average value of the d-th gray value class, represents the probability of occurrence of a pixel with gray level d, Represents the probability of the occurrence of a pixel with gray level d-1, represents the average gray value of the pixel at gray level d-1, and ; According to the occurrence probability of the d-th gray value class and the average value of the d-th gray value class, the inter-class variance is obtained. When the inter-class variance is 2, the binarization threshold of the denoised container image is obtained, the pixel gray value of the pixel point in the denoised container image matrix whose gray value is less than the binarization threshold is set to 0, and the pixel gray value of the pixel point in the denoised container image matrix whose gray value is greater than the binarization threshold is set to 255, and the image binarization processing is completed to obtain the binarized container image, which constitutes a binarized container image set; S14, setting the Laplacian operator formula to f, recording the horizontal direction of the binary container image as the x direction, recording the vertical direction of the binary container image as the y direction, using the differential equation to perform second-order partial derivatives in the x direction and the y direction respectively, bringing the second-order partial derivative results into the Laplacian operator formula to obtain the Laplacian edge operator, and performing a convolution operation on the binary container image through a Gaussian filter, outputting a processed container image, and forming a processed container image set; S2, extracting the pixel gradient magnitude and gradient direction of the processed container image in the processed container image set, and obtaining a HOG feature vector set using a HOG feature extraction method according to the gradient magnitude and gradient direction; The S2 comprises the following steps: S21, selecting pixel points on the processed container images in the processed container image set, and the pixel point coordinates , pixel gray value , pixel The calculation formulas for the horizontal and vertical gradients at are as follows: , ; in, Represents pixel The horizontal gradient at Represents pixel The vertical gradient at Represents pixel coordinates The gray value of the pixel at Represents pixel coordinates The gray value of the pixel at Represents pixel coordinates The gray value of the pixel at Represents pixel coordinates Gray value of pixel point; Then the pixel The gradient size at , pixel The gradient direction at ; S22, converting the processed container image into a processed container image matrix, selecting a size of A pixel block is formed by cell units, the gradient direction of the cell unit in the 0-360 degree direction is calculated, and then the gradient direction of the cell unit in the 0-360 degree direction is evenly divided into 9 boxes, and the gradient information of the 9 boxes is calculated, and the gradient information includes the gradient magnitude and the gradient direction. The gradient information of the cell unit is counted by using a histogram to obtain the gradient information of the pixel block; the gradient information of other pixel blocks in the processed container image matrix is ​​calculated to obtain a processed container image gradient set, and the processed container image gradient set is recorded as a HOG feature vector set; S3, training the initial SVM classifier to obtain the final SVM classifier, inputting the HOG feature vector set into the final SVM classifier for classification, and completing container recognition; The S3 comprises the following steps: S31, obtaining container images from the Internet to form a container sample image set, performing image preprocessing on the container sample images in the container sample image set, extracting HOG feature vectors, obtaining a HOG feature vector sample set, training an initial SVM classifier, and obtaining a final SVM classifier. The specific steps are as follows: S311, setting the kernel function of the SVM classifier to a radial basis function, judging whether the input sample set is linearly separable by using the classification surface equation, converting it into a classification function using the Lagrangian function, and converting the sample set into a high-dimensional feature vector by introducing a penalty factor, a kernel function, and a relaxation factor, obtaining an optimal classification function, and realizing classification of the sample set; S312, dividing the HOG feature vector sample set into a feature vector sample training set and a feature vector sample test set, inputting the feature vector sample training set into an initial SVM classifier, iterating continuously until the initial SVM classifier converges, and obtaining a trained SVM classifier; then inputting the feature vector sample training set and the feature vector sample test set that failed to be identified in the feature vector sample training set into the trained SVM classifier, and setting the error threshold to , when the output result error is less than the error threshold, the final SVM classifier is obtained; otherwise, the feature vector sample training set that failed to be recognized is repeatedly input into the trained SVM classifier for training until the output result error is less than the error threshold; S32, using the final SVM classifier to identify the container, inputting the HOG feature vector set into the final SVM classifier, outputting the identification result, and completing the container identification; S4. After completing the container recognition, a container top image set is obtained from the processed container image set, and after image distortion correction and background segmentation processing are performed on the container top images in the container top image set, a container corner fitting lock hole image set is obtained; a convolutional neural network is trained to obtain a convolutional neural network model, and the position of the container corner fitting lock hole is located to realize container loading and unloading recognition and positioning; The S4 comprises the following steps: S41, after completing the container recognition, selecting the top-related images of the container from the processed container image set to obtain container top images, forming a container top image set, performing image distortion correction and background segmentation processing on the container top images in the container top image set, and obtaining a container corner fitting lock hole image set, the specific steps are as follows: S411, setting a cubic B-spline function, converting the container top image into a container top image matrix, selecting the coordinates of distorted pixel points in the container top image matrix, offsetting the distorted pixel points on the x-axis and the y-axis, and performing surface interpolation using the cubic B-spline function to obtain the offsets of the distorted pixel points on the x-axis and the y-axis; using the distorted pixel point coordinates plus the offsets of the distorted pixel points on the x-axis and the y-axis to obtain the actual pixel point coordinates; S412, using bilinear interpolation method to calculate the linear interpolation of the actual pixel point on the x-axis and y-axis, converting the actual pixel point into the interpolation pixel point, recorded as , select neighboring pixels around the interpolation pixel, respectively denoted as , , and , correcting the interpolated pixel grayscale value according to the grayscale value of the adjacent pixel point to obtain the distortion-corrected pixel point; sequentially performing distortion correction on all the distorted pixel points in the container top image matrix to obtain the distortion-corrected container top image, forming a set of distortion-corrected container top images; S413, converting the distortion-corrected container top image into a distortion-corrected container top image matrix, and assigning weights to the connected edges in the distortion-corrected container top image matrix. The calculation formula is as follows: ; in, Represents pixel and pixels The weight of the connected edges, Represents pixel Gray value, Represents pixel Gray value, Represents parameters; Calculate the pixel The weight of the edge connected to the end pixel is calculated to obtain the pixel difference corresponding to the distortion-corrected container top image and the background image, and then the background image in the distortion-corrected container top image is segmented according to the pixel difference to obtain the container corner fitting keyhole image, which constitutes the container corner fitting keyhole image set; S42, obtaining a sample image of the top of the container from the Internet, extracting the lock holes of the corner fittings of the container from the sample image of the top of the container, obtaining a set of images of the lock holes of the corner fittings of the container, and training a convolutional neural network using the set of images of the lock holes of the corner fittings of the container to obtain a convolutional neural network model, the specific steps are as follows: S421. Add a detection area detection window to the convolutional neural network and set the step size to , the convolutional neural network scans the target in the detection window of the detection area, and then moves to the right with a step size Slide the detection area detection window until the detection area detection window slides to the leftmost end of the image and the entire image is scanned; S422, dividing the container corner fitting lock hole images in the container corner fitting lock hole image set to obtain half container corner fitting lock hole images, complete container corner fitting lock hole images, and container corner fitting lock hole images in the up, down, left, and right directions, and mixing them to form a container corner fitting lock hole image training set and a container corner fitting lock hole image test set; setting the convolutional neural network activation function to be a Sigmoid function, inputting the container corner fitting lock hole image training set into the convolutional neural network for training, setting the maximum number of iterations to be g, and obtaining a trained convolutional neural network when the current number of iterations reaches the maximum number of iterations; then inputting the container corner fitting lock hole image test set into the trained convolutional neural network, setting the accuracy threshold to , when the accuracy of the output result is greater than the accuracy threshold, stop the iteration and obtain the convolutional neural network model; otherwise, adjust and update the weight parameters until the accuracy of the output result is greater than the accuracy threshold; S43, inputting the container corner fitting lock hole image set into a convolutional neural network model, outputting the container corner fitting lock hole positioning image, and forming a container loading corner fitting lock hole positioning image set; according to the positions of the container corner fitting lock hole positioning in the container corner fitting lock hole positioning image, the crane hoist loads and unloads the container to realize container loading and unloading identification and positioning.

[0027] Example 2 The present invention also discloses a system for a container loading and unloading positioning and identification method based on machine vision, which specifically comprises: a container image preprocessing module, a HOG feature extraction module, a container identification module and a container corner piece lock hole position positioning module; The container image preprocessing module is used to perform noise reduction, binarization and edge detection on the container image; The HOG feature extraction is used to obtain a HOG feature vector set using a HOG feature extraction method on the processed container image; The container identification module is used to identify the container using an SVM classifier; The container corner fitting lock hole position positioning module is used to process the container top image and use a convolutional neural network model to locate the container corner fitting lock hole position.

[0028] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0029] The preferred embodiments of the invention disclosed above are only used to help explain the invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the invention, so that those skilled in the art can understand and use the invention well.

Claims

1. Container loading and unloading positioning and recognition method based on machine vision, It is characterized in that The steps include: S1. Shoot a batch of container images to obtain container images to be processed, which constitute a set of container images to be processed. Perform noise reduction, binarization and edge detection on the container images to be processed to obtain processed container images, which constitute a set of processed container images. S2, extracting the pixel gradient magnitude and gradient direction of the processed container image in the processed container image set, and obtaining a HOG feature vector set using a HOG feature extraction method according to the gradient magnitude and gradient direction; S3, training the initial SVM classifier to obtain the final SVM classifier, inputting the HOG feature vector set into the final SVM classifier for classification, and completing container recognition; S4. After completing container recognition, a container top image set is obtained from the processed container image set, and after image distortion correction and background segmentation processing are performed on the container top images in the container top image set, a container corner fitting lock hole image set is obtained; a convolutional neural network is trained to obtain a convolutional neural network model, and the position of the container corner fitting lock hole is located to realize container loading and unloading recognition and positioning.

2. According to the method for positioning and identifying container loading and unloading based on machine vision in claim 1, It is characterized in that The S1 comprises the following steps: S11, taking a batch of container images to obtain container images to be processed, forming a set of container images to be processed, and processing the container images to be processed using a weighted average method to obtain grayscale container images; S12, using an adaptive median filtering algorithm to perform noise reduction processing on the grayscale container image to obtain a noise-reduced container image to form a noise-reduced container image set; S13, determining a binarization threshold of the container image after noise reduction, completing the image binarization process, obtaining a binarized container image, and forming a binarized container image set; S14. Set the Laplacian edge operator to perform edge detection on the binary container image, output the processed container image, and form a processed container image set.

3. The container loading and unloading positioning and identification method based on machine vision according to claim 2, It is characterized in that The S12 comprises the following steps: S121, setting a sliding window, and calculating the average grayscale value of all pixels in the sliding window, which is recorded as the sliding window grayscale value mean; S122, set the gray value threshold to , compare the gray value mean of the sliding window with the gray value threshold to find the noise pixel point; S123, assigning pixel weights, and using an improved median filter algorithm to process the noise pixels to obtain a denoised container image to form a denoised container image set.

4. The container loading and unloading positioning and identification method based on machine vision according to claim 3, It is characterized in that The S2 comprises the following steps: S21, selecting pixel points on the processed container image in the processed container image set, and calculating the gradient magnitude and gradient direction of the pixel points; S22, setting a pixel block, using a histogram to count the gradient size and gradient direction of the pixel block, and using a HOG feature extraction method to obtain a HOG feature vector set.

5. The container loading and unloading positioning and identification method based on machine vision according to claim 4, It is characterized in that The S3 comprises the following steps: S31, acquiring container images from the Internet to form a container sample image set, performing image preprocessing on the container sample images in the container sample image set, extracting HOG feature vectors, obtaining a HOG feature vector sample set, training an initial SVM classifier, and obtaining a final SVM classifier; S32, using the final SVM classifier to identify the container, inputting the HOG feature vector set into the final SVM classifier, outputting the identification result, and completing the container identification.

6. The container loading and unloading positioning and identification method based on machine vision according to claim 5, It is characterized in that The training of the initial SVM classifier to obtain the final SVM classifier comprises the following steps: The kernel function of the SVM classifier is set as the radial basis function, the HOG feature vector sample set is divided into a feature vector sample training set and a feature vector sample test set, the feature vector sample training set is input into the initial SVM classifier, and it is continuously iterated until the initial SVM classifier converges to obtain a trained SVM classifier; then the feature vector sample training set and the feature vector sample test set that failed to be identified in the feature vector sample training set are input into the trained SVM classifier, and the error threshold is set as , when the output result error is less than the error threshold, the final SVM classifier is obtained; Otherwise, the feature vector sample training set that failed to be recognized is repeatedly input into the trained SVM classifier for training until the output result error is less than the error threshold.

7. The container loading and unloading positioning and identification method based on machine vision according to claim 6, It is characterized in that The S4 comprises the following steps: S41, after completing the container recognition, selecting container top images from the processed container image set to form a container top image set, performing image distortion correction and background segmentation processing on the container top images in the container top image set, and obtaining a container corner fitting lock hole image set; S42, obtaining a sample image of the top of the container from the Internet, extracting the lock holes of the corner fittings of the container from the sample image of the top of the container, obtaining a set of images of the lock holes of the corner fittings of the container, and training a convolutional neural network using the set of images of the lock holes of the corner fittings of the container to obtain a convolutional neural network model; S43, inputting the container corner fitting lock hole image set into a convolutional neural network model, outputting the container corner fitting lock hole positioning image, and forming a container loading corner fitting lock hole positioning image set; according to the positions of the container corner fitting lock hole positioning in the container corner fitting lock hole positioning image, the crane hoist loads and unloads the container to realize container loading and unloading identification and positioning.

8. The container loading and unloading positioning and identification method based on machine vision according to claim 7, It is characterized in that The image distortion correction and background segmentation processing of the container top images in the container top image set comprises the following steps: A cubic B-spline function is set, and the distorted pixel points in the container top image are interpolated using the cubic B-spline function to obtain the actual pixel coordinates; the grayscale value of the actual pixel point is corrected to obtain the distortion-corrected container top image, which constitutes a set of distortion-corrected container top images; the pixel difference corresponding to the distortion-corrected container top image and the background image is calculated, and then the background image in the distortion-corrected container top image is segmented according to the pixel difference to obtain the container corner fitting keyhole image, which constitutes a set of container corner fitting keyhole images.

9. The container loading and unloading positioning and identification method based on machine vision according to claim 8, It is characterized in that The S42 comprises the following steps: S421. Add a detection area detection window to the convolutional neural network and set the step size to , the convolutional neural network scans the target in the detection window of the detection area, and then moves to the right with a step size Slide the detection area detection window until the detection area detection window slides to the leftmost end of the image and the entire image is scanned; S422, dividing the container corner fitting lock hole images in the container corner fitting lock hole image set to obtain half container corner fitting lock hole images, complete container corner fitting lock hole images, and container corner fitting lock hole images in the up, down, left, and right directions, and mixing them to form a container corner fitting lock hole image training set and a container corner fitting lock hole image test set; setting the convolutional neural network activation function to be a Sigmoid function, inputting the container corner fitting lock hole image training set into the convolutional neural network for training, setting the maximum number of iterations to be g, and obtaining a trained convolutional neural network when the current number of iterations reaches the maximum number of iterations; then inputting the container corner fitting lock hole image test set into the trained convolutional neural network, setting the accuracy threshold to , when the accuracy of the output result is greater than the accuracy threshold, stop the iteration and obtain the convolutional neural network model; otherwise, adjust and update the weight parameters until the accuracy of the output result is greater than the accuracy threshold.

10. A system for implementing the container loading and unloading positioning and identification method based on machine vision according to any one of claims 1 to 9, It is characterized in that Specifically include: Container image preprocessing module, HOG feature extraction module, container recognition module and container corner fitting lock hole location module; The container image preprocessing module is used to perform noise reduction, binarization and edge detection on the container image; The HOG feature extraction is used to obtain a HOG feature vector set using a HOG feature extraction method on the processed container image; The container identification module is used to identify the container using an SVM classifier; The container corner fitting lock hole position positioning module is used to process the container top image and use a convolutional neural network model to locate the container corner fitting lock hole position.

Images