A container lock hole identification and positioning method and system

Abstract

The present invention discloses a method and system for identifying and locating a container lock hole, the method comprising: constructing a sample data set; constructing and training a target recognition model including a feature extraction network, a global self-attention encoding network and a classification prediction network to obtain a container lock hole recognition model; acquiring image data of a region to be detected; inputting the image data of the region to be detected into the container lock hole recognition model to obtain a container lock hole category and location region detection result. The technical solution provided by the present invention identifies and locates the container lock hole by constructing a target recognition model including a feature extraction network, a global self-attention encoding network and a classification prediction network; extracting the lock hole image, using the maximum inter-class variance method to perform binarization processing on the image, searching and drawing the lock hole contour, and obtaining the container lock hole accurate positioning result; it can improve the accuracy of container lock hole recognition and positioning, and improve the accuracy and operation efficiency of grabbing containers.

Description

Technical Field

[0001] The present invention relates to the field of container identification, and in particular to a container lock hole identification and positioning method and system. Background Art

[0002] Seaside container grabbing refers to the process of grabbing and moving containers on ships moored at the seaside of the container terminal using quay cranes or other lifting equipment. This process is an important part of the automated operation of container terminals and is of great significance for improving the operating efficiency and safety of the terminal.

[0003] In the seaside container grabbing task, identifying and locating the container lock hole is the key to quickly completing the container grabbing task. However, the existing container lock hole identification and positioning method is not accurate enough. The lock holes of adjacent containers will interfere with the identification and positioning, and there is a certain phenomenon of missed detection and false detection, which greatly affects the efficiency of the container grabbing task. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the current technology, the present invention provides a method for identifying and locating the keyhole of a container. By constructing a target recognition model based on a position-constrained global attention mechanism, and utilizing a feature extraction network, a global self-attention encoding network, and a classification prediction network to identify and locate the keyhole of the container, the model can enhance its correct understanding of the scene, reduce missed detections and false detections, and improve the accuracy and efficiency of container grabbing.

[0005] To achieve the above object, the embodiments of the present invention adopt the following technical solutions:

[0006] A container lock hole identification and positioning method comprises the following steps:

[0007] Obtain container keyhole image sample data and construct a sample data set;

[0008] Based on the sample data set, a target recognition model including a feature extraction network, a global self-attention encoding network, and a classification prediction network was constructed and trained to obtain a container keyhole recognition model.

[0009] Acquire image data of the area to be detected;

[0010] The image data of the area to be detected is input into the container keyhole recognition model to obtain the container keyhole category and location area detection results.

[0011] According to one aspect of the present invention, the container lock hole identification and positioning method further includes: performing noise reduction processing on the container lock hole image sample data and the image data of the area to be detected by Gaussian filtering.

[0012] According to one aspect of the present invention, the feature extraction network comprises:

[0013] Build a backbone network based on the residual network;

[0014] Input the image data into the feature extraction network to obtain the feature vector;

[0015] On the basis of extracting feature vectors, learnable relation encoding parameters are added;

[0016] Relation encoding calculates the position deviation, scale ratio, and direction information between the container lock holes to generate a relationship vector as the output of the feature extraction network.

[0017] According to one aspect of the present invention, the global self-attention encoding network comprises:

[0018] Built on top of a self-attention encoder;

[0019] The core part of the self-attention encoder is the multi-head attention module;

[0020] The modules of the self-attention encoder are connected through residual connections;

[0021] The input of the self-attention encoder is first normalized by the layer, and then three weight matrices Q, K, and V are generated through linear transformation to calculate the self-attention and obtain the relation-aware features as the output of the global self-attention encoding network;

[0022] The input of the global self-attention encoding network is the output of the feature extraction network.

[0023] According to one aspect of the present invention, the classification prediction network comprises:

[0024] Built on multi-layer perceptrons;

[0025] The multilayer perceptron consists of a fully connected layer, a Dropout layer, and a GELU activation function;

[0026] The input of the classification prediction network is the relation-aware features output by the global self-attention encoding network;

[0027] The classification prediction network performs target detection based on the relationship-aware features and outputs the container keyhole category and location area detection results.

[0028] According to one aspect of the present invention, the container lock hole identification and positioning method further includes:

[0029] Obtaining a container lock hole image according to the container lock hole category and location area detection results;

[0030] According to the container keyhole image, the keyhole outline is found and drawn to obtain the precise positioning result of the container keyhole.

[0031] According to one aspect of the present invention, the step of searching and drawing the outline of the lock hole according to the image of the lock hole of the container to obtain the precise positioning result of the lock hole of the container includes:

[0032] Perform histogram equalization and smoothing filter noise reduction on the container keyhole image to obtain the keyhole grayscale image;

[0033] According to the maximum inter-class variance method, the binary threshold of the keyhole grayscale image is obtained;

[0034] According to the thresholding function, the keyhole grayscale image is converted into a black and white binary image, the keyhole outline is segmented, and the precise positioning result of the container keyhole is obtained.

[0035] According to one aspect of the present invention, obtaining a binarization threshold of the keyhole grayscale image according to the maximum inter-class variance method includes:

[0036] Get the grayscale histogram of the keyhole grayscale image;

[0037] Traverse the grayscale, assuming that the current grayscale is the binary threshold, and calculate the frequency and average grayscale of the current threshold division method;

[0038] Calculate the inter-class variance based on the frequency and average grayscale of the current threshold division method;

[0039] Traverse all gray levels and find the gray level that maximizes the inter-class variance, which is the optimal binarization threshold.

[0040] A container lock hole identification and positioning system, based on the container lock hole identification and positioning method as described above, comprises:

[0041] A sample construction module is used to obtain container keyhole image sample data and construct a sample data set;

[0042] A model building module is used to build and train a target recognition model including a feature extraction network, a global self-attention encoding network, and a classification prediction network based on a sample data set to obtain a container keyhole recognition model;

[0043] An image acquisition module, used to acquire image data of the area to be detected;

[0044] The detection module is used to input the image data of the area to be detected into the container lock hole recognition model to obtain the container lock hole category and location area detection results.

[0045] Advantages of the present invention:

[0046] The present invention provides a container lock hole identification and positioning method. By constructing a target recognition model based on a position-constrained global attention mechanism, a feature extraction network, a global self-attention encoding network and a classification prediction network are used to identify and position the container lock hole. This can enhance the model's correct understanding of the scene, reduce missed detections and false detections, and improve the accuracy and operating efficiency of container grabbing.

[0047] This method also extracts the container keyhole image detected by the target recognition model, performs histogram equalization and noise reduction processing, and obtains the keyhole grayscale image; uses the maximum inter-class variance method to obtain the optimal image binarization threshold, and then binarizes the image, searches and draws the keyhole contour, and obtains the precise positioning result of the container keyhole; it can further improve the accuracy of container keyhole recognition and positioning, and improve the accuracy of container grabbing. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 This is a flow chart of a container lock hole identification and positioning method according to Embodiment 1 of the present invention;

[0050] Figure 2 This is an example of the sea side grab box described in the present invention;

[0051] Figure 3 is a structural diagram of the feature extraction network described in the present invention;

[0052] Figure 4 : is a structural diagram of the self-attention encoder of the present invention;

[0053] Figure 5 This is a flow chart of a container lock hole identification and positioning method according to Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Embodiment 1

[0056] like Figure 1As shown, a container lock hole identification and positioning method includes the following steps:

[0057] S1: Obtain container keyhole image sample data and build a sample dataset.

[0058] like Figure 2 The figure shows an example of a container and a seaside container grabbing. When grabbing a container on the seaside, the lock hole of the container needs to be identified and located so as to successfully grab the container. When identifying and locating the lock hole of the container, the lock hole of the adjacent container will interfere with the identification and positioning of the lock hole of the target container, resulting in missed detection and false detection.

[0059] In actual application, a sample image of a container is acquired through a camera, the type of the keyhole is marked, and the position of the keyhole is marked. In order to avoid interference with subsequent processing due to noise generated during image acquisition, the sample image needs to be subjected to noise reduction processing, and a Gaussian filtering method is used in this embodiment.

[0060] The container lock hole identification and positioning method further includes: performing noise reduction processing on image data of container lock hole image sample data by Gaussian filtering.

[0061] S2: Based on the sample data set, a target recognition model including a feature extraction network, a global self-attention encoding network and a classification prediction network is constructed and trained to obtain a container keyhole recognition model.

[0062] The container keyhole recognition model constructed by this method is a target recognition model built based on a position-constrained global attention mechanism. This model extracts multiple initial target points in the multiple images currently collected, and determines the final target point based on the positional relationship between the multiple initial target points, where the target point is the four lock positions of the container.

[0063] The target recognition model consists of three levels of networks: feature extraction network, global self-attention encoding network and classification prediction network.

[0064] The feature extraction network includes:

[0065] Build a backbone network based on the residual network;

[0066] Input the image data into the feature extraction network to obtain the feature vector;

[0067] On the basis of extracting feature vectors, learnable relation encoding parameters are added;

[0068] Relation encoding calculates the position deviation, scale ratio, and direction information between the container lock holes to generate a relationship vector as the output of the feature extraction network.

[0069] In practical applications, the first-level feature extraction network is built based on the ResNet-18 residual network as the backbone network. This network includes four residual modules for extracting feature information of different scales. This network structure can be referred to Figure 3 shown.

[0070] The container image is input into the feature extraction network to obtain the feature vector. In order to retain the texture and position information of the original feature vector, a learnable relational encoding parameter is added based on the extracted feature vector. , calculated as follows:

[0071]

[0072] in represents the output vector of the feature extraction network, is the original feature vector.

[0073] Relation encoding generates relationship vectors by calculating the position deviation, scale ratio, direction and other information between objects, which are used as the output vectors of the feature extraction network. These relationship vectors will be used as the input of the attention mechanism of the next level network, so that the model can understand the geometric relationship between objects.

[0074] The global self-attention encoding network includes:

[0075] Built on top of a self-attention encoder;

[0076] The core part of the self-attention encoder is the multi-head attention module;

[0077] The modules of the self-attention encoder are connected through residual connections;

[0078] The input of the self-attention encoder is first normalized by the layer, and then three weight matrices Q, K, and V are generated through linear transformation to calculate the self-attention and obtain the relation-aware features as the output of the global self-attention encoding network;

[0079] The input of the global self-attention encoding network is the output of the feature extraction network.

[0080] The convolution receptive field in the feature extraction network is small, resulting in the lack of global correlation information in the extracted features. Therefore, a second-level global self-attention encoding network is added to the target recognition model to supplement it. The global self-attention encoding network mainly extracts the global relationship information of feature points, so that the allocation of attention is not only based on the features of each target, but also takes into account the geometric relationship between them. This network consists of four layers of self-attention encoders. The structure of the self-attention encoder can be referred to Figure 4 shown.

[0081] The core part of the self-attention encoder is the multi-head attention module. The input of the encoder is first normalized by the layer, and then three weight matrices Q, K, and V are generated through linear transformation to calculate the self-attention. The calculation method is expressed based on the following formula:

[0082]

[0083] Among them, Q is the query vector; K is the key vector; R is the relation encoding vector; d k is the scaling factor. The object of calculation is multi-head self-attention, which can project the input feature vector into different subspaces to learn different features. By calculating self-attention, the correlation between global features can be established.

[0084] This method embeds the position information of elements into the existing attention mechanism and can reduce non-target feature expressions through position constraints.

[0085] In the current container grabbing task on the sea side of the quay crane, there is an obvious geometric and semantic connection between the spreader target point, the container target point and the surrounding cabin structure within the container grabbing working area. By inserting the attention module into the network, the model's correct understanding of the scene is improved and the phenomenon of missed detection and false detection is reduced.

[0086] The classification prediction network includes:

[0087] Built on multi-layer perceptrons;

[0088] The multilayer perceptron consists of a fully connected layer, a Dropout layer, and a GELU activation function;

[0089] The input of the classification prediction network is the relation-aware features output by the global self-attention encoding network;

[0090] The classification prediction network performs target detection based on the relationship-aware features and outputs the container keyhole category and location area detection results.

[0091] Finally, we normalize the output of the multi-head self-attention module of the previous level network and send it to the multilayer perceptron (MLP) of the third-level classification prediction network to realize target detection based on relational perception features and output the final target category and position.

[0092] S3: Acquire image data of the area to be detected.

[0093] The container keyhole identification and positioning method further includes: performing noise reduction processing on the image data of the detection area by Gaussian filtering.

[0094] In practical applications, an image containing multiple container locations within the field of view of the current position is obtained by a camera. In order to avoid interference with subsequent processing due to noise generated during image acquisition, the image needs to be denoised before target point recognition. In this embodiment, a Gaussian filtering method is used.

[0095] S4: Input the image data of the area to be detected into the container lock hole recognition model to obtain the container lock hole category and location area detection results.

[0096] The filtered image is input into the target recognition model constructed in step S2. The target recognition model extracts high-level features of the image through the first-level feature extraction network. The high-level features are multiple keyhole points and keyhole similarity points within the field of view. This process is feature extraction, and the features that meet the requirements are identified.

[0097] Because not all features extracted by the feature extraction network are target features, they include features of non-target positions and non-keyhole features. In order to eliminate the above interfering features, positional relationship constraints are introduced in the second-level global self-attention encoding network, and multiple features are screened by their positional relationships to obtain the final features. By configuring a relationship-aware global attention module in the model, the extracted feature map is input into the relationship-aware global attention module, the attention weight is calculated based on the relationship between the targets, and the original features are weighted summed to generate a new relationship-aware feature representation.

[0098] The feature representation obtained by the second-level network is used to perform target detection on the relationship-aware features using the third-level classification prediction network. These enhanced features are used to generate the final target category and position output to obtain the container keyhole category and position selection area detection results.

[0099] The beneficial effect of this embodiment is that this method constructs a target recognition model based on a position-constrained global attention mechanism, and uses a feature extraction network, a global self-attention encoding network, and a classification prediction network to identify and locate the container lock hole, which can enhance the model's correct understanding of the scene, reduce missed detections and false detections, and improve the accuracy and efficiency of container grabbing.

[0100] Embodiment 2

[0101] like Figure 5 As shown, a container lock hole identification and positioning method includes the following steps:

[0102] S1: Obtain container keyhole image sample data and build a sample dataset.

[0103] like Figure 2The figure shows an example of a container and a seaside container grabbing. When grabbing a container on the seaside, the lock hole of the container needs to be identified and located so as to successfully grab the container. When identifying and locating the lock hole of the container, the lock hole of the adjacent container will interfere with the identification and positioning of the lock hole of the target container, resulting in missed detection and false detection.

[0104] In actual application, a sample image of a container is acquired through a camera, the type of the keyhole is marked, and the position of the keyhole is marked. In order to avoid interference with subsequent processing due to noise generated during image acquisition, the sample image needs to be subjected to noise reduction processing, and a Gaussian filtering method is used in this embodiment.

[0105] The container lock hole identification and positioning method further includes: performing noise reduction processing on image data of container lock hole image sample data by Gaussian filtering.

[0106] S2: Based on the sample data set, a target recognition model including a feature extraction network, a global self-attention encoding network and a classification prediction network is constructed and trained to obtain a container keyhole recognition model.

[0107] The container keyhole recognition model constructed by this method is a target recognition model built based on a position-constrained global attention mechanism. This model extracts multiple initial target points in the multiple images currently collected, and determines the final target point based on the positional relationship between the multiple initial target points, where the target point is the four lock positions of the container.

[0108] The target recognition model consists of three levels of networks: feature extraction network, global self-attention encoding network and classification prediction network.

[0109] The feature extraction network includes:

[0110] Build a backbone network based on the residual network;

[0111] Input the image data into the feature extraction network to obtain the feature vector;

[0112] On the basis of extracting feature vectors, learnable relation encoding parameters are added;

[0113] Relation encoding calculates the position deviation, scale ratio, and direction information between the container lock holes to generate a relationship vector as the output of the feature extraction network.

[0114] In practical applications, the first-level feature extraction network is built based on the ResNet-18 residual network as the backbone network. This network includes four residual modules for extracting feature information of different scales. This network structure can be referred to Figure 3 shown.

[0115] The container image is input into the feature extraction network to obtain the feature vector. In order to retain the texture and position information of the original feature vector, a learnable relational encoding parameter is added based on the extracted feature vector. , calculated as follows:

[0116]

[0117] in, represents the output vector of the feature extraction network, is the original feature vector.

[0118] Relation encoding generates relationship vectors by calculating the position deviation, scale ratio, direction and other information between objects, which are used as the output vectors of the feature extraction network. These relationship vectors will be used as the input of the attention mechanism of the next level network, so that the model can understand the geometric relationship between objects.

[0119] The global self-attention encoding network includes:

[0120] Built on top of a self-attention encoder;

[0121] The core part of the self-attention encoder is the multi-head attention module;

[0122] The modules of the self-attention encoder are connected through residual connections;

[0123] The input of the self-attention encoder is first normalized by the layer, and then three weight matrices Q, K, and V are generated through linear transformation to calculate the self-attention and obtain the relation-aware features as the output of the global self-attention encoding network;

[0124] The input of the global self-attention encoding network is the output of the feature extraction network.

[0125] The convolution receptive field in the feature extraction network is small, resulting in the lack of global correlation information in the extracted features. Therefore, a second-level global self-attention encoding network is added to the target recognition model to supplement it. The global self-attention encoding network mainly extracts the global relationship information of feature points, so that the allocation of attention is not only based on the features of each target, but also takes into account the geometric relationship between them. This network consists of four layers of self-attention encoders. The structure of the self-attention encoder can be referred to Figure 4 shown.

[0126] The core part of the self-attention encoder is the multi-head attention module. The input of the encoder is first normalized by the layer, and then three weight matrices Q, K, and V are generated through linear transformation to calculate the self-attention. The calculation method is expressed based on the following formula:

[0127]

[0128] Among them, Q is the query vector; K is the key vector; R is the relation encoding vector; d k is the scaling factor. The object of calculation is multi-head self-attention, which can project the input feature vector into different subspaces to learn different features. By calculating self-attention, the correlation between global features can be established.

[0129] This method embeds the position information of elements into the existing attention mechanism and can reduce non-target feature expressions through position constraints.

[0130] In the current container grabbing task on the sea side of the quay crane, there is an obvious geometric and semantic connection between the spreader target point, the container target point and the surrounding cabin structure within the container grabbing working area. By inserting the attention module into the network, the model's correct understanding of the scene is improved and the phenomenon of missed detection and false detection is reduced.

[0131] The classification prediction network includes:

[0132] Built on multi-layer perceptrons;

[0133] The multilayer perceptron consists of a fully connected layer, a Dropout layer, and a GELU activation function;

[0134] The input of the classification prediction network is the relation-aware features output by the global self-attention encoding network;

[0135] The classification prediction network performs target detection based on the relationship-aware features and outputs the container keyhole category and location area detection results.

[0136] Finally, we normalize the output of the multi-head self-attention module of the previous level network and send it to the multilayer perceptron (MLP) of the third-level classification prediction network to realize target detection based on relational perception features and output the final target category and position.

[0137] S3: Acquire image data of the area to be detected.

[0138] The container keyhole identification and positioning method further includes: performing noise reduction processing on the image data of the detection area by Gaussian filtering.

[0139] In practical applications, an image containing multiple container locations within the field of view of the current position is obtained by a camera. In order to avoid interference with subsequent processing due to noise generated during image acquisition, the image needs to be denoised before target point recognition. In this embodiment, a Gaussian filtering method is used.

[0140] S4: Input the image data of the area to be detected into the container lock hole recognition model to obtain the container lock hole category and location area detection results.

[0141] The filtered image is input into the target recognition model constructed in step S2. The target recognition model extracts high-level features of the image through the first-level feature extraction network. The high-level features are multiple keyhole points and keyhole similarity points within the field of view. This process is feature extraction, and the features that meet the requirements are identified.

[0142] Because not all features extracted by the feature extraction network are target features, they include features of non-target positions and non-keyhole features. In order to eliminate the above interfering features, positional relationship constraints are introduced in the second-level global self-attention encoding network, and multiple features are screened by their positional relationships to obtain the final features. By configuring a relationship-aware global attention module in the model, the extracted feature map is input into the relationship-aware global attention module, the attention weight is calculated based on the relationship between the targets, and the original features are weighted summed to generate a new relationship-aware feature representation.

[0143] The feature representation obtained by the second-level network is used to perform target detection on the relationship-aware features using the third-level classification prediction network. These enhanced features are used to generate the final target category and position output to obtain the container keyhole category and position selection area detection results.

[0144] The above process can identify the keyhole at the target location, but can only obtain the container keyhole type and the approximate location selection area. To obtain the position of the keyhole outline and accurately locate the keyhole, the next step of processing is required.

[0145] S5: Acquire a container lock hole image according to the container lock hole category and location area detection results.

[0146] According to the container lock hole category and position frame selection area detection results obtained in step S4, the identified frame selection area is extracted to obtain the container lock hole image.

[0147] S6: According to the container lock hole image, the lock hole contour is searched and drawn to obtain the precise positioning result of the container lock hole.

[0148] Step S6 includes:

[0149] S61: Performing histogram equalization and smoothing filter noise reduction processing on the container keyhole image to obtain a keyhole grayscale image.

[0150] Because there is a certain amount of noise in the recognized container keyhole image, after obtaining the container keyhole image, it needs to be processed to improve the image quality so as to facilitate the subsequent positioning of the container keyhole.

[0151] This process first performs a histogram equalization operation on the keyhole image. The grayscale value distribution of the keyhole image is adjusted by histogram equalization, so that an image with uniform grayscale frequency distribution and clear keyhole details can be obtained. The grayscale histogram is obtained by normalization. The probability of grayscale occurrence is calculated as follows:

[0152]

[0153] Where n is the total number of image pixels; k = 0, 1, ..., L-1, L is the total number of gray levels; r k is the gray level; h´(r k ) is the gray level r k Probability of occurrence; n k is the gray level r k Number of occurrences.

[0154] The cumulative probability S of each gray level and all previous gray levels appearing k , calculated as follows:

[0155]

[0156] Through After merging and inverse transformation, the grayscale value of the keyhole image is obtained, which is calculated as follows:

[0157]

[0158] After the histogram equalization operation is performed on the image of container keyhole recognition, the contrast between the keyhole and the background area in the image changes.

[0159] After the keyhole image is processed by histogram equalization, there will be some noise in the image, which will affect the quality of the image. This method eliminates the noise and distortion in the image through image smoothing. Smoothing filtering can effectively weaken the high-frequency components in the image.

[0160] After preprocessing, the container keyhole image is enhanced and the noise is eliminated.

[0161] After the above processing, the image is converted from color to grayscale, represented by grayscale values ​​between 0 and 255. However, in fact, the information range of such an image array is still very large, which will affect our positioning of the container keyhole. Therefore, it is necessary to convert the image into a black and white binary image, and then search and draw the outline of the keyhole, and finally complete the precise positioning of the container keyhole.

[0162] S62: Obtain a binarization threshold of the keyhole grayscale image according to the maximum inter-class variance method.

[0163] Step S62 includes:

[0164] Get the grayscale histogram of the keyhole grayscale image;

[0165] Traverse the grayscale, assuming that the current grayscale is the binary threshold, and calculate the frequency and average grayscale of the current threshold division method;

[0166] Calculate the inter-class variance based on the frequency and average grayscale of the current threshold division method;

[0167] Traverse all gray levels and find the gray level that maximizes the inter-class variance, which is the optimal binarization threshold.

[0168] In practical applications, the selection of the binarization threshold T is the key point for image segmentation. In this embodiment, the maximum inter-class variance method is used to obtain the optimal binarization threshold T. The principle of this method is to divide the image into two parts by the optimal threshold and maximize the inter-class variance of the two parts.

[0169] The total number of pixels in the acquired image is N, the total number of gray levels is L, and the total number of pixels with gray value i is ,and is the probability of the occurrence of grayscale levels from 0 to k pixels, is the average gray value of the grayscale from 0 to k pixels, then:

[0170]

[0171]

[0172] in, , .

[0173] Assume there are M thresholds , , …, , divide the image into M+1 classes . The frequency, average grayscale and variance are , and , calculated as follows:

[0174]

[0175] Total occurrence probability of image pixels With average gray The calculation method is as follows:

[0176]

[0177]

[0178] The between-class variance is calculated as follows:

[0179]

[0180] The binarization threshold T that maximizes the inter-class variance is obtained, which is the optimal binarization threshold.

[0181] S63: According to the threshold function, the keyhole grayscale image is converted into a black and white binary image, the keyhole outline is segmented, and the precise positioning result of the container keyhole is obtained.

[0182] The image binarization function is expressed as follows:

[0183]

[0184] Among them, T is the binarization threshold, f(x,y) is the grayscale value of the original image pixel, and g(x,y) is the value of the image pixel after binarization.

[0185] The keyhole image is segmented by a thresholding function, wherein the thresholding function can be determined based on a function provided in OpenCV. The determined threshold value can be used to determine the contour, which is the contour of the keyhole, thereby achieving accurate positioning of the keyhole.

[0186] The beneficial effects of this embodiment are as follows: this method also obtains a keyhole grayscale image by extracting the container keyhole image detected by the target recognition model, performing histogram equalization and noise reduction processing; using the maximum inter-class variance method to obtain the optimal image binarization threshold, and then binarizing the image, searching and drawing the keyhole contour, and obtaining the precise positioning result of the container keyhole; it can further improve the accuracy of container keyhole recognition and positioning, and improve the accuracy of container grabbing.

[0187] Embodiment 3

[0188] A container lock hole identification and positioning system, based on the container lock hole identification and positioning method as described in Embodiment 1 or 2, comprises:

[0189] A sample construction module is used to obtain container keyhole image sample data and construct a sample data set;

[0190] A model building module is used to build and train a target recognition model including a feature extraction network, a global self-attention encoding network, and a classification prediction network based on a sample data set to obtain a container keyhole recognition model;

[0191] An image acquisition module, used to acquire image data of the area to be detected;

[0192] The detection module is used to input the image data of the area to be detected into the container lock hole recognition model to obtain the container lock hole category and location area detection results.

[0193] Preferably, the system further comprises:

[0194] The Gaussian filtering module is used to perform noise reduction processing on the container keyhole image sample data and the image data of the area to be detected through Gaussian filtering.

[0195] Preferably, the system further comprises:

[0196] A keyhole image extraction module is used to obtain a container keyhole image based on the container keyhole category and location area detection results;

[0197] The keyhole contour acquisition module is used to find and draw the keyhole contour according to the container keyhole image, and obtain the precise positioning result of the container keyhole.

[0198] Embodiment 4

[0199] A computer program, when executed, implements the steps of the container lock hole identification and positioning method as described in embodiment 1 or 2.

[0200] Embodiment 5

[0201] A readable storage medium stores a computer program as described in Example 4, and when the computer program is executed, the steps of the container lock hole identification and positioning method as described in Example 1 or 2 are implemented.

[0202] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A container lock hole identification and positioning method, characterized in that: The following steps are involved: Obtain container keyhole image sample data and construct a sample data set; Based on the sample data set, a target recognition model including a feature extraction network, a global self-attention encoding network, and a classification prediction network was constructed and trained to obtain a container keyhole recognition model. Acquire image data of the area to be detected; The image data of the area to be detected is input into the container lock hole recognition model to obtain the container lock hole category and location area detection results; The target recognition model is constructed based on the position-constrained global attention mechanism. This model extracts multiple initial target points in the multiple images currently collected, and determines the final target point according to the positional relationship between the multiple initial target points, where the target point is the four lock positions of the container; The feature extraction network includes: Build a backbone network based on the residual network; Input the image data into the feature extraction network to obtain the feature vector; On the basis of extracting feature vectors, learnable relation encoding parameters are added; Relation encoding calculates the position deviation, scale ratio, and direction information between the container lock holes to generate a relationship vector as the output of the feature extraction network, enabling the model to understand the geometric relationship between the objects; The output vector of the feature extraction network for: in, Encode the parameters for the relation, is the feature vector; The global self-attention encoding network includes: Built on top of a self-attention encoder; The core part of the self-attention encoder is the multi-head attention module; The modules of the self-attention encoder are connected through residual connections; The input of the self-attention encoder is first normalized by the layer, and then three weight matrices Q, K, and V are generated through linear transformation to calculate the self-attention and obtain the relation-aware features as the output of the global self-attention encoding network; The input of the global self-attention encoding network is the output of the feature extraction network; The global self-attention encoding network extracts the global relationship information of feature points, so that the allocation of attention is not only based on the features of each target, but also takes into account the geometric relationship between them. In the global self-attention encoding network, position relationship constraints are introduced to screen multiple features based on their positional relationships to obtain the final features. The container lock hole identification and positioning method further includes: Obtaining a container lock hole image according to the container lock hole category and location area detection results; According to the container lock hole image, find and draw the lock hole outline to obtain the precise positioning result of the container lock hole; The step of searching and drawing the outline of the lock hole according to the image of the lock hole of the container to obtain the precise positioning result of the lock hole of the container includes: Perform histogram equalization and smoothing filter noise reduction on the container keyhole image to obtain the keyhole grayscale image; According to the maximum inter-class variance method, the binary threshold of the keyhole grayscale image is obtained; According to the thresholding function, the keyhole grayscale image is converted into a black and white binary image, the keyhole outline is segmented, and the precise positioning result of the container keyhole is obtained.

2. The container lock hole identification and positioning method according to claim 1, characterized in that: The container lock hole identification and positioning method further includes: performing noise reduction processing on the container lock hole image sample data and the image data of the area to be detected by Gaussian filtering.

3. The container lock hole identification and positioning method according to claim 1, characterized in that: The classification prediction network includes: Built on top of multi-layer perceptrons; The multilayer perceptron consists of a fully connected layer, a Dropout layer, and a GELU activation function; The input of the classification prediction network is the relation-aware features output by the global self-attention encoding network; The classification prediction network performs target detection based on the relationship-aware features and outputs the container keyhole category and location area detection results.

4. The container lock hole identification and positioning method according to claim 1, characterized in that: The step of obtaining the binarization threshold of the keyhole grayscale image according to the maximum inter-class variance method includes: Get the grayscale histogram of the keyhole grayscale image; Traverse the grayscale, assuming that the current grayscale is the binary threshold, and calculate the frequency and average grayscale of the current threshold division method; Calculate the inter-class variance based on the frequency and average grayscale of the current threshold division method; Traverse all gray levels and find the gray level that maximizes the inter-class variance, which is the optimal binarization threshold.

5. A container lock hole identification and positioning system, characterized in that: The container lock hole identification and positioning method according to any one of claims 1 to 4 comprises: A sample construction module is used to obtain container keyhole image sample data and construct a sample data set; A model building module is used to build and train a target recognition model including a feature extraction network, a global self-attention encoding network, and a classification prediction network based on a sample data set to obtain a container keyhole recognition model; An image acquisition module, used to acquire image data of the area to be detected; The detection module is used to input the image data of the area to be detected into the container lock hole recognition model to obtain the container lock hole category and location area detection results.

Images