A dirty detection method and system for an automated port horizontal transportation device

Abstract

The application discloses a kind of automatic wharf horizontal transport equipment dirty detection method and device, construct first level convolutional neural network for detecting IGV from image and second level convolutional neural network for detecting dirty area from IGV, first level convolutional neural network and second level convolutional neural network are constructed into double-branch cascade detection model, the original image obtained from the automatic wharf operation scene is input as the double-branch cascade detection model, and the first level convolutional neural network identifies IGV from complex scene, and outputs the image labeled IGV boundary box, according to the boundary box, IGV subgraph containing only IGV image is cut out from image, so as to remove background interference and transmit to second level convolutional neural network, and second level convolutional neural network focuses on detecting dirty area from IGV subgraph, so as to improve the detection efficiency and detection precision of IGV dirty area.

Description

Technical Field

[0001] This invention belongs to the field of automated port technology, specifically, it relates to a method and system for detecting dirt and grime in automated terminal horizontal transport equipment. Background Technology

[0002] With the development of automated terminals, the automated management of horizontal transport equipment has become key to improving terminal operational efficiency. However, horizontal transport equipment (IGV) is prone to accumulating dirt during operation, causing interference to optical components and dust accumulation in electrical and cooling systems, thus affecting the normal operation of the equipment and the overall efficiency of the terminal.

[0003] Current methods for detecting IGV contamination mostly rely on manual inspections or neural network models, which have the following problems:

[0004] 1. Manual inspection is inefficient and costly.

[0005] 2. Neural network models for detecting and simultaneously identifying IGVs and their contaminated areas in complex dock operation scenarios have the following problems:

[0006] (1) It is difficult to quantitatively define the distinction between clean IGV and dirty IGV in the model, resulting in low detection accuracy.

[0007] (2) The background of the dock (such as the surrounding environment of IGV, changes in light, etc.) is mixed with the characteristics of IGV, which leads to confusion between the dirty areas of IGV and the background characteristics, thereby reducing the detection accuracy (especially small dirty areas).

[0008] (3) The dirty areas on IGV are usually small areas. A single model is easily affected by the overall features of IGV, resulting in a high rate of missed detection of dirt.

[0009] (4) It is difficult to simultaneously achieve high-precision detection of IGV and its contaminated areas, and the model cannot be lightweight.

[0010] (5) The dataset has poor generalization ability. Summary of the Invention

[0011] This invention proposes a method and system for detecting contamination in horizontal transport equipment at automated terminals. By combining a dual-model hierarchical detection mechanism and image segmentation technology, it achieves accurate positioning of IGVs and their contaminated areas. The coordinate remapping mechanism further improves the positioning efficiency and accuracy of contaminated areas in IGVs.

[0012] The present invention is implemented using the following technical solutions:

[0013] A method for detecting contamination in automated terminal horizontal transport equipment is proposed, including:

[0014] S1, Construct the IGV location dataset and the IGV dirty area dataset;

[0015] S2, Build and train a first-level convolutional neural network for detecting IGV from images;

[0016] S3, build and train a second-level convolutional neural network for detecting contaminated regions from IGV;

[0017] S4, the first-level convolutional neural network and the second-level convolutional neural network are cascaded to form a dual-branch cascaded detection model. The first-level convolutional neural network model detects IGV from the input image, and the second-level convolutional neural network model performs dirty area detection on the IGV image detected by the first-level convolutional neural network model.

[0018] S5 remaps the coordinates of all predicted bounding boxes for dirty areas sequentially, annotates the detected areas in the original image, and visualizes them.

[0019] In some embodiments of the present invention, the method further includes:

[0020] During the training of S3, an expression for the angular error is established. , The angle between the line connecting the center points of the dirt prediction box and the label box and the horizontal line is calculated when the center points of the two boxes are... When the values ​​of y and y are equal, the angle error between the two boxes is at its minimum. The error reaches its maximum when the value is equal to π / 4.

[0021] Based on the height difference between the center point coordinates of the prediction box and the center point coordinates of the annotation box And the distance between the center point coordinates of the predicted bounding box and the center point coordinates of the labeled bounding box. get ,Establish and , The expression:

[0022] ;

[0023] Establish the distance error loss expression ;in, Center point of the annotation box coordinates Subtract the center point of the prediction box coordinates With two frames Difference between maximum and minimum coordinates The square of the ratio; Center point of the annotation box coordinates Subtract the center point of the prediction box coordinates With two frames Difference between maximum and minimum coordinates The square of the ratio; These are hyperparameters, ranging from 0 to 2;

[0024] The final bounding box regression loss function is:

[0025] ;in, This only occurs when the prediction box and the label box overlap:

[0026] Wherein, TP represents the area where the predicted bounding box and the labeled bounding box overlap, indicating a true positive result; FP represents the area where the predicted bounding box and the labeled bounding box do not overlap, indicating a false positive result; and FN represents the area where the labeled bounding box and the predicted bounding box do not overlap, indicating a false negative result.

[0027] In some embodiments of the present invention, S4 includes:

[0028] The original image is input into the first-level convolutional neural network model to obtain the IGV bounding box.

[0029] IGV sub-images are cropped from the original image based on the IGV bounding boxes; if there are multiple IGVs, they are cropped separately and then resized to form multiple IGV sub-images.

[0030] The cropped IGV sub-image is input into the second-level convolutional neural network model to obtain the relative coordinates and category of the dirty region in the IGV sub-image.

[0031] In some embodiments of the present invention, S5 includes:

[0032] Define two-level scaling factor and ;in, , , ; and The resolutions of the input images are respectively , value; , , , and These are the resolutions of the cropped sub-images. , value;

[0033] The coordinate remapping mechanism for obtaining the IGV detection box is as follows:

[0034] ;

[0035] The coordinate remapping mechanism of the IGV contaminated area detection box is as follows:

[0036] ;

[0037] Based on the coordinate remapping mechanism, the coordinate information of all predicted bounding boxes is mapped sequentially, and the detection results are annotated in real time in the original image.

[0038] An automated terminal horizontal transport equipment dirt detection device is proposed, comprising:

[0039] Image acquisition unit, used to acquire images of the dock operation site;

[0040] The dual-branch cascaded detection unit is used to detect the dirty areas of IGVs in the dock operation site images acquired by the IGV image acquisition unit. The dual-branch cascaded detection model is pre-trained to detect the dirty areas of IGVs. In the dual-branch cascaded detection model, the first-level convolutional neural network model detects IGVs from the input image, and the second-level convolutional neural network model detects the dirty areas in the IGV image detected by the first-level convolutional neural network model.

[0041] The dirty area annotation unit is used to sequentially map the coordinate information of all dirty area prediction boxes, annotate the detection areas in the original image, and visualize them.

[0042] In some embodiments of the present invention, the apparatus further includes:

[0043] The prompting unit is used to count the number of dirty areas of an IGV and issue a maintenance prompt when the number of dirty areas exceeds a threshold.

[0044] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In the method and device for detecting contamination in horizontal transport equipment at automated terminals proposed in this invention, a first-level convolutional neural network is constructed for detecting IGVs from images, and a second-level convolutional neural network is constructed for detecting contaminated areas from IGVs. The first-level and second-level convolutional neural networks are constructed into a dual-branch cascaded detection model. The original image obtained from the automated terminal operation scene is used as the input of the dual-branch cascaded detection model. The first-level convolutional neural network identifies IGVs from complex scenes and outputs images with labeled IGV bounding boxes. Based on the bounding boxes, an IGV sub-image containing only IGV images is cropped from the image to remove background interference and transmit it to the second-level convolutional neural network. The second-level convolutional neural network focuses on detecting contaminated areas from the IGV sub-image, thereby improving the detection efficiency and accuracy of contaminated areas of IGVs.

[0045] Furthermore, this invention independently trains two neural network models and achieves the detection of dirty areas in IGV through cascading calls. It can make targeted improvements and optimizations to individual models, has strong scalability, and can add new levels, such as adding a vehicle type classification layer. It can also replace the detection layer algorithm, such as using a segmentation network to refine the dirty areas.

[0046] Furthermore, the dual-model detection mechanism achieves a lightweight design for the detection system. The first-level convolutional neural network model is trained with a large dataset to create a low-parameter and low-GFLOPS target detection model, while simultaneously meeting the requirements of high accuracy, lightweight design, and low latency for targets like IGV with obvious features and low differentiation, thus saving computing power. The second-level convolutional neural network model only detects dirty regions when IGV is detected, resulting in high detection efficiency.

[0047] Furthermore, compared to the shortcomings of existing technologies where the binary classification of clean IGV and dirty IGV is easily affected by viewing angle and lighting, resulting in blurred boundaries, the device of this invention sets a prompt threshold by statistically counting the number of dirty IGV areas (rather than global binary classification: clean IGV or dirty IGV), thereby quantifying the number of local dirty areas and transforming a subjective problem into an objective statistical problem, effectively avoiding subjective disputes over the overall degree of dirtiness.

[0048] Furthermore, the first-level neural network model of this invention only needs to learn the geometric features of IGV, while the second-level convolutional neural network model only needs to focus on the texture features of IGV. The two are decoupled, which reduces the learning difficulty and makes the accuracy of vehicle dirt detection higher than that of end-to-end dirt classification.

[0049] Furthermore, the annotation during training in this invention is performed in a hierarchical manner, with secondary annotation based on primary annotation. Secondary annotation only requires incremental annotation of the stolen goods area, thereby reducing the annotation cost of the system.

[0050] Other features and advantages of the present invention will become clearer after reading the detailed description of the embodiments of the present invention in conjunction with the accompanying drawings. Attached Figure Description

[0051] The accompanying drawings, as part of this invention, are provided to further illustrate the invention. The illustrative embodiments and descriptions are used to explain the invention but do not constitute an undue limitation thereof. Clearly, the drawings described below are merely some embodiments; those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0052] Figure 1 The following are the execution steps of the method for detecting dirt in automated horizontal transport equipment at a terminal proposed in this invention;

[0053] Figure 2 This is a schematic diagram illustrating the execution process of the method for detecting contamination in automated horizontal transport equipment at a wharf, as proposed in this invention.

[0054] Figure 3 This is a schematic diagram of the functional structure of the dirt detection device for automated dock horizontal transport equipment proposed in this invention;

[0055] Figure 4 This diagram illustrates the positional relationship between the predicted bounding box and the labeled bounding box during the training of a convolutional neural network using the method of the present invention.

[0056] It should be noted that these accompanying drawings and textual descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art by referring to specific embodiments. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the accompanying drawings. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0058] The present invention proposes a method for detecting contamination in automated terminal horizontal transport equipment, such as... Figure 1 As shown, it includes:

[0059] S1: Construct the IGV location dataset and the IGV dirty area dataset.

[0060] The construction of the IGV localization dataset includes:

[0061] (1) Collect environmental images containing IGV: Collect IGV images by scene changes, environmental changes, different working states, different models of IGV, different degrees of occlusion, and different working angles.

[0062] (2) Label the bounding boxes of IGVs in each image and generate a labeling file. The labeling file shall include at least the coordinates of the IGV bounding boxes, the IGV model, and the size.

[0063] (3) Divide the training set, validation set and test set according to the proportion.

[0064] The construction of the IGV contaminated area dataset includes:

[0065] (1) From the IGV localization dataset, crop the IGV region sub-map according to the labeled bounding box to ensure that the sub-map only contains IGV and its surface details.

[0066] (2) Label the contaminated areas in each IGV region sub-map and generate a label file. The label file shall include at least the coordinates of the contaminated areas, the type of contamination, and the percentage of contamination area.

[0067] (3) Divide the training set, validation set and test set according to the proportion.

[0068] S2: Build and train a first-level convolutional neural network for detecting IGV from images.

[0069] like Figure 2 The illustrated embodiment shows that the first-level convolutional neural network consists of an input layer, a group of convolutional and feature extraction layers, and a fully connected layer. The input layer receives the original image, including IGV, with an input image size of 640. 640 Taking 3 as an example; the convolution and feature extraction layer group extracts deep features of the input image step by step through multiple rounds of convolution operations, providing feature support for subsequent IGV judgment. It includes an initial convolutional layer and multiple rounds of convolution and pooling / downsampling layers; the initial convolutional layer contains several convolutional kernels, which perform preliminary convolution on the input image, compress the image size and extract low-level features, and reduce the output feature map size to 320. 320 N (N is the number of convolutional kernels); the multi-round convolution and pooling / downsampling layer consists of multiple alternating convolutional and pooling layers. Through repeated convolution-pooling operations, the feature map size is continuously reduced and the number of feature channels is increased, while extracting more abstract mid-level features, such as the contours and local structures of IGV. After multiple rounds of operations, the final output is 160. 160 The feature map is of size M (M is the current number of channels). The fully connected layer flattens the feature map extracted by the convolutional layer into a one-dimensional feature vector. It performs feature fusion and final classification through fully connected operations. It consists of a first fully connected layer, a second fully connected layer, and a third fully connected layer. The first fully connected layer performs the first fully connected fusion on the features extracted by the convolutional layer, mapping the two-dimensional features into a high-dimensional one-dimensional feature vector, capturing more global feature associations, and the output dimension is 4096. The second fully connected layer further performs deep fusion and non-linear transformation on the features to enhance the expressive power of the features and provide more discriminative features for the final classification. The output dimension is still 4096. The third fully connected layer outputs a scalar value based on the features fused by the first two fully connected layers. This value is mapped to between 0 and 1 through an activation function, representing the probability of IGV in the input image. The output judgment layer compares the probability value output by the third fully connected layer with a preset threshold to complete the final classification decision from feature extraction to whether IGV exists, providing a preliminary judgment basis for subsequent secondary dirty region detection. That is, the second-level convolutional neural network will be triggered to detect dirty regions of IGV only when IGV exists.

[0070] After constructing the first-level convolutional neural network model, the model is trained using a training set, including:

[0071] 1. Training data preparation: Geometric transformation and pixel transformation are used to enhance the IGV image training set to improve the model's generalization ability; the annotation files are converted to network input format, and the training set is divided according to batch size.

[0072] 2. Network initialization: The weight parameters of all convolutional and fully connected layers in the network are initialized using the normal initialization method, and the bias parameters are initialized to 0.

[0073] 3. Loss Function Definition: For IGV detection tasks (target localization and category determination), a composite loss function is designed. ;in, For bounding box regression loss, For category classification loss, The weighting coefficients are used to balance the regression and classification losses.

[0074] The bounding box regression loss can be calculated using the CIoU (Complete Intersection over Union) loss: ;in, For the prediction box, For the annotation box, For Euclidean distance, The length of the diagonal of the smallest rectangle enclosing the two frames. This is a shape similarity penalty term. It is used to optimize the positional deviation between the predicted bounding box and the labeled bounding box.

[0075] The category classification loss is calculated using multi-class cross-entropy with Sigmoid: middle, The i-th element in the one-hot vector of the true label (1 indicates IGV is present, 0 indicates it is absent). The value corresponding to the i-th class in the logits output by the network. This represents the predicted probability after applying the Sigmoid function. It is used to optimize classification accuracy for the presence or absence of IGV.

[0076] 4. Selecting the optimizer and learning rate strategy: The Adam optimizer and cosine annealing learning rate scheduling strategy are adopted.

[0077] 5. Iterative Training Process: Set the total number of training rounds and the optimal validation accuracy; in each round, input all training batches sequentially, and output the predicted bounding boxes for each batch of images through the network's forward computation. and classification value Based on labeled data ( Calculate the composite loss ; for composite loss The gradients of the parameters of each layer are obtained by taking the derivative, and the network weights are updated by the Adam optimizer. After every 5 rounds of training, the model performance is evaluated on the validation set.

[0078] 6. Model convergence determination and output: When the maximum number of training rounds is met or the early stopping mechanism is triggered, the weight file of the optimal model is output.

[0079] The first-level convolutional neural network is trained with a large dataset, resulting in low parameters and low GFLOPS (Gigafloating-point operations per second). This can simultaneously meet the requirements of high precision, lightweight design, and low latency for targets like IGV, which have distinct features and low differentiation, while saving computing power.

[0080] Furthermore, the first-level convolutional neural network accurately locates IGVs in complex scenes, providing reliable regional constraints for the second-level dirt detection and solving the problem of low localization accuracy caused by background interference in traditional single models.

[0081] S3: Build and train a second-level convolutional neural network for detecting contaminated regions from IGV.

[0082] The construction and training of the second-level convolutional neural network can refer to the content of the first-level convolutional neural network, and will not be elaborated upon in this embodiment of the invention. Of course, the first / second-level convolutional neural network can also be implemented using networks such as Fast R-CNN and YOLOv8, and this invention does not specifically limit it.

[0083] In the training of the second-level convolutional neural network model, this invention improves the bounding box regression loss function to enhance the localization accuracy of dirty areas:

[0084] The angle between the line connecting the center points of the dirt prediction box and the label box and the horizontal line. ,like Figure 4 As shown, it is known that when the center points of the two boxes are... Equal values ​​( =0) or the y-value is equal ( When =π / 2), the angular error between the two boxes is at its minimum. The error reaches its maximum when the angle is equal to π / 4. Based on this, the expression for the angle error is established as follows:

[0085] ;

[0086] Based on the height difference between the center point coordinates of the prediction box and the center point coordinates of the annotation box And the distance between the center point coordinates of the predicted bounding box and the center point coordinates of the labeled bounding box. get:

[0087] ;therefore ;

[0088] According to the formula for the difference of cosine squares:

[0089] ,

[0090] ;

[0091] Can be established and , The expression:

[0092] .

[0093] The distance error loss expression is:

[0094] ;

[0095] Center point of the annotation box Coordinate value minus the center point of the prediction box Coordinate values ​​and two boxes The square of the ratio of the difference between the maximum and minimum coordinates; where Center point of the annotation box Coordinate values Center point of the prediction box Coordinate values; Two frames The difference between the maximum and minimum coordinates.

[0096] Center point of the annotation box Coordinate value minus the center point of the prediction box Coordinate values ​​and two boxes The square of the ratio of the difference between the maximum and minimum coordinates; This is a hyperparameter, ranging from 0 to 2. Center point of the annotation box Coordinate values Prediction box center point Coordinate values Two frames The difference between the maximum and minimum coordinates is also the height difference between the center point coordinates of the predicted box and the center point coordinates of the labeled box.

[0097] The final bounding box regression loss function is:

[0098] ;in, This only occurs when the prediction box and the label box overlap:

[0099] Wherein, TP represents the area where the predicted bounding box and the labeled bounding box overlap, indicating a true positive result; FP represents the area where the predicted bounding box and the labeled bounding box do not overlap, indicating a false positive result; and FN represents the area where the labeled bounding box and the predicted bounding box do not overlap, indicating a false negative result.

[0100] S4: Concatenate the first-level convolutional neural network and the second-level convolutional neural network to form a dual-branch cascaded detection model. The first-level convolutional neural network model detects IGV from the input image, and the second-level convolutional neural network model performs dirty area detection on the IGV image detected by the first-level convolutional neural network model.

[0101] Specifically, this includes:

[0102] (1) Input the original image into the first-level convolutional neural network model to obtain the IGV bounding box.

[0103] (2) Based on the IGV bounding boxes, crop the IGV sub-images from the original image, resize them to 640×640, and use them as input to the second-level convolutional neural network model. If there are multiple IGVs, crop them separately and resize them to form multiple IGV sub-images.

[0104] (3) Input the cropped IGV sub-image into the second-level convolutional neural network model to obtain the relative coordinates and category of the dirty area in the IGV sub-image.

[0105] After the first-level convolutional neural network detects IGV, it segments the image within the IGV detection box, extracts the IGV region, separates the IGV region from the background, and transmits it to the second-level convolutional neural network to improve detection efficiency and accuracy, and can isolate false positive results in the environment.

[0106] S5: Remap all predicted bounding box coordinates sequentially, annotate the detection areas in the original image, and visualize them.

[0107] This embodiment uses coordinate remapping to accurately visualize and locate dirty areas. Specifically:

[0108] Define two-level scaling factor and ;in, , , ; and The resolutions of the input images are respectively , value;

[0109] , , , and These are the resolutions of the cropped sub-images. , value.

[0110] The final coordinate remapping mechanism for the IGV detection box is as follows:

[0111] ;

[0112] The coordinate remapping mechanism of the IGV contaminated area detection box is as follows:

[0113] .

[0114] Based on the coordinate remapping mechanism, the coordinate information of all predicted boxes is mapped sequentially, and the detection results are annotated in real time in the original image, accurately visualizing and locating the dirty areas on the IGV.

[0115] S6: Based on the test results, publish the cleaning and maintenance plan in the intelligent operation and maintenance system.

[0116] Based on the aforementioned IGV contamination detection method, this invention also proposes a contamination detection device for automated terminal horizontal transport equipment, such as... Figure 3 As shown, it includes:

[0117] The image acquisition unit is used to acquire images of the dock operation site.

[0118] The dual-branch cascaded detection unit is used to detect the dirty areas of IGVs in the dock operation site images acquired by the IGV image acquisition unit. The dual-branch cascaded detection model is pre-trained to detect the dirty areas of IGVs. In the dual-branch cascaded detection model, the first-level convolutional neural network model detects IGVs from the input image, and the second-level convolutional neural network model detects the dirty areas in the IGV image detected by the first-level convolutional neural network model.

[0119] The dirty area annotation unit is used to sequentially map the coordinate information of all dirty area prediction boxes, annotate the detection areas in the original image, and visualize them.

[0120] The prompting unit is used to count the number of dirty areas of an IGV and issue a maintenance prompt when the number of dirty areas exceeds a threshold.

[0121] The specific methods for detecting dirt in this device have been described in detail and will not be repeated here.

[0122] It should be noted that, in the specific implementation process, the above-mentioned control part can be implemented by a hardware processor executing computer-executable instructions in software form stored in memory, which will not be elaborated here. The programs corresponding to the actions performed by the above control circuit can all be stored in the computer-readable storage medium of the system in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0123] The computer-readable storage media mentioned above may include volatile memory, such as random access memory; may also include non-volatile memory, such as read-only memory, flash memory, hard disk or solid-state drive; and may also include combinations of the above types of memory.

[0124] The term "processor" as mentioned above can also refer to a collective of multiple processing elements. For example, a processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor, and it can also be a special-purpose processor.

[0125] It should be noted that the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for detecting dirt and grime in automated terminal horizontal transport equipment, characterized in that, include: S1, Construct the IGV location dataset and the IGV dirty area dataset; S2, Build and train a first-level convolutional neural network for detecting IGV from images; S3, build and train a second-level convolutional neural network for detecting contaminated regions from IGV; S4, the first-level convolutional neural network and the second-level convolutional neural network are cascaded to form a dual-branch cascaded detection model. The first-level convolutional neural network model detects IGV from the input image, and the second-level convolutional neural network model performs dirty area detection on the IGV image detected by the first-level convolutional neural network model. S5 remaps the coordinates of all predicted bounding boxes for dirty areas sequentially, annotates the detected areas in the original image, and visualizes them.

2. The method for detecting contamination in automated terminal horizontal transport equipment according to claim 1, characterized in that, The method further includes: During the training of S3, an expression for the angular error is established. , The angle between the line connecting the center points of the dirt prediction box and the label box and the horizontal line is calculated when the center points of the two boxes are... When the values ​​of y and y are equal, the angle error between the two boxes is at its minimum. The error reaches its maximum when the value is equal to π / 4. Based on the height difference between the center point coordinates of the prediction box and the center point coordinates of the annotation box And the distance between the center point coordinates of the predicted bounding box and the center point coordinates of the labeled bounding box. get ,Establish and , The expression: ； Establish the distance error loss expression ;in, Center point of the annotation box coordinates Subtract the center point of the prediction box coordinates With two frames Difference between maximum and minimum coordinates The square of the ratio; Center point of the annotation box coordinates Subtract the center point of the prediction box coordinates With two frames Difference between maximum and minimum coordinates The square of the ratio; These are hyperparameters, ranging from 0 to 2; The final bounding box regression loss function is: ;in, This only occurs when the prediction box and the label box overlap: Wherein, TP represents the area where the predicted bounding box and the labeled bounding box overlap, indicating a true positive result; FP represents the area where the predicted bounding box and the labeled bounding box do not overlap, indicating a false positive result; and FN represents the area where the labeled bounding box and the predicted bounding box do not overlap, indicating a false negative result.

3. The method for detecting dirt and grime in automated terminal horizontal transport equipment according to claim 1, characterized in that, S4 include: The original image is input into the first-level convolutional neural network model to obtain the IGV bounding box. IGV sub-images are cropped from the original image based on the IGV bounding boxes; if there are multiple IGVs, they are cropped separately and then resized to form multiple IGV sub-images. The cropped IGV sub-image is input into the second-level convolutional neural network model to obtain the relative coordinates and category of the dirty region in the IGV sub-image.

4. The method for detecting dirt and grime in automated terminal horizontal transport equipment according to claim 1, characterized in that, S5 include: Define two-level scaling factor and ;in, , , ; and The resolutions of the input images are respectively , value; , , , and These are the resolutions of the cropped sub-images. , value; The coordinate remapping mechanism for obtaining the IGV detection box is as follows: ； The coordinate remapping mechanism of the IGV contaminated area detection box is as follows: ； Based on the coordinate remapping mechanism, the coordinate information of all predicted bounding boxes is mapped sequentially, and the detection results are annotated in real time in the original image.

5. A dirt detection device for automated horizontal transport equipment at a dock, characterized in that, include: Image acquisition unit, used to acquire images of the dock operation site; The dual-branch cascaded detection unit is used to detect the dirty areas of IGVs in the dock operation site images acquired by the IGV image acquisition unit. The dual-branch cascaded detection model is pre-trained to detect the dirty areas of IGVs. In the dual-branch cascaded detection model, the first-level convolutional neural network model detects IGVs from the input image, and the second-level convolutional neural network model detects the dirty areas in the IGV image detected by the first-level convolutional neural network model. The dirty area annotation unit is used to sequentially map the coordinate information of all dirty area prediction boxes, annotate the detection areas in the original image, and visualize them.

6. The dirt detection device for automated terminal horizontal transport equipment according to claim 5, characterized in that, The device further includes: The alert unit is used to count the number of dirty areas on a single IGV and issue a maintenance alert when the number of dirty areas exceeds a threshold.

Images