Gate groove sill foreign matter detection method based on machine vision

Abstract

The invention provides a gate slot sill foreign matter detection method based on machine vision. The method comprises the following steps: acquiring the last three frames of a real-time video of a gate slot sill of a target gate at a certain frequency each time; performing bilateral filtering preprocessing on the last three frames of the real-time video to relieve image noise caused by insufficient underwater light and turbid water; an improved detection model insensitive to a target position is adopted to perform foreign matter detection on the last three frames of a real-time video, and the influence of underwater foreign matter shaking and foreign matter position real-time change on the detection precision is relieved; whether foreign matters exist in the last three frames of the real-time video or not is judged, if no foreign matters exist, other gate detection work is carried out, and if foreign matters exist in more than one frame of image, foreign matter position information is transmitted back, and follow-up gate groove sill cleaning work is carried out; the foreign matter detection effect on the gate number and the gate groove sill is more remarkable in severe scenes such as insufficient underwater light and turbid water.

Description

Technical Field

[0001] The present invention relates to the technical field of foreign body detection on the bottom sill of a gate slot of an underwater gate, and in particular to a method for detecting foreign body on the bottom sill of a gate slot based on machine vision. Background Art

[0002] As a device for controlling the flow of water or liquid, the gate has been widely used in water conservancy projects, ship locks, river management and other fields. It is made of strong and durable metal components and can withstand large water pressure and flow. The main function of the gate is to regulate and control the water flow. It is often used in flood control, water storage and irrigation, where the water level needs to be adjusted. When the water level needs to be controlled to rise, the gate opens to allow the water to pass through; when the water level needs to be controlled to drop, the gate closes to prevent the water from passing through. In the gate system, the gate slot is usually located at the bottom or side wall of the waterway or river to accommodate and support the gate, while the gate slot sill is the concave area at the bottom of the gate slot, usually presenting a horizontal or vertical concave shape. Its design and construction are crucial to the operation and sealing performance of the gate. It is usually made of concrete or metal materials, with sufficient strength and durability to cope with the pressure of the water flow and the weight of the gate, provide the support and guidance functions of the gate, and ensure the stability and sealing of the gate.

[0003] With the rapid development of my country's economy, water conservancy projects are increasing, and the use of gates and gate slots is becoming more and more extensive. Therefore, in some large-scale automation projects, the problem of foreign body detection on the bottom sill of the gate slot of the gate has become more and more important. Therefore, it is necessary to regularly check and maintain the flatness of the bottom sill of the gate slot, as well as detect and clean possible foreign objects or blockages to ensure the normal operation of the gate system. At present, manual methods are usually used for on-site investigation and regular inspection, but this method is inefficient and not only has serious safety hazards, but is also easily constrained by adverse conditions such as insufficient light and turbid water.

[0004] A Chinese patent with publication number CN110244373A discloses a "gate slot detection system and dedicated underwater robot", which is characterized in that the system includes a dedicated underwater robot for gate slot detection, a control unit and a lifting mechanism; the control unit controls the dedicated underwater robot for gate slot detection by transmitting power and control signals through a watertight cable; the robot is manually controlled to complete the detection of the gate slot and the cleaning of surface marine attachments. Relevant patents for automatic detection of foreign objects at the bottom of the gate slot in a complex underwater environment have not yet been proposed, and traditional target detection methods are difficult to cope with harsh scenes such as insufficient underwater light and turbid water, especially when the flow of water causes the position of foreign objects to change, the recognition and detection effect is very unsatisfactory. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a method for detecting foreign objects in the bottom sill of an underwater gate slot based on machine vision, which aims to solve the problem of how to accurately detect foreign objects in the bottom sill of a gate slot under adverse conditions such as insufficient underwater light, turbid water, and foreign objects shaking due to water flow.

[0006] In order to solve the above problems, the present invention proposes a method for detecting foreign objects at the bottom sill of a door groove based on machine vision, comprising the following steps:

[0007] S1, obtaining a video of the bottom sill area of ​​the underwater gate slot;

[0008] S2, intercept the last three frames of the real-time video of the gate bottom area at a certain frequency, marked as M t-1 、M t 、M t+1 , and the bilateral filtering method is used to preprocess the image;

[0009] S3, through the backbone feature extraction network, the improved detection model YOLOv7-tiny network of the lightweight network MobileNetV3 is used to detect foreign objects in the pre-processed images;

[0010] S4. Determine whether there are foreign objects in the last three frames of the real-time video. If there are no foreign objects, perform other gate detection work. If there is a foreign object in at least one frame of the image, send back the foreign object location information and perform subsequent gate slot bottom sill cleaning work.

[0011] Preferably, step S2 includes the following sub-steps:

[0012] S2.1, scan image M, M selection range is M t-1 、M t 、M t+1 , obtain the two-dimensional coordinates (i, j) and two-dimensional neighborhood coordinates (k, l) of each pixel processed by bilateral filtering noise reduction;

[0013] S2.2, calculate the Euclidean distance between the pixel point (i, j) and a point (k, l) in the neighborhood, and then obtain the spatial domain kernel d(i, j, k, l) of each pixel in the image M;

[0014] S2.3, calculate the absolute value of the pixel value difference between the pixel point (i, j) and a point (k, l) in the neighborhood, and then obtain the value range kernel r(i, j, k, l) of each pixel of the image M;

[0015] S2.4. Calculate the weight coefficient w(i,j,k,l) ​​of each pixel through the spatial domain kernel d(i,j,k,l) ​​and the range kernel r(i,j,k,l) ​​of the image M;

[0016] S2.5. According to the weight coefficient w(i,j,k,l) ​​of each pixel, image M t-1 、M t 、M t+1 Each pixel of the image is subjected to bilateral filtering, and the pixel value processed by bilateral filtering replaces the original pixel value.

[0017] Further preferably, the step S3 includes the following sub-steps:

[0018] S3.1. The processed image M t-1 、M t 、M t+1 Input to the improved detection model YOLOv7-tiny network and use the self-attention-based dynamic detection head Dyhead;

[0019] S3.2. Use the weighted intersection-over-union loss function WIoU, which is insensitive to the target position, to calculate the model bounding box loss to ensure that the model predicts the bounding box more focused on the foreign object target.

[0020] Further preferably, in step S3.1, MobileNetV3 converts the scale D F ×D F ×N feature maps after n D K ×D K ×1 convolution kernel operation to obtain m features, then input point convolution, and then after N convolution kernels of size 1×1×m operation, obtain a scale of D K ×D K ×1 features.

[0021] Further preferably, in the step S3.1, the improved detection model YOLOv7-tiny network includes a backbone feature extraction network Backbone, a neck network Neck, and a detection head Head. After the input foreign body image input is subjected to feature extraction through the simplified Backbone, feature maps of different depths are obtained, and then the feature maps are injected into the Neck part composed of the pyramid network FPN and the path aggregation network PAN to achieve multi-scale feature fusion, and then sent to the Head part for identification and positioning of the target foreign body.

[0022] Further preferably, in step S3.1, CBL is used to extract features of input data, ELAN is used to extract image features and generate pixel-level segmentation results, the size of the feature map is reduced by a pooling operation, and feature tensors from different sources or different sizes are connected together by a connection operation.

[0023] Further preferably, in step S3.1, Dyhead converts the attention function into three sequential attentions, each of which focuses on only one feature dimension, and the three types of attention are applied to detect the head in sequence and can be superimposed and utilized multiple times.

[0024] Further preferably, the step S4 further includes the following sub-steps:

[0025] S4.1. Determine the image M processed by the above steps t-1 、M t 、M t+1 Whether the prediction box P appears t-1 , P t , P t+1 ;

[0026] S4.2. If there is P t-1 , P t , P t+1 If there is at least one prediction box in the image, the coordinates of the upper left corner of the prediction box P and the length and width of the prediction box are returned as the location information of the foreign object and cleaned up;

[0027] S4.3. If no prediction box appears on the image, the bottom sills of other gate slots are detected.

[0028] Further preferably, the step S4.2 further includes the following sub-steps:

[0029] S4.2.1. If there is P t-1 , P t , P t+1 If any prediction box is found, the coordinates of the upper left corner of the prediction box P and the length and width of the prediction box are returned as the location information of the foreign object;

[0030] S4.2.2 If there is P t-1 , P t , P t+1 If any two prediction boxes or all prediction boxes are found, the intersecting area of ​​these prediction boxes is returned as a new prediction box P, and the coordinates of the upper left corner of the prediction box P and the length and width of the prediction box are returned as the location information of the foreign object;

[0031] S4.2.3. Carry out subsequent cleaning of the bottom sill of the gate slot.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The self-attention-based dynamic detection head Dyhead is used to enhance the model's ability to characterize foreign body features under different background interferences such as turbid water.

[0034] The lightweight network MobileNetV3 is used to perform convolution kernel operations on the feature map. Compared with the standard 3×3 convolution kernel, this depth-separable convolution is more sensitive to feature extraction, greatly improving the efficiency of feature extraction of foreign objects at the bottom of the door groove.

[0035] By improving the detection model YOLOv7-tiny network to extract the features of the input data, extract image features, generate pixel-level segmentation results, reduce the size of the feature map, and connect feature tensors from different sources or different sizes together, and then use the weighted intersection-over-union loss function WIoU, which is insensitive to the target position, to calculate the model bounding box loss, the model predicts that the bounding box is more focused on the foreign object target, thereby improving the detection accuracy when the foreign object is shaking. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of a method for detecting foreign matter on the bottom sill of an underwater gate slot based on machine vision according to the present invention;

[0037] Figure 2 A structural diagram of the specific process of realizing image feature mapping in the MobileNetV3 module in the improved detection model YOLOv7-tiny network of the present invention;

[0038] Figure 3 This is a structural diagram of the feature fusion layer and multi-scale detection output layer of the real-time detection algorithm in the improved detection model YOLOv7-tiny network in the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0040] The embodiment of the present invention proposes a method for detecting foreign matter at the bottom sill of a door groove based on machine vision. Figure 1 As shown, the following steps are included:

[0041] Step 1: Obtain a video of the bottom sill area of ​​the underwater gate;

[0042] Step 2: Capture the last three frames of the real-time video of the gate bottom area at a certain frequency, marked as M t-1 、M t 、M t+1 , and the bilateral filtering method is used to preprocess the image;

[0043] Step 3: Use the backbone feature extraction network to detect foreign objects in the preprocessed images using the improved detection model YOLOv7-tiny network of the lightweight network MobileNetV3;

[0044] Step 4: Determine whether there are foreign objects in the last three frames of the real-time video. If there are no foreign objects, perform other gate inspections. If there is a foreign object in at least one frame of the image, send back the foreign object location information and perform subsequent gate slot bottom sill cleaning.

[0045] Specifically, in step 1, a video of the gate slot sill area is captured by an underwater camera:

[0046] As a further solution, in step 2, when preprocessing the image using the bilateral filtering method, the following sub-steps are included:

[0047] (2.1) Scan image M, the range of M is M t-1 、M t 、M t+1 , obtain the two-dimensional coordinates (i, j) and two-dimensional neighborhood coordinates (k, l) of each pixel processed by bilateral filtering denoising; as data support for the subsequent calculation of the spatial domain kernel, value range kernel, and weight coefficient of each pixel;

[0048] (2.2) Calculate the Euclidean distance between the pixel point (i, j) and a point (k, l) in the neighborhood, and then obtain the spatial domain kernel d(i, j, k, l) of each pixel in the image M;

[0049] First, the Euclidean distance between the pixel point (i, j) and a point in the neighborhood (k, l) is calculated. The Euclidean distance is then calculated by summing the squares of the position differences. To adjust the attenuation rate of the distance, we can get the spatial kernel of each pixel. The mathematical description of the spatial kernel is as follows:

[0050]

[0051] Where i, j represent the horizontal and vertical coordinates of the pixel, k, l represent the horizontal and vertical coordinates of a point in the neighborhood of the pixel. d(i,j,k,l) ​​represents the spatial domain kernel of pixel (i,j). It is a parameter in the smoothing formula, which indicates the decay rate of spatial distance;

[0052] (2.3) Calculate the absolute value of the pixel value difference between the pixel point (i, j) and a point (k, l) in the neighborhood, and then obtain the range kernel r(i, j, k, l) of each pixel in the image M;

[0053] First, the absolute value of the pixel value difference between the pixel (i, j) and a point (k, l) in the neighborhood is calculated, and the absolute value of the pixel value difference is calculated by the square difference, and then To adjust the decay rate of the difference, we get the range kernel of each pixel. The mathematical description of the range kernel is as follows:

[0054]

[0055] Where i, j represent the horizontal and vertical coordinates of the pixel point, k, l represent the horizontal and vertical coordinates of a point in the neighborhood of the pixel point. f(k,l) represents the pixel value at the pixel point (k,l) in the image, r(i,j,k,l) ​​represents the value range kernel of the pixel (i,j), It is a parameter in the smoothing formula, which indicates the decay speed of pixel value difference;

[0056] (2.4) Calculate the weight coefficient w(i,j,k,l) ​​of each pixel through the spatial domain kernel d(i,j,k,l) ​​and the range kernel r(i,j,k,l) ​​of the image M;

[0057] The weight coefficient is the product of the spatial domain kernel d and the value domain kernel r, which is used to calculate the weight of the pixel (i, j) and a point (k, l) in the neighborhood. The size of the weight coefficient depends on the combined influence of the spatial distance and the pixel value difference. The mathematical description of the weight coefficient is as follows:

[0058] w(i,j,k,l)=d(i,j,k,l)*r(i,j,k,l)

[0059] Where i, j represent the horizontal and vertical coordinates of the pixel, k, l represent the horizontal and vertical coordinates of a point in the neighborhood of the pixel. w(i,j,k,l) ​​represents the weight coefficient of the pixel, and the size of the weight coefficient depends on the combined influence of spatial distance and pixel value difference. d(i,j,k,l) ​​represents the spatial domain kernel of pixel (i,j), and r(i,j,k,l) ​​represents the value domain kernel of pixel (i,j);

[0060] (2.5) According to the weight coefficient w(i,j,k,l) ​​of each pixel, the image M t-1 、M t 、M t+1 Perform bilateral filtering on each pixel, and replace the original pixel value with the pixel value processed by bilateral filtering; eliminate the noise caused by turbid water and insufficient light;

[0061] Bilateral filtering is a compromise between the spatial proximity of an image and the similarity of pixel values, while taking into account both the spatial difference and intensity difference of pixels. It is effective in dealing with problems such as insufficient underwater light and underwater noise affecting image quality. The mathematical description of bilateral filtering is as follows:

[0062]

[0063] Where i, j represent the horizontal and vertical coordinates of the pixel, k, l represent the horizontal and vertical coordinates of a point in the neighborhood of the pixel. f(k,l) represents the pixel value at the pixel point (k,l) in the image, w(i,j,k,l) ​​represents the pixel weight coefficient, and g(i,j) represents the new pixel value at the pixel (i,j) after bilateral filtering.

[0064] As a further solution, when using the improved detection model YOLOv7-tiny network for foreign body detection in step 3, the following sub-steps are included:

[0065] (3.1) The processed image M t-1 、M t 、M t+1 Input into the improved detection model YOLOv7-tiny network, and use the self-attention-based dynamic detection head Dyhead; enhance the model's ability to characterize foreign body features under different background interferences such as turbid water;

[0066] MobileNetV3 implements the feature mapping of images. The specific process is as follows Figure 2 As shown;

[0067] MobileNetV3 scales to D F ×D F ×N feature maps after n D K ×D K ×1 convolution kernel operation to obtain m features, then input point convolution, and then after N convolution kernels of size 1×1×m operation, obtain a scale of D K ×D K ×1 features; compared with the standard 3×3 convolution kernel, this depth-separable convolution is more sensitive to feature extraction, greatly improving the efficiency of feature extraction of foreign objects at the bottom of the door slot;

[0068] The specific operation steps of self-attention are as follows:

[0069] Dyhead converts the attention function into three sequential attentions, each of which focuses only on one feature dimension. Given a three-dimensional feature tensor F∈RL×S×C, we have:

[0070] W(F)=π C (π S (π L (F)·F)·F)·F

[0071] where π L (·),π s (·) and π c (·) represent different attention functions for three different dimensions of and C. The expressions of attention functions are:

[0072]

[0073] π c (F)·F-max(α 1 (F)·F c +β l (F),α 2 (F)·F c +β 2 (F)

[0074] where f(·) represents a linear function approximated by a 1×1 convolutional layer, and σ(·) represents the Hard-sigmoid function, indicating the number of sparse sampling positions; p k +Δp k In order to focus on the discrimination area, the offset Δp is increased through the self-learning space k The change position, Δm k Indicates position p k An important measure of self-learning. c represents the c-th channel feature slice, and max(·) represents a hyperfunction that first reduces the dimension of the global feature and then uses the output of the fully connected layer and the normalized layer. The three types of attention are applied to the detection head in sequence and can be superimposed multiple times to enhance the model's ability to express foreign body features under different background interferences such as turbid water.

[0075] The improved target detection model can more effectively adapt to the identification of foreign objects at the bottom of the gate slot in complex underwater environments. The feature fusion layer and multi-scale detection output layer structure of the improved YOLOv7-tiny real-time detection algorithm are as follows: Figure 3 As shown;

[0076] The improved YOLOv7-tiny is mainly composed of the backbone feature extraction network Backbone, the neck network Neck and the detection head Head. Among them, CBL stands for Convolutional Block Layer, which is the abbreviation of convolutional block layer. It consists of a series of convolutional layers, batch normalization layers and activation function layers, and is used to extract the features of the input data. ELAN stands for Encoder-Decoder with Lateral Connections, which is an encoder-decoder neural network model, in which the encoder is responsible for extracting image features, and the decoder is responsible for mapping the features back to the original image size and generating pixel-level segmentation results. MP stands for Max Pooling, which is a pooling operation used to reduce the size of the feature map. Concat is a connection operation that connects feature tensors from different sources or different sizes together. After the input foreign body image input is subjected to feature extraction by the simplified Backbone, feature maps of different depths are obtained, and then the feature maps are injected into the Neck part composed of the pyramid network FPN and the path aggregation network PAN to achieve multi-scale feature fusion, and then sent to the Head part for identification and positioning of the target foreign body;

[0077] (3.2) In order to solve the problem that the position of foreign objects at the bottom of the door slot changes in real time due to water flow, thus affecting the model detection anomaly, the weighted intersection-over-union loss function WIoU, which is insensitive to the target position, is used to calculate the model bounding box loss, so that the model predicts the bounding box to focus more on the foreign object target and improve the detection accuracy when the foreign object is shaking. The weighted intersection-over-union loss is calculated as follows:

[0078]

[0079]

[0080] Where W s and H s is the width and height of the minimum closed area of ​​the predicted box and the real box, * indicates that W s and H s Separate from the graph, locate the constant, and prevent gradients that hinder convergence.

[0081] As a further solution, when determining whether there is a foreign object in the door slot sill in step 4, the following sub-steps are also included:

[0082] (4.1) Determine the image M processed by the above steps t-1 、M t 、M t+1 Whether the prediction box P appears t-1 , P t , P t+1 ;

[0083] (4.2.1) If there exists P t-1 , P t , P t+1 If any prediction box is found, the coordinates of the upper left corner of the prediction box P and the length and width of the prediction box are returned as the location information of the foreign object;

[0084] (4.2.2) If there exists P t-1 , P t , P t+1 If any two prediction boxes or all prediction boxes are found, the intersecting area of ​​these prediction boxes is returned as a new prediction box P, and the coordinates of the upper left corner of the prediction box P and the length and width of the prediction box are returned as the location information of the foreign object;

[0085] (4.2.3) Carry out subsequent cleaning of the gate slot sill;

[0086] (4.3) If no prediction box appears on the image, the bottom sills of other gate slots are detected.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for detecting foreign matter in door groove sill based on machine vision, characterized in that: The steps include: S1, obtaining a video of the bottom sill area of ​​the gate slot of the underwater gate; S2, intercept the last three frames of the real-time video of the gate bottom area at a certain frequency, marked as M t-1 、M t 、M t+1 , and the bilateral filtering method is used to preprocess the image; S3, through the backbone feature extraction network, the improved detection model YOLOv7-tiny network of the lightweight network MobileNetV3 is used to detect foreign objects in the pre-processed images; S4. Determine whether there are foreign objects in the last three frames of the real-time video. If there are no foreign objects, perform other gate detection work. If there is a foreign object in at least one frame of the image, send back the foreign object location information and perform subsequent gate slot bottom sill cleaning work.

2. The method for detecting foreign matter in door groove sill based on machine vision as claimed in claim 1, characterized in that: The step S2 comprises the following sub-steps: S2.1, scan image M, M selection range is M t-1 、M t 、M t+1 , obtain the two-dimensional coordinates (i, j) and two-dimensional neighborhood coordinates (k, l) of each pixel processed by bilateral filtering denoising; S2.2, calculate the Euclidean distance between the pixel point (i, j) and a point (k, l) in the neighborhood, and then obtain the spatial domain kernel d(i, j, k, l) of each pixel in the image M; S2.3, calculate the absolute value of the pixel value difference between the pixel point (i, j) and a point (k, l) in the neighborhood, and then obtain the value range kernel r(i, j, k, l) of each pixel of the image M; S2.

4. Calculate the weight coefficient w(i,j,k,l) ​​of each pixel through the spatial domain kernel d(i,j,k,l) ​​and the range kernel r(i,j,k,l) ​​of the image M; S2.

5. According to the weight coefficient w(i,j,k,l) ​​of each pixel, image M t-1 、M t 、M t+1 Each pixel of the image is subjected to bilateral filtering, and the pixel value processed by bilateral filtering replaces the original pixel value.

3. The method for detecting foreign matter in door groove sill based on machine vision as claimed in claim 1, characterized in that: The step S3 comprises the following sub-steps: S3.

1. The processed image M t-1 、M t 、M t+1 Input to the improved detection model YOLOv7-tiny network and use the self-attention-based dynamic detection head Dyhead; S3.

2. Use the weighted intersection-over-union loss function WIoU, which is insensitive to the target position, to calculate the model bounding box loss to ensure that the model predicts the bounding box more focused on the foreign object target.

4. The method for detecting foreign matter in door groove sill based on machine vision as claimed in claim 3, characterized in that: In step S3.1, MobileNetV3 converts the scale to D F ×D F ×N feature maps after n D K ×D K ×1 convolution kernel operation to obtain m features, then input point convolution, and then after N convolution kernels of size 1×1×m operation, obtain a scale of D K ×D K ×1 features.

5. The method for detecting foreign matter in door groove sill based on machine vision as claimed in claim 4, characterized in that: In the step S3.1, the improved detection model YOLOv7-tiny network includes a backbone feature extraction network Backbone, a neck network Neck, and a detection head Head. After the input foreign body image input is subjected to feature extraction by the simplified Backbone, feature maps of different depths are obtained, and then the feature maps are injected into the Neck part composed of the pyramid network FPN and the path aggregation network PAN to achieve multi-scale feature fusion, and then sent to the Head part for identification and positioning of the target foreign body.

6. The method for detecting foreign matter in door groove sill based on machine vision as claimed in claim 5, characterized in that: In step S3.1, CBL is used to extract features of input data, ELAN is used to extract image features and generate pixel-level segmentation results, the size of the feature map is reduced through a pooling operation, and feature tensors from different sources or different sizes are connected together through a connection operation.

7. The method for detecting foreign matter in door groove sill based on machine vision as claimed in claim 3, characterized in that: In step S3.1, Dyhead converts the attention function into three sequential attentions, each of which focuses on only one feature dimension. The three types of attention are applied to detect the head in sequence and can be superimposed and utilized multiple times.

8. The method for detecting foreign matter in door groove sill based on machine vision as claimed in claim 6, characterized in that: The step S4 further comprises the following sub-steps: S4.

1. Determine the image M processed by the above steps t-1 、M t 、M t+1 Whether the prediction box P appears t-1 , P t , P t+1 ; S4.

2. If there is P t-1 , P t , P t+1 If there is at least one prediction box in the image, the coordinates of the upper left corner of the prediction box P and the length and width of the prediction box are returned as the location information of the foreign object and cleaned up; S4.

3. If no prediction box appears on the image, the bottom sills of other gate slots are detected.

9. The method for detecting foreign matter in door groove sill based on machine vision as claimed in claim 8, characterized in that: The step S4.2 further includes the following sub-steps: S4.2.

1. If there is P t-1 , P t , P t+1 If any prediction box is found, the coordinates of the upper left corner of the prediction box P and the length and width of the prediction box are returned as the location information of the foreign object; S4.2.2 If there is P t-1 , P t , P t+1 If any two prediction boxes or all prediction boxes are found, the intersecting area of ​​these prediction boxes is returned as a new prediction box P, and the coordinates of the upper left corner of the prediction box P and the length and width of the prediction box are returned as the location information of the foreign object; S4.2.

3. Carry out subsequent cleaning of the bottom sill of the gate slot.

Images