Gap detection method, device, equipment and storage medium based on visual detection

By combining laser irradiation and YOLOV5 model image processing technology in gap detection, the problems of low efficiency, limited accuracy and high cost of traditional gap detection methods are solved, and automated, fast and accurate gap detection is achieved.

CN118505699BActive Publication Date: 2024-11-22无锡宇宁科技集团股份有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410962405.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-22
Estimated Expiration
2044-07-18

AI Technical Summary

Technical Problem

Traditional gap detection methods rely on manual visual inspection and mechanical measurement tools, and have problems such as subjectivity, low efficiency, accuracy limitation and high cost, especially on high-yield production lines, which are difficult to meet demand.

Method used

Using a visual detection method, the image data is extracted and analyzed by laser light and the pre-trained model YOLOV5 model (including the bottom module, the middle module and the top module), and the gap is triangulated by combining the line laser and the camera. , reduce the demand for high-precision cameras.

Benefits of technology

It realizes automated gap detection, improves detection speed and accuracy, reduces detection costs, overcomes the expensive problem of high-precision cameras, and is suitable for the detection of complex shapes and tiny gaps.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118505699B_ABST
    Figure CN118505699B_ABST
Patent Text Reader

Abstract

The present application discloses a gap detection method, device, equipment and storage medium based on visual detection. The present application relates to the field of image processing technology. The gap detection method includes irradiating the position to be tested of the object to be tested by a line laser, and recording the image data corresponding to the position to be tested by a camera; inputting the image data into a pre-trained model for judgment, and outputting the judgment result, the judgment result includes the gap distance of the object to be tested, the pre-trained model is a YOLOV5 model, and the pre-trained model includes a bottom module, a middle module and a top module; according to the judgment result, the feedback result is obtained by comparing with the preset distance threshold. The present application realizes the intelligent processing of gap detection through the pre-trained model, which can reduce the cost of analysis and detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a gap detection method, device, equipment and storage medium based on visual detection. Background Art

[0002] Traditional gap detection methods mainly rely on manual visual inspection or simple mechanical measurement tools. These methods have multiple limitations, including but not limited to:

[0003] Subjectivity: Manual visual inspection is highly dependent on the operator’s experience and concentration and is susceptible to individual differences and fatigue, leading to inconsistent and unreliable inspection results.

[0004] Inefficiency: Manual inspection is slow and cannot meet the needs of large-scale production, especially on high-volume production lines.

[0005] Accuracy limitations: While mechanical measurement tools can provide consistency, they often cannot accommodate complex or irregularly shaped objects and have limited ability to detect tiny gaps.

[0006] Cost issue: With the rise of labor costs and the intensification of market competition, companies are increasingly in need of seeking more cost-effective testing solutions.

[0007] With the development of computer vision and deep learning technologies, automated detection methods have gradually become possible. These technologies can simulate the human visual system and automatically extract features from images and analyze them through algorithms. However, the application of algorithms in gap detection still faces challenges: high-precision cameras are required to observe the objects to be tested, but high-precision cameras are expensive and have not significantly reduced costs compared to manual inspection.

[0008] Therefore, how to reduce the cost of gap detection is a problem that needs to be solved at present. Summary of the invention

[0009] The main purpose of this application is to provide a gap detection method, device, equipment and storage medium based on visual detection, aiming to solve the technical problem of how to reduce the cost of gap detection.

[0010] To achieve the above objectives, the present application provides a gap detection method based on visual detection, and the gap detection method based on visual detection includes:

[0011] Acquire image data corresponding to a position to be measured, where the position to be measured is a position on the object to be measured where a line laser is irradiated;

[0012] Input the image data into a pre-trained model for judgment, and output a judgment result, wherein the judgment result includes a gap distance of the object to be measured, wherein the pre-trained model is a YOLOV5 model, and the pre-trained model includes a bottom module, a middle module, and a top module;

[0013] The judgment result is compared with a preset distance threshold to obtain a feedback result.

[0014] In one embodiment, the step of inputting the image data into a pre-trained model for judgment and outputting the judgment result comprises:

[0015] Extracting features of the image data through the bottom module to obtain image features, wherein the image features are extracted through at least one learning channel;

[0016] Up-sampling and down-sampling the image features through the intermediate module to obtain up-sampled features and down-sampled features;

[0017] The up-sampled features and the down-sampled features are concatenated to obtain a feature pyramid;

[0018] The top module performs calculations based on the feature pyramid to obtain the judgment result and outputs the judgment result.

[0019] In one embodiment, a compression activation module is added after each bottleneck layer of the bottom module, wherein the compression activation module comprises a fully connected layer;

[0020] The step of extracting features from the image data by the bottom module to obtain image features comprises:

[0021] The spatial dimension of the image features is compressed by global average pooling to obtain a pooling vector;

[0022] Inputting the pooled vector into the fully connected layer, learning the dependency between the learning channels through the fully connected layer, and generating channel weights corresponding to the learning channels;

[0023] Normalizing the channel weights by using a preset activation function to obtain activation channel weights;

[0024] The image features are weighted by the activated channel weights to obtain channel-weighted image features.

[0025] In one embodiment, a convolutional block attention module is added to the intermediate module;

[0026] The step of weighting the image features by the activation channel weights to obtain the weighted image features comprises:

[0027] The spatial relationship of image features is learned through maximum pooling and average pooling to obtain the spatial weight of each pixel;

[0028] Normalizing the spatial weights by using a preset activation function to obtain activation spatial weights;

[0029] The channel-weighted image features are spatially weighted by using the activation space weights to obtain channel- and space-weighted image features.

[0030] In one embodiment, the bottom module includes a convolution block configuration module, and the convolution block configuration module includes a routing block unit, a first convolution layer, a second convolution layer, and a linear correction unit;

[0031] The method further comprises:

[0032] Merging the image features through the routing block unit to obtain merged image features;

[0033] Performing a convolution operation on the merged image features through the first convolution layer, and activating through the linear correction unit to obtain a first activation output, wherein the size of the filter of the first convolution layer is 1x1, the step value is 1, and the padding value is 0;

[0034] Performing a convolution operation on the first activation output through the second convolution layer, and activating through the linear correction unit to obtain a second activation output, wherein the filter size of the second convolution layer is 3x3, the step value is 2, and the padding value is 1;

[0035] The second activation output is merged through the routing block unit to obtain multi-scale image features.

[0036] In one embodiment, the top module performs calculations according to the feature pyramid to obtain the judgment result, including:

[0037] Constructing a bounding box based on the feature pyramid;

[0038] Performing non-maximum suppression calculation on the bounding box to obtain a confidence score for each bounding box;

[0039] sorting the bounding boxes based on the confidence score of each bounding box to obtain a sorted list;

[0040] Calculating the overlap of the bounding boxes according to the sorted list to obtain the overlap corresponding to each bounding box;

[0041] Adjusting the confidence score according to the overlap to obtain an adjusted confidence score;

[0042] Reordering the bounding boxes based on the adjusted confidence scores, and selecting a to-be-detected box, wherein the to-be-detected box is the bounding box with the highest confidence score after the reordering;

[0043] A distance measurement calculation is performed on the image data based on the frame to be detected to obtain a determination result.

[0044] In one embodiment, the step of extracting features from the image data by the bottom module to obtain image features further includes:

[0045] Performing data enhancement processing on the image data to obtain enhanced image data, wherein the data enhancement processing includes at least one of random cropping, brightness adjustment, and adding noise;

[0046] Feature extraction is performed on the enhanced image data to obtain image features.

[0047] In addition, to achieve the above-mentioned purpose, the present application also provides a gap detection device, the gap detection device comprising:

[0048] An image data acquisition module, used to acquire image data corresponding to a position to be measured, where the position to be measured is a position on the object to be measured where the line laser is irradiated;

[0049] A judgment result output module, used for inputting the image data into a pre-trained model for judgment, and outputting a judgment result, wherein the judgment result includes a gap distance of the object to be measured, and the pre-trained model is a YOLOV5 model, and the pre-trained model includes a bottom module, a middle module and a top module;

[0050] The distance comparison module is used to compare the judgment result with a preset distance threshold to obtain a feedback result.

[0051] In addition, to achieve the above-mentioned purpose, the present application also provides a gap detection device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the gap detection device is used to execute the steps of the gap detection method based on visual detection as described above.

[0052] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and the computer-readable storage medium stores a program for implementing a gap detection method based on visual detection, and the program for implementing the gap detection method is executed by a processor to implement the steps of the gap detection method as described above.

[0053] The present application provides a gap detection method based on visual detection. The present application first irradiates the position to be tested of the object to be tested with a laser, and records the laser reflection at the position to be tested with a camera. The image data recorded by the camera is then input into a pre-trained model for judgment to obtain the gap distance of the position to be tested. The gap distance of the position to be tested is compared with the preset distance threshold to determine whether the object to be tested meets the requirements. The present application reduces the demand for camera clarity by using a simple combination of laser irradiation and a camera, thereby reducing the cost of gap detection.

[0054] In summary, the present application combines laser irradiation and a camera to measure the gap using triangulation, thereby overcoming the technical defect of the prior art that a high-precision camera is required and reducing the cost of gap detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0057] Figure 1 A schematic diagram of a flow chart of an embodiment of a gap detection method based on visual detection provided by the present application;

[0058] Figure 2 A schematic diagram of a common scenario provided for an embodiment of a gap detection method based on visual detection of the present application;

[0059] Figure 3 A schematic diagram of a flow chart of an embodiment of a gap detection method based on visual detection in the present application;

[0060] Figure 4 A distance measurement schematic diagram provided for an embodiment of a gap detection method based on visual detection in the present application;

[0061] Figure 5 This is a schematic diagram of the module structure of the gap detection device according to an embodiment of the present application;

[0062] Figure 6 It is a schematic diagram of the structure of the gap detection device in the embodiment of the present application.

[0063] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0064] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0065] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0066] The main solution of the present application is: obtaining image data corresponding to the position to be measured, where the position to be measured is the position where the laser is irradiated on the object to be measured; inputting the image data into a pre-trained model for judgment, and outputting a judgment result, where the judgment result includes the gap distance of the object to be measured; and comparing the judgment result with a preset distance threshold to obtain a feedback result.

[0067] At present, the detection equipment of the automated gap detection method relies on the quality of the data. It can be said that the original data has a great influence on the final training results of the model. Therefore, it is necessary to obtain high-definition image data through a high-definition camera to ensure the accuracy of recognition.

[0068] The gap detection method based on visual detection in the present application combines line laser and camera to perform triangulation measurement of gaps, thus overcoming the technical defect of the prior art that high-precision cameras are required and reducing the cost of gap detection.

[0069] It should be noted that the execution subject of this embodiment may be a gap detection device, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a gap detection device capable of realizing the above functions, etc., and this embodiment does not specifically limit this. The following takes the gap detection device as the execution subject as an example to illustrate this embodiment and the following embodiments.

[0070] Based on this, the present application proposes a gap detection method based on visual detection according to an embodiment, please refer to Figure 1 , the gap detection method includes steps S10 to S30:

[0071] Step S10, obtaining image data corresponding to the position to be measured, where the position to be measured is the position on the object to be measured where the laser is irradiated;

[0072] It should be noted that the object to be tested refers to any type of object or material that needs to be tested for gaps, which may have gaps due to defects in their production, processing or use. The object to be tested can be an industrial product, a mechanical part or other entity that needs to be tested to ensure integrity and safety, and the image data refers to the digital image information of the object to be tested under laser irradiation conditions.

[0073] It can be understood that the line laser is emitted by a laser emission source, which can be a solid-state laser, a semiconductor laser, a fiber laser or other types of lasers, and the image data can be acquired by a camera.

[0074] Through the combination of a laser emission source and a camera, the camera clarity requirements can be lowered compared to obtaining image data through a single camera, and there is no need for an expensive high-definition camera, thereby reducing the cost of gap detection.

[0075] For example, in order to help understand the technical concept or technical principle of the gap detection method based on visual detection in this embodiment, please refer to FIG. Figure 2 A common scenario diagram of a gap detection method based on visual detection is provided as follows:

[0076] A is a laser pen used to emit line laser, B is a camera used to record the image of the position to be tested, and C is the mobile phone to be tested.

[0077] First, a laser pen A emits a line laser to the product C to be tested, irradiating the position to be tested, and then a camera B obtains image data of the position to be tested.

[0078] It should be noted that the above examples are only used to assist in understanding the present application and do not constitute a limitation on the gap detection method based on visual detection in the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0079] Step S20, inputting the image data into a pre-trained model for judgment, and outputting a judgment result, wherein the judgment result includes a gap distance of the object to be measured, wherein the pre-trained model is a YOLOV5 model, and the pre-trained model includes a bottom module, a middle module, and a top module;

[0080] It should be noted that the pre-trained model is a deep learning model that is pre-trained on a large amount of labeled data to learn the recognition of surface features and gaps of the object to be tested. The judgment result refers to the data obtained after analyzing the acquired image data through the pre-trained model. The judgment result reflects key information such as the position, size and distance of the gap on the object to be tested.

[0081] It is understandable that the pre-trained model can be a YOLOV5 model or a deep learning model improved based on the YOLOV5 model.

[0082] By inputting image data into the pre-trained model and outputting the judgment result including the gap distance of the object to be tested, this process is highly automated, significantly improving the detection speed and accuracy, while reducing the dependence on manual detection and further reducing the cost of gap detection.

[0083] In a feasible embodiment, the pre-trained model includes a bottom module, a middle module and a top module.

[0084] It should be noted that the bottom module corresponds to Backbone, which is the basic part of the network and is responsible for extracting preliminary features from the input data. In this embodiment, the bottom module may include a series of convolutional layers, activation functions, and pooling layers to build the cornerstone of feature extraction for the entire network. The middle module corresponds to Neck, which is the middle part connecting Backbone and Head, and is responsible for further feature processing and information fusion. The top module corresponds to Head, which is the final part of the convolutional neural network and is responsible for making specific predictions or classifications based on the features transferred from Backbone and Neck.

[0085] For the same or similar contents as those in the above embodiments, please refer to the above description and will not be described in detail later. Figure 3 Step S20 inputs the image data into the pre-trained model for judgment and outputs the judgment result, including steps S201 to S204:

[0086] Step S201, extracting features of image data through a bottom module to obtain image features, where the image features are extracted through at least one learning channel;

[0087] It should be noted that the bottom module refers to the network structure that forms the basis of deep learning, which is responsible for extracting image features from the input image data. The learning channel refers to the independent processing path used for feature extraction in the deep neural network. Each learning channel corresponds to a specific attribute of the original image, such as color, texture or shape. Image features refer to the feature representation obtained through feature extraction. Each learning channel will generate corresponding image features. In the case of multiple learning channels, there will also be multiple corresponding image features.

[0088] It can be understood that feature extraction is a crucial process, which involves automatically discovering and extracting important information from input data. In this embodiment, feature extraction is used to extract image data to obtain image features that can reflect the basic characteristics of the image data, such as image features that reflect the shape of the image data.

[0089] In a feasible embodiment, a compression activation module is added after each bottleneck layer of the bottom module, and the compression activation module includes a fully connected layer;

[0090] It should be noted that the bottom module is equivalent to the backbone network, which refers to the feature extraction framework that constitutes the core of the model. It automatically extracts key features from the input image data through a series of structured layers. This embodiment is an improvement on the backbone network, and can be improved based on the existing backbone network, such as Darknet-53 or Vision Transformer. The bottleneck layer is a special convolutional layer design that can reduce the number of parameters and computational complexity while maintaining feature expression capabilities.

[0091] It can be understood that the Squeeze-and-Excitation module is a structure that explicitly models dependencies in the learning channel dimension. It measures the importance of each learning channel by learning a set of channel weights and re-adjusts the image features of the learning channel accordingly. This method allows the network to pay more attention to important feature channels.

[0092] Step S201, after extracting features of the image data through the bottom module and obtaining the image features, it also includes steps A10 to A40:

[0093] Step A10, compressing the spatial dimension of the image features by global average pooling to obtain a pooling vector;

[0094] It should be noted that global average pooling is a pooling operation commonly used in deep learning. Its function is to compress the spatial dimensions (height and width) of each channel into a single global feature (pooling vector) while retaining the learning channel.

[0095] Step A20, inputting the pooled vector into the fully connected layer, learning the dependency between the learning channels through the fully connected layer, and generating channel weights corresponding to the learning channels;

[0096] It should be noted that the fully connected layer is a layer that does not consider the spatial structure of the input data. In the fully connected layer, each neuron is connected to all neurons in the previous layer, and each neuron has its own weight. A neuron is an element in a learning channel, and a learning channel can be regarded as a collection of multiple neurons. Therefore, the fully connected layer connection can capture the dependencies between learning channels.

[0097] It is understandable that the channel weights obtained through the fully connected layer can reflect the importance of each learning channel to the model decision, but the channel weights are the original values ​​output by the fully connected layer and do not have nonlinear characteristics. The true relationship of the data is nonlinear, so a model with only linear features may not be able to accurately capture the relationship between channels.

[0098] Step A30, normalizing the channel weights by a preset activation function to obtain activation channel weights;

[0099] It should be noted that the preset activation function refers to the sigmoid activation function, which is used to normalize the channel weights, compress the channel weights to the (0,1) interval, and obtain the activated channel weights, so as to more accurately represent the importance of each learning channel.

[0100] Step A40, weighting the image features by using the activated channel weights to obtain channel-weighted image features.

[0101] This embodiment weights the image features by activating channel weights. The weighted image features contain a quantitative representation of the importance of each channel, thereby enhancing the model's focus on important features while suppressing unimportant features, thereby reducing the introduction of additional parameters and computational costs.

[0102] In a feasible embodiment, a convolutional block attention module is added to the intermediate module;

[0103] It should be noted that the convolutional block attention module refers to the CBAM (Convolutional Block Attention Module) module, which can focus on important information in both channel and spatial dimensions.

[0104] Step A40, after weighting the image features by the activation channel weights to obtain the channel-weighted image features, further includes steps B10 to B30:

[0105] Step B10, learning the spatial relationship of image features through maximum pooling and average pooling to obtain the spatial weight of each pixel;

[0106] It should be noted that max pooling is a technique that enhances the feature recognition capability of deep learning models through a spatial attention mechanism. By dividing the image features into windows and selecting the largest pixel value in each window as the representative value of the window, max pooling emphasizes the most significant features, which helps the network identify important spatial information. Pixels are the smallest unit in a digital image and can be understood as the "atom" of an image. Each pixel usually corresponds to a point on a display or represents a specific location in an image file. As the basic building block of an image, pixels provide a rich data foundation for image analysis through their color and position information.

[0107] It can be understood that the spatial relationship between pixels is understood by extracting the local maximum and average values ​​of the image features through maximum pooling and average pooling respectively. The image features of the maximum pooling and the image features of the average pooling are concatenated to generate new image features, and then the new image features are learned through the convolution layer to obtain the spatial weights.

[0108] Step B20, normalizing the spatial weights by a preset activation function to obtain activation spatial weights;

[0109] It should be noted that the preset activation function refers to the Sigmoid activation function, and the activation space weight refers to the normalized space weight.

[0110] It can be understood that by normalizing the output of the convolutional layer using the Sigmoid activation function, it will ensure that the weight value is between 0 and 1. The spatial weight of each pixel can be regarded as a measure of the importance of that pixel.

[0111] Step B30, spatially weighting the channel-weighted image features by activating spatial weights to obtain channel- and spatial-weighted image features.

[0112] It can be understood that the activation space weight is multiplied pixel by pixel with the channel-weighted image feature to obtain a new image feature, which contains a quantized representation of the spatial importance of each pixel, thereby enhancing the model's attention to spatial positions.

[0113] Since the spatial weighting is based on channel-weighted image features, the CBAM module focuses on important information in both channel and spatial dimensions, thereby improving the performance of the pre-trained model.

[0114] In a feasible embodiment, the bottom module includes a convolution block configuration module, and the convolution block configuration module includes a routing block unit, a first convolution layer, a second convolution layer, and a linear correction unit;

[0115] It should be noted that the convolution block configuration module is part of the bottom module, which is responsible for defining and configuring the convolution block parameters. The routing block unit is responsible for routing data in the network, which may involve merging or diverting data to meet different processing requirements. In this embodiment, it is used to merge feature maps from different layers to build a multi-scale feature pyramid. The first convolution layer is used to adjust the number of channels while keeping the spatial dimension unchanged to achieve linear transformation of features. The second convolution layer is used to extract more abstract features and may change the spatial size of the feature map. Linear correction unit refers to activation functions such as ReLU, which are used to introduce nonlinearity to help the network learn complex feature representations.

[0116] The gap detection method based on visual detection in this embodiment includes steps C10 to C40:

[0117] Step C10, merging the image features through the routing block unit to obtain merged image features;

[0118] It is understandable that the features of different learning channels need to be deeply learned through different convolution blocks. The convolution blocks can be named, and different convolution blocks can be selected through the routing block to combine and merge the image features.

[0119] Step C20, performing a convolution operation on the merged image features through a first convolutional layer, and activating through a linear correction unit to obtain a first activation output, the filter size of the first convolutional layer is 1x1, the step value is 1, and the padding value is 0;

[0120] It should be noted that the first activation output refers to the output of the first convolutional layer after being processed by the activation function.

[0121] It can be understood that setting the step value to 1 and the padding value to 0 can ensure that the spatial size of the image features remains unchanged. Activating the linear correction unit can enhance the network's expressiveness and solve the gradient vanishing problem.

[0122] Step C30, performing a convolution operation on the first activation output through a second convolutional layer, and activating through a linear correction unit to obtain a second activation output, the filter size of the second convolutional layer is 3x3, the step value is 2, and the padding value is 1;

[0123] It should be noted that the second activation output refers to the result of the output of the second convolutional layer after being processed by the activation function.

[0124] It can be understood that the filter size of the second convolutional layer is 3x3 to extract deeper feature information, the step value is 2, and the padding value is 1 to reduce the spatial size of the image features and reduce the computational complexity. As in step C20, the output of the second convolutional layer is activated using a linear rectification unit to obtain a second activated output.

[0125] Step C40: merging the second activation outputs through the routing block unit to obtain multi-scale image features.

[0126] It should be noted that multi-scale image features refer to image features extracted at different scales. Multi-scale image features can reflect the information of images at different scales, including but not limited to shape, texture, color and spatial relationship.

[0127] It is understandable that the routing block unit here may be different from the routing block unit in step C10, and different convolution blocks may be selected for synthesis to obtain features with more size fusion.

[0128] By configuring the convolution block parameters of the gap detection method based on visual detection in this embodiment, image features can be captured at different scales without designing a complete network path for each scale separately, thereby reducing the overall parameter amount of the model, thereby achieving lightweight model, so that the model can complete training and reasoning tasks faster on resource-constrained devices, and the use of activation functions can improve the training efficiency of the network and reduce the training time, thereby indirectly contributing to the lightweight model.

[0129] In addition, the number of parameters also affects the generalization ability and robustness of the model. Too many parameters may cause the model to overfit and reduce its generalization ability; while reducing the number of parameters helps prevent overfitting and improves the robustness of the pre-trained model.

[0130] In another feasible embodiment, step S201 includes steps D10 to D20:

[0131] Step D10, performing data enhancement processing on the image data to obtain enhanced image data, wherein the data enhancement processing includes at least one of random cropping, brightness adjustment, and noise addition;

[0132] It should be noted that data enhancement processing refers to a processing method that randomly transforms image data to increase data diversity. Enhanced image data refers to an image data set that is expanded after data enhancement processing, including image data before processing and image data obtained by data enhancement processing.

[0133] It can be understood that random cropping is specifically to randomly crop the original image to generate image fragments of different sizes and positions. Brightness adjustment is to change the brightness of the image to simulate target detection scenarios under different lighting conditions. Adding noise is to add random noise to the image.

[0134] Step D20, extracting features from the enhanced image data to obtain image features.

[0135] It should be noted that the image features in this embodiment refer to image features extracted from enhanced image data after data enhancement.

[0136] The gap detection method in this embodiment is based on visual detection. By increasing the image data through data enhancement processing, that is, increasing the number of samples, the pre-trained model's detection capability for targets of different scales is increased, and the pre-trained model is helped to learn to accurately detect targets even under noise interference, thereby further improving the accuracy and stability of detection, realizing end-to-end deployment, and reducing its negative impact on inference latency.

[0137] Step S202, upsampling and downsampling the image features through the intermediate module to obtain upsampled features and downsampled features;

[0138] It should be noted that upsampling refers to a sampling method that increases the spatial resolution of a feature map, and downsampling refers to a sampling method that reduces the spatial size of a feature map. Upsampled features are image features extracted by upsampling, and downsampled features are image features extracted by downsampling.

[0139] By upsampling, the spatial resolution of image features can be restored and important details can be retained; by downsampling, higher-level abstract features can be extracted. This design not only improves the pre-trained model's ability to detect objects of different sizes, but also enhances the pre-trained model's comprehensive understanding of image content, thereby improving overall performance and accuracy.

[0140] Step S203, concatenating the upsampled features and the downsampled features to obtain a feature pyramid;

[0141] It should be noted that the feature pyramid is a network structure that represents the multi-scale features of an image through convolutional image features at different levels. This structure allows the model to capture both local details and global context information of the image.

[0142] Step S204: the top module performs calculations based on the feature pyramid, obtains a judgment result, and outputs the judgment result.

[0143] Since the feature pyramid contains the features of the image data, the position, size and distance of the gap can be obtained by calculating the feature pyramid.

[0144] By flexibly constructing a feature pyramid through up- and down-sampling, and then performing calculations based on the feature pyramid, it is possible to process multi-scale features and improve the ability to detect objects of different sizes. The construction of the feature pyramid enables the pre-trained model to show higher stability and accuracy when facing challenges such as scale changes and occlusion. In addition, this multi-scale feature fusion strategy also improves the generalization ability of the model.

[0145] In a feasible embodiment, step S204, the top module performs calculation according to the feature pyramid, obtains the judgment result and outputs the judgment result, including steps E10 to E70:

[0146] Step E10, constructing a bounding box based on the feature pyramid;

[0147] It should be noted that a bounding box is a rectangular area that is defined by four parameters: two coordinate points (usually the coordinates of the upper left corner and the lower right corner) to determine the position and size of the rectangle, or it can be defined by four coordinate points, i.e. the four corners of the rectangle. In an image, a bounding box is usually used to enclose a target object for further analysis or processing.

[0148] Step E20, performing non-maximum suppression calculation on the bounding box to obtain a confidence score for each bounding box;

[0149] It should be noted that Non-Maximum Suppression (NMS) is a greedy algorithm used to remove bounding boxes with too much overlap during target detection and retain the bounding boxes with the highest confidence. The confidence score is a numerical value assigned by the pre-trained model to each predicted bounding box, indicating the probability or confidence level that the pre-trained model believes that the bounding box is correctly classified and contains the target object.

[0150] Step E30, sorting the bounding boxes based on the confidence score of each bounding box to obtain a sorted list;

[0151] It should be noted that the sorted list refers to a list formed by sorting the bounding boxes obtained during the target detection process from high to low according to the confidence scores of the bounding boxes.

[0152] Step E40, calculating the overlap of the bounding boxes according to the sorted list to obtain the overlap corresponding to each bounding box;

[0153] It is understandable that the overlap between each bounding box in the sorted list and other bounding boxes is calculated, usually using the Intersection over Union (IoU) as a metric, which is the ratio of the intersection area of ​​a bounding box with another bounding box to the union area.

[0154] It can be understood that the intersection-over-union ratio is a calculation of the overlap between two bounding boxes. When the number of bounding boxes is 2, one calculation is required, and when the number of bounding boxes is 3, three calculations are required, and so on.

[0155] Step E50, adjusting the confidence score according to the overlap to obtain an adjusted confidence score;

[0156] It should be noted that after obtaining the overlap, the confidence score of the bounding box with high overlap will be reduced.

[0157] It can be understood that the confidence score adjustment is obtained by calculating the adjustment weight through a mathematical formula and then multiplying the confidence score and the adjustment weight.

[0158] In one feasible embodiment, the confidence score calculation weight may be adjusted by a linear method. In another feasible embodiment, the confidence score calculation weight may be adjusted by a Gaussian distribution.

[0159] Step E60, reordering the bounding boxes based on the adjusted confidence scores, and selecting a to-be-detected box, where the to-be-detected box is the bounding box with the highest confidence score after the reordering;

[0160] Step E70, performing distance measurement calculation on the image data based on the frame to be detected to obtain a determination result.

[0161] It should be noted that if there is no gap in the frame to be detected, there will be no gap in the line laser. If there is an abnormal gap, such as a gap caused by the misalignment of the line laser, then this gap is the gap of the object to be detected, and the abnormal gap is measured. The judgment result refers to the result of the pre-trained model finding the gap position and measuring the distance according to the frame to be detected, that is, the judgment result contains the position information and size information of the gap.

[0162] Through the gap detection method based on visual detection in this embodiment, overlapping bounding boxes are not completely removed, but their scores are reduced, which can improve the recall rate while maintaining a certain accuracy rate.

[0163] Step S30, comparing the judgment result with a preset distance threshold to obtain a feedback result.

[0164] It should be noted that the preset distance threshold is a pre-set maximum value of the gap. When the gap size exceeds the preset distance threshold, it means that the detected product does not meet the generation standard. The preset distance threshold needs to be defined by specific product and production line standards. This embodiment does not limit the size of the preset distance threshold.

[0165] The gap detection method based on visual detection in this embodiment is combined with laser instruments and models for processing to avoid directly using cameras to obtain data. Gap observation has very high requirements for cameras, and the corresponding cameras are expensive. Using line lasers instead saves costs and improves accuracy.

[0166] For example, in order to help understand the technical concept or technical principle of the gap detection method based on visual detection in this embodiment, please refer to FIG. Figure 4 A distance measurement schematic diagram of a gap detection method based on visual detection is provided, as follows:

[0167] The rectangular box S in the figure is the boundary box to be tested, and the solid line part is the normal situation where there is no gap in the image recorded by the camera when the line laser is irradiated on the product to be tested. Under normal circumstances, when the product has no bumps and depressions, the line laser in the image is a straight line. The dotted line part indicates that the line laser in this part is missing, and the missing part is the gap. The width of the dotted line is 0.01mm, that is, the gap width is 0.01mm.

[0168] It should be noted that the above examples are only used to assist in understanding the present application and do not constitute a limitation on the gap detection method based on visual detection in the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0169] The present application also provides a gap detection device, please refer to Figure 5 , the gap detection device comprises:

[0170] An image data acquisition module 10 is used to acquire image data corresponding to a position to be measured, where the position to be measured is a position on the object to be measured where the line laser is irradiated;

[0171] A judgment result output module 20 is used to input the image data into a pre-trained model for judgment and output a judgment result, wherein the judgment result includes a gap distance of the object to be measured, and the pre-trained model is a YOLOV5 model, and the pre-trained model includes a bottom module, a middle module and a top module;

[0172] The distance comparison module 30 is used to compare the judgment result with a preset distance threshold to obtain a feedback result.

[0173] Optionally, the judgment result output module 20 is further used for:

[0174] The step of inputting the image data into a pre-trained model for judgment and outputting the judgment result comprises:

[0175] Extracting features of the image data through the bottom module to obtain image features, wherein the image features are extracted through at least one learning channel;

[0176] Upsampling and downsampling the image features through the intermediate module to obtain upsampled features and downsampled features;

[0177] The up-sampled features and the down-sampled features are concatenated to obtain a feature pyramid;

[0178] The top module performs calculations based on the feature pyramid to obtain the judgment result and outputs the judgment result.

[0179] Optionally, a compression activation module is added after each bottleneck layer of the bottom module, and the compression activation module includes a fully connected layer;

[0180] The judgment result output module 20 is also used for:

[0181] The spatial dimension of the image features is compressed by global average pooling to obtain a pooling vector;

[0182] Inputting the pooled vector into the fully connected layer, learning the dependency between the learning channels through the fully connected layer, and generating channel weights corresponding to the learning channels;

[0183] Normalizing the channel weights by using a preset activation function to obtain activation channel weights;

[0184] The image features are weighted by the activated channel weights to obtain channel-weighted image features.

[0185] Optionally, a convolutional block attention module is added to the intermediate module;

[0186] The judgment result output module 20 is also used for:

[0187] The step of weighting the image features by the activation channel weights to obtain the channel-weighted image features comprises:

[0188] The spatial relationship of image features is learned through maximum pooling and average pooling to obtain the spatial weight of each pixel;

[0189] Normalizing the spatial weights by using a preset activation function to obtain activation spatial weights;

[0190] The channel-weighted image features are spatially weighted by using the activation space weights to obtain channel- and space-weighted image features.

[0191] Optionally, the bottom module includes a convolution block configuration module, and the convolution block configuration module includes a routing block unit, a first convolution layer, a second convolution layer and a linear correction unit;

[0192] The judgment result output module 20 is also used for:

[0193] Merging the image features through the routing block unit to obtain merged image features;

[0194] Performing a convolution operation on the merged image features through the first convolution layer, and activating through the linear correction unit to obtain a first activation output, wherein the size of the filter of the first convolution layer is 1x1, the step value is 1, and the padding value is 0;

[0195] Performing a convolution operation on the first activation output through the second convolution layer, and activating through the linear correction unit to obtain a second activation output, wherein the filter size of the second convolution layer is 3x3, the step value is 2, and the padding value is 1;

[0196] The second activation output is merged through the routing block unit to obtain multi-scale image features.

[0197] Optionally, the judgment result output module 20 is further used for:

[0198] Constructing a bounding box based on the feature pyramid;

[0199] Performing non-maximum suppression calculation on the bounding box to obtain a confidence score for each bounding box;

[0200] sorting the bounding boxes based on the confidence score of each bounding box to obtain a sorted list;

[0201] Calculating the overlap of the bounding boxes according to the sorted list to obtain the overlap corresponding to each bounding box;

[0202] Adjusting the confidence score according to the overlap to obtain an adjusted confidence score;

[0203] Reordering the bounding boxes based on the adjusted confidence scores, and selecting a to-be-detected box, wherein the to-be-detected box is the bounding box with the highest confidence score after the reordering;

[0204] A distance measurement calculation is performed on the image data based on the frame to be detected to obtain a determination result.

[0205] Optionally, the judgment result output module 20 is further used for:

[0206] Performing data enhancement processing on the image data to obtain enhanced image data, wherein the data enhancement processing includes at least one of random cropping, brightness adjustment, and adding noise;

[0207] Feature extraction is performed on the enhanced image data to obtain image features.

[0208] The gap detection device provided in the embodiment of the present application adopts the gap detection method based on visual detection in the above embodiment, which can solve the technical problem of how to reduce the cost of gap detection. Compared with the prior art, the beneficial effects of the gap detection device provided in the embodiment of the present application are the same as the beneficial effects of the gap detection method based on visual detection provided in the above embodiment, and other technical features in the gap detection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0209] The present application provides a gap detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the gap detection method based on visual detection in the above-mentioned embodiment.

[0210] Reference below Figure 6 , which shows a schematic structural diagram of a gap detection device suitable for implementing an embodiment of the present application. Figure 6 The gap detection device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0211] like Figure 6 As shown, the gap detection device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the gap detection device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the gap detection device to communicate with other devices wirelessly or wired to exchange data. Although the gap detection device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0212] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0213] The gap detection device provided by the present application adopts the gap detection method based on visual detection in the above embodiment, which can solve the technical problem of how to reduce the cost of gap detection. Compared with the prior art, the beneficial effects of the gap detection device provided by the present application are the same as the beneficial effects of the gap detection method based on visual detection provided by the above embodiment, and the other technical features in the gap detection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0214] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0215] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0216] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the gap detection method based on visual detection in the above-mentioned embodiment.

[0217] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0218] The computer-readable storage medium may be included in the gap detection device; or may exist independently without being assembled into the gap detection device.

[0219] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the gap detection device, the gap detection device: obtains image data corresponding to the position to be measured, and the position to be measured is the position of the laser irradiation on the object to be measured; inputs the image data into a pre-trained model for judgment, and outputs a judgment result, and the judgment result includes the gap distance of the object to be measured. The pre-trained model is a YOLOV5 model, and the pre-trained model includes a bottom module, a middle module and a top module; compares the judgment result with a preset distance threshold to obtain a feedback result.

[0220] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0221] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0222] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0223] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned gap detection method based on visual detection, and can solve the technical problem of how to reduce the cost of gap detection. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the gap detection method provided in the above-mentioned embodiment, and will not be repeated here.

[0224] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A gap detection method based on visual detection, characterized in that: The gap detection method based on visual detection includes: Acquire image data corresponding to a position to be measured, where the position to be measured is a position on the object to be measured where a line laser is irradiated; Input the image data into a pre-trained model for judgment, and output a judgment result, wherein the judgment result includes a gap distance of the object to be measured, wherein the pre-trained model is a YOLOV5 model, and the pre-trained model includes a Backbone module, a Neck module, and a Head module; Comparing the judgment result with a preset distance threshold, and obtaining a feedback result; The step of inputting the image data into a pre-trained model for judgment and outputting the judgment result comprises: Extracting features of the image data through the Backbone module to obtain image features, wherein the image features are extracted through at least one learning channel; Upsampling and downsampling the image features by the Neck module to obtain upsampled features and downsampled features; The up-sampled features and the down-sampled features are concatenated to obtain a feature pyramid; The Head module performs calculations according to the feature pyramid, obtains a judgment result and outputs the judgment result; The Head module performs calculations according to the feature pyramid to obtain a judgment result, including: Constructing a bounding box based on the feature pyramid; Performing non-maximum suppression calculation on the bounding box to obtain a confidence score for each bounding box; sorting the bounding boxes based on the confidence score of each bounding box to obtain a sorted list; Calculating the overlap of the bounding boxes according to the sorted list to obtain the overlap corresponding to each bounding box; Adjusting the confidence score according to the overlap to obtain an adjusted confidence score; Reordering the bounding boxes based on the adjusted confidence scores, and selecting a to-be-detected box, the to-be-detected box being the bounding box with the highest confidence score after the reordering; A distance measurement calculation is performed on the image data based on the frame to be detected to obtain a determination result.

2. The method according to claim 1, characterized in that Adding a compression activation module after each bottleneck layer of the Backbone module, wherein the compression activation module includes a fully connected layer; The step of extracting features from the image data by the Backbone module to obtain image features comprises: The spatial dimension of the image features is compressed by global average pooling to obtain a pooling vector; Inputting the pooling vector into the fully connected layer, learning the dependency between the learning channels through the fully connected layer, and generating channel weights corresponding to the learning channels; Normalizing the channel weights by using a preset activation function to obtain activation channel weights; The image features are weighted by the activated channel weights to obtain channel-weighted image features.

3. The method according to claim 2, characterized in that Add a convolutional block attention module in the Neck module; The step of weighting the image features by the activation channel weights to obtain the channel-weighted image features comprises: The spatial relationship of image features is learned through maximum pooling and average pooling to obtain the spatial weight of each pixel; Normalizing the spatial weights by using a preset activation function to obtain activation spatial weights; The channel-weighted image features are spatially weighted by using the activation space weights to obtain channel- and space-weighted image features.

4. The method according to claim 1, characterized in that The Backbone module includes a convolution block configuration module, and the convolution block configuration module includes a routing block unit, a first convolution layer, a second convolution layer and a linear correction unit; The method further comprises: Merging the image features through the routing block unit to obtain merged image features; Performing a convolution operation on the merged image features through the first convolution layer, and activating through the linear correction unit to obtain a first activation output, wherein the size of the filter of the first convolution layer is 1x1, the step value is 1, and the padding value is 0; Performing a convolution operation on the first activation output through the second convolution layer, and activating through the linear correction unit to obtain a second activation output, wherein the filter size of the second convolution layer is 3x3, the step value is 2, and the padding value is 1; The second activation output is merged through the routing block unit to obtain multi-scale image features.

5. The method according to claim 1, characterized in that The step of extracting features from the image data through the Backbone module to obtain image features also includes: Performing data enhancement processing on the image data to obtain enhanced image data, wherein the data enhancement processing includes at least one of random cropping, brightness adjustment, and adding noise; Feature extraction is performed on the enhanced image data to obtain image features.

6. A gap detection device, characterized in that: The gap detection device comprises: An image data acquisition module, used to acquire image data corresponding to a position to be measured, where the position to be measured is a position on the object to be measured where the line laser is irradiated; A judgment result output module, used for inputting the image data into a pre-trained model for judgment, and outputting a judgment result, wherein the judgment result includes a gap distance of the object to be measured, and the pre-trained model is a YOLOV5 model, and the pre-trained model includes a Backbone module, a Neck module and a Head module; A distance comparison module is used to compare the judgment result with a preset distance threshold to obtain a feedback result; Wherein, the judgment result output module is also used for: Extracting features of the image data through the Backbone module to obtain image features, wherein the image features are extracted through at least one learning channel; Upsampling and downsampling the image features by the Neck module to obtain upsampled features and downsampled features; The up-sampled features and the down-sampled features are concatenated to obtain a feature pyramid; The Head module performs calculations according to the feature pyramid, obtains a judgment result and outputs the judgment result; The judgment result output module is also used for: Constructing a bounding box based on the feature pyramid; Performing non-maximum suppression calculation on the bounding box to obtain a confidence score for each bounding box; sorting the bounding boxes based on the confidence score of each bounding box to obtain a sorted list; Calculating the overlap of the bounding boxes according to the sorted list to obtain the overlap corresponding to each bounding box; Adjusting the confidence score according to the overlap to obtain an adjusted confidence score; Reordering the bounding boxes based on the adjusted confidence scores, and selecting a to-be-detected box, the to-be-detected box being the bounding box with the highest confidence score after the reordering; A distance measurement calculation is performed on the image data based on the frame to be detected to obtain a determination result.

7. A gap detection device, characterized in that: The gap detection device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the gap detection method based on visual detection as claimed in any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the gap detection method based on visual detection as described in any one of claims 1 to 5 are implemented.