Sliding shoe appearance detection and size measurement method and system

Abstract

The invention discloses a sliding shoe appearance detection and size measurement method and a sliding shoe appearance detection and size measurement system, which solve the problem that height customization is required for different types of flaws, so that the system has higher universality and adaptability and can meet the detection requirements of various types of flaws. According to the technical scheme, a measurement template is created; inputting a detection text; after the workpieces placed on the conveying belt are detected, the workpieces are transferred to a detection station; the bottom surface of the workpiece located on the detection station is photographed, and bottom surface defect detection is carried out; after bottom defect detection is completed, the workpiece located on the detection station is rotated by an angle, the front face of the workpiece is photographed, the size of the workpiece is measured, and whether the measured size of the workpiece is within the tolerance range or not is judged.

Description

Technical Field

[0001] The invention relates to the field of quality inspection and automated production, and in particular to a method for appearance inspection and size measurement of a sliding shoe. Background Art

[0002] In the industrial production process of skid boots, the stability and consistency of product quality are crucial. However, due to the complexity of the manufacturing process, the diversity of raw material properties and other factors, skid boots often have problems such as appearance defects or dimensional deviations. These problems not only affect the appearance of the product, but may also pose a potential threat to its performance. Therefore, strict quality inspection of products during the production process is a key step to ensure product quality.

[0003] Traditionally, these quality inspection tasks rely mainly on manual re-inspection. However, manual re-inspection has many limitations. On the one hand, manual inspection is time-consuming and inefficient, and it is difficult to meet the rapid inspection needs of large-scale production lines; on the other hand, manual inspection is costly and easily affected by the subjective judgment of inspectors, resulting in instability and inconsistency of inspection results.

[0004] With the rapid development of mechanical automation technology, automated inspection systems that combine advanced software algorithms and hardware equipment have emerged, providing new solutions for quality inspection of industrial products such as slippers. This type of automated system uses high-precision sensors and advanced image processing algorithms to quickly and accurately detect product appearance defects and dimensional deviations, greatly improving inspection efficiency and accuracy and reducing labor costs.

[0005] However, despite the many advantages of automated inspection systems, they still face many challenges in practical applications. In particular, for products such as slippers that have complex surface characteristics and a variety of defect types, traditional inspection algorithms are often difficult to adapt to all scenarios and require customized development for different types of defects (such as scratches, over-processing, uneven color, etc.), which undoubtedly increases the difficulty and cost of algorithm development.

[0006] In addition, in terms of dimensional measurement, industrial products such as slippers have extremely high requirements for measurement accuracy, especially in micron-level measurements, where even a tiny error can have a serious impact on the final quality of the product. Therefore, how to improve the robustness and measurement accuracy of the algorithm while ensuring measurement speed is a key issue that the current automated inspection system needs to solve.

[0007] In summary, although the existing automated inspection system has achieved certain results in the quality inspection of industrial products such as slippers, there are still many challenges and problems to be solved when facing complex defect judgment and high-precision dimensional measurement. Therefore, it is necessary to develop a more efficient, accurate and adaptable slipper appearance inspection and dimensional measurement method to meet the high requirements of industrial production for product quality inspection. Summary of the invention

[0008] A brief summary of one or more aspects is given below to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceived aspects, and is neither intended to identify the key or critical elements of all aspects nor to define the scope of any or all aspects. Its only purpose is to give some concepts of one or more aspects in a simplified form as a prelude to a more detailed description that will be given later.

[0009] The purpose of the present invention is to provide a method and system for detecting and measuring the appearance of a slipper, which solves the problem of requiring high customization for different types of defects, so that the system has higher versatility and adaptability and can cope with various types of defect detection needs.

[0010] The technical solution of the present invention is: the present invention discloses a method for detecting the appearance and measuring the size of a sliding shoe, the method comprising:

[0011] Step S1: Create a measurement template;

[0012] Step S2: input detection text;

[0013] Step S3: After being detected, the workpiece placed on the conveyor belt is transferred to the detection station;

[0014] Step S4: photographing the bottom surface of the workpiece located at the inspection station and performing bottom surface defect inspection;

[0015] Step S5: After the bottom defect detection is completed, the workpiece located at the detection station is rotated by an angle, the front of the workpiece is photographed, the workpiece size is measured, and it is determined whether the measured workpiece size is within the tolerance range.

[0016] According to an embodiment of the method for detecting appearance and measuring dimensions of a sliding shoe of the present invention, step S1 further comprises:

[0017] Step S11: input a front camera photo of a qualified part as input data for creating a measurement template;

[0018] Step S12: Create a front dimension template: Select the measurement areas in the input image in sequence to generate a front dimension template, where the measurement area includes four dimensions: step height, full height, minor diameter, and major diameter.

[0019] According to an embodiment of the method for detecting and measuring the appearance of a sliding shoe of the present invention, the contents of the detection text in step S2 include: scratches, dents, burrs, and non-through holes.

[0020] According to an embodiment of the method for detecting the appearance and measuring the size of a sliding shoe of the present invention, step S4 further comprises:

[0021] Step S41: After the workpiece arrives at the inspection station, the clamping claw clamps the workpiece, and the camera located at the bottom of the inspection station takes a picture of the bottom surface of the workpiece, and the data is transmitted to the open data set target detection model based on the improved Yolov8;

[0022] Step S42: training an open dataset target detection model based on the improved Yolov8;

[0023] Step S43: Use the trained open dataset object detection model based on the improved Yolov8 to detect the bottom photo of the workpiece, and determine whether there are defects on the bottom according to the output results.

[0024] According to one embodiment of the method for detecting and measuring the appearance of a slipper and its size of the present invention, an open data set target detection model based on an improved Yolov8 is constructed by combining a Clip model with an improved Yolov8 model, including: improving a PSA module and a C3 module, improving a Backbone network of Yolov8, and integrating language and visual patterns into the open data set target detection model.

[0025] According to an embodiment of the method for detecting appearance and measuring dimensions of a sliding shoe of the present invention, step S43 further comprises:

[0026] Step S431: extracting text features using the Clip model;

[0027] Step S432: Use the improved Yolov8 Backbone to extract image features;

[0028] Step S433: The extracted text features and image features are respectively input into the self-attention module and the deformable self-attention module, and fused, wherein the image-to-text cross-attention module and the text-to-image cross-attention module in the Glip model are used to align the features of different modalities between the text and the image to achieve feature fusion;

[0029] Step S434: inputting the fused and updated text features and image features into the FNN, further learning and nonlinear mapping the fused features, and outputting the final image features and text features;

[0030] Step S435: The open dataset target detection model matches the predicted regression box to the category or noun appearing in the input text, and outputs the text embedding detection target as the final detection result.

[0031] According to an embodiment of the method for detecting the appearance and measuring the size of a sliding shoe of the present invention, step S5 further comprises:

[0032] Step S51: After the bottom measurement is completed, the area array camera located in front of the inspection station takes a photo;

[0033] Step S52: performing template matching on the image, obtaining the center point position of the template, and obtaining measurement areas of various sizes according to the position of the center point, namely, the local measurement area;

[0034] Step S53: Each local measurement area is evenly divided into a number of sub-measurement areas at intervals of a set number of pixels in the horizontal or vertical direction;

[0035] Step S54: Find the color transition area in each sub-measurement area, and take the derivative in the vertical direction of the transition area according to the actual measurement position, perform Gaussian smoothing on the derivative after taking the derivative, and find the maximum or minimum value on the smoothed curve, and use it as the basis for selecting the boundary transition point;

[0036] Step S55: each local measurement area will obtain multiple boundary transition points, perform outlier analysis on the multiple boundary transition points, take the vertical direction of the edge of the workpiece in the area as the standard, perform K-means clustering on the horizontal coordinates or vertical coordinates of these points, confirm the outliers according to the set threshold value and remove the outliers;

[0037] Step S56: averaging the denoised boundary transition points, and using the average value point as a calculation point representing the boundary;

[0038] Step S57: repeat the operations of the above steps S53-S56 to obtain the calculation points of all boundaries of the area to be measured, and calculate the various dimensional parameters of the workpiece based on the horizontal and vertical coordinate differences;

[0039] Step S58: Repeat the operation of step S57 to finally obtain the true results of all workpiece sizes;

[0040] Step S59: judge the actual result of the workpiece size and output the final detection result.

[0041] The present invention also discloses a system for detecting and measuring the appearance of a sliding shoe, the system comprising:

[0042] Template creation module, used to create measurement templates;

[0043] A detection text input module, used for inputting detection text;

[0044] An inspection station placement module is used to transfer the workpiece placed on the conveyor belt to the inspection station after being inspected;

[0045] A bottom surface defect detection module is used to take a picture of the bottom surface of the workpiece located at the detection station and perform bottom surface defect detection;

[0046] The size measurement and detection module is used to rotate the workpiece on the detection station by an angle after the bottom defect detection is completed, and the front of the workpiece is photographed to measure the size of the workpiece and determine whether the measured workpiece size is within the tolerance range.

[0047] The present invention also discloses a computer system for detecting the appearance and measuring the size of a slipper, comprising a memory, a processor, and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to implement the steps of the method for detecting the appearance and measuring the size of a slipper as described above.

[0048] The present invention also discloses a computer-readable storage medium for detecting the appearance and measuring the size of a sliding shoe, which stores program instructions executable by a processor to implement the steps of the method for detecting the appearance and measuring the size of a sliding shoe as described above.

[0049] The present invention also discloses a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the steps of the above-mentioned method for detecting the appearance of the sliding shoe and measuring its size are implemented.

[0050] Compared with the prior art, the present invention has the following beneficial effects: Compared with the prior art, the technical innovations of the method of the present invention are mainly reflected in the following two aspects:

[0051] 1. Open dataset target detection model combining Clip model and improved Yolov8 model:

[0052] This invention combines the Clip model with the improved Yolov8 model for the first time, constructs an open dataset target detection model, and successfully applies it to product appearance defect detection. The open dataset target detection model can fully utilize the text-image feature fusion capability of the Clip model and the efficiency and accuracy of the improved Yolov8 model in target detection to achieve in-depth learning and extraction of product appearance features. This innovation not only solves the problem that traditional algorithms need to be highly customized for different types of defects, but also significantly improves the versatility and adaptability of the system, enabling it to flexibly respond to various types of defect detection needs and reduce the cost of algorithm development and maintenance.

[0053] 2. High-precision measurement based on sub-pixel precision information of images:

[0054] In terms of dimensional measurement, the present invention uses sub-pixel precision information based on images, and achieves high-precision measurement of product dimensions by capturing and analyzing sub-pixel level detail information in images, effectively improving the accuracy of traditional pixel-level measurement methods. This enables the present invention to accurately capture tiny dimensional changes that are ignored by traditional methods, thereby ensuring accurate measurement of product dimensions, and is particularly suitable for industrial production environments with strict tolerance requirements.

[0055] Therefore, the above technical innovations of the present invention bring about remarkable technical effects, which are specifically manifested in the following aspects:

[0056] 1. Improved the versatility and adaptability of defect detection:

[0057] The open data set target detection model that combines the Clip model with the improved Yolov8 model enables the present invention to cope with various types of defect detection needs without the need for highly customized algorithm development for different defects, thereby reducing the cost of algorithm development and maintenance and improving the efficiency and accuracy of defect detection.

[0058] 2. Achieved high-precision product size measurement:

[0059] The sub-pixel precision information based on the image enables the present invention to achieve high-precision measurement of product dimensions, effectively improving the accuracy of traditional pixel-level measurement methods. It not only meets the needs of product dimension measurement with strict tolerance requirements in industrial production, but also helps to improve the quality stability and consistency of products.

[0060] 3. Reduce the errors and time costs caused by manual operation:

[0061] The automated detection process of the present invention reduces the errors and time costs caused by manual operation, making the detection process more efficient and reliable. This not only improves production efficiency, but also reduces production costs, bringing significant economic benefits to the enterprise.

[0062] In summary, the present invention combines traditional image processing technology with deep learning methods to form a product defect detection and dimension measurement solution, providing strong technical support for quality inspection in industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The above features and advantages of the present invention can be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components with similar related properties or features may have the same or similar reference numerals.

[0064] Figure 1A flow chart of an embodiment of a method for detecting appearance and measuring dimensions of a sliding shoe according to the present invention is shown.

[0065] Figure 2 The network structure diagram of the open dataset target detection model based on the improved Yolov8 is shown.

[0066] Figure 3 The structural diagram of the improved C3PSA model is shown.

[0067] Figure 4 A system module diagram of an embodiment of a system for detecting and measuring the appearance of a slipper shoe according to the present invention is shown. DETAILED DESCRIPTION

[0068] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. Note that the aspects described below in conjunction with the accompanying drawings and specific embodiments are only exemplary and should not be construed as limiting the scope of protection of the present invention in any way.

[0069] Figure 1 The flow chart of an embodiment of the method for detecting the appearance and measuring the size of a sliding shoe of the present invention is shown. Figure 1 , the implementation steps of the method of this embodiment are described in detail as follows.

[0070] Step S1: Create a measurement template.

[0071] In step S1, the steps of creating a measurement template include:

[0072] Step S11: input a front camera photo of a qualified part as input data for creating a measurement template;

[0073] Step S12: Create a front dimension template: select measurement areas of four dimensions, namely, step height, full height, minor diameter, and major diameter, in the input image in sequence to generate a front dimension template.

[0074] Step S2: Input detection text.

[0075] The contents of the inspection text include: scratches, dents, burrs, missing holes, etc.

[0076] Step S3: A plurality of workpieces are placed on the conveyor belt. After the infrared switch detects the workpieces on the conveyor belt, it sends a clamping signal to the mobile bracket to clamp the workpieces to the inspection station.

[0077] Step S4: The clamping jaws clamp the workpiece located on the inspection station, and the camera located at the bottom of the inspection station takes a picture of the bottom surface of the workpiece and performs bottom defect detection.

[0078] Step S4 further includes the following processing:

[0079] Step S41: After the workpiece arrives at the inspection station, the clamping claw clamps the workpiece, and the camera located at the bottom of the inspection station takes a picture of the bottom surface of the workpiece, and transmits the data to the open data set target detection model based on the improved Yolov8.

[0080] Step S42: training an open dataset target detection model based on the improved Yolov8.

[0081] First, the open dataset target detection model based on the improved Yolov8 is constructed by combining the Clip model with the improved Yolov8 model. Figure 2 As shown, the specific construction characteristics of the open dataset target detection model are as follows.

[0082] 1. Improve the PSA (Partial self-attention) module and the C3 module. The C3 module is based on the C3 module, which is used for feature extraction in image processing or computer vision tasks. It usually contains multiple submodules or layers to gradually extract feature information from the image. It does not contain an attention mechanism, but relies on traditional convolutional layers or other feature extraction methods. The PSA module is introduced into the C3 module. The PSA module is a partial self-attention mechanism that can process the input features based on the attention mechanism, thereby obtaining the C3PSA module, which enables the C3PSA module to use the attention mechanism to enhance the sensitivity to key areas in the image. The C3PSA module improves the model's sensitivity to key areas in the image, thereby enhancing the ability to extract image features.

[0083] The improved C3PSA module adopts the original structure of the C3 module, replacing the Bottleneck module with the PSA module. The improved C3PSA model structure is shown in FIG. Figure 3 Since the original C3 module does not have an attention mechanism, the Bottleneck module is used for feature extraction, which limits the expressiveness of the features. PSA is a self-attention module that first divides the features into two parts along the channel direction, and only inputs part of the data into the convolution layer (Conv) and the MHSA layer (Multi-Head Self-Attention) ( Figure 3The right side of the image is shown in the figure) for feature extraction based on the attention mechanism, and then after nonlinear mapping by FNN (Feedforward Neural Network) (FNN is a neural network structure in which information only propagates forward without loops or feedback. It usually contains multiple layers, each layer contains multiple neurons, and these neurons are connected to the neurons in the previous layer through weights and biases. In the PSA module, FNN is used to perform nonlinear mapping on the features processed by the MHSA layer, thereby further enhancing the expressive power of the features), the two parts of the features are concatenated (Concat) and convolutionally fused. This structure can better highlight the key parts of the image and help the model learn more meaningful features. By embedding the PSA module into the C3 module, stacking multiple PSA layers and performing residual connections, the feature representation capability can be effectively enhanced.

[0084] 2. Improve the Backbone of Yolov8. Backbone is a key component in deep learning models. It is a deep convolutional neural network (CNN) in computer vision tasks, which is used to extract features from input images. In tasks such as object detection and image classification, Backbone is responsible for converting the original image into a series of high-level feature maps, which are then used in subsequent detection or classification tasks. Yolov8 is a version of the YOLO (You Only Look Once) series of object detection models. The YOLO series is known for its high efficiency and accuracy and is widely used in real-time object detection tasks. Yolov8 inherits the core idea of ​​the YOLO series, that is, a single forward propagation can complete object detection and classification. It further improves the performance and accuracy of the model by optimizing the network structure, loss function, etc. The original Yolov8 consists of a convolutional (Conv) module and a C2f module. The C2f module is an efficient feature extraction method that achieves a balance between lightweight and performance through diversion and feature fusion. The present invention adds a C3PSA module to the Backbone of Yolov8 to obtain an improved Backbone of Yolov8. Compared with the original Yolov8 model, the improved Yolov8Backbone model effectively enhances the model's perception of key information, enabling the model to focus on different positions of input features from multiple spatial angles and capture the association between positions in the spatial dimension. In this way, the Yolov8Backbone model can dynamically select important parts of the image (such as defective areas) for processing while ignoring irrelevant information (such as background).

[0085] In Yolov8's original Backbone, the last layer is the SPPF layer (Spatial Pyramid Pooling-Fast). The SPPF layer divides the input feature map into multiple grids of different scales through repeated max pooling operations, and performs max pooling operations in each grid. This can extract multi-scale feature information and enhance the model's ability to detect objects of different scales, but it does not pay significant attention to important feature channels. In order to better extract features, the C3PSA module with attention is added after it, which can perform deeper feature enhancement on the multi-scale features of its output, thereby improving the detection ability of the model.

[0086] 3. The open dataset object detection model combining the Clip model and the improved Yolov8 model of the present invention integrates language and visual modes, and can flexibly cope with the detection of different workpieces and different defects. At the same time, it is more concise in use, and there is no need to input complex proper nouns. Only (natural language) text and detection images need to be input to automatically detect the required results.

[0087] The model training process includes shooting training data with a bottom camera, data classification (good products, data with multiple defects, etc.), data labeling, and then inputting training images and training text into the model, and finally outputting a trained open dataset target detection model.

[0088] Step S43: Use the trained open dataset object detection model to detect the bottom photo of the product, and determine whether there are defects on the bottom based on the output results.

[0089] Step S43 further includes the following detailed process.

[0090] Step S431: Use the Clip (Contrastive Language-Image Pre-training) model to extract text features, that is, extract text features from the detection text input before the detection task starts.

[0091] Step S432: Use the improved Yolov8 Backbone to extract image features, that is, extract image features from the photos taken by the bottom camera.

[0092] Step S433: The extracted text features and image features are respectively input into the self-attention module (input text features) and the deformable self-attention module (input image features), and fused. The image-to-text cross-attention module and the text-to-image cross-attention module in the Glip (Grounded Language-Image Pre-training) model are used to align the features of different modalities between text and image to achieve feature fusion. The Glip (Grounded Language-Image Pre-training) model is a multimodal pre-training model that combines image and text data and aims to enhance the understanding of images and text by jointly learning the association between images and text. The image-to-text cross-attention module allows the model to focus on the part of the image that is most relevant to the text description. It generates attention weights based on image features, which are used to weight text features to highlight the text information most relevant to the image content. The text-to-image cross-attention module works in the opposite way, allowing the model to focus on the part of the text description that is most relevant to the image. By generating attention weights based on text features and applying these weights to weight image features, the model is able to focus on the image regions that best match the text description.

[0093] Step S434: Input the fused and updated text features and image features into the FNN (Feedforward Neural Network), further learn and nonlinearly map the fused features, and output the final image features and text features. After the text features are embedded in the image perception operation, the image features and text features are sent to the final text comparison head and detection box head respectively. The text comparison head is a specially designed network layer for calculating the similarity between text features. The detection box head is a network layer responsible for generating the target detection box, which predicts the location (usually the coordinates of the bounding box) and category of the target based on the extracted image features (which may also include the fused text features).

[0094] Step S435: The open dataset target detection model matches the predicted regression box to the category or noun that appears in the input text, embeds the text into the detection target, and outputs it as the final detection result. Among them, the predicted regression box is a key output in the target detection task. It is usually a rectangular box (or bounding box) used to locate the target object in the image. The coordinates of the predicted regression box (usually the coordinates of the upper left and lower right corners) and may also include the category label of the target are predicted by the detection box head. The predicted regression box needs to be matched with the category or noun that appears in the input text, and the text is embedded in the detection target to generate the final detection result.

[0095] Step S5: After the bottom defect detection is completed, the servo motor drives the dividing plate to rotate an angle (for example, 1 / 4 turn), and the area array camera located in front of the detection station takes a picture of the front of the workpiece, measures the size of the workpiece (including the four dimensions of step height, full height, minor diameter, and major diameter), and determines whether the measured workpiece size is within the tolerance range. For example, the measurement accuracy of the step height dimension is within ±0.001mm, and the measurement accuracy of the full height, minor diameter, and major diameter dimensions is within ±0.01mm.

[0096] Step S5 further includes the following processing steps.

[0097] Step S51: After the bottom measurement is completed, the area array camera located in front of the inspection station takes a picture and imports the picture into the system.

[0098] Step S52: Perform template matching on the image to obtain the center point position of the template, and obtain measurement areas of various sizes based on the position of the center point, which are called local measurement areas.

[0099] Step S53: Each local measurement area is evenly divided into a number of sub-measurement areas in the horizontal or vertical direction (vertical direction of the edge of the workpiece in the area) at intervals of a set number of pixels (for example, 20 pixels).

[0100] Step S54: Find the color transition area in each sub-measurement area, and take the derivative (horizontally or vertically) in the vertical direction of the transition area according to the actual measurement position. After taking the derivative, perform Gaussian smoothing on the derivative, find the maximum or minimum value on the smoothed curve, and use it as the basis for selecting the boundary transition point.

[0101] For the search of color transition area, the center point position and scaling ratio of the object to be matched are found based on template matching, and the color transition area of ​​the object to be matched is obtained based on the color transition area in the original template. Then, bilinear interpolation is performed on the area at the sub-pixel accuracy level to improve the accuracy of the pixels and help achieve higher detection accuracy.

[0102] Step S55: Each local measurement area will get n boundary transition points. Perform outlier analysis on these points, and use the vertical direction of the edge of the workpiece in the area as the standard to perform K-means clustering on the horizontal or vertical coordinates of these points. According to the set threshold, identify those outliers that are too large or too small and remove them.

[0103] Compared with the direct output method commonly used in industry, the present invention can effectively solve the problem of inaccurate measurement caused by abnormal conditions such as dust or special workpieces (such as local depression or protrusion of the edge). By removing some outliers, it not only reduces the impact of impurities such as dust on the measurement accuracy, but also can adapt to more complex measurement scenarios. This method shows stronger robustness when facing a variety of workpiece types and different environmental conditions, and can ensure that the measurement results are more consistent and reliable, thus meeting the requirements for high precision and high adaptability in industrial detection.

[0104] The above K-Means clustering calculates an adaptive threshold. The initial K-Means requires manual determination of the threshold to perform outlier denoising, but due to the automation and precision requirements in industrial applications, using a fixed threshold for denoising is unreliable. Therefore, in the K-Means of this method, the number of clusters (or clusters) formed is first set to K = 2, and the points are divided into two clusters using the initial K-Means algorithm. Then, the distance (distance) from each data point in each cluster to its cluster center is calculated, the mean distance (Mean distance) and the standard deviation (Distancestandard deviation) are calculated, and an adaptive threshold (Auto threshold) is set based on these values. Finally, the points that are too far from the cluster center are screened out according to the adaptive threshold and removed as outliers, and the two clusters after removing the outliers are merged to obtain the final boundary points. The formula is as follows.

[0105] K=2

[0106]

[0107] Auto threshold=Mean distance+Distance standard deviation

[0108] In the above formula: K is the number of clusters; n is the number of points; distance is the distance from each data point in each cluster to its cluster center; mean distance is the mean distance; distance standard deviation is the standard deviation of distance; auto threshold is the adaptive threshold; distance iis the distance value of the i-th point; Represents the mean distance.

[0109] Step S56: Calculate the average value of the denoised boundary transition points, and use the average value point as a calculation point k1 representing the boundary.

[0110] Repeat the operations of steps S53-S55 to obtain the boundary transition points of all the areas to be measured. In this method, one size parameter requires two groups of boundary transition points for calculation, so the boundary points are divided into two categories for different processing.

[0111] Step S57: Repeat the operations of the above steps S53-S56 to obtain the calculation points k1-kn of all boundaries of the area to be measured, and calculate the various dimensional parameters of the product based on the horizontal and vertical coordinate differences.

[0112] Taking a parameter as an example, a set of boundary transition points are averaged to obtain a calculation point P1(u,v) representing the boundary; another set of boundary transition points P is retained. P1 and P, which are points based on the image coordinate system, are calculated through the camera's intrinsic parameters (CameraParameters) and extrinsic parameters (CameraPose) to obtain points P1_world and P_world based on the world coordinate system. The boundary transition point P_world in the world coordinate is then used to fit the line L1. Finally, the distance from the boundary calculation point P1_world in the world coordinate to the line L1 is calculated, and the final result is the actual size D1 of the workpiece, as shown in the following formula.

[0113]

[0114] P1_world=P1*CameraParameters*CameraPose=(x_world,y_world)

[0115] L1:A*X+B*Y+C=0

[0116]

[0117] In the above formula: CameraParameters is the intrinsic parameter of the camera; CameraPose is the extrinsic parameter of the camera; f x and f y is the focal length, c x and c y is the coordinate of the center point of the image, r 11 -r 33is the rotation matrix, t1-t3 is the translation matrix; P1 is the point after averaging a set of boundary transition points; P1_world(x_world,y_world) is the point of P1 converted to the world coordinate system; A and B are the X and Y coefficients of the line L1.

[0118] Step S58: Repeat the operation of step S57 to finally obtain the true results of all workpiece sizes.

[0119] Step S59: Determine the size of the workpiece and output the final test result. If it is qualified, it will output OK, if it is not qualified, it will output NG (Not Good).

[0120] Figure 4 The module of the shoe appearance inspection and size measurement system of the present invention is shown. Figure 4 The system of this embodiment includes: a template creation module, a detection text input module, a detection station placement module, a bottom surface defect detection module, and a dimension measurement detection module.

[0121] The template creation module is used to create a measurement template. Its specific implementation is the same as step S1 of the method embodiment, which will not be repeated here.

[0122] The detection text input module is used to input the detection text. Its specific implementation is the same as step S2 of the method embodiment, which will not be repeated here.

[0123] The detection station placement module is used to transfer the workpiece placed on the conveyor belt to the detection station after being detected. Its specific implementation is the same as step S3 of the method embodiment, which will not be repeated here.

[0124] The bottom surface defect detection module is used to take a picture of the bottom surface of the workpiece located at the detection station and perform bottom surface defect detection. Its specific implementation is the same as step S4 of the method embodiment, which will not be repeated here.

[0125] The size measurement and detection module is used to rotate the workpiece on the detection station by an angle after the bottom defect detection is completed, and the front of the workpiece is photographed to measure the size of the workpiece and determine whether the measured size of the workpiece is within the tolerance range. Its specific implementation is the same as step S5 of the method embodiment, which will not be repeated here.

[0126] In addition, the present invention also discloses a computer system for detecting the appearance and measuring the size of a slipper, comprising a memory, a processor, and program instructions stored in the memory for the processor to execute, wherein the processor executes the program instructions to implement the steps of the embodiment of the method for detecting the appearance and measuring the size of a slipper as described above.

[0127] In addition, the present invention also discloses a computer-readable storage medium for detecting the appearance and measuring the size of a slipper, which stores program instructions executable by a processor to implement the steps of the embodiment of the method for detecting the appearance and measuring the size of a slipper as described above.

[0128] In addition, the present invention also discloses a computer program product, including a computer program, which implements the steps of the above-mentioned embodiment of the method for detecting the appearance and measuring the size of the sliding shoe when the computer program is executed by a processor.

[0129] Although the above methods are illustrated and described as a series of actions for simplicity of explanation, it should be understood and appreciated that these methods are not limited by the order of the actions, because according to one or more embodiments, some actions may occur in a different order and / or concurrently with other actions from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art.

[0130] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technician may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0131] The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in cooperation with a DSP core, or any other such configuration.

[0132] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, a storage medium may be integrated into a processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components.

[0133] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented as a computer program product in software, each function may be stored on or transmitted by a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one place to another. Storage media may be any available medium that can be accessed by a computer. As an example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, a server, or other remote source using a coaxial cable, a fiber optic cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wherein disk often reproduces data magnetically, while disc reproduces data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0134] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein, but should be granted the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting the appearance and measuring the size of a sliding shoe, characterized in that: Methods include: Step S1: Create a measurement template; Step S2: input detection text; Step S3: After being detected, the workpiece placed on the conveyor belt is transferred to the detection station; Step S4: photographing the bottom surface of the workpiece located at the inspection station and performing bottom surface defect inspection; Step S5: After the bottom defect detection is completed, the workpiece located at the detection station is rotated by an angle, the front of the workpiece is photographed, the workpiece size is measured, and it is determined whether the measured workpiece size is within the tolerance range.

2. The method for detecting and measuring the appearance of a sliding shoe according to claim 1, characterized in that: Step S1 further comprises: Step S11: input a front camera photo of a qualified part as input data for creating a measurement template; Step S12: Create a front dimension template: Select the measurement areas in the input image in sequence to generate a front dimension template, where the measurement area includes four dimensions: step height, full height, minor diameter, and major diameter.

3. The method for detecting and measuring the appearance of a sliding shoe according to claim 1, characterized in that: The contents of the inspection text in step S2 include: scratches, dents, burrs, and non-through holes.

4. The method for detecting and measuring the appearance of a sliding shoe according to claim 1, characterized in that: Step S4 further comprises: Step S41: After the workpiece arrives at the inspection station, the clamping claw clamps the workpiece, and the camera located at the bottom of the inspection station takes a picture of the bottom surface of the workpiece, and the data is transmitted to the open data set target detection model based on the improved Yolov8; Step S42: training an open dataset target detection model based on the improved Yolov8; Step S43: Use the trained open dataset object detection model based on the improved Yolov8 to detect the bottom photo of the workpiece, and determine whether there are defects on the bottom according to the output results.

5. The method for detecting the appearance and measuring the size of the sliding shoe according to claim 4, characterized in that: The open dataset target detection model based on the improved Yolov8 is constructed by combining the Clip model with the improved Yolov8 model, including: improving the PSA module and the C3 module, improving the Backbone network of Yolov8, and integrating language and visual modes.

6. The method for detecting appearance and measuring dimensions of a sliding shoe according to claim 4, characterized in that: Step S43 further includes: Step S431: extracting text features using the Clip model; Step S432: Use the improved Yolov8 Backbone to extract image features; Step S433: The extracted text features and image features are respectively input into the self-attention module and the deformable self-attention module, and fused, wherein the image-to-text cross-attention module and the text-to-image cross-attention module in the Glip model are used to align the features of different modalities between the text and the image to achieve feature fusion; Step S434: inputting the fused and updated text features and image features into the FNN, further learning and nonlinear mapping the fused features, and outputting the final image features and text features; Step S435: The open dataset target detection model matches the predicted regression box to the category or noun appearing in the input text, and outputs the text embedding detection target as the final detection result.

7. The method for detecting the appearance and measuring the size of the sliding shoe according to claim 1, characterized in that: Step S5 further comprises: Step S51: After the bottom measurement is completed, the area array camera located in front of the inspection station takes a photo; Step S52: performing template matching on the image, obtaining the center point position of the template, and obtaining measurement areas of various sizes according to the position of the center point, namely, the local measurement area; Step S53: Each local measurement area is evenly divided into a number of sub-measurement areas at intervals of a set number of pixels in the horizontal or vertical direction; Step S54: Find the color transition area in each sub-measurement area, and take the derivative in the vertical direction of the transition area according to the actual measurement position, perform Gaussian smoothing on the derivative after taking the derivative, and find the maximum or minimum value on the smoothed curve, and use it as the basis for selecting the boundary transition point; Step S55: each local measurement area will obtain multiple boundary transition points, perform outlier analysis on the multiple boundary transition points, take the vertical direction of the edge of the workpiece in the area as the standard, perform K-means clustering on the horizontal coordinates or vertical coordinates of these points, confirm the outliers according to the set threshold value and remove the outliers; Step S56: averaging the denoised boundary transition points, and using the average value point as a calculation point representing the boundary; Step S57: repeat the operations of the above steps S53-S56 to obtain the calculation points of all boundaries of the area to be measured, and calculate the various dimensional parameters of the workpiece based on the horizontal and vertical coordinate differences; Step S58: Repeat the operation of step S57 to finally obtain the true results of all workpiece sizes; Step S59: judge the actual result of the workpiece size and output the final detection result.

8. A system for detecting and measuring the appearance of a skid shoe, characterized in that: The system includes: Template creation module, used to create measurement templates; A detection text input module, used for inputting detection text; An inspection station placement module is used to transfer the workpiece placed on the conveyor belt to the inspection station after being inspected; A bottom surface defect detection module is used to take a picture of the bottom surface of the workpiece located at the detection station and perform bottom surface defect detection; The size measurement and detection module is used to rotate the workpiece on the detection station by an angle after the bottom defect detection is completed, and the front of the workpiece is photographed to measure the size of the workpiece and determine whether the measured workpiece size is within the tolerance range.

9. A computer system for appearance inspection and size measurement of sliding shoes, characterized in that: The method comprises a memory, a processor and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to implement the steps of the method for detecting the appearance and measuring the size of the slipper according to any one of claims 1 to 7.

10. A computer-readable storage medium for skid shoe appearance inspection and size measurement, characterized in that: It stores program instructions executable by a processor to implement the steps of the method for detecting the appearance and measuring the size of the slipper as claimed in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for detecting appearance and measuring dimensions of a shoe as claimed in any one of claims 1 to 7 are implemented.

Images