Part defect detection method and equipment based on deep learning, namely medium

Abstract

The embodiment of the invention discloses a deep learning-based part defect detection method, equipment and a medium, belongs to the technical field of industrial product detection, and solves the problems of defect missing detection and low detection accuracy in a micro part detection process. Determining a target running speed of the conveyor belt based on a detection capability coefficient-part movement speed relation model; after the speed of the conveying belt is adjusted, image collection is conducted on the parts on the conveying belt, and images of the to-be-detected parts are obtained; determining a part area in the to-be-detected part image, and dividing the part area into a plurality of grid units for feature information extraction; inputting the feature information into a preset SSD part defect detection network to obtain part defect information; performing time sequence association on the detection intermediate variables to obtain an intermediate variable sequence; and predicting the detection state of the industrial real-time detection platform based on the intermediate variable sequence, starting a leak detection mechanism based on the predicted detection state, and carrying out secondary detection on the part defect information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial product detection, and in particular to a part defect detection method and device based on deep learning and a medium. BACKGROUND

[0002] In industrial production, part defect detection is a core link to ensure product quality.

[0003] Traditional part defect detection methods usually use filtering detection and ultrasonic detection technology. Filtering detection relies on signal processing algorithms to identify defects, but in the face of complex electromagnetic interference, mechanical vibration and other noises in the industrial field, its defect feature separation ability from interference signals is limited, resulting in poor detection result stability and high misjudgment rate. Ultrasonic detection uses the reflection characteristics of sound waves to detect defects on the surface of parts to some extent, but as the depth of the defect increases, the sound wave attenuates and scatters in the medium, the detection signal is severely distorted, and the positioning and quantitative analysis error of deep defects significantly increases, making it difficult to achieve accurate detection of full size and full depth.

[0004] With the development of industrial manufacturing towards precision, micro parts are increasingly widely used. Such parts are small in size and fine in features, and traditional detection methods lack real-time perception and adaptive adjustment ability for the dynamic motion state of the parts, making it difficult to accurately capture defect features. During the detection process, micro defect omission and detection delay often occur, and in the high-speed operation scene of the production line, the detection accuracy and efficiency are in conflict, making it difficult to meet the efficiency and accuracy requirements of modern industrial production for quality control. SUMMARY

[0005] The embodiments of the present application provide a part defect detection method and device based on deep learning and a medium, which are used to solve the following technical problems: the existing technology often has micro defect omission and detection delay in the detection process of micro parts, and in the high-speed operation scene of the production line, the detection accuracy and efficiency are in conflict, making it difficult to meet the efficiency and accuracy requirements of modern industrial production for quality control.

[0006] The embodiments of the present application adopt the following technical solutions:

[0007] The embodiment of the application provides a part defect detection method based on deep learning. The method comprises the following steps: acquiring running data and part data of an industrial real-time detection platform; determining a target running speed of a conveying belt based on a detection capability coefficient-part movement speed relationship model; after adjusting the conveying belt based on the target running speed of the conveying belt, acquiring images of parts on the conveying belt through a data acquisition device to obtain to-be-inspected part images; performing edge detection on the to-be-inspected part images to determine part regions, and dividing the part regions into a plurality of grid units to extract feature information based on the grid units; inputting the feature information into a preset SSD part defect detection network to obtain part defect information; within a preset detection period, performing time sequence correlation on intermediate variables in the detection to obtain an intermediate variable sequence; and based on the intermediate variable sequence, predicting a detection state of the industrial real-time detection platform, starting a missed detection mechanism based on the predicted detection state, and performing secondary detection on the part defect information.

[0008] The embodiment of the application optimizes the parameter setting of the detection model by establishing the detection capability coefficient-part movement speed relationship model, improves the accuracy and stability of the micro part defect detection, and the SSD-based micro part defect detection algorithm can effectively identify various defects of the micro part, and has higher accuracy than the traditional algorithm. Secondly, the missed detection algorithm based on the intermediate variable reduces the missed detection rate of the micro part in the dynamic detection process, and improves the reliability of the detection model. The embodiment of the application combines the industrial real-time detection platform with the detection algorithm based on deep learning, realizes real-time detection of the micro part, improves the efficiency of the micro part quality detection on the production line, and meets the actual needs of industrial production.

[0009] In one implementation manner of the application, the running data and the part data of the industrial real-time detection platform are acquired, and the target running speed of the conveying belt is determined based on the detection capability coefficient-part movement speed relationship model. Specifically, the field of view range and the detection response time length corresponding to the data acquisition device are determined based on the running data of the industrial real-time detection platform; the part size and the part running distance are determined based on the part data; the detection capability coefficient-part movement speed relationship model is constructed through the running data and the part data based on the comparison relationship between the image acquisition time length and the detection response time length; and the optimal speed of the detection capability coefficient-part movement speed relationship model is solved to obtain the target running speed of the conveying belt.

[0010] In one implementation manner of the application, the detection capability coefficient-part movement speed relationship model is constructed through the running data and the part data. Specifically, when the detection response time length is not greater than the image acquisition time length, a first relationship function is constructed as follows:

[0011]

[0012] When the detection response duration is greater than the image acquisition duration, a second relationship function is constructed:

[0013]

[0014] Based on the first relationship function and the second relationship function, a detection capability coefficient-part movement speed relationship model is constructed; wherein v is the running speed of the conveying belt; θ is a quadratic function of the conveying belt speed; T is the detection response duration; t is the image acquisition duration; D1 is the part spacing; and W1 is the part width value.

[0015] In an implementation manner of the present application, edge detection is performed on the to-be-inspected part image to determine a part region, and the part region is divided into a plurality of grid units, so as to extract feature information based on the grid units. Specifically, the to-be-inspected part image is converted into a binary image, and a part boundary is determined based on an edge detection algorithm. Edge pixel points are determined based on image gradient values corresponding to the part boundary, so as to determine the part region based on the edge pixel points. Defect distribution of the part is determined based on historical detection data, and the part region is divided into a plurality of risk regions based on the defect distribution. The to-be-inspected part image is divided into a plurality of grid units based on the plurality of risk regions in the part region, and the plurality of grid units are divided into different densities of grid based on risk levels respectively corresponding to the plurality of risk regions. Feature information is extracted for each grid unit. The extracted feature information at least includes a gray value and an edge geometric feature.

[0016] In an implementation manner of the present application, before the feature information is input into the preset SSD part defect detection network, the method further includes: extracting feature maps of different levels corresponding to the preset part image through a convolutional neural network; setting a plurality of default boxes of different sizes in each feature map, and mapping coordinates of the default boxes on the feature maps to the preset part image; determining probabilities of defect targets in the default boxes belonging to each type of defect through multi-classification cross-entropy, and determining position deviations of the predicted boxes from the real defect boxes using a smooth L1 loss; and optimizing the preset SSD part defect detection network based on the probabilities and the position deviations.

[0017] In an implementation manner of the present application, in a preset detection period, a time sequence correlation is performed on intermediate variables to obtain an intermediate variable sequence. Specifically, in a part detection process, an intermediate variable dataset is obtained. The intermediate variable dataset at least includes image acquisition data, detection data, and platform running data. Time stamps corresponding to the intermediate variable dataset are mapped to a same time axis based on a data acquisition time corresponding to the detection data. A window sliding is performed on the time axis based on a fixed-length time window, and data features of each window are extracted. In the preset detection period, the extracted data features are arranged based on a time sequence to obtain the intermediate variable sequence.

[0018] In an implementation form of the present application, the detection state of the industrial real-time detection platform is predicted based on the sequence of intermediate variables, specifically comprising: inputting the sequence of intermediate variables generated in the current detection period into a preset long short-term memory network; extracting statistical features in the sequence of intermediate variables through the long short-term memory network, and outputting a probability distribution of the detection state based on the statistical features; comparing the probability distribution with a preset state threshold, and outputting the detection state of the industrial real-time detection platform based on the comparison result; wherein the detection state includes a normal state, a warning state and a risk state.

[0019] In an implementation form of the present application, the detection state predicted is used to start a missed detection mechanism, specifically comprising: mapping different detection states to different missed detection risk levels; starting different missed detection mechanisms based on different missed detection risk levels; wherein, when the detection state is the normal state, the existing detection process is maintained; when the detection state is the warning state, a higher resolution shooting is enabled for the part image to be detected, and a warning information is sent to the management personnel; when the detection state is the risk state, a backup detection algorithm is started to perform secondary detection on the part image to be detected, and the production line is suspended in the case that the detection state is the risk state for a plurality of preset detection periods in succession.

[0020] The embodiment of the present application provides a part defect detection device based on deep learning, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: acquire running data of an industrial real-time detection platform and part data, determine a target running speed of a conveying belt based on a detection capacity coefficient-part movement speed relationship model; after adjusting the conveying belt based on the target running speed of the conveying belt, acquire images of parts on the conveying belt through a data acquisition device to obtain part images to be detected; perform edge detection on the part images to be detected to determine part regions, and divide the part regions into a plurality of grid units to extract feature information based on the grid units; input the feature information into a preset SSD part defect detection network to obtain part defect information; perform time sequence correlation on detection intermediate variables in a preset detection period to obtain a sequence of intermediate variables; predict a detection state of the industrial real-time detection platform based on the sequence of intermediate variables, and start a missed detection mechanism based on the predicted detection state to perform secondary detection on the part defect information.

[0021] The nonvolatile computer storage medium provided by the embodiment of the application stores computer executable instructions, and the computer executable instructions are configured to: acquire running data and part data of an industrial real-time detection platform, determine a target running speed of a conveying belt based on a detection capability coefficient-part movement speed relationship model, after adjusting the conveying belt based on the target running speed of the conveying belt, acquire images of parts on the conveying belt through a data acquisition device to obtain to-be-detected part images, perform edge detection on the to-be-detected part images to determine part regions, divide the part regions into a plurality of grid units, perform feature information extraction based on the grid units, input the feature information into a preset SSD part defect detection network to obtain part defect information, perform time sequence correlation on intermediate variables in a preset detection period to obtain an intermediate variable sequence, and predict a detection state of the industrial real-time detection platform based on the intermediate variable sequence, to start a missed detection mechanism based on the predicted detection state to perform secondary detection on the part defect information.

[0022] The above at least one technical solution adopted by the embodiment of the application can achieve the following beneficial effects: the embodiment of the application optimizes the parameter setting of the detection model by establishing a detection capability coefficient-part movement speed relationship model, and improves the accuracy and stability of micro part defect detection. The SSD-based micro part defect detection algorithm can effectively identify various defects of micro parts, and has higher accuracy than traditional algorithms. Secondly, the missed detection algorithm based on the intermediate variable in the embodiment of the application reduces the missed detection rate of micro parts in the dynamic detection process, and improves the reliability of the detection model. The embodiment of the application combines the industrial real-time detection platform with the detection algorithm based on deep learning, realizes real-time detection of micro parts, improves the efficiency of micro part quality detection on the production line, and meets the actual needs of industrial production. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative labor. In the drawings:

[0024] Figure 1 A schematic diagram of an industrial real-time detection platform is provided for the embodiment of the present application;

[0025] Figure 2 A flowchart of a part defect detection method based on deep learning is provided for the embodiment of the present application;

[0026] Figure 3 A schematic diagram of micro part detection is provided for the embodiment of the present application;

[0027] Figure 4 A defect detection process schematic diagram provided for an embodiment of the present application;

[0028] Figure 5 A structure schematic diagram of a part defect detection equipment based on deep learning provided for an embodiment of the present application.

[0029] Reference signs:

[0030] 200: part defect detection equipment based on deep learning, 201: processor, 202: memory. DETAILED DESCRIPTION

[0031] The embodiment of the present application provides a part defect detection method, equipment and medium based on deep learning.

[0032] In order to enable personnel in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor shall fall within the scope of protection of the present application.

[0033] The technical solutions of the embodiments of the present application will be described in detail below with reference to the drawings.

[0034] The embodiment of the present application is mainly used for detecting small parts in industrial production. In most cases, parts with a length, width or height less than 10mm are regarded as small parts. Figure 1 An industrial real-time detection platform schematic diagram provided for an embodiment of the present application is shown in Figure 1 On the industrial production line, an industrial real-time detection platform is built for defect detection of small parts (such as 0.8cm knitting needle). The platform includes a speed-regulating conveyor belt, a data processor, a data acquisition sensor, a light source, a gear motor, a mechanical support component and a visual screen, etc. A high-speed industrial camera is used to collect small part images, and a Raspberry Pi B3 is used as the data processor, which is used for data analysis in cooperation with a workstation.

[0035] Figure 2 A part defect detection method flowchart based on deep learning provided for an embodiment of the present application is shown in Figure 2 The part defect detection method based on deep learning includes the following steps:

[0036] S101, obtain the running data of the industrial real-time detection platform and the part data, and determine the target running speed of the conveyor belt based on the detection capability coefficient-part movement speed relationship model.

[0037] In an implementation form of the present application, based on the running data of the industrial real-time detection platform, the field of view range corresponding to the data acquisition device and the detection response time are determined. And based on the part data, the part size and the part running distance are determined. Based on the comparison relationship between the image acquisition time and the detection response time, the detection capability coefficient-part movement speed relationship model is constructed by the running data and the part data. The optimal speed of the detection capability coefficient-part movement speed relationship model is solved to obtain the target running speed of the conveying belt.

[0038] Specifically, the running data of the industrial real-time detection platform contains the key parameter information of the equipment running. By analyzing the configuration parameters, historical acquisition records and other running data of the data acquisition device, the corresponding field of view range can be determined. The field of view range determines the area size covered by one shooting. The detection response time reflects the time required from the start of obtaining image data by the data acquisition device to the completion of analysis and output of the detection algorithm. The size information of the part, including length, width, height, etc., can be directly obtained from the design drawings, specifications and other data of the part. The part running distance is related to the movement of the conveying belt. By recording the starting position and target detection position of the part on the conveying belt, the running distance of the part during the detection process can be calculated.

[0039] Further, when the residence time of the part in the field of view is greater than or equal to the detection response time, the detection algorithm has enough time to complete the analysis of the part; otherwise, it may appear missed detection. Combined with the determined running data and part data, the detection capability coefficient-part movement speed relationship model is constructed. The function extreme point is determined by mathematical method, that is, the part movement speed that makes the detection capability coefficient reach the maximum value, that is, the optimal speed, is obtained. The optimal speed can ensure that the detection efficiency is considered while the detection accuracy is maximized and the risk of missed detection is reduced. The optimal speed is taken as the target running speed of the conveying belt.

[0040] Further, Figure 3 A micro part detection schematic diagram provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the size of the micro part is determined: length L1, width W1; camera field of view: length L2, width W2; distance d1 of the part to the bottom edge of the field of view; part spacing D1. Figure 3 view view

[0041] When the detection response time is not greater than the image acquisition time, the first relationship function is constructed:

[0042]

[0043] ​​​When the detection response time is longer than the image acquisition time, the second relationship function is constructed:

[0044]

[0045] Based on the first relationship function and the second relationship function, a detection capability coefficient-part movement speed relationship model is constructed. Through the analysis of this model, it is found that when the lens field of view W view When parameters such as model response time t and part spacing D1 are fixed, θ is a quadratic function of the conveyor speed v and has a maximum value. At this time, v is the optimal conveyor speed.

[0046] Where v is the conveyor belt speed; θ is a quadratic function of the conveyor belt speed; T is the detection response time; t is the image acquisition time; D1 is the part spacing; and W1 is the part width.

[0047] S102: After adjusting the conveyor belt based on the target running speed of the conveyor belt, the data acquisition device is used to acquire images of the parts on the conveyor belt to obtain images of the parts to be inspected.

[0048] In one implementation of the present application, when the conveyor belt is running stably at a target speed, the data acquisition device starts to work. The device starts image acquisition according to a preset trigger mechanism to obtain an image of the part to be inspected.

[0049] S103 , performing edge detection on the image of the part to be inspected to determine the part area, and dividing the part area into a plurality of grid units to extract feature information based on the grid units.

[0050] In one implementation of the present application, the image of the part to be inspected is converted into a binary image, and the boundary of the part is determined based on an edge detection algorithm. The edge pixel points are determined by the image gradient value corresponding to the boundary of the part, so as to determine the part area based on the edge pixel points. Based on historical inspection data, the defect distribution of the part is determined, and the part area is divided into multiple risk areas based on the defect distribution. Within the part area, based on the multiple risk areas, the image of the part to be inspected is divided into multiple grid units, and based on the risk levels corresponding to the multiple risk areas, each grid unit is grid-divided with different densities, and feature information is extracted for each grid unit; wherein the extracted feature information includes at least grayscale values ​​and edge geometric features.

[0051] Specifically, since the image of the part to be detected contains a large amount of redundant information and complex background, which is not conducive to defect detection, the embodiment of the application converts the pixel points in the image into a binary image according to the gray value difference through image binarization technology. Specifically, a threshold segmentation algorithm is used to set a suitable gray threshold, and the pixels higher than the threshold are set as white, representing the part area; and the pixels lower than the threshold are set as black, representing the background, thereby simplifying the image information. Secondly, an edge detection algorithm is used to identify the positions where the gray value changes sharply by calculating the image gradient, and the pixel points corresponding to these positions are the part boundary, thereby outlining the contour range of the part. The edge pixel points are further selected according to the image gradient value corresponding to the boundary. The image gradient reflects the rate of change of the pixel gray value, and the points with larger gradient values are usually located at the part edge. By setting a gradient threshold, the pixel points greater than the threshold are retained as edge pixel points, thereby constituting the contour boundary of the part, and the complete part area is determined.

[0052] Further, by statistically analyzing the historical detection data, the regularity and distribution pattern of part defects are determined. For example, it is analyzed that the crack defects of a certain type of part are concentrated in the edge region, and the hole defects are mostly distributed in the central part. Based on these analysis results, the part area is divided into multiple risk areas, each area corresponding to a different defect occurrence probability. According to the divided risk areas and the corresponding risk levels, the part area is divided into grids. For areas with high risk levels, such as the edge area with high defect incidence, high-density grid division is used to capture defect features more finely; and for areas with low risk levels, low-density grid division is used to reduce unnecessary calculation. After the division is completed, multiple information including gray value and edge geometric features are extracted for each grid unit. The gray value reflects the light and dark degree of the pixels in the grid, which can be used to identify surface stains, wear and other defects; and the edge geometric features such as edge curvature, angle, length, etc. are used to detect crack, notch and other shape abnormal defects. The embodiment of the application improves the accuracy of defect detection by multi-dimensional feature extraction.

[0053] S104, input the feature information into the preset SSD part defect detection network to obtain part defect information.

[0054] In an implementation manner of the application, different levels of feature maps corresponding to the preset part image are extracted by a convolutional neural network. Multiple default boxes with different sizes are set in each feature map, and the coordinates of the default boxes on the feature map are mapped to the preset part image. The probability that the defect target in the default box belongs to each type of defect is determined by multi-classification cross-entropy, and the positional deviation of the prediction box and the real defect box is determined by using smooth L1 loss. The preset SSD part defect detection network is optimized based on the probability and the positional deviation.

[0055] Specifically, the convolutional neural network (CNN) performs layer-by-layer abstract processing on the preset part image through the combination of multiple convolutional layers and pooling layers. The bottom convolutional layer retains the basic information of the image, such as edge, texture and other detailed features, and is suitable for detecting surface defects of small parts; the high-level convolutional layer extracts semantic information and can identify the overall shape and complex defect pattern of the part. Through the hierarchical extraction mechanism, the CNN outputs feature maps of different levels. On each level feature map, according to the size of the part and the distribution law of the defect, multiple default boxes of different sizes and proportions are set. Small size default boxes are used to capture small defects, and large size default boxes are suitable for larger range defects or the whole part. The coordinates of each default box on the feature map are mapped to the actual position of the original preset part image through step calculation.

[0056] Further, the feature information in each default box is input into a classification subnetwork, and the Softmax function is used to output the probability distribution of the region belonging to each category of defects or background. The multi-classification cross-entropy loss function measures the difference between the predicted probability and the true label, and the greater the probability deviation, the higher the loss value, so as to drive the network to learn more accurate defect classification ability. The regression subnetwork adjusts the default box to generate a predicted box. The smooth L1 loss function quantifies the position deviation by calculating the difference between the predicted box and the real defect box in the center coordinates, width and height dimensions. The defect category probability calculated by the multi-classification cross-entropy and the position deviation calculated by the smooth L1 loss are combined to form the total loss function of the SSD network. Through the back propagation algorithm, the total loss is reversely transmitted along the network level, and the parameters of each layer of the network are automatically adjusted, so that the category probability of the predicted box is closer to the true label, and the position is closer to the real defect box.

[0057] Further, the obtained features are input into the preset SSD part defect detection network to output complete information including defect position, category and confidence.

[0058] S105、In the preset detection period, the time sequence of the intermediate variable is associated to obtain the intermediate variable sequence.

[0059] In an implementation manner of the present application, during the part detection process, an intermediate variable dataset is obtained; wherein the intermediate variable dataset at least includes image acquisition data, detection data and platform running data. Taking the data acquisition time corresponding to the detection data as the benchmark, the time stamps corresponding to the intermediate variable dataset are mapped to the same time axis. Based on the fixed length time window, the time axis is windowed and slid, and the data features of each window are extracted. In the preset detection period, the extracted data features are arranged based on the time sequence to obtain the intermediate variable sequence.

[0060] Specifically, in the whole process of part detection, intermediate variable data generated in key links is collected and summarized in real time to form an intermediate variable dataset. Among them, image acquisition data covers shooting parameters of an industrial camera, image transmission time consumption, a timestamp of triggering acquisition, etc., which affect image quality and detection basis; detection data includes input features of an SSD network, prediction box confidence, defect classification results, etc., which reflect the running state of the detection algorithm; platform running data records conveyor belt speed, device load, memory occupation, etc. system parameters, which embody the overall running condition of the industrial real-time detection platform.

[0061] Further, with the data acquisition time corresponding to the detection data as the core reference point, the timestamps of all intermediate variable data such as the triggering time of the image acquisition data and the recording time of the platform running data are mapped to the same time axis through a time calibration algorithm, ensuring that each data is aligned in the time dimension and eliminating analysis errors caused by time deviation.

[0062] Further, a fixed-length time window is used to divide the unified time axis. The window moves along the time axis at a set step, and each time a data segment is intercepted. For the data in each window, the system extracts multi-dimensional features. In the preset detection period, the data features extracted by each sliding window are arranged in time sequence to form an intermediate variable sequence.

[0063] S106, based on the intermediate variable sequence, the detection state of the industrial real-time detection platform is predicted, and the detection state based on the prediction is used to start the missed detection mechanism to perform secondary detection on the part defect information.

[0064] In an implementation manner of the present application, the intermediate variable sequence generated in the current detection period is input into a preset long short-term memory network. Statistical features are extracted from the intermediate variable sequence by the long short-term memory network, and a probability distribution of the detection state is output based on the statistical features. The probability distribution is compared with a preset state threshold, and the detection state of the industrial real-time detection platform is output based on the comparison result, wherein the detection state includes a normal state, a warning state and a risk state.

[0065] Specifically, the intermediate variable sequence generated in the current detection period is input into a pre-trained long short-term memory network, and the long short-term memory network automatically learns and extracts key statistical features in the sequence, such as long-term trend, short-term fluctuation and abnormal point information of data. Based on the extracted statistical features, the output layer calculates and outputs the probability distribution of the detection state through an activation function, i.e. the possibility of the detection platform being in a normal state, a warning state and a risk state, which quantifies the possibility of different states appearing in the form of probability values.

[0066] Further, the detection state probability distribution output by the LSTM network is compared with a preset state threshold. For example, when the probability of the detection platform being in a normal state is higher than 0.8, the normal state is determined; when the probability of the normal state is lower than 0.6 and the probability of the early warning state is higher than 0.3, the early warning state is determined; and when the probability of the risk state is higher than 0.5, the risk state is determined. According to the comparison result, the current detection state of the industrial real-time detection platform is output.

[0067] Further, different detection states correspond to different subsequent processing strategies: maintaining the existing detection process in the normal state; starting preliminary investigation and data marking in the early warning state; and immediately triggering the missed detection mechanism, performing secondary detection or manual intervention in the risk state, so as to realize intelligent monitoring and risk prevention and control of the detection platform state.

[0068] In an implementation manner of the present application, the embodiment of the present application takes 0.8 cm knitting needles as experimental objects, collects a data set containing four common defects (bent shape, length size error, end point size error and twist), a total of 3000 images, of which 2140 are training images and 860 are test images. The image data is subjected to data enhancement operations such as rotation (0°, 22.5°, 45°, 67.5°, 90°) to improve the model generalization ability.

[0069] According to the established relationship model and the experimental equipment and part structure, the experimental parameters are set, the relationship between the conveyor belt speed and the model defect detection ability coefficient is simulated and analyzed by using MATLAB, and the theoretical derivation is verified.

[0070] Comparing the algorithm of the embodiment of the present application with classic algorithms such as YOLO V3, Faster-RCNN and FPN, the results show that the embodiment of the present application has higher accuracy, shorter positioning time and lower missed detection rate under different defect types.

[0071] Meanwhile, the embodiment of the present application studies the influence of different camera fields of view and conveyor belt speeds on the detection results, and determines that the model detection capability is best when the field of view is 36.00 mm and the speed is 7.67 m / min.

[0072] The comparison data are shown in Table 1.

[0073]

[0074] Table 1

[0075] Figure 4 A defect detection process schematic diagram provided by the embodiment of the present application is shown in FIG. 1. Figure 4As shown, based on the detection capability coefficient-part movement speed relationship model, the target running speed of the conveying belt is determined, the image acquisition device is used to acquire the image of the parts on the conveying belt, the features of the acquired part image are extracted, and the extracted features are input into the SSD part defect detection network to output complete information containing the defect position, category and confidence. And based on the intermediate variable sequence, the detection state of the industrial real-time detection platform is predicted, and the missed detection mechanism is started based on the predicted detection state to perform secondary detection on the part defect information.

[0076] Figure 5 A structure schematic diagram of a part defect detection equipment based on deep learning provided by the embodiment of the application. The part defect detection equipment based on deep learning 200 comprises at least one processor 201 and a memory 202 in communication connection with the at least one processor 201. The memory 202 stores instructions executable by the at least one processor 201. The instructions are executed by the at least one processor 201 to enable the at least one processor 201 to acquire running data and part data of an industrial real-time detection platform, determine a target running speed of a conveying belt based on a detection capability coefficient-part movement speed relationship model, acquire an image of a part on the conveying belt by a data acquisition device after adjusting the conveying belt based on the target running speed of the conveying belt, determine a part region by edge detection on the part image, divide the part region into a plurality of grid units, extract feature information based on the grid units, input the feature information into a preset SSD part defect detection network, obtain part defect information, perform time sequence correlation on detection intermediate variables in a preset detection period to obtain an intermediate variable sequence, predict a detection state of the industrial real-time detection platform based on the intermediate variable sequence, start a missed detection mechanism based on the predicted detection state, and perform secondary detection on the part defect information.

[0077] A non-volatile computer storage medium provided by an embodiment of the present application stores computer-executable instructions, wherein the computer-executable instructions are configured to: obtain operating data and part data of an industrial real-time detection platform, and determine a target operating speed of a conveyor belt based on a detection capability coefficient-part movement speed relationship model; after adjusting the conveyor belt based on the target operating speed of the conveyor belt, perform image acquisition of parts on the conveyor belt through a data acquisition device to obtain images of parts to be inspected; perform edge detection on the images of the parts to be inspected to determine a part area, and divide the part area into a plurality of grid units, so as to extract feature information based on the grid units; input the feature information into a preset SSD part defect detection network to obtain part defect information; perform time-series association on detection intermediate variables within a preset detection cycle to obtain an intermediate variable sequence; predict the detection status of the industrial real-time detection platform based on the intermediate variable sequence, and activate a missed detection mechanism based on the predicted detection status to perform secondary detection on the part defect information.

[0078] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For relevant portions, refer to the descriptions of the method embodiments.

[0079] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. It will be apparent to those skilled in the art that various modifications and variations may be made to the embodiments of the present application. However, such modifications or substitutions do not deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A method for detecting part defects based on deep learning, characterized in that: The method comprises: Obtain the operating data and part data of the industrial real-time detection platform, and determine the target operating speed of the conveyor belt based on the detection capability coefficient-part movement speed relationship model; After adjusting the conveyor belt based on the target running speed of the conveyor belt, the data acquisition device is used to acquire images of the parts on the conveyor belt to obtain images of the parts to be inspected; Performing edge detection on the image of the part to be inspected to determine a part area, and dividing the part area into a plurality of grid units to extract feature information based on the grid units; Inputting the feature information into a preset SSD part defect detection network to obtain part defect information; In a preset detection period, time series correlation is performed on the detection intermediate variables to obtain an intermediate variable sequence; The detection state of the industrial real-time detection platform is predicted based on the intermediate variable sequence, and a missed detection mechanism is activated based on the predicted detection state to perform a secondary detection on the part defect information.

2. The method for detecting part defects based on deep learning according to claim 1, characterized in that: The acquisition of the operation data and part data of the industrial real-time detection platform and determination of the target conveyor belt speed based on the detection capability coefficient-part movement speed relationship model specifically includes: Based on the operating data of the industrial real-time detection platform, determining the field of view and detection response time corresponding to the data acquisition device; and, based on the part data, determining a part size and a part running distance; Based on the comparison relationship between the image acquisition time and the detection response time, the detection capability coefficient-part movement speed relationship model is constructed using the operation data and the part data; The optimal speed of the detection capability coefficient-part movement speed relationship model is solved to obtain the target running speed of the conveyor belt.

3. The method for detecting part defects based on deep learning according to claim 2, wherein: The step of constructing the detection capability coefficient-part movement speed relationship model using the operation data and the part data specifically includes: When the detection response duration is not greater than the image acquisition duration, a first relationship function is constructed: When the detection response time is longer than the image acquisition time, a second relationship function is constructed: Based on the first relationship function and the second relationship function, constructing the detection capability coefficient-part movement speed relationship model; Where v is the conveyor belt speed; θ is a quadratic function of the conveyor belt speed; T is the detection response time; t is the image acquisition time; D1 is the part spacing; and W1 is the part width.

4. The method for detecting part defects based on deep learning according to claim 1, wherein: The performing edge detection on the image of the part to be inspected to determine a part area, and dividing the part area into a plurality of grid units to extract feature information based on the grid units, specifically includes: Converting the image of the part to be inspected into a binary image and determining the part boundary based on an edge detection algorithm; Determine edge pixel points by using the image gradient value corresponding to the part boundary, so as to determine the part area based on the edge pixel points; Determining defect distribution of a part based on historical inspection data, and dividing the part area into a plurality of risk areas based on the defect distribution; In the part area, based on the multiple risk areas, the image of the part to be inspected is divided into multiple grid units, and based on the risk levels corresponding to the multiple risk areas, each grid unit is divided into grids with different densities; Feature information is extracted from each grid unit respectively; wherein the extracted feature information includes at least grayscale value and edge geometric features.

5. The method for detecting part defects based on deep learning according to claim 1, wherein: Before inputting the feature information into a preset SSD part defect detection network, the method further includes: Extract feature maps of different levels corresponding to the preset part images through convolutional neural networks; Setting a plurality of default boxes of different sizes in each of the feature maps, and mapping the coordinates of the default boxes on the feature maps to the preset part images; The probability that the defect target in the default box belongs to each category of defect is determined by multi-classification cross entropy, and the position deviation between the predicted box and the actual defect box is determined by smooth L1 loss; The preset SSD part defect detection network is optimized based on the probability and the position deviation.

6. The method for detecting part defects based on deep learning according to claim 1, wherein: The process of performing time-series correlation on the detection intermediate variables within the preset detection period to obtain the intermediate variable sequence specifically includes: During the part inspection process, an intermediate variable data set is obtained; wherein the intermediate variable data set includes at least image acquisition data, inspection data, and platform operation data; Based on the data acquisition time corresponding to the detection data, the timestamps corresponding to the intermediate variable data sets are mapped to the same time axis; Based on a time window of fixed length, the time axis is window-sliding, and data features are extracted for each window; In the preset detection period, the extracted data features are arranged based on chronological order to obtain the intermediate variable sequence.

7. The method for detecting part defects based on deep learning according to claim 1, characterized in that: The predicting of the detection status of the industrial real-time detection platform based on the intermediate variable sequence specifically includes: Inputting the intermediate variable sequence generated in the current detection cycle into a preset long short-term memory network; Extracting statistical features from the intermediate variable sequence through the long short-term memory network, and outputting a probability distribution of the detection state based on the statistical features; Comparing the probability distribution with a preset state threshold, and outputting a detection state of the industrial real-time detection platform based on the comparison result; The detection status includes normal status, warning status and risk status.

8. The method for detecting part defects based on deep learning according to claim 7, characterized in that: The starting of the missed detection mechanism based on the predicted detection status specifically includes: Mapping different detection states to different missed detection risk levels; Based on different levels of missed detection risk, different missed detection mechanisms are activated; Wherein, when the detection state is the normal state, the existing detection process is maintained; When the detection state is the warning state, a higher-resolution image of the part to be inspected is taken, and a warning message is sent to a manager; When the detection state is the risk state, the backup detection algorithm is started to perform a secondary detection on the image of the part to be inspected, and when the risk state is maintained for multiple consecutive preset detection cycles, the production line is suspended.

9. A part defect detection device based on deep learning, characterized in that: The device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute the method according to any one of claims 1 to 8.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions can execute the method according to any one of claims 1 to 8.

Images