Industrial product visual detection method and detector based on context feature fusion

Abstract

The application relates to an industrial product visual detection method and detector based on context feature fusion, belongs to the field of industrial product optical image processing, and is used for solving the problems that current detection algorithms exist in single detection of products, misjudgment and missed detection, secondary detection leads to slow detection speed, and specific defect detection lacks universality. The method of the application adopts a deep neural network and combines context information to perform an industrial product surface defect detection method, improves the detection speed and accuracy of industrial product surface defects, and has good universality.

Description

Technical Field

[0001] This disclosure relates to optical image processing of industrial products, and more particularly to a visual inspection method and detector for industrial products based on contextual feature fusion. Background Technology

[0002] Currently, product defect detection on industrial production lines is often done on a piece-by-piece basis, meaning that products have similar but independent characteristics. Optical imaging is used to detect defects in a specific product. Using deep learning technology for surface defect detection in industrial products can effectively replace semi-automated detection methods that rely on manual labor, improving both detection speed and accuracy.

[0003] Whether a product is defective often needs to be determined by the context of the previous or next product, such as defect detection of semi-finished belts or oil seal chains. Existing deep learning target detection algorithms detect products individually, which can lead to misjudgments and missed detections. The secondary confirmation of defects after the product inspection is completed can also slow down the detection speed and has poor universality. Summary of the Invention

[0004] In view of the above-mentioned prior art, the technical problem solved by the present invention is at least as follows:

[0005] (1) Current deep learning object detection algorithms for industrial product defect detection are performed on a piece-by-piece basis. For some industrial products that require contextual information to determine defects, this can lead to a large number of misjudgments and missed detections, reducing the accuracy of automated detection. Specifically, this manifests as a large number of misjudgments or omissions of defects that may exist at the edges. Taking the defect detection of oil seal chains as an example, if a chain roller appears at the edge and only half of the roller is captured within the current camera's field of view, the existing defect detection method will judge this half of the roller as a defective half and cut off its defect location. If the other half of the roller is in an area not captured by the camera (in the previous or next frame), then this segment of the chain is considered a good chain and does not have a defect. Because the roller appears at the edge, the existing algorithm makes a misjudgment, resulting in the waste of chain products. Even if appearing at the edge is a probabilistic event and accounts for a small percentage, the waste of industrial products is still a large amount when hundreds of thousands of industrial products are defect-detected every day on the industrial production line, and improvement is needed.

[0006] (2) Performing secondary verification after the deep learning object detection algorithm identifies a defect increases the complexity of the automated detection process. Furthermore, waiting for the detection results of the next image before performing secondary verification at the result level delays the cropping time, reducing the speed of automated detection and leading to inaccurate cropping positions. Additionally, secondary verification requires writing automated screening scripts for specific defect categories. This method only applies to half-roller defects within the judgment program, requiring consideration of all possible scenarios for screening specific defects. When other defects appear, similar scripts need to be written for judgment, making secondary verification lack universality.

[0007] Based on the above problems, the purpose of this invention is to provide a universal method for visual inspection of industrial products. This method utilizes the contextual information of optical images of industrial products and combines it with deep neural networks to detect defects in industrial products, thereby improving the detection speed and accuracy of defects in industrial production line products.

[0008] To achieve the above objectives, the technical solution of the present invention is as follows:

[0009] In a first aspect, the present invention proposes a visual inspection method for industrial products based on contextual feature fusion, the method comprising the following steps:

[0010] Acquire an optical image of the surface of the industrial product to be inspected, and extract features based on a preset first deep neural network to obtain a feature image;

[0011] Based on a pre-defined second deep neural network, defect detection is performed on the feature image to obtain the defect category and defect location;

[0012] Based on a pre-defined third deep neural network, the two feature images are fused to obtain fused features, and the defect categories of industrial products in the fused image are detected and judged.

[0013] Based on the detection results of the second and third deep neural networks, edge defects are analyzed and judged.

[0014] The context information refers to the optical image features of the previous or next image of the optical image feature map of the surface of the product to be inspected.

[0015] In the technical solution of the above method, the deep neural network used comprises three parts: image feature extraction, defect category and location generation, and context feature fusion classification. These three parts together form a context feature fusion detector for defect detection in specific industrial products. After defect classification, the detector further uses context information to determine whether there has been a missed detection or a false positive, thereby improving the reliability and accuracy of the defect detection results, accelerating the detection speed, and saving production costs.

[0016] In the above method, one implementation of the first deep neural network is that the first deep neural network is composed of N residual networks, where N is a set value;

[0017] The second deep neural network employs a single-stage object detection algorithm, using fully connected convolutional layers in the output layer to regress the location of the target box and its category;

[0018] The third deep neural network uses a long short-term memory network.

[0019] In the above method, one implementation of the second deep neural network is: generating anchor boxes of different sizes on the feature map, performing binary classification on each anchor box, and outputting M proposed detection boxes as defect locations;

[0020] Perform target multi-classification on M suggested detection boxes to predict the specific defect type at the defect location, where M is an integer value greater than 0.

[0021] In the above method, the fusion feature is obtained by splicing the front and back ends of two adjacent feature maps and obtaining the features of the two adjacent image data according to the determined context feature acquisition ratio. The length of the front and back ends of the splicing can be set. For the first image, some randomly generated features are used for splicing.

[0022] In the above method, the third deep neural network updates its parameters during training using the following loss function until the loss value is less than a third set threshold.

[0023]

[0024] In the formula: L cls () represents the loss function for binary classification of the target; Pr and Pr * These represent the probability of predicting the target and the true label value, respectively. This represents the weighting coefficients for multiple classifications of the target.

[0025] In the above method, the first deep neural network and the second deep neural network are connected in series, and their respective parameters are updated simultaneously during training using the following loss function until the loss value is less than the second set threshold.

[0026]

[0027] In the formula: L cls () represents the loss function for binary classification of the target; L reg () represents the loss function for bounding box regression localization; Pr and Pr * These represent the probability of predicting the target and the true label value, respectively; t r =(trx , t ry , t rw , t rh ), t r Let t be the four position coordinate vectors of the r-th predicted bounding box. rx t ry t rw t rh These are the center coordinates of the r-th prediction box and the width and height of the prediction box, respectively. Let be the coordinate vectors of the four positions of the r-th ground truth bounding box. Let be the center coordinates of the r-th ground truth bounding box, and be the width and height of the ground truth bounding box, respectively. and , respectively, are the weight coefficients for the two tasks of target multi-class classification and bounding box regression localization, and λ is the balancing parameter.

[0028] In the technical solution of the above method, the analysis and determination of edge defects includes:

[0029] If the second deep neural network determines that there is a defect at the edge, but the third deep neural network also determines that there is a defect at the edge, then the defect at the edge is true.

[0030] If the second deep neural network determines that there is no defect at the edge, but the third deep neural network determines that there is a defect at the edge, then the defect at the edge is a missed detection.

[0031] If the second deep neural network determines that there is a defect at the edge, but the third deep neural network also determines that there is no defect at the edge, then the defect at the edge is a misjudgment.

[0032] If the second deep neural network determines that the edge has no defects, but the third deep neural network also determines that the edge has no defects, then the defect at the edge is false.

[0033] In the above method, the second deep neural network uses no fewer than 300 defect labels during training to ensure that the deep neural network can learn the effective features of various defects, thereby ensuring the quality of defect detection.

[0034] Secondly, the present invention proposes an industrial product visual detector based on context feature fusion, the detector comprising a first deep neural network, a second deep neural network, a third deep neural network, and an analysis and judgment module;

[0035] The first deep neural network is configured to extract features from optical images of industrial product surfaces to obtain feature images;

[0036] The second deep neural network is configured to perform defect detection on the feature image map to obtain the defect category and defect location;

[0037] The third deep neural network is configured to fuse two feature images to obtain fused features and detect and determine the defect category of industrial products in the fused image.

[0038] The analysis and judgment module is configured to analyze and judge edge defects based on the detection results of the second deep neural network and the third deep neural network.

[0039] The context information refers to the optical image features of the previous or next image of the optical image feature map of the surface of the product to be inspected.

[0040] In the above-mentioned detector's technical solution, the first deep neural network consists of N residual networks, where N is a set value; the second deep neural network adopts a single-stage target detection algorithm, using fully connected convolutional layers in the output layer to regress the target box position and its category; the third deep neural network adopts a long short-term memory network.

[0041] In the above-mentioned detector's technical solution, the defect analysis in the analysis and judgment module includes:

[0042] If the second deep neural network determines that there is a defect at the edge, but the third deep neural network also determines that there is a defect at the edge, then the defect at the edge is true.

[0043] If the second deep neural network determines that there is no defect at the edge, but the third deep neural network determines that there is a defect at the edge, then the defect at the edge is a missed detection.

[0044] If the second deep neural network determines that there is a defect at the edge, but the third deep neural network also determines that there is no defect at the edge, then the defect at the edge is a misjudgment.

[0045] If the second deep neural network determines that the edge has no defects, but the third deep neural network also determines that the edge has no defects, then the defect at the edge is false. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 , one A schematic diagram of the context feature fusion method for visual inspection of industrial products in one embodiment;

[0048] Figure 2 , one A schematic diagram of the context feature fusion detector training process in one embodiment;

[0049] Figure 3 , one A schematic diagram of the context feature fusion detector detection process in one embodiment;

[0050] Figure 4 , one A schematic diagram of roller detection in one embodiment. Detailed Implementation

[0051] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," and "third" may explicitly or implicitly include one or more of that feature. The operations of the flowchart may not be implemented in sequence. Instead, the operations may be implemented in reverse order or simultaneously. In addition, one or more other operations may be added to the flowchart. One or more operations may be removed from the flowchart.

[0052] A schematic diagram of the overall process of this invention in one embodiment is shown below. Figure 1 As shown, an industrial production line includes:

[0053] Step 1: Acquisition of optical image data for industrial products;

[0054] Step 2: Image dataset production;

[0055] Step 3: Generation of the context feature fusion detector;

[0056] Step 4: Training the context feature fusion detector;

[0057] Step 5: Defect detection results for industrial products.

[0058] Each step will be explained in detail below.

[0059] Step 1: Acquisition of optical image data for industrial products.

[0060] Adding optical imaging equipment to industrial product production lines allows for high-quality, comprehensive optical imaging of each industrial product while the production line is moving at high speed. The images are then stored to obtain raw image data for surface defect detection.

[0061] Step 2: Generate an image dataset.

[0062] The industrial optical image data obtained in step 1 is labeled and processed into a dataset format usable by the context feature fusion detector, as follows:

[0063] Step 2-1: Defect labeling.

[0064] Specialized dataset annotation tools can be used to manually annotate each defect in the image data using bounding boxes, according to the format of the PASCAL VOC dataset. At least 300 defects of each type should be annotated to ensure that the deep neural network can learn effective features of various defects, thereby guaranteeing defect detection quality.

[0065] Step 2-2: Data processing.

[0066] Since the contextual feature information in this embodiment comes from the two adjacent image data before or after the current image data, to ensure the accuracy and effectiveness of the obtained contextual feature information, the image data and defect labels obtained in step 2-1 are uniformly processed. The image data is renamed and sorted according to the adjacent positions of the images to ensure that the contextual information of the image comes from the adjacent images. The defect labels are normalized, that is, the pixel position of the label is normalized to a number within [0, 1] according to the image size to ensure that the defect label information of adjacent parts can be obtained when obtaining the contextual features.

[0067] Steps 2-3: Generate integrated tag information.

[0068] Based on the requirements for defect detection in industrial products, determine the proportion of contextual features to be acquired and the definition standards for positive and negative samples after feature fusion.

[0069] The defect dataset is traversed sequentially. The presence of defects in two adjacent image data within the feature range is obtained according to the feature ratio. The fused feature data is determined to be a defect sample according to the positive and negative sample definition criteria. If it is not a defect sample, it is defined as a negative sample and a fusion label of 0 is added. Otherwise, it is defined as a positive sample and a fusion label of 1 is added.

[0070] Taking the misjudgment of half-roller defects in oil seal chain defect detection as an example, the feature acquisition ratio is determined based on the size of the roller in the image to ensure that the roller for judgment can be captured in two adjacent images. The presence of a defect is then determined within the defect annotation information according to this ratio. If a half-roller defect exists at the tail edge of the previous image and also at the head edge of the current image, both half-rollers are considered misjudged, and this fused feature is defined as a negative sample, assigned a fusion label of 0. All other cases are considered positive samples, assigned a fusion label of 1. This step ultimately generates an industrial image dataset containing image data, defect label information, and fusion label information.

[0071] Step 3: Context feature fusion detector generation.

[0072] The context feature fusion detector consists of three parts: a feature extraction network structure as the first deep neural network, a second deep neural network for identifying defect categories and locations, and a third deep neural network for further defect judgment through context feature fusion.

[0073] Step 4: Training the context feature fusion detector.

[0074] The industrial product image dataset generated in step 2 is fed into the context feature fusion detector generated in step 3 for training, resulting in a context feature fusion detector for detecting defects in specific industrial products. A schematic diagram of the training process is shown below. Figure 2 As shown.

[0075] Step 4-1: Initialize the parameters of the context feature fusion detector and the training parameters.

[0076] The parameters of the context feature fusion detector mainly include the parameters of three main network structures. The first deep neural network is a 50-layer residual network, which is a commonly used network structure for feature extraction in the field of object detection. The initialization parameters are the parameters of its pre-trained model trained on a public dataset.

[0077] The second deep neural network uses a single-stage object detection algorithm, which directly uses fully connected convolutional layers in the output layer to regress the location of the target box and its category. The initial parameters are pre-trained parameters trained on a public dataset.

[0078] The third deep neural network uses a long short-term memory network to perform binary classification on sequentially fused feature information. The initial parameters are those of a pre-trained model trained on a public dataset. This step also requires initializing training parameters such as image size, number of images per batch, defect category names and numbers, and number of training iterations.

[0079] Step 4-2: Obtain data in batches.

[0080] Image data, original label information, and fused label information are obtained from industrial product image datasets by batch size.

[0081] Step 4-3: Feature extraction.

[0082] Image data is fed into a feature extraction network after being standardized in size. The convolutional, activation, and pooling layers of the feature extraction network are used to perform convolution operations on the image data to obtain multiple feature maps, which are then used for subsequent operations and processing at the feature level.

[0083] Step 4-4: Defect location and category generation.

[0084] In step 4-3, anchor boxes of various sizes are generated for each pixel on the feature map. The SoftMax function is used to perform binary classification on each anchor box, finally outputting M proposed detection boxes as the target defect region. Further, target multi-classification is performed on the M proposed detection boxes to predict the specific defect type of the target defect region, and the position of each proposed detection box is calculated for bounding box regression localization. The SoftMax function is then used to perform target multi-classification on the features within the proposed detection boxes to determine the specific defect type of the target defect; the position of the proposed detection boxes is then calculated for bounding box regression localization. The input parameters of the objective function used for bounding box regression are the coordinates of the top-left corner of the proposed detection box and the width and height of the proposed detection box.

[0085] Steps 4-5: Calculation of defect location and category loss and parameter update.

[0086] Based on the target multi-class classification and bounding box regression localization tasks described in steps 4-4, the predicted values ​​of defect locations and categories in the current batch of industrial product images are obtained. These predicted values ​​are then compared with the actual values ​​of defect locations and categories in the defect labeling information. The calculation aims to minimize the difference between the predicted and actual values ​​of the target region. The loss function Loss2 of the defect location and category generation network branch is as follows:

[0087]

[0088] In the formula: L cls () represents the loss function for binary classification of the target; L reg () represents the loss function for bounding box regression localization; Pr and Pr * These represent the probability of predicting the target and the true label value, respectively; t r =(t rx , t ry , t rw , t rh ), t r Let t be the four position coordinate vectors of the r-th predicted bounding box. rx t ry t rw t rh These are the center coordinates of the r-th prediction box and the width and height of the prediction box, respectively. Let be the coordinate vectors of the four positions of the r-th ground truth bounding box. Let be the center coordinates of the r-th ground truth bounding box, and be the width and height of the ground truth bounding box, respectively. and These are the weight coefficients for the target multi-class classification and bounding box regression localization tasks, respectively, with λ being the balancing parameter. The loss value is calculated using the loss function, and the network parameters are updated in the direction that reduces the loss value. This part of the network parameters that needs to be updated includes the parameters of the second deep neural network and the parameters of the first deep neural network.

[0089] Steps 4-6: Intra-batch feature fusion and classification.

[0090] The image feature maps obtained in step 4-3 are traversed sequentially. Each time, the feature maps of two adjacent images are traversed. The front and back ends of the adjacent feature maps are extracted as needed and spliced ​​and fused. The fused feature data is sent to the third deep neural network for binary classification to obtain the predicted value of whether the fused feature information has defects.

[0091] Steps 4-7: Calculate the loss of the classification branch and update the parameters after fusing contextual features.

[0092] Based on the predicted value of whether there are defects in the current fused feature information obtained from the target binary classification task described in steps 4-6, a calculation is performed with the true value in the fused label information to minimize the difference between the predicted value and the true value of the fused feature information. The loss function Loss3 of the third deep neural network is obtained as follows:

[0093]

[0094] In the formula: L cls () represents the loss function for binary classification of the target; Pr and Pr * These represent the probability of predicting the target and the true label value, respectively. This represents the weighting coefficients for multiple classifications of the target.

[0095] The loss value is calculated using the loss function, and the network parameters are updated in the direction that the loss value decreases. The network parameters that need to be updated in this part are the parameters of the third deep neural network.

[0096] Steps 4-8 determine whether the training process is complete. If the loss value no longer decreases or reaches the number of training iterations set in the initialization, the training process is complete, and the model parameters of the current context fusion detector are saved; otherwise, the training process is not complete, and batch images are taken from the industrial product image dataset for training.

[0097] The trained context feature fusion detector can be applied to defect detection. To ensure the effectiveness of the context feature fusion detector, it should be tested before application, such as... Figure 3 As shown, the steps are as follows:

[0098] Step 5-1: Image preprocessing.

[0099] The input image data is preprocessed to obtain raw image data that can be used for detection by the context feature fusion detector.

[0100] Step 5-2: Model loading.

[0101] Deploy context feature fusion detectors on different industrial production lines, load pre-trained network parameters, and learn using a dataset specific to each industrial production line. The learned detectors can then be used to detect various industrial products on the production line.

[0102] Step 5-3: Feature extraction.

[0103] The feature extraction network in the context feature fusion detector is used to extract features from the image to be detected and generate a feature map.

[0104] Step 5-4: Defect location and defect classification generation.

[0105] The feature map extracted in step 5-3 is fed into the defect location and category generation network to detect the defect information of the current industrial product, and the defect location and defect classification are generated for subsequent analysis and screening.

[0106] Step 5-5: Contextual feature fusion.

[0107] The feature maps extracted in step 5-3 are fused with contextual features. According to the specific circumstances of this embodiment, the feature maps that need to be stitched together are those from the tail edge of the previous image and the head edge of the current image. This is used to determine whether there are any missed detections or misjudgments due to photographic issues at the connection point. For the first image, randomly generated partial features are used for stitching. One specific method is to use an existing Python matrix library to generate entirely black or entirely white partial feature maps according to the required matrix size for stitching. For other images, partial features from the saved previous image can be used for stitching. The last image is stitched with the features saved from the previous image, and since there are no subsequent images that need to be detected, the saved partial feature maps can be deleted. The stitched feature maps are then fed into a third deep neural network for defect detection, generating a category indicating whether the fused feature map contains defects.

[0108] Steps 5-6: Result analysis and screening.

[0109] Based on the specific defect locations and categories detected in step 5-4 and the categories of defects detected at the edges in step 5-5, determine whether the defects at the edges are false positives and filter them out.

[0110] Specifically, if both the second and third deep neural networks determine that there is a defect at the edge, the defect is considered true, and the result is returned. If the second deep neural network determines that there is a defect at the edge but the third deep neural network determines that there is no defect, the defect is a false positive and should be deleted before returning the result. If the second deep neural network determines that there is no defect at the edge but the third deep neural network determines that there is a defect, the defect is a missed detection and the defect information should be added to the head edge of the current image before returning the result. If both the second and third deep neural networks determine that there is no defect at the edge, the edge is normal, and the result is returned.

[0111] Steps 5-6: Outputting results and subsequent processing.

[0112] After analyzing and judging the detection results of the context feature fusion detector, the defect location and category are output as the final model detection result, and corresponding adjustments are made to the subsequent industrial process.

[0113] like Figure 4 As shown, the dashed line represents the first image, and the solid line represents the second image. The images were taken sequentially, with a time interval of t between the two images. The original industrial process was that the end of the first image only captured half a roller, which was considered a defective half-roller. A rejection signal was issued to remove the roller at the end of the first image, resulting in misjudgment and waste.

[0114] According to the method of the present invention, a specific implementation of an industrial process adjustment is as follows: after a defect is detected in the product at the end of the first image, since the defect is at the end, the rejection signal for this defect is first cached and delayed until the detection result of the second image information is obtained, i.e., a delay time t. If the context detection still determines that there is a defect, the roller at the beginning of the second image is removed, becoming the same roller rejected in the original process. If the context detection determines that there is no defect, and it is only because the end was captured, the delayed rejection signal is cleared, reducing the rejection of one roller, reducing waste, and increasing the yield rate compared to the original industrial process. Another feasible industrial process is that the rejection signal is only generated after the edge defect is analyzed and judged based on the detection results of the second and third deep neural networks. Here, the industrial product is taken as a roller, but when the method of the present invention is applied, it can also be a belt semi-finished product, an oil seal chain, etc.

[0115] According to step 5, the trained context feature fusion detector is tested in an actual production environment and then fine-tuned, so that it can be used for actual defect detection.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that the method and detector of this disclosure can be implemented by means of software plus necessary general-purpose hardware, or by special-purpose hardware including application-specific integrated circuits, dedicated CPUs, dedicated memory, dedicated components, etc. Generally, any function performed by a computer program can be easily implemented using corresponding hardware, and the specific hardware structure used to implement the same function can be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the purposes of this disclosure, software program implementation is more often a preferred implementation method.

[0117] Although embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of the present invention, and all of these are within the scope of protection of the present invention.

Claims

1. A visual inspection method for industrial products based on contextual feature fusion, characterized in that, The method includes the following steps: An optical image of the surface of the industrial product to be inspected is acquired, and feature extraction is performed based on a preset first deep neural network to obtain a feature image; Based on a pre-defined second deep neural network, defect detection is performed on the feature image to obtain the defect category and defect location; Based on a pre-defined third deep neural network, which employs a long short-term memory network, two adjacent feature images are spliced ​​and fused to obtain fused features. The defect categories of industrial products in the fused image are then detected and determined. The fused features are obtained by splicing the front and back ends of two adjacent feature images and acquiring features of the adjacent image data according to a determined context feature acquisition ratio. The length of the front and back ends of the splicing can be set. For the first image, randomly generated partial features are used for splicing. Based on the detection results of the second and third deep neural networks, edge defects are analyzed and judged, including: if the second deep neural network judges that there is a defect at the edge, but the third deep neural network judges that there is a defect at the edge, then the defect at the edge is real; If the second deep neural network determines that the edge has no defects, but the third deep neural network determines that the edge has defects, then the defect at the edge is a missed detection; if the second deep neural network determines that the edge has defects, but the third deep neural network also determines that the edge has no defects, then the defect at the edge is a false positive; if the second deep neural network determines that the edge has no defects, but the third deep neural network also determines that the edge has no defects, then the defect at the edge is a false positive. The context information refers to the optical image features of the previous or next image of the optical image feature map of the surface of the product to be inspected.

2. The method according to claim 1, characterized in that, The first deep neural network consists of N residual networks, where N is a set value; The second deep neural network employs a single-stage object detection algorithm, using fully connected convolutional layers in the output layer to regress the location of the target box and its category.

3. The method according to claim 1, characterized in that, The second deep neural network generates anchor boxes of different sizes on the feature map, performs binary classification on each anchor box, and outputs M proposed detection boxes as defect locations; Perform target multi-classification on M suggested detection boxes to predict the specific defect type at the defect location.

4. The method according to claim 1, characterized in that, The third deep neural network updates its parameters during training using the following loss function until the loss value is less than a third set threshold. In the formula: The loss function for binary classification of the target; and These represent the probability of predicting the target and the true label value, respectively. This represents the weighting coefficients for multiple classifications of the target.

5. The method according to claim 1, characterized in that, The first deep neural network and the second deep neural network are connected in series. During training, their respective parameters are updated simultaneously using the following loss function until the loss value is less than the second set threshold. In the formula: The loss function for binary classification of the target; The loss function for bounding box regression localization; and These represent the probability of predicting the target and the true label value, respectively. , Let be the four position coordinate vectors of the r-th predicted bounding box. , , , These are the center coordinates of the r-th prediction box and the width and height of the prediction box, respectively. , Let be the coordinate vectors of the four positions of the r-th ground truth bounding box. , , , Let be the center coordinates of the r-th ground truth bounding box, and be the width and height of the ground truth bounding box, respectively. and These are the weight coefficients for the two tasks: target multi-class classification and bounding box regression localization. These are the balancing parameters.

6. An industrial product visual detector based on contextual feature fusion according to claim 1, characterized in that, The detector includes a first deep neural network, a second deep neural network, a third deep neural network, and an analysis and judgment module; The first deep neural network is configured to extract features from optical images of industrial product surfaces to obtain feature images; The second deep neural network is configured to perform defect detection on the feature image map to obtain the defect category and defect location; The third deep neural network is configured to fuse two feature images to obtain fused features and detect and determine the defect category of industrial products in the fused image. The analysis and judgment module is configured to analyze and judge edge defects based on the detection results of the second deep neural network and the third deep neural network. The context information refers to the optical image features of the previous or next image of the optical image feature map of the surface of the product to be inspected.

7. The detector according to claim 6, characterized in that, The first deep neural network consists of N residual networks, where N is a set value; the second deep neural network uses a single-stage object detection algorithm, and uses fully connected convolutional layers in the output layer to regress the position of the target box and its category; the third deep neural network uses a long short-term memory network.

Images