Method and device for detecting foreign matters on surface of bath solution of production line

Abstract

The invention relates to a production line bath liquid surface foreign matter detection method and device, and the method comprises the following steps: a camera is installed on a perforated surface light source to form a detection device, the detection device is moved right above bath liquid, and the light source is turned on to form a bath liquid image with a camera inverted image on the surface of the bath liquid; the method comprises the following steps: collecting a picture, dividing the picture into a deep color region and other regions, and dividing each region into small image blocks with different synchronization lengths and starting points; in a training stage, performing image data enhancement on the small image blocks, and expanding training samples; manually marking data sets with foreign matters and data sets without foreign matters; carrying out ResNet-50 deep learning network model training to obtain training parameters; in an actual measurement stage, threshold segmentation and morphological segmentation are combined to distinguish a dark grey area and other areas, corresponding training parameters are loaded respectively, and foreign matter positions are broken and marked through a ResNet-50 network model. The device comprises an area light source, an industrial camera, a control background and a display terminal. And real-time detection of the foreign matters on the surface of the bath solution is realized through mutual cooperation of all parts. The automatic detection of the foreign matters on the surface of the production line bath solution can be realized, and a new way for automatically detecting the foreign matters on the surface of the production line bath solution is provided.

Description

Technical Field

[0001] The present invention relates to a method and device for detecting foreign objects on the surface of bath solution in a production line, which can realize automatic detection of foreign objects on the surface of bath solution in the production line. Background Art

[0002] In the production lines of aluminum alloy or stainless steel, it is necessary to perform processes such as degreasing, water washing, pickling, and surface conditioning on the processed workpieces. These steps are usually achieved by arranging different working baths, in which solutions such as boric acid and sulfuric acid, nitric acid, etc. are placed according to different production processes. However, problems such as the adverse effects of foreign objects such as oil stains and residues generated after long-term use of the bath solution in the production line on the quality of workpieces. In the actual production scenario, since the bath solution is opaque and foreign objects will float on the surface, there are special personnel to conduct regular inspections, and manual cleaning is carried out after the foreign objects are found. During the processing of metals such as aluminum alloy and stainless steel, low-contrast foreign objects such as oil films, saponified products, and flocculants often accumulate on the surface of the bath solution. The detection of such foreign objects is extremely difficult due to the following characteristics:

[0003] (1) When foreign objects such as oil stains are distributed on the surface of the bath solution, the image contrast is low and it is difficult to distinguish.

[0004] (2) The surface of the bath solution is interfered by ambient light and specular reflection will occur, masking the characteristics of foreign objects.

[0005] (3) The sizes of production baths vary, with the size distribution ranging from 1 meter to 4 meters, and the distribution of foreign objects is random. As a result, the larger the size of the bath solution, the more difficult it is for manual visual inspection, and at the same time, the labor intensity is high, and visual fatigue and quality fluctuations are likely to occur.

[0006] In the research on the automated detection of foreign impurities on the surface of opaque bath solutions in industrial production lines, there is almost a blank at present. Most of the existing solution foreign matter detection equipment is only applicable to small bottled transparent solutions. For example, the automatic lamp inspection machine developed by Seidenader Company in Germany uses the method of image contrast subtraction to detect the appearance defects and visible foreign matters in solutions and suspensions. However, it is only applicable to medicines or small bottled transparent liquids and cannot be used for the detection of foreign matters on the surface of the working tank structure of the production line. Domestically, such as the intelligent lamp inspection machine developed by the Wang Yaonan team of Hunan University for the detection of foreign matters in large infusion solutions for medical use, uses the black background imaging method, combines the method of high-speed rotation and rapid stop to collect multiple frame image sequences, and then uses the method of inter-frame second-order difference and energy accumulation to detect tiny foreign matters in the liquid medicine. Fang Jifei of Hunan University further proposed a method to obtain the depth information of foreign matters using binocular vision to improve the recognition accuracy. This method can effectively overcome the problems of dynamic background interference and narrow depth of field, but does not classify foreign matters according to the depth information. The Gao Chunming team of the University of Electronic Science and Technology designed a working mode of rapid rotation and stop to collect the changing sequence images of the solution, and used SVM for detection and classification. However, this detection method that intervenes in the measurement object is not applicable to the production line. Summary of the Invention

[0007] In view of the existing actual needs, the technical problem to be solved by the present invention is to provide a method for detecting foreign matters on the surface of the bath solution in a production line. By hoisting an opening surface light source and a camera directly above the bath solution, collecting pictures of the bath solution surface, combining threshold segmentation and morphological segmentation to distinguish the dark gray area from other areas, loading corresponding training parameters for different areas, and classifying and analyzing different areas in the form of small image blocks with different step sizes and starting points, the detection of foreign matters on the surface of the bath solution is realized.

[0008] The technical solution adopted by the present invention to achieve the above object is as follows:

[0009] A method for detecting foreign matters on the surface of the bath solution in a production line, comprising the following steps:

[0010] S1. Image acquisition and preprocessing stage:

[0011] Turn on the surface light source and adjust the brightness so that the light irradiates the surface of the bath solution to be measured; control the industrial camera to collect images of the bath solution surface;

[0012] Analyze the distinguishing features between the camera reflection and the surrounding bath solution environment, and use image processing methods to separate the camera reflection area from other areas; divide the camera reflection area and other areas into two types of image block sequences respectively;

[0013] S2. Training stage:

[0014] Perform data enhancement on the two types of image block sequences, expand the image block sample set and save it as an image file;

[0015] Manually annotate the image files, classify them into four categories according to whether there are foreign objects in the camera reflection area and other areas, with or without foreign objects, and obtain a four-category image dataset with labeled classification required for training;

[0016] Use the deep learning model framework ResNet-50 to iteratively learn the sample sets of the camera reflection area and other areas with labeled classification, and obtain their respective network structure parameters and ideal models I and II when the training cutoff condition is met;

[0017] S3. Measurement stage:

[0018] Collect and obtain two types of image patches to be detected in real time according to step S1; respectively load the obtained network structure parameters, and use ideal models I and II to infer and classify the two types of image patches to judge whether there are foreign objects;

[0019] Draw a frame annotation on the original image for the image patches judged to have "foreign objects";

[0020] Take the union of adjacent image frames and redraw the frame annotation on the original image to complete the detection.

[0021] The distinguishing feature between the camera reflection and the surrounding bath solution environment is that the bath solution area where the camera reflection is located shows unequal contrast compared to the surrounding bath solution environment.

[0022] The entire collected picture is separated into the camera reflection area and other areas by means of binarization and morphological filtering.

[0023] The two types of image patch sequences are divided as follows: within the dark gray area and other areas respectively, traverse and divide them into small image patches with unequal pixel sizes at a step size of N / 2, where the small image patches are square image patches with a size of N*N pixels.

[0024] The data augmentation in the training stage is as follows: for the obtained image patch sequence, flip and linearly enhance each one by one, and then rotate each image patch clockwise or counterclockwise around the center point at a preset angle as the step interval until 90 degrees, so as to obtain more image patches, which are saved as image files as the sample set.

[0025] The improvement of the deep learning model framework based on the traditional ResNet-50 includes:

[0026] Prune the last two residual blocks of the traditional ResNet-50, and retain the single convolutional layer STAGE0 and the first two residual blocks: the first residual block STAGE1 and the second residual block STAGE2;

[0027] Reduce the two convolutional layers in the first residual block STAGE1 to one convolutional layer;

[0028] The number of filters in each convolutional layer of the first residual block STAGE1 and the second residual block STAGE2 is halved;

[0029] An SE attention mechanism module is embedded after each convolutional layer of the second residual block STAGE2, and the output result of the excitation part is used as the weight value of each feature channel.

[0030] In the actual measurement stage, the image blocks judged as "with foreign objects" are framed and marked on the original image, including:

[0031] For the image blocks judged as "with foreign objects", the center coordinates of the image blocks are located from the small image blocks to the original whole image. Taking the center coordinates at this time as the center of the square, a frame is marked on the whole image;

[0032] The above-mentioned obtaining the union of adjacent image frames and re-framing and marking on the original image to complete the detection includes:

[0033] Taking the union of the annotation frames of adjacent small image blocks with a distance of no more than 1 small image block and merging them into a large annotation frame, which is marked on the image.

[0034] A foreign object detection device for the surface of production line tank liquid, including: a surface light source, an industrial camera, a control background, and a display terminal;

[0035] The surface light source is placed above the tank liquid to be measured in a hanging manner, and is used to provide light to the tank liquid to be measured;

[0036] One or several industrial cameras are installed through a central opening on the surface of the surface light source, and the industrial cameras face downward to collect images of the surface of the tank liquid to be measured;

[0037] The control background is a PC or intelligent terminal machine set in the control room; it is internally provided with a memory and a processor. A program module is stored in the memory. When the processor loads the program module, the above method steps are executed to identify and locate foreign objects on the surface of the tank liquid; and statistical calculations are performed according to the foreign object identification and location results;

[0038] The display terminal is used to provide a visual interface for users, collect human input parameters and instructions, display various image data during the process of identifying foreign objects in the tank liquid, and the statistical calculation results.

[0039] The parameters of the above-mentioned human-computer interaction include:

[0040] During the image preprocessing process: the window pixel size, starting pixel point, and pixel step size for dividing small image blocks; optional image block flipping, linear enhancement, and image enhancement operations such as the rotation direction and angle step size of each image block;

[0041] The network parameters of the ResNet-50 structure corresponding to the gray area and other areas during the training phase: number and size of convolution kernel layers, activation function, training batch, number of iterations, probability confidence threshold, and loss function threshold;

[0042] The instructions for human-computer interaction include: a surface light source start-up instruction and an image acquisition trigger instruction issued to the industrial camera; the statistical calculation results are probability confidence, loss function, false alarm rate, and accuracy calculation.

[0043] The present invention has the following beneficial effects and advantages:

[0044] The present invention provides a method for detecting foreign matter on the surface of production line tank liquid. The method improves the image contrast between foreign matter and tank liquid by reflecting imaging with a large plane light source. The method divides the image into dark gray and two other different areas by combining threshold segmentation with morphological segmentation. Different training parameters are used to detect foreign matter on the surface of tank liquid, thereby reducing missed detections. The method divides the image into small image blocks by setting different step sizes and starting points for classification analysis, thereby converting the target detection problem into a target classification problem and reducing the difficulty of detection. During the training phase, the small image blocks obtained by division are subjected to enhancement methods such as rotation, flipping, and linear transformation at different angles and step sizes of 1-90 degrees to expand the sample set and reduce the lengthy sample collection work in the early stage. The present invention can automatically inspect the foreign matter situation on the surface of production line tank liquid and can be applied to surface foreign matter detection scenarios of other solutions, greatly improving production efficiency and saving labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of the method of the present invention;

[0046] Figure 2 Schematic diagram of the arrangement of the camera and the open-hole surface light source of the method of the present invention;

[0047] Figure 3 Schematic diagram of the ResNet-50 network model structure improved by the method of the present invention;

[0048] Figure 4 The calculation steps of the ResNet-50 network of the method of the present invention;

[0049] Figure 5 This is a diagram showing the actual detection effect of broken paper shells using the method of the present invention;

[0050] Figure 6 This is a diagram showing the actual oil pollution detection effect of the method of the present invention;

[0051] Figure 7 This is a diagram showing the actual on-site detection effect of the method of the present invention. DETAILED DESCRIPTION

[0052] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe in detail the specific implementation methods of the present invention with reference to the accompanying drawings. Many specific details are set forth in the following description in order to fully understand the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the invention. Therefore, the present invention is not limited by the specific implementations disclosed below.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The following will further describe the present invention in detail with reference to the accompanying drawings and embodiments.

[0054] Technical solution: Taking the crushing of paper shells, oil stains, and cotton fluffs as foreign objects as an example, the detection of foreign objects on the surface of the production line tank liquid is described in detail.

[0055] Example 1, a method for detecting foreign objects on the surface of the production line tank liquid of the present invention, includes the following steps:

[0056] Step 1: Turn on the light source to form a reflection on the surface of the tank liquid, and control the camera to collect pictures of the surface of the tank liquid. The reflection of the camera is in the dark gray area in the middle of the image.

[0057] Step 2: Use one camera to collect the entire picture; or use multiple cameras to collect small-angle pictures respectively, and detect whether there are foreign objects in each image collected respectively.

[0058] Extract the middle dark gray area by combining threshold segmentation and morphological segmentation. The dark gray area and other areas are divided into different small image blocks according to different starting points and step lengths. The size of the small image block is a square image block of 100*100 pixels.

[0059] Step 3: In the training stage, flip and linearly enhance the image blocks obtained in Step 2, and then rotate each image block clockwise by 1 degree at a time interval with the center point as the center until 90 degrees, so as to obtain more image blocks, which are saved as image files as the sample set.

[0060] Step 4: Manually label the image files obtained in Step 3, classify them into 4 folders of having foreign objects and no foreign objects in the dark gray area, and having foreign objects and no foreign objects in other areas. The folder names are the labels, and the labeled data set required for training is obtained.

[0061] Step 5: Use two ResNet-50 deep learning models I and II to train the dark gray area sample set and the other area sample set respectively, obtain the training parameters of their respective ResNet-50 deep learning models, and the ideal models I' and II' corresponding to the training parameters;

[0062] Step 6: In the actual measurement stage, extract the middle dark gray area and the other areas through the combination of threshold segmentation and morphological segmentation, divide them into multiple small image blocks as detection objects respectively, load the training parameters of the ResNet-50 deep learning models I and II obtained in Step 5, use the ideal models I' and II' to infer the small image blocks, classify the small image blocks, and determine whether there are foreign objects;

[0063] Step 7: Draw a frame annotation on the original image for the image blocks determined to have "foreign objects";

[0064] Step 8: Take the union of adjacent image frames obtained in Step 7, draw a frame annotation on the original image again to complete the detection.

[0065] The following is a detailed description of the above key steps:

[0066] Step 2: For the entire collected image, extract the middle dark gray area representing the camera reflection through the combination of threshold segmentation and morphological segmentation. The dark gray area and the other areas are divided into different small image blocks according to different starting points and step lengths. The small image blocks are square image blocks with a size of 100*100 pixels, including:

[0067] The size of the collected image is 3000*4096 pixels. In order to obtain sufficient samples in the training stage and accurate results in the actual measurement stage, the entire image is divided into square image blocks with a size of 100*100 pixels. Note that the size of the image blocks can be adjusted by yourself, such as 64*64 pixels, 128*128 pixels, 500*500 pixels, etc. The small image blocks in this step are obtained by looping from the entire image after setting the starting point and step length of the traversal division. For example, the starting point is (1, 1) and the step length is 50 pixels, that is, the four vertices of the first small image block are (1, 1), (101, 1), (1, 101), (101, 101), and the four vertices of the second small image block are (51, 1), (151, 1), (51, 151), (151, 151), and so on, to obtain small image blocks with overlapping areas. Note that the starting point and step length of the image can be set according to needs. Since the camera is embedded on the open-hole surface light source, the reflection of the camera in the middle of the image is a dark gray area, and the gray value difference from the other areas is large, so binary threshold segmentation and morphological segmentation methods need to be applied for separate extraction and processing.

[0068] Step 3: In the training stage, flip and linearly enhance the image patches obtained in the previous step. Then, for each image patch, rotate it clockwise around its center point by 1 degree at a time interval until it reaches 90 degrees, to obtain more image patches, which are saved as image files in the sample set, including:

[0069] This step is required in the training stage. The purpose is to expand the number of samples. Since the training stage is only carried out once, this step is also only carried out once. First, flip each of the small image patches obtained in Step 2 around the rows and columns of the image respectively, so as to obtain image patches that are 3 times the number of the previous small image patches. Then, perform linear enhancement to obtain the image patches after image enhancement. At this time, the above image patches each rotate clockwise around their respective image center points by 1 degree at a time interval until they reach 90 degrees, so as to obtain more image patches, which are used as the sample set and saved as image files.

[0070] Step 4: Manually label the image files obtained in Step 3 and classify them into 4 folders: foreign objects in the dark gray area with and without foreign objects, and foreign objects in other areas with and without foreign objects. The folder names are the labels, and the required training dataset is obtained, including:

[0071] This step is required in the training stage. Since the training stage is only carried out once, this step is also only carried out once.

[0072] Step 5: Improve the network structure based on the traditional ResNet-50 deep learning model, train the sample sets in the dark gray area and other areas, and obtain their respective training parameters, including:

[0073] Use the typical ResNet-50 structure. The input layer is the small image patch. This network includes a residual network with 50 convolutional layers. The residual module realizes a very high-accuracy image recognition ability through a deep network. The ResNet-50 network consists of a single convolutional layer (STAGE 0), 4 groups of modules in the middle (STAGE 1, STAGE 2, STAGE 3, STAGE 4), and a final fully connected layer. Among them, there are 3 small modules in STAGE 1, each small module contains 3 convolutional layers, and each convolutional layer has 64 convolutional kernels; there are 4 small modules in STAGE 2, each small module contains 3 convolutional layers, and each convolutional layer has 128 convolutional kernels; there are 6 small modules in STAGE 3, each small module contains 3 convolutional layers, and each convolutional layer has 256 convolutional kernels; there are 3 small modules in STAGE 4, each small module contains 3 convolutional layers, and each convolutional layer has 512 convolutional kernels, and the activation function is tanh.

[0074] The small image patch dataset is pre-divided into two categories: "no foreign object" and "with foreign object", and the ResNet-50 neural network model is pre-trained. Due to the small dataset, the model is improved as follows: the two convolutional layers in the first residual block STAGE1 of the traditional ResNet-50 are reduced to one convolutional layer; the four residual blocks in ResNet-50 are reduced to two residual blocks, that is, the redundant levels of STAGE3 and STAGE4 are deleted; the number of filters in each convolutional layer is halved, for example, the 256 in BTNK1 of STAGE2 is halved to 128; and an SE module is added in STAGE2 of ResNet-50, introducing an attention mechanism. The output result of the excitation part is used as the weight value of each feature channel, and the accuracy of the model can be greatly improved with only a small amount of additional computational cost, so as to improve the precision and accuracy of the model.

[0075] This step is required in the training stage, and the training stage is only carried out once, so this step is also only carried out once.

[0076] Step 6: In the actual measurement stage, the middle dark gray area and other areas are extracted by combining threshold segmentation and morphological segmentation, and each is divided into multiple small image patches as detection objects. The training parameters obtained in step 5 are respectively loaded, and the ResNet-50 model is used to infer the small image patches and classify the small image patches to determine whether there are foreign objects, including:

[0077] In the actual measurement stage, the middle dark gray area and other areas are extracted by combining threshold segmentation and morphological segmentation, and each is divided into multiple small image patches as detection objects. The training parameters corresponding to the ResNet-50 model obtained in step 5 are loaded, and for each small image patch, the ideal models I’ and II’ are respectively used to infer to obtain the corresponding inference classification results.

[0078] Step 7: Draw a frame annotation on the original image for the image patches judged as "with foreign object", including:

[0079] For the image patches judged as "with foreign object", the center coordinates of the image patches are restored from the small image patches to the original entire image. With the center coordinates at this time as the center of the square, a red square marking frame with a side length of 100 pixels is drawn on the entire image.

[0080] Step 8: Take the union of the adjacent image frames obtained in step 7, and redraw the frame annotation on the original image to complete the detection, including:

[0081] The foreign matters in the bath solution may be oil stains, cotton fluffs, etc. Their characteristic is that they will be dispersed over a large area, manifested as the area occupied in the image being larger than the area of the small image blocks. Therefore, in the result of the box selection in step 8, there will be a situation where the same foreign matter is marked with multiple boxes, which affects the visual perception of the image. Therefore, the bounding boxes within a distance of no more than 1 small image block from each other are unioned and merged into a large bounding box, which is marked on the image.

[0082] Experimental verification of two types of foreign matters is carried out. Currently, the detection results of the foreign matters on the surface of the bath solution have all reached the standard: for the broken cardboard, the correct rate reaches 95.8% under the false alarm rate of 4.0%; for the oil stain, the correct rate reaches 98.0% under the false alarm rate of 7.7%.

[0083] Example 2, a device for detecting foreign matters on the surface of the bath solution in a production line according to the present invention includes: a surface light source inlaid with an industrial camera, a control background, and a display terminal;

[0084] By erecting a bracket and a slide rail above the bath solution to be detected, the surface light source is suspended above the bath solution to be detected by means of hoisting, and is used to provide light to the bath solution to be detected;

[0085] One or a plurality of industrial cameras are installed through central holes on the surface of the surface light source, and the viewing angles of the industrial cameras face downward to collect images of the surface of the bath solution to be detected;

[0086] The control background is a PC or an intelligent terminal machine set in the control room; it is provided with a memory and a processor, and program modules are stored in the memory. When the processor loads the program modules, the above-mentioned method steps are executed to identify and locate foreign matters on the surface of the bath solution; and calculate the false alarm rate, correct rate, etc. according to the results of the foreign matter identification and location.

[0087] The display terminal is used to provide a visual interface for the user, collect the parameters and instructions input by humans, and display various image data, statistical calculation results such as false alarm rate and correct rate during the identification process of the foreign matters in the bath solution. The parameters include: in the process of image preprocessing, the window pixel size, starting pixel point, and pixel step length for dividing small image blocks; optional image enhancement operations (image block flipping, linear enhancement, rotation direction and angle step length of each image block); network parameters of the ResNet-50 structure corresponding to the gray area and other areas respectively (number of convolutional kernel layers and size, activation function, training batch, number of iterations, probability confidence threshold, loss function threshold); the instructions include: surface light source turn-on instruction, image acquisition trigger instruction issued to the industrial camera.

[0088] The above is only a preferred embodiment of the present invention, and does not impose any limitation on the present invention. Any simple modification, change, and equivalent structural change made to the above embodiments according to the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for detecting foreign matters on the surface of the trough liquid of a production line, characterized in that, It includes the following steps: S1. Image acquisition and preprocessing stage: Turn on the surface light source and adjust the brightness so that the light irradiates the surface of the bath solution to be measured; control the industrial camera to collect the image of the bath solution surface; Analyze the difference features between the camera reflection and the surrounding bath solution environment, and use image processing methods to separate the camera reflection area from other areas; Divide the camera reflection area and other areas into two types of image block sequences respectively; S2. Training stage: Perform data augmentation on the two types of image block sequences, expand the image block sample set and save it as an image file; Manually label the image file, classify it as having or not having foreign objects in the camera reflection area and having or not having foreign objects in other areas, and obtain four types of image data sets with label classifications required for training; Use the deep learning model framework ResNet-50 to iteratively learn the sample sets of the camera reflection area and other areas with label classifications, and obtain their respective network structure parameters and ideal models I and II when the training cut-off conditions are met; S3. Actual measurement stage: Collect and obtain two types of image blocks to be detected in real time according to step S1; respectively load the obtained network structure parameters, and use the ideal models I and II to infer and classify the two types of image blocks to judge whether there are foreign objects; Draw a frame and label the image blocks judged as "having foreign objects" on the original image; Take the union of adjacent image frames and redraw a frame and label it on the original image to complete the detection.

2. The method for detecting foreign objects on the surface of the tank liquid of a production line according to claim 1, characterized in that, The difference feature between the camera reflection and the surrounding bath solution environment is that the bath solution area where the camera reflection is located shows unequal contrast compared with the surrounding bath solution environment.

3. A method for detecting foreign objects on the surface of the tank liquid of a production line according to claim 1, characterized in that, For the entire collected picture, the camera reflection area and other areas are separated by means of binarization and morphological filtering.

4. The foreign matter detection method for the surface of the tank solution of a production line according to claim 1, wherein, The division into two types of image block sequences is as follows: within the dark gray area and other areas respectively, traverse and divide into small image blocks with unequal pixel sizes at a step size of N / 2, where the small image blocks are square image blocks with a size of N*N pixels.

5. A method for detecting foreign objects on the surface of the tank liquid of a production line according to claim 1, characterized in that, The data augmentation in the training stage is as follows: for the obtained image block sequence, flip and linearly enhance each one by one, and then rotate each image block clockwise or counterclockwise around the center point at a preset angle as the step interval until it reaches 90 degrees, so as to obtain more image blocks, which are saved as an image file as the sample set.

6. The foreign object detection method on the surface of the production line tank liquid according to claim 1, wherein, The improvement of the deep learning model framework based on the traditional ResNet-50 includes: Prune the last two residual blocks of the traditional ResNet-50, and retain the single convolutional layer STAGE0 and the first two residual blocks: the first residual block STAGE1 and the second residual block STAGE2; Reduce the two convolutional layers in the first residual block STAGE1 to one convolutional layer; Halve the number of filters in each convolutional layer of the first residual block STAGE1 and the second residual block STAGE2; Embed the SE attention mechanism module after each convolutional layer of the second residual block STAGE2, and use the output result of the excitation part as the weight value of each feature channel.

7. A method for detecting foreign matters on the surface of the bath solution in a production line according to claim 1, characterized in that, In the actual measurement stage, drawing a frame and labeling the image blocks judged as "having foreign objects" on the original image includes: For the image block determined to have "foreign objects", the center coordinates of the image block are located from the small image block to the original entire image. Taking the center coordinates at this time as the center of the square, a frame is marked on the entire image. Taking the union of the obtained adjacent image frames and redrawing and annotating the frame on the original image to complete the detection includes: Taking the union of the annotation frames within a distance of no more than 1 small image block from adjacent ones and merging them into a large annotation frame, which is annotated on the image.

8. A foreign object detection device for the surface of the tank solution in a production line, characterized in that, Including: Surface light source, industrial camera, control background, display terminal; The surface light source is placed above the liquid in the tank to be measured in a suspended manner for providing light to the liquid in the tank to be measured; One or several industrial cameras are installed through holes at the center of the surface of the surface light source, and the industrial cameras are oriented downward for collecting images of the surface of the liquid in the tank to be measured; The control background is a PC or intelligent terminal machine set in the control room; it is provided with a memory and a processor. Program modules are stored in the memory. When the processor loads the program modules, the above method steps are executed to identify and locate foreign objects on the surface of the liquid in the tank; and statistical calculations are performed according to the foreign object identification and location results; The display terminal is used to provide a visual interface for users, collect manually input parameters and instructions, display various image data during the foreign object identification process of the liquid in the tank, and the statistical calculation results.

9. A method for detecting foreign objects on the surface of the trough liquid of a production line according to claim 8, characterized in that, The parameters of the human-computer interaction include: During the image preprocessing process: the window pixel size, starting pixel point, and pixel step size for dividing small image blocks; optional image block flipping, linear enhancement, and image enhancement operations such as the rotation direction and angle step size of each image block; During the training stage, the network parameters of the ResNet-50 structure corresponding to the gray area and other areas respectively: the number and size of convolutional kernel layers, activation function, training batch, number of iterations, probability confidence threshold, loss function threshold.

10. A method for detecting foreign objects on the surface of the trough liquid of a production line according to claim 8, characterized in that, The instructions of the human-computer interaction include: surface light source turn-on instruction, image acquisition trigger instruction sent to the industrial camera; the statistical calculation results are probability confidence, loss function, false alarm rate, and accuracy calculation.

Images