Visual Recognition Detection System and Detection Method for Tape Cutting and Groove Insertion Positions

Abstract

The present invention provides a visual recognition detection system and a detection method for tape cutting and slotting positions. The detection system includes a light source illumination module for providing stable and reliable illumination; a camera acquisition module for taking and obtaining a tape image located in the tape slot detection area of the tape wrapping machine; a visual positioning module for visually positioning the tape slot detection area of the tape image of the tape wrapping machine; an image feature extraction module for extracting features from the image of the tape slot detection area of the tape wrapping machine cropped according to the result of visual positioning to form an image to be detected; a classification and recognition module for classifying and recognizing the image to be detected; the present invention realizes the real-time detection of tape cutting and tape slotting of the tape wrapping machine and sends real-time feedback of the tape to the intelligent control terminal of the tape wrapping machine through the network port communication module, forming a closed loop of the overall tape wrapping system, making the tape wrapping system more stable and reliable; compared with manual visual inspection, it reduces detection errors and improves detection efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of encapsulation, and particularly relates to a visual recognition detection system for the tape cutting and slotting positions and a detection method thereof. Background Art

[0002] At present, during the manufacturing process of wire harnesses, encapsulation is the most time-consuming process, which highly depends on manual labor, resulting in poor encapsulation consistency and great difficulty in quality control. For the overall automated flexible mechanical encapsulation process, tape detection is a crucial step. After each section of wire harness encapsulation, tape detection is required. Moreover, the cutting and slotting of the tape will directly have a serious impact on the subsequent wire harness encapsulation. Therefore, tape detection for encapsulation is an important guarantee for closing the loop of the encapsulation system and ensuring the stability and reliability of the encapsulation system. Traditionally, manual visual inspection is used, but it has high labor costs, low detection efficiency, and detection errors caused by human factors. Summary of the Invention

[0003] The purpose of the present invention is to overcome the defects of the prior art and propose a visual recognition detection system for the tape cutting and slotting positions and a detection method thereof, which can improve the efficiency of tape detection, reduce costs and save labor.

[0004] To achieve the above purpose, the present invention adopts the following specific technical solutions:

[0005] The visual recognition detection system for the tape cutting and slotting positions provided by the present invention includes a front-end part and a rear-end part. The front-end part includes a light source illumination module and a camera acquisition module, and the rear-end part includes a visual positioning module, an image feature extraction module, and a classification and recognition module. Among them,

[0006] The light source illumination module is used to provide stable and reliable illumination;

[0007] The camera acquisition module is used to capture the tape image located in the tape slot detection area of the encapsulation machine;

[0008] The visual positioning module is used to perform visual positioning on the tape slot detection area of the tape image of the encapsulation machine;

[0009] The image feature extraction module is used to extract features from the image of the tape slot detection area of the encapsulation machine cropped according to the result of visual positioning to form an image to be detected;

[0010] The classification and recognition module is used to perform classification and recognition on the image to be detected;

[0011] Preferably, it further includes a network port communication module, which is used to implement network port communication between different processes on a computer.

[0012] Preferably, the light source illumination module includes an annular light source and an annular light source brightness adjustment controller, where;

[0013] The annular light source is used to provide stable illumination;

[0014] The annular light source brightness adjustment controller is used to adjust the brightness of the annular light source.

[0015] Preferably, the annular light source adopts an LED (light-emitting diode) lamp bead array and is fixed inside the annular diffuser plate.

[0016] Preferably, the camera acquisition module includes a distortion-free camera and a housing. The housing contains a fixed bracket and is fixed at the front end of the mechanical arm of the encapsulation machine. The distortion-free camera is fixed at the center inside the annular light source through the fixed bracket, and the distortion-free camera is perpendicular to the plane of the tape slot of the encapsulation machine.

[0017] Preferably, the vision positioning module includes an image grayscale conversion sub-module, an image cropping sub-module, an image preprocessing sub-module, an image downsampling sub-module, and an image matching and positioning sub-module; where,

[0018] The image grayscale conversion sub-module is used to convert the acquired original image into a grayscale image;

[0019] The image cropping sub-module is used to crop the detection area of the tape slot of the encapsulation machine in the grayscale image as a template image for matching and positioning;

[0020] The image preprocessing sub-module is used to preprocess the template image for matching and positioning according to the image grayscale information;

[0021] The image downsampling sub-module is used to perform pyramid downsampling on the detection image of the tape slot of the encapsulation machine and the template image for matching and positioning;

[0022] The image matching and positioning sub-module is used to match the image of the tape slot detection area of the encapsulation machine with the template image for matching and positioning.

[0023] Preferably, the image downsampling sub-module performs multiple bilinear downsamplings based on the following formula:

[0024]

[0025] where, level is the number of pyramid layers; the side length of the minimum matching template is d; the side length of the original template image is m.

[0026] Preferably, the side length d of the minimum matching template is less than the side length m of the original template image.

[0027] Preferably, the image matching and positioning sub-module calculates the optimal matching and positioning position based on the following formula:

[0028]

[0029] where η i is the normalized cross - correlation coefficient for the image matching of the i - th layer. When η i takes the maximum value, it is the optimal matching and positioning location. is the gray - level mean of the i - th layer template image; templateStd_ is the biased sample standard deviation of the i - th layer template image; sampleMean_i is the gray - level mean of the detection image area of the same size as the template in the i - th layer; sampleStd_i is the biased sample standard deviation of the detection image area of the same size as the template in the i - th layer; SumPix(el) is the sum of image pixels in the detection area of the same size as the template in the i - th layer; y is the pixel convolution of the i - th layer template and the detection image of the same size.

[0030] Preferably, the image feature extraction module includes an image acquisition sub - module, a color - space transformation sub - module, an image masking processing sub - module, an image feature transformation sub - module, and an image compression sub - module. Among them,

[0031] The image acquisition sub - module is used to crop a square tape image with side length m centered on the tape slot detection area of the encapsulation machine and extract the detection area of the region of interest;

[0032] The color - space transformation sub - module is used to transform the RGB color - space image into an HSV color - space image;

[0033] The image masking processing sub - module is used to mask the converted HSV color - space image to obtain the image feature color area;

[0034] The feature transformation sub - module is used to make the feature image of the tape slot detection area of the encapsulation machine change significantly when the tape blocks the features of the tape slot detection area;

[0035] The image compression sub - module is used to compress the image to obtain a small - data - volume image containing feature information.

[0036] Preferably, the image masking processing module performs iterative processing using morphological opening operations.

[0037] Preferably, the classification and recognition module includes a tape cutting recognition sub - module and a tape slotting recognition sub - module. Among them,

[0038] The tape cutting recognition sub - module is used to perform visual recognition detection on whether the tape is cut for the encapsulated object after encapsulation is completed. The area for performing the tape cutting visual recognition detection is the tape break detection area;

[0039] The tape slotting recognition sub - module is used to perform visual recognition detection of tape slotting on the tape slot area cropped from the visual positioning result in the case of tape cutting.

[0040] Preferably, the tape cutting identification sub-module includes an image processing unit for the tape break detection area, a data acquisition unit for the tape break detection area, a classifier training unit for the tape break detection area, and a classifier identification unit for the tape break detection area, where;

[0041] The image processing unit for the tape break detection area is used to grayscale the image of the tape break detection area;

[0042] The data acquisition module for the tape break detection area is used to collect the data of the image of the tape break detection area as positive and negative samples;

[0043] The classifier training module for the tape break detection area is used to calculate the grayscale gradient histogram of the positive and negative samples to train the visual recognition classifier for tape cutting of the rubber wrapping machine;

[0044] The classifier identification module for the tape break detection area is used to judge whether the tape is cut after the rubber wrapping is completed.

[0045] Preferably, the tape slotting identification sub-module includes an image detection unit for the tape slot area, a data acquisition unit for the tape slotting detection area, a classifier training unit for the tape slotting detection area, and a classifier identification unit for the tape slotting detection area, where;

[0046] The image detection unit for the tape slot area is used to obtain the image of the tape slot detection area of the rubber wrapping machine;

[0047] The data acquisition unit for the tape slotting detection area is used to collect the image data of the slotting detection area as positive and negative samples;

[0048] The classifier training unit for the tape slotting detection area is used to calculate the grayscale gradient histogram of the positive and negative samples to train the visual recognition classifier for tape slotting of the rubber wrapping machine;

[0049] The classifier identification unit for the tape slotting detection area is used to judge the tape slotting identification of the rubber wrapping machine.

[0050] The detection method implemented by using the above-mentioned visual recognition detection system for tape cutting and slotting positions provided by the present invention includes the following steps:

[0051] S1. Obtain image data by shooting through the camera shooting module, and perform visual positioning on the tape slot detection area of the rubber wrapping machine;

[0052] S2. Crop and extract the tape slot detection area according to the visual positioning result of the tape slot detection area of the rubber wrapping machine, and use the image processing method to extract the features of the image of the tape slot detection area;

[0053] S3. Based on the machine learning model of SVM, train the image data containing feature information to obtain an image classification recognition classifier, and perform visual recognition detection of cutting and slotting on the tape image of the rubber wrapping machine successively.

[0054] Preferably, step S1 specifically includes the following steps:

[0055] S110. Using the method of image grayscale conversion, convert the collected original image into a grayscale image;

[0056] S120. In the grayscale image, intercept the tape slot detection area of the encapsulation machine as the template image for matching and positioning;

[0057] S130. Preprocess the template image for matching and positioning according to the image grayscale information of the template image for matching and positioning;

[0058] S140. According to the template image for matching and positioning and the detection image of the tape slot detection area of the encapsulation machine, calculate the number of pyramid layers through the following formula, and then perform multiple downsamplings based on bilinear interpolation according to the number of pyramid layers;

[0059]

[0060] S150. Adopt the method based on the normalized cross-correlation coefficient to perform layer-by-layer matching and positioning of the multi-layer template and the tape slot detection area of the encapsulation machine.

[0061] Preferably, step S2 specifically includes the following steps:

[0062] S210. Using the method of extracting the region of interest, crop it into a square region image centered on the tape slot area of the encapsulation machine;

[0063] S220. Based on the color space transformation method, convert the RGB color space image of the square region image into an HSV color space image;

[0064] S230. Perform masking processing on the HSV color space image, set the color threshold based on the characteristic color, and extract the characteristic color region;

[0065] S240. Based on the image after masking processing, perform iterative processing using morphological opening operation;

[0066] S250. Compress the image obtained by performing multiple iterative processes of morphological opening operation to obtain a small data volume image containing characteristic information.

[0067] Preferably, step S3 specifically includes the following steps:

[0068] S310. Using the method of extracting the region of interest, visually locate the tape slot area of the encapsulation machine, crop the original image to obtain the tape break detection area image on the right side of the tape slot of the encapsulation machine, and perform grayscale processing on the tape break detection area image;

[0069] S320. Based on the image of the tape break detection area and the actual cutting situation, collect positive and negative samples, classify the positive and negative samples, and set labeled samples;

[0070] S330. Calculate the gray gradient histogram based on the positive and negative samples, and use the gray gradient histogram as the sample feature descriptor to train the visual recognition classifier for tape cutting of the rubber covering machine;

[0071] S340. According to the visual recognition classifier for tape cutting of the rubber covering machine, recognize the images obtained during the actual rubber covering process, and judge and cut the tape cutting timing after the rubber covering is completed;

[0072] S350. Once again, adopt the method of extracting the region of interest to visually locate the tape slot area of the rubber covering machine, and crop the original image to obtain the image of the tape slot entry detection area of the rubber covering machine;

[0073] S360. Based on the image of the entry detection area, collect positive and negative samples, classify the positive and negative samples, and set labeled samples;

[0074] S370. Calculate the gray gradient histogram based on the positive and negative samples, and use the gray gradient histogram as the sample feature descriptor to train the visual recognition classifier for tape entry into the slot of the rubber covering machine;

[0075] S380. According to the visual recognition classifier for tape entry into the slot of the rubber covering machine, after the tape break detection is qualified, perform the visual recognition detection for tape entry into the slot.

[0076] The present invention can achieve the following technical effects: realizing the real-time detection of the cutting of the rubber covering machine and the entry of the tape into the slot, and sending the real-time feedback of the tape to the intelligent control terminal of the rubber covering machine through the network port communication module, forming a closed loop of the overall packaging system, making the rubber covering system more stable and reliable; compared with manual visual inspection, reducing the detection errors and improving the detection efficiency. Description of the Drawings

[0077] Figure 1 is the framework structure diagram of the visual recognition detection system for tape cutting and entry position according to the embodiment of the present invention.

[0078] Figure 2a is the internal structure schematic diagram of the light source illumination module according to the embodiment of the present invention.

[0079] Figure 2b is the external structure schematic diagram of the light source illumination module according to the embodiment of the present invention.

[0080] Figure 3a is the internal structure schematic diagram of the camera acquisition module according to the embodiment of the present invention.

[0081] Figure 3bIt is a schematic structural diagram of the forward view captured by the camera provided in the embodiment of the present invention.

[0082] Figure 3c It is a schematic structural diagram of the reverse view captured by the camera provided in the embodiment of the present invention.

[0083] Figure 4 Among them, (a) is the grayscale image converted by the image grayscale conversion sub-module provided in the embodiment of the present invention.

[0084] Figure 4 Among them, (b) is the template image for matching and positioning the screenshot by the image cropping sub-module provided in the embodiment of the present invention.

[0085] Figure 4 Among them, (c) is the visual positioning result image of the image matching and positioning sub-module provided in the embodiment of the present invention.

[0086] Figure 5 Among them, (a) is the image of the tape slot opening after rotation by the image acquisition sub-module provided in the embodiment of the present invention.

[0087] Figure 5 Among them, (b) is the image of the slot detection area acquired by the image acquisition sub-module provided in the embodiment of the present invention.

[0088] Figure 5 Among them, (c) is the HSV image converted by the color space transformation sub-module provided in the embodiment of the present invention.

[0089] Figure 5 Among them, (d) is the image after masking and opening operation by the image masking processing sub-module provided in the embodiment of the present invention.

[0090] Figure 5 Among them, (e) is the image after image feature compression by the image feature compression sub-module provided in the embodiment of the present invention.

[0091] Figure 6 Among them, (a) is a schematic diagram of the state where the tape is inside the slot opening provided in the embodiment of the present invention.

[0092] Figure 6 Among them, (b) is the feature image of the state where the tape is inside the slot opening without performing the opening operation provided in the embodiment of the present invention.

[0093] Figure 6 Among them, (c) is the feature image of the state where the tape is inside the slot opening after performing the opening operation k times provided in the embodiment of the present invention.

[0094] Figure 6 Among them, (d) is a schematic diagram of the position where the tape is at the extreme edge of the slot opening provided in the embodiment of the present invention.

[0095] Figure 6 Among them, (e) is the characteristic image of the tape at the extreme edge position of the notch without performing opening operation provided according to an embodiment of the present invention.

[0096] Figure 6 Among them, (f) is the characteristic image of the tape at the extreme edge position of the notch after performing k times of opening operation provided according to an embodiment of the present invention.

[0097] Figure 7a is the detection image of the tape being successfully cut provided according to an embodiment of the present invention.

[0098] Figure 7b is the detection image of the tape not being successfully cut provided according to an embodiment of the present invention.

[0099] Figure 8a is the detection image of the tape successfully entering the groove provided according to an embodiment of the present invention.

[0100] Figure 8b is the detection image of one side of the tape not entering the groove provided according to an embodiment of the present invention.

[0101] Figure 8c is the detection image of the tape not completely entering the groove provided according to an embodiment of the present invention.

[0102] Figure 8d is the detection image of the tape completely exiting the groove provided according to an embodiment of the present invention.

[0103] Figure 9 Among them, (a) is the image containing all characteristics on both sides provided according to an embodiment of the present invention.

[0104] Figure 9 Among them, (b) is the image with the left - hand side characteristics semi - occluded provided according to an embodiment of the present invention.

[0105] Figure 9 Among them, (c) is the image with the right - hand side characteristics semi - occluded provided according to an embodiment of the present invention.

[0106] Figure 9 Among them, (d) is the image with the left - hand side characteristics fully occluded provided according to an embodiment of the present invention.

[0107] Figure 9 Among them, (e) is the image with the right - hand side characteristics fully occluded provided according to an embodiment of the present invention.

[0108] Figure 9 Among them, (f) is the image with both - side characteristics fully occluded provided according to an embodiment of the present invention.

[0109] Figure 10It is a schematic flowchart of a visual recognition detection method for tape cutting and slotting positions according to an embodiment of the present invention.

[0110] Among them, the reference numerals include: lamp bead 1, distortion-free camera 2, annular diffuser 3, annular light source 4, fixed bracket 5, rubber coating machine robotic arm 6, housing device 7, above the camera lens 8, below the camera lens 9, detachable rubber coating manipulator 10, tape 11, rubber coating machine tape slot 12, slot feature area 12-1, slot feature small area 12-2, rubber coating manipulator rotation structure 13, rubber coating machine tape slot plane 14, rubber coating machine tape slot detection area 15, tape cutting blade 16, housing 17, light source illumination module 18, camera acquisition module 19, visual positioning module 20, image feature extraction module 21, classification and recognition module 22, network port communication module 23. Detailed implementation manners

[0111] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, the same modules are denoted by the same reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, their detailed descriptions will not be repeated.

[0112] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention.

[0113] Figure 1 It is a frame structure of a visual recognition detection system for tape cutting and slotting positions according to an embodiment of the present invention.

[0114] As Figure 1 shown, the visual recognition detection system for tape cutting and slotting positions provided by the embodiment of the present invention includes a front-end part and a back-end part. The front-end part includes a light source illumination module 18 and a camera acquisition module 19. The back-end part includes a visual positioning module 20, an image feature extraction module 21 and a classification and recognition module 22. Among them, the light source illumination module 18 is used to provide stable and reliable illumination; the camera acquisition module 19 is used to take and obtain a tape image located in the rubber coating machine tape slot detection area 15; the visual positioning module 20 is used to perform visual positioning on the rubber coating machine tape slot detection area 15 of the tape image; the image feature extraction module 21 is used to extract features from the image of the rubber coating machine tape slot detection area 15 cropped according to the result of visual positioning to form an image to be detected; the classification and recognition module 22 is used to perform classification and recognition on the image to be detected; it further includes a network port communication module 23, and the network port communication module 23 is used to implement network port communication between different processes on a computer.

[0115] Figure 2a and Figure 2b are the structural schematic diagrams of the light source illumination module provided by the embodiments of the present invention.

[0116] As Figure 2a and Figure 2b shown, the light source illumination module 18 includes an annular light source 4 and an annular light source brightness adjustment controller, wherein; the annular light source 4 is used to provide stable illumination and illuminate perpendicularly to the surface of the tape slot 12 of the encapsulation machine. When selecting the model of the ambient light source 4, the outer diameter R1 of the distortion-free camera 2 should be less than the inner diameter R2 of the annular light source. The annular light source 4 adopts an array of LED (light-emitting diode) lamp beads 1 and is fixed inside the annular diffuser plate 3; the annular light source brightness adjustment controller is used to adjust the brightness of the annular light source 4.

[0117] Figure 3a is the internal structure of the camera acquisition module 19 provided by the embodiments of the present invention.

[0118] As Figure 3a shown, the camera acquisition module 19 includes a distortion-free camera 2 and a housing 17. The housing 17 contains a fixed bracket 5 fixed to the front end of the mechanical arm 6 of the encapsulation machine. The distortion-free camera 2 is fixed at the center inside the annular light source 4 through the fixed bracket 5, and the distortion-free camera 2 is perpendicular to the plane of the tape slot 12 of the encapsulation machine.

[0119] Figure 3b and Figure 3c are the structural diagrams of the viewing angles captured by the distortion-free camera provided by the embodiments of the present invention.

[0120] As Figure 3b and Figure 3c shown, the distortion-free camera 2 is perpendicular to the tape slot plane 14 of the encapsulation machine, and the upper part 8 and the lower part 9 of the camera lens are kept horizontal or vertical, so that the camera can capture and obtain the picture of the tape 11 in the tape slot detection area 15 of the encapsulation machine.

[0121] Figure 4 from (a) to Figure 4 from (c) in are the processes of the visual positioning module provided by the embodiments of the present invention.

[0122] As Figure 4 from (a) to Figure 4 from (c) shown, the visual positioning module 20 includes an image grayscale conversion sub-module, an image interception sub-module, an image preprocessing sub-module, an image downsampling sub-module, and an image matching and positioning sub-module;

[0123] Among them, the image grayscale conversion sub-module is used to convert the collected original image into a grayscale image. The width of the detected grayscale image is width, and the height is height;

[0124] The image cropping sub-module is used to crop the detection area 15 of the tape slot of the encapsulation machine in the grayscale image as the template image for matching and positioning. Both the width and height of the template image are m;

[0125] The image preprocessing sub-module is used to preprocess the template image for matching and positioning according to the image grayscale information, including the template grayscale mean templateMean and the biased sample standard deviation templateStd;

[0126] The image downsampling sub-module is used to perform pyramid downsampling on the detection image of the tape slot detection area 15 of the encapsulation machine and the template image for matching and positioning, so as to improve the speed of visual positioning. The image downsampling sub-module performs multiple bilinear downsamplings based on the following formula. Let the layer with the smallest compressed image size be the top layer of the pyramid image, that is, the first layer, and the layer with the original image size be the bottom layer, that is, the level layer:

[0127]

[0128] where level is the number of pyramid layers; the side length of the smallest matching template is d; the side length of the original template image is m, and the side length d of the smallest matching template is less than the side length m of the original template image;

[0129] The image matching and positioning sub-module is used to match the image of the tape slot detection area 15 of the encapsulation machine with the template image for matching and positioning. First, start from the top layer of the image pyramid and layer by layer match the detection image and the template image based on the normalized cross-correlation coefficient η to locate the position of the template image in the detection image. The position with the largest η value in each layer is the optimal matching and positioning position. Among them, when matching in the top first layer, the global sample image is matched with the template; when matching in other layers, according to the matching position result of the previous layer, after expanding the image pixel matching result coordinates (xl, yl) by 1 time, local matching and positioning are performed in the vicinity of the detection image matching result coordinates. Let the matching neighborhood radius be r, and the local matching range is as follows:

[0130] x ∈ [xl·2 - r, xl·2 + r]

[0131] y ∈ [yl·2 - r, yl·2 + r]

[0132] Specifically, when matching, the normalized cross-correlation coefficient η of the i-th layer i is calculated by the following formula:

[0133]

[0134] Among them, templateMean_i is the grayscale mean of the template image in the i-th layer; templateStd_ is the biased sample standard deviation of the template image in the i-th layer; sampleMean_i is the grayscale mean of the detection image area with the same size as the template in the i-th layer; sampleStd_i is the biased sample standard deviation of the detection image area with the same size as the template in the i-th layer; SumPixel(S) is the sum of the image pixels in the detection area with the same size as the template in the i-th layer; is the pixel convolution of the template in the i-th layer and the detection image with the same size.

[0135] Figure 5 from (a) to Figure 5 in (e) is the process of the image feature extraction module provided by the embodiment of the present invention.

[0136] Such as Figure 5 from (a) to Figure 5 as shown in (e), the image feature extraction module 21 includes an image acquisition sub-module, a color space transformation sub-module, an image masking processing sub-module, an image feature transformation sub-module, and an image compression sub-module;

[0137] Figure 5 In (a), it is the tape notch image after rotation by the image acquisition sub-module provided by the embodiment of the present invention.

[0138] Such as Figure 5 as shown in (a), according to the position coordinates of visual positioning, the video image of width×height is cropped into a square area image centered on the notch area with a size of m×m (m < width, m < height), and rotated 90 degrees clockwise (no rotation when the camera lens is horizontally installed).

[0139] Figure 5 In (b), it is the notch detection area image obtained by the image acquisition sub-module provided by the embodiment of the present invention.

[0140] Such as Figure 5 as shown in (b), the cropped image contains the feature color areas on both sides of the notch. At the same time, on the premise of not losing the image clarity, the detection area of interest is extracted, effectively reducing the burden of the subsequent image processing process and improving the image processing speed.

[0141] Figure 5 In (c), it is the HSV image after conversion by the color space transformation sub-module provided by the embodiment of the present invention.

[0142] Such as Figure 5As shown in (c), the present invention converts the intercepted image represented in the R (red), G (green), and B (blue) color channel modes into an image represented in the H (hue), S (saturation), and V (brightness) modes by using the functions in the OpenCV library.

[0143] Figure 5 In (d), it is the image after the mask and opening operation of the image mask processing sub-module provided according to the embodiment of the present invention.

[0144] As Figure 5 shown in (d), let the color of the obvious feature regions on both sides of the notch in the cropped image be color1 (the feature color can be sprayed as other colors according to the actual situation), that is, the color to be masked during masking is color1. The HSV value range of color1 is H: h1 - h2; S: s1 - s2; V: v1 - v2. Within the value range of colo1r, experimentally adjust and set the upper and lower color thresholds: lower = [h1′, s1′, v1′]; upper = [h2′, s2′, v2′]. Then call the functions in the library to mask the converted HSV image to obtain the color1 color region therein;

[0145] Figure 6 From (a) to Figure 6 In (f), it is a schematic diagram of the change in the image features of the morphological opening operation iterative method for adding terms.

[0146] As Figure 6 From (a) to Figure 6 shown in (f), on the basis of obtaining the mask image of the tape notch detection region 15 of the encapsulation machine, perform multiple iterative processes of morphological opening operation on the masked image, so that for the feature image of the tape notch detection region 15 of the encapsulation machine, when the tape blocks the notch feature region 12 - 1, the feature change of the small notch feature occlusion region 12 - 2 is obvious.

[0147] Figure 5 In (e), it is the image after the image feature compression of the image feature compression sub-module provided according to the embodiment of the present invention.

[0148] As Figure 5 shown in (e), the size of the feature image after the image mask and opening operation is compressed from m × m to n × n (n < m), reducing the amount of data to be processed without losing the image features and improving the efficiency of recognition and detection.

[0149] Figure 7a It is the detection image of the successful cutting of the tape provided according to the embodiment of the present invention.

[0150] As Figure 7aAs shown, when the tape wrapping machine successfully cuts the tape 11, the detachable tape wrapping manipulator 10 controls the movement of the tape wrapping manipulator rotation structure 13 to a specified angle, and makes the tape slot detection area 15 captured by the camera relatively fixed.

[0151] Figure 7b It is a detection image of the tape not being successfully cut according to an embodiment of the present invention.

[0152] As Figure 7b shown, when the tape wrapping machine fails to cut the tape 11, the tape wrapping manipulator rotation mechanism 13 will move to the position as Figure 8b shown under the influence of the tape tension.

[0153] Figure 8a It is a detection image of the tape successfully entering the slot according to an embodiment of the present invention.

[0154] Figure 8b It is a detection image of one side of the tape not entering the slot according to an embodiment of the present invention.

[0155] Figure 8c It is a detection image of the tape not being fully inserted into the slot according to an embodiment of the present invention.

[0156] Figure 8d It is a detection image of the tape completely out of the slot according to an embodiment of the present invention.

[0157] Figure 9 In (a) of , it is an image with all features on both sides according to an embodiment of the present invention.

[0158] Figure 9 In (b) of , it is an image with the left side features semi-occluded according to an embodiment of the present invention.

[0159] Figure 9 In (c) of , it is an image with the right side features semi-occluded according to an embodiment of the present invention.

[0160] Figure 9 In (d) of , it is an image with the left side features fully occluded according to an embodiment of the present invention.

[0161] Figure 9 In (e) of , it is an image with the right side features fully occluded according to an embodiment of the present invention.

[0162] Figure 9 In (f) of , it is an image with all features on both sides fully occluded according to an embodiment of the present invention.

[0163] As Figures 8a to 8d and Figure 9 shown in (a) of , the classification and recognition module 22 includes a tape cutting recognition sub-module and a tape slotting recognition sub-module, wherein,

[0164] The tape cutting recognition sub-module is used to visually recognize and detect whether the tape of the rubberized object after the rubberizing is completed. The area for the visual recognition and detection of tape cutting is the tape cutting inspection area; the tape slotting recognition sub-module is used to visually recognize and detect whether the tape enters the slot in the tape slot area cropped from the visual positioning result in the case where the tape 11 is cut.

[0165] The tape cutting recognition sub-module includes an image processing unit for the tape cutting inspection area, a data acquisition unit for the tape cutting inspection area, a classifier training unit for the tape cutting inspection area, and a classifier recognition unit for the tape cutting inspection area. Among them, the image processing unit for the tape cutting inspection area is used to grayscale the image of the tape cutting inspection area; the data acquisition module for the tape cutting inspection area is used to collect the data of the image of the tape cutting inspection area as positive and negative samples; the classifier training module for the tape cutting inspection area is used to calculate the grayscale gradient histogram of the positive and negative samples to train the visual recognition classifier for tape cutting of the rubberizing machine; the classifier recognition module for the tape cutting inspection area is used to judge whether the tape cutting blade 16 cuts the tape 11 after the rubberizing is completed.

[0166] The tape slotting recognition sub-module includes an image detection unit for the tape slot area, a data acquisition unit for the tape slotting detection area, a classifier training unit for the tape slotting detection area, and a classifier recognition unit for the tape slotting detection area. Among them, the image detection unit for the tape slot area is used to obtain the image of the tape slotting detection area 15 of the rubberizing machine tape slot; the data acquisition unit for the tape slotting detection area is used to collect the image data of the slotting detection area as positive and negative samples; the classifier training unit for the tape slotting detection area is used to calculate the grayscale gradient histogram of the positive and negative samples to train the visual recognition classifier for tape slotting of the rubberizing machine; the classifier recognition unit for the tape slotting detection area is used to judge the tape slotting recognition of the rubberizing machine.

[0167] The above content details the visual recognition and detection system for tape cutting and slotting positions provided by the embodiments of the present invention. The embodiments of the present invention also provide a detection method using the visual recognition and detection system for tape cutting and slotting positions.

[0168] Figure 10 Shows the flow of the visual recognition and detection method for tape cutting and slotting positions provided by the embodiments of the present invention.

[0169] As Figure 10 shown, the visual recognition and detection method for tape cutting and slotting positions provided by the embodiments of the present invention includes the following steps:

[0170] Step S1, acquire image data by the camera acquisition module 19 at the front end, and perform visual positioning on the tape slot detection area 15 of the rubberizing machine.

[0171] Step S2: Crop out the visual positioning result of the tape slot detection area 15 of the encapsulation machine, and use the image processing method to extract the features of the image of the tape slot detection area 15 of the encapsulation machine. The width of the detected grayscale image is width, and the height is height.

[0172] Step S3: Based on the machine learning model of SVM, train the image data containing feature information to obtain an image classification and recognition device, and successively perform visual recognition and detection on the cutting and slotting of the tape image of the encapsulation machine.

[0173] Step S1 specifically includes the following steps:

[0174] Step S110 (as shown in (a) of Figure 5 ): Use the method of image grayscale conversion to convert the collected original image into a grayscale image. The width of the detected grayscale image is width, and the height is height.

[0175] Step S120 (as shown in (b) of Figure 5 ): In the grayscale image, intercept the tape slot detection area 015 of the encapsulation machine as the template image for matching and positioning. The width and height of the template image are both m.

[0176] Step S130: Preprocess the template image for matching and positioning according to the image grayscale information of the template image for matching and positioning; template grayscale mean templateMean and biased sample standard deviation templateStd.

[0177] Step S140: According to the template image for matching and positioning and the detection image of the tape slot detection area 15 of the encapsulation machine, calculate the number of pyramid layers through the following formula, and then perform multiple downsamplings based on bilinear interpolation according to the number of pyramid layers;

[0178]

[0179] Step S150: Adopt the normalized cross-correlation coefficient to perform layer-by-layer matching and positioning of the multi-layer template and the tape slot detection area of the encapsulation machine; match the image of the tape slot detection area 15 of the encapsulation machine with the template image for matching and positioning. First, start from the top layer of the image pyramid, and layer by layer, based on the normalized cross-correlation coefficient η, match the detection image with the template image to locate the position of the template image in the detection image. The position with the largest η value in each layer is the optimal matching and positioning position. Among them, when matching in the top first layer, use the global sample image to match with the template; when matching in other layers, according to the matching position result of the previous layer, expand the image pixel matching result coordinates (xl, yl) by 1 time, and perform local matching and positioning near the neighborhood of the detection image matching result coordinates. Let the matching neighborhood radius be r, and the local matching range is as follows:

[0180] x ∈ [xl·2 - r, xl·2 + r]

[0181] y ∈ [yl·2 - r, yl·2 + r]

[0182] During specific matching, the normalized cross - correlation coefficient η of the i - th layer i is calculated by the following formula:

[0183]

[0184] where, templateMean_i is the gray - level mean of the template image of the i - th layer; templateStd_ is the biased sample standard deviation of the template image of the i - th layer; sampleMean_i is the gray - level mean of the detection image area of the same size as the template in the i - th layer; sampleStd_i is the biased sample standard deviation of the detection image area of the same size as the template in the i - th layer; SumPixel(S) is the sum of image pixels in the detection area of the same size as the template in the i - th layer; is the pixel convolution of the i - th layer template and the detection image of the same size.

[0185] As shown in (a) - Figure 6 (e) in Figure 6 Based on the above - mentioned image feature extraction module 21,

[0186] Step S2 specifically includes the following steps:

[0187] Step S210 (as shown in (a) - 6b in Figure 6 )、According to the position coordinates of visual positioning, crop the video image of width×height into a square - shaped area image centered on the tape slot detection area 15 of the encapsulation machine with a size of m×m (m < width, m < height), and perform a 90 - degree clockwise rotation (no rotation when the camera lens is horizontally installed), and crop it into a square - shaped area image centered on the tape slot detection area 15 of the encapsulation machine by using the method of extracting the region of interest.

[0188] Step S220 (as shown in (c) in Figure 6 )、Based on the color - space transformation method, transform the RGB color - space image of the square - shaped area image into an HSV color - space image.

[0189] Step S230 (as shown in ( Figure 6As shown in (d) therein, perform masking on the HSV color space image, set a color threshold based on the characteristic color, and extract the characteristic color region; set the color of the obvious characteristic regions on both sides of the notch in the cropped image as color1 (the characteristic color can be sprayed as other colors according to the actual situation), that is, the color to be masked during masking is color1. The HSV value range of color1 is H: h1 - h2; S: s1 - s2; V: v1 - v2. Within the value range of color1, experimentally adjust and set the upper and lower color thresholds: lower = [h1′, s1′, v1′]; upper = [h2′, s2′, v2′]. Then call the function in the library to perform masking on the converted HSV image to obtain the color1 color region therein.

[0190] Step S240: Based on the image after masking processing, perform iterative processing using morphological opening operation; make the characteristic image of the notch detection region 15 of the encapsulation machine have obvious characteristic changes in the small region 12 - 2 of the notch feature when the notch feature region 12 - 1 is blocked by the tape.

[0191] Step S250 (as shown in (e) therein) Figure 6 Perform image compression on the image obtained by performing multiple iterative processes of morphological opening operation, compress the size of the characteristic image after image masking and opening operation from m×m to n×n (n < m) to obtain a small - data - volume image containing characteristic information.

[0192] Step S3 specifically includes the following steps:

[0193] Step S310 (as shown in (a) - 9b therein) Figure 9 Locate the notch detection region 15 of the encapsulation machine tape visually by extracting the region of interest, crop the original image to obtain the image of the tape break - detection region on the right side of the encapsulation machine tape, and perform grayscale processing on the tape break - detection region image.

[0194] Step S320 (as shown therein) Figure 9 Collect positive and negative samples based on the actual cutting situation according to the tape break - detection region image, and classify the positive and negative samples and set label samples; in the present invention, when collecting positive and negative samples of image data, directly perform grayscale processing on the tape break - detection region image, directly use the grayscale image of the encapsulation machine tape cutting image as the positive sample, use the grayscale image of the uncut tape as the negative sample, and then set the sample labels.

[0195] Calculate the grayscale gradient histogram according to the positive and negative samples, and use the grayscale gradient histogram as the sample feature descriptor to train the visual recognition classifier for the encapsulation machine tape cutting; call the SVM model in the OpenCV library, set the kernel function as RBF, and train the visual recognition classifier for the encapsulation machine tape into the slot.

[0196] Step S340: Identify the images obtained during the actual encapsulation process according to the visual recognition classifier for tape cutting of the encapsulation machine, and determine the tape cutting timing and cut after the encapsulation is completed.

[0197] Step S350: Once again, use the method of extracting the region of interest to visually locate the tape slot area 15 of the encapsulation machine, and crop the original image to obtain the image of the tape slot detection area 15 of the encapsulation machine.

[0198] Step S360 (as shown in Figure 9 ): Collect positive and negative samples according to the image of the slot-in detection area, classify the positive and negative samples, and set the labeled samples; there are two solutions that can be adopted when collecting positive and negative samples of image data in the present invention: (1) Use the n×n compressed image as shown in (e) of Figure 6 obtained by the above-mentioned image feature extraction module 21, classify the compressed feature image as shown in each sub-image of Figure 9 , and set the sample labels, where Figure 9 a is a positive sample, Figure 9 and the remaining sub-images are negative samples. (2) Directly grayscale the located image of the slot-in detection area, directly use the grayscale image of the tape slot-in image of the encapsulation machine as the positive sample, use the grayscale image of the tape not in the slot as the negative sample, and then set the sample labels.

[0199] Step S370: Calculate the gray gradient histogram according to the positive and negative samples, and use the gray gradient histogram as the sample feature descriptor to train the visual recognition classifier for tape slot-in of the encapsulation machine.

[0200] Step S380 (as shown in (c) of Figure 9 - Figure 9 (d) shown): Further judge the images initially judged to have the tape 11 successfully inserted into the slot according to the visual recognition classifier for tape slot-in of the encapsulation machine, select the rectangular area on the right side of the slot to calculate the sum of pixels in the area, and screen out the images that are not fully inserted into the slot and the images where the tape 11 is not inserted into the slot at all through the pixel sum threshold; when directly using the grayscale image of the slot-in detection area as the positive and negative samples, the tape 11 slot-in can be directly recognized and judged according to the tape slot-in recognition classifier.

[0201] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples", etc., mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0202] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0203] The above specific implementation manners of the present invention do not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and deformations made according to the technical concept of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A visual recognition and detection system for tape cutting and slotting positions, characterized in that, it includes a front-end part and a back-end part. The front-end part includes a light source illumination module and a camera acquisition module, and the back-end part includes a visual positioning module, an image feature extraction module, and a classification and recognition module. Among them, the light source illumination module is used to provide stable and reliable illumination; the camera acquisition module is used to capture and obtain the tape image located in the tape slot detection area of the tape wrapping machine; the visual positioning module is used to perform visual positioning on the tape slot detection area of the tape image of the tape wrapping machine; the image feature extraction module is used to extract features from the image of the tape slot detection area of the tape wrapping machine cropped according to the result of the visual positioning, forming an image to be detected; the classification and recognition module is used to perform classification and recognition on the image to be detected; the classification and recognition module includes a tape cutting recognition sub-module and a tape slotting recognition sub-module. Among them, the tape cutting recognition sub-module is used to perform visual recognition and detection on whether the tape of the wrapped object after tape wrapping is cut, and the area for performing the visual recognition and detection is the tape break detection area; the tape slotting recognition sub-module is used to perform visual recognition and detection of tape slotting on the tape slot area cropped according to the visual positioning result in the case of tape cutting.

2. The visual recognition and detection system for tape cutting and slotting positions according to claim 1, characterized in that, it further includes an Ethernet communication module, and the Ethernet communication module is used to realize Ethernet communication between different processes on one computer.

3. The visual recognition and detection system for tape cutting and slotting positions according to claim 1, characterized in that, the light source illumination module includes an annular light source and an annular light source brightness adjustment controller. Among them; the annular light source is used to provide stable illumination; the annular light source brightness adjustment controller is used to adjust the brightness of the annular light source.

4. The visual recognition and detection system for tape cutting and slotting positions according to claim 3, characterized in that, the annular light source adopts a light-emitting diode lamp bead array and is fixed inside the annular diffuser plate.

5. The visual recognition and detection system for tape cutting and slotting positions according to claim 3 or 4, characterized in that, the camera acquisition module includes a distortion-free camera and a housing. The housing contains a fixed bracket and is fixed at the front end of the mechanical arm of the tape wrapping machine. The distortion-free camera is fixed at the center inside the annular light source through the fixed bracket, and the distortion-free camera is perpendicular to the plane of the tape slot of the tape wrapping machine.

6. The visual recognition and detection system for tape cutting and slotting positions according to claim 1, characterized in that, the visual positioning module includes an image grayscale conversion sub-module, an image cropping sub-module, an image preprocessing sub-module, an image downsampling sub-module, and an image matching and positioning sub-module; among them, the image grayscale conversion sub-module is used to convert the collected original image into a grayscale image; the image cropping sub-module is used to crop the tape slot detection area in the grayscale image as a template image for matching and positioning; The image preprocessing sub-module is used to preprocess the template image for matching and positioning according to the image grayscale information; The image downsampling sub-module is used to perform pyramid downsampling on the detection image of the tape slot detection area of the rubber coating machine and the template image for matching and positioning; The image matching and positioning sub-module is used to match the image of the tape slot detection area of the rubber coating machine with the template image for matching and positioning.

7. The visual recognition and detection system for tape cutting and slotting position as described in claim 6, wherein, the image downsampling sub-module performs multiple bilinear downsamplings based on the following formula: Among them, is the number of pyramid layers; the minimum matching template size is ; the side length of the original template image is .

8. The visual recognition and detection system for tape cutting and slotting position as described in claim 7, wherein, The side length of the minimum matching template is less than the side length of the original template image .

9. The visual recognition and detection system for tape cutting and slotting position as described in claim 6, wherein, the image matching and positioning sub-module calculates the optimal matching and positioning position based on the following formula: Among them, is the normalized cross-correlation coefficient for layer image matching, and the optimal matching positioning position is when the maximum value is taken, is the gray mean value of the layer template image; is the biased sample standard deviation of the layer template image; is the biased sample standard deviation of the layer and the detection image area of the same size as the template; is the sum of the pixels of the layer and the detection area image of the same size as the template; is the pixel convolution of the layer template and the detection image of the same size.

10. The visual recognition and detection system for tape cutting and slotting position as described in claim 1, wherein, the image feature extraction module includes an image acquisition sub-module, a color space transformation sub-module, an image masking processing sub-module, an image feature transformation sub-module, and an image compression sub-module, wherein, the image acquisition sub-module is used to crop a square tape image with a side length of m centered on the tape slot detection area of the rubber coating machine and extract the detection area of the region of interest; the color space transformation sub-module is used to transform the RGB color space image into an HSV color space image; the image masking processing sub-module is used to mask the transformed HSV color space image to obtain the image feature color area; the feature transformation sub-module is used to make the feature image of the tape slot detection area of the rubber coating machine change significantly when the tape blocks the features of the tape slot detection area; the image compression sub-module is used to compress the image to obtain a small data volume image containing feature information.

11. The visual recognition and detection system for tape cutting and slotting position as described in claim 10, wherein, the image masking processing module performs iterative processing using morphological opening operation.

12. The visual recognition and detection system for tape cutting and slotting position as described in claim 1, wherein, the tape cutting recognition sub-module includes a tape break detection area image processing unit, a tape break detection area data acquisition unit, a tape break detection area classifier training unit, and a tape break detection area classifier recognition unit, wherein; the tape break detection area image processing unit is used to grayscale the tape break detection area image; the tape break detection area data acquisition module is used to collect the data of the image of the tape slot area cropped from the visual positioning result as positive and negative samples; the tape break detection area classifier training module is used to calculate the grayscale gradient histogram of the positive and negative samples to train the visual recognition classifier for tape cutting of the rubber coating machine; the tape break detection area classifier recognition module is used to judge whether the tape is cut after the rubber coating is completed.

13. The visual recognition and detection system for tape cutting and slotting position as described in claim 1, wherein, The tape slot-in recognition sub-module includes a tape slot area image detection unit, a tape slot-in detection area data acquisition unit, a tape slot-in detection classifier training unit, and a tape slot-in detection classifier recognition unit, where; The tape slot area image detection unit is used to obtain the image of the tape slot detection area of the tape wrapping machine; The tape slot-in detection area data acquisition unit is used to collect the image data of the slot-in detection area as positive and negative samples; The tape slot-in detection classifier training unit is used to calculate the gray gradient histogram of the positive and negative samples to train the visual recognition classifier for the tape slot-in of the tape wrapping machine; The tape slot-in detection classifier recognition unit is used to judge the tape slot-in recognition of the tape wrapping machine.

14. A visual recognition and detection method for tape cutting and slot-in position, implemented by using the visual recognition and detection system for tape cutting and slot-in position according to claims 1 to 13, characterized in that, it includes the following steps: S1. Obtain image data by shooting through the camera acquisition module, and perform visual positioning on the tape slot detection area of the tape wrapping machine; S2. Crop and extract the tape slot detection area according to the visual positioning result of the tape slot detection area of the tape wrapping machine, and use an image processing method to extract the features of the image of the tape slot detection area; S3. Based on the machine learning model of SVM, train the image data containing feature information to obtain an image classification and recognition classifier, and perform visual recognition and detection of tape cutting and slot-in on the tape image of the tape wrapping machine successively.

15. The visual recognition and detection method for tape cutting and slot-in position according to claim 14, characterized in that, step S1 specifically includes the following steps: S110. Adopt the method of image grayscale conversion to convert the collected original image into a grayscale image; S120. Intercept the tape slot detection area in the grayscale image as a template image for matching and positioning; S130. Preprocess the template image for matching and positioning according to the image grayscale information of the template image for matching and positioning; S140. According to the template image for matching and positioning and the detection image of the tape slot detection area of the tape wrapping machine, calculate the number of pyramid layers through the following formula, and then perform multiple downsamplings based on bilinear interpolation according to the number of pyramid layers; S150. Adopt the normalized cross-correlation coefficient to perform layer-by-layer matching and positioning of the multi-layer template and the tape slot detection area of the tape wrapping machine.

16. The visual recognition and detection method for tape cutting and slot-in position according to claim 14, characterized in that, step S2 specifically includes the following steps: S210. Adopt the method of extracting the region of interest to crop into a square region image centered on the tape slot area of the tape wrapping machine; S220. Based on the color space transformation method, convert the RGB color space image of the square region image into an HSV color space image; S230. Perform masking processing on the HSV color space image, set a color threshold based on the characteristic color, and extract the characteristic color region; S240. Based on the image after the masking processing, perform iterative processing using morphological opening operation; S250. Iteratively process the image obtained by the morphological opening operation multiple times, and use image compression to obtain a small-data-volume image containing the feature information.

17. The visual recognition and detection method for tape cutting and slotting position as described in claim 14, characterized in that, Step S3 specifically includes the following steps: S310. By using the method of extracting the region of interest, visually locate the tape slot area of the tape wrapping machine, crop the original image to obtain the tape break detection area image on the right side of the tape slot of the tape wrapping machine, and perform grayscale processing on the tape break detection area image; S320. According to the tape break detection area image, collect positive and negative samples based on the actual cutting situation, classify the positive and negative samples, and set label samples; S330. Calculate the grayscale gradient histogram of the positive and negative samples, and use the grayscale gradient histogram as the sample feature descriptor to train the tape cutting visual recognition classifier of the tape wrapping machine; S340. According to the tape cutting visual recognition classifier of the tape wrapping machine, identify the image obtained during the actual tape wrapping process, and judge the tape cutting timing and cut after the tape wrapping is completed; S350. Again, by using the method of extracting the region of interest, visually locate the tape slot area of the tape wrapping machine, and crop the original image to obtain the tape slotting detection area image of the tape wrapping machine; S360. According to the slotting detection area image, collect the positive and negative samples, classify the positive and negative samples, and set label samples; S370. Calculate the grayscale gradient histogram of the positive and negative samples, and use the grayscale gradient histogram as the sample feature descriptor to train the tape slotting visual recognition classifier of the tape wrapping machine; S380. According to the tape slotting visual recognition classifier of the tape wrapping machine, perform the tape slotting visual recognition detection after the tape break detection is qualified.

Images