Large-area plate processing quality inspection method based on linear array camera and computer vision and application

Through the quality inspection method based on line array cameras and computer vision, the problems of low efficiency and difficult to ensure the detection of large-area sheets are solved, and fast and accurate defect detection and real-time quality control are achieved, and production efficiency and product quality are improved.

CN119027375APending Publication Date: 2024-11-26CHONGQING UNIV OF TECH

Patent Information

Application Number
CN202411031852.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The prior art has low efficiency and difficult to guarantee the quality inspection of large-area sheets. Especially metal sheets and plastic sheets have difficulty in detection due to the high-gloss reflection characteristics. Traditional artificial visual inspection cannot meet the modern high-speed production line and high-quality requirements.

Method used

Using quality inspection methods based on line array cameras and computer vision, images are collected and stitched and cut into small images through line array cameras, combined with instance segmentation detection algorithms and digital image processing, pixel-level segmentation information is obtained, processing quality is judged and output is not qualified. Area location. Specific steps include image stitching, cutting, instance segmentation detection, digital image processing and result display.

Benefits of technology

It realizes fast, accurate and real-time defect detection of large-area sheets, significantly improves image processing speed and target area recognition accuracy, ensures real-time quality control during the production process, reduces the defect rate of finished products, and improves production efficiency and product quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027375A_ABST
    Figure CN119027375A_ABST
Patent Text Reader

Abstract

The invention provides a line-scan digital camera and computer vision-based large-area plate processing quality inspection method and application, and the method comprises the steps: S1, splicing images collected by a line-scan digital camera, and obtaining a complete image; s2, the complete image is cut into small images in sequence through an SAHI algorithm, and the small images are partially overlapped; s3, sending the cut small image into an instance segmentation detection algorithm to obtain pixel-level segmentation information of the target; and then digital image processing is carried out based on the pixel-level segmentation information of the target so as to realize detection of the processing quality. And finally judging whether the detection result is qualified or not according to the detection result. The method has better detection performance and higher lightweight degree, achieves better balance between computing resource requirements and detection precision, and is beneficial to deployment on intelligent industrial assembly lines such as laser cutting and the like.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of plate detection, and in particular to a large-area plate processing quality inspection method and application based on a linear array camera and computer vision. Background Art

[0002] Large-area plates are processed into different sizes and shapes and are widely used in many fields such as construction, aerospace, and automobile manufacturing. Since the materials of the plates are different and the area is large, modern manufacturing has strict requirements on the processing quality and specifications of products. In particular, metal plates, plastic plates, and some composite materials have high light reflection characteristics due to surface coatings or their own material properties, which brings many difficulties to their quality inspection. At present, the processing quality inspection of plates is mainly carried out by manual visual inspection, but this method is inefficient, and its inspection standards are often affected by human factors, and the accuracy is difficult to guarantee. This backward method is difficult to apply to modern high-speed production lines, and it cannot meet the high quality requirements of modern manufacturing.

[0003] With the development of industrial camera equipment and computer vision technology, product quality inspection technology based on computer vision has been widely used in the manufacturing industry due to its advantages of high automation, fast computing speed, and low system cost, and has gradually become one of the important quality inspection methods in modern manufacturing. The industrial product quality inspection system based on computer vision generally consists of two parts: hardware subsystem and software subsystem.

[0004] The hardware subsystem is the image acquisition platform. There are currently two mature image acquisition solutions on the market: the first is the area array camera image acquisition system, which has the advantages of fast acquisition speed and intuitive imaging results. However, it has disadvantages such as large image distortion, low image resolution, and difficulty in controlling the illumination of large-area images; the second is the line array camera image acquisition system, which has the advantages of high image acquisition accuracy, large-area objects can be imaged at one time, easy to control illumination, and small distortion.

[0005] The software subsystem mainly refers to the quality inspection algorithm, which is mainly divided into two categories: deep learning algorithm and traditional algorithm. Deep learning algorithm has ushered in major changes in recent years due to the significant improvement of graphics card computing power. This type of algorithm, especially lightweight models, has gradually gained market advantages in data-rich application scenarios due to its advantages such as fast processing speed, high recognition accuracy, and low demand for video memory. However, deep learning algorithms rely on a large amount of data support, and their deployment effect is limited when faced with industrial environments with limited data sets, difficult image acquisition, or poor working conditions. On the other hand, traditional algorithms have been continuously applied in the industrial field due to their transparent data processing process based on strict mathematical derivation, low deployment cost, and algorithm requirements for a small number of samples. The advantages of this type of algorithm are its stability and low resource requirements, but the main challenges they face include poor generalization ability and cumbersome debugging process, which increases the development difficulty for R&D personnel. More importantly, it is difficult for traditional algorithms to strike a balance between detection accuracy and speed, and usually requires algorithm optimization and adjustment according to specific project requirements. Summary of the invention

[0006] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a large-area plate processing quality inspection method and application based on a linear array camera and computer vision.

[0007] In order to achieve the above-mentioned object of the present invention, the present invention provides a large-area plate processing quality inspection method based on a line array camera and computer vision, comprising:

[0008] S1, stitching the images collected by the line array camera to obtain a complete image;

[0009] S2, the complete image is sequentially cut into small images of 640*640 size through the SAHI algorithm to adapt to the multiple 2x upsampling and downsampling set in the algorithm model, and there is partial overlap between the small images;

[0010] S3, the small image after cutting is sent to the instance segmentation detection algorithm to obtain the pixel-level segmentation information of the target; then digital image processing is performed based on the pixel-level segmentation information of the target, so as to realize the detection of processing quality; finally, whether the detection result is qualified is judged according to the detection result, if it is unqualified, the position coordinates of the unqualified area are output.

[0011] S4, both qualified and unqualified results are displayed on the host computer.

[0012] Furthermore, the step of sending the cut small image to an instance segmentation detection algorithm to obtain pixel-level segmentation information of the target includes:

[0013] S31, input the image into a backbone network built based on a reversible column network, the backbone network includes a stem and two columns, the stem uses a convolutional layer to extract a feature map with a shape of 320*320*16 from the input image, each column contains four levels, namely level1, level2, level3, level4, for feature extraction and fusion;

[0014] The first column downsamples the feature map step by step, and obtains feature maps of sizes 160*160*32, 80*80*64, 40*40*128, and 20*20*256 at four levels respectively; in the second column, level1~level3 will fuse the output feature map of the next layer of the previous column by upsampling, and downsample the fused feature map to the next level, and finally obtain output feature maps of four sizes, which are the same as the feature maps output by the four levels in the first column;

[0015] S32, the four feature maps extracted by the backbone network will be sent to the neck fusion network for fusion of multi-scale feature maps. The network has two paths: bottom-up and top-down.

[0016] S33, the output feature map of level 4 will be sent to the bottom-up path. First, it will be extracted by fast feature pyramid SPPF using pooling kernels of different sizes to enhance the algorithm's detection ability for objects of different sizes, and obtain a feature map of size 20*20*512.

[0017] The feature map after SPPF is then input into the pyramid spatial channel attention mechanism PSCA. The PSCA attention mechanism first extracts multi-scale feature maps through four sizes of convolution kernels, then extracts multi-scale attention weights through the channel attention branch and the coordinate attention branch, and recalibrates the extracted attention weights using softmax. Finally, the calibrated attention weights are weighted on the input feature map to obtain a feature map of size 20*20*512.

[0018] S34, the feature map recalibrated by PSCA is upsampled by 2 times using Upsample_1 to obtain a feature map of size 40*40*256, and then concatenated with the feature map of size 40*40*128 output by level3 through Concat_1 to obtain a feature map of size 40*40*640;

[0019] The feature map output by Concat_1 will obtain rich gradient flow information through C2F_1, and then the output feature map of C2F_1 will be further upsampled to obtain a feature map with a shape of 80*80*256. This feature map is concatenated with the feature map of size 80*80*64 output by level2 to obtain a feature map of size 80*80*320. The concatenated feature map is then fused through C2F_2 and a feature map of size 80*80*128 is output;

[0020] S35, after the upsampling path is calculated, the multi-scale features will be further fused through the downsampling path. For the feature map output by C2F_2, the convolution layer Conv_1 with a step size of 2 will downsample it to obtain a feature map of size 40*40*128;

[0021] Concat_3 concatenates the outputs of Conv_1 and C2F_1 to obtain a feature map of size 80*80*384, and fuses this feature map using C2F_3 to obtain a feature map of size 80*80*256. This feature map is then downsampled using the convolutional layer Conv_2 with a stride of 2 to obtain a feature map of size 40*40*256.

[0022] Conv_4 concatenates the output of Conv_2 with the output of the PSCA attention mechanism to obtain a feature map of size 20*20*768, and fuses this feature map through C2F_4 to obtain a feature map of size 20*20*512;

[0023] S36, after the neck network completes the fusion of multi-scale features, level 1 in C2F_3, C2F_2, and Column2 is input into the semantic segmentation branch for feature fusion. By integrating features of three different scales, the processing capability of multi-scale targets is improved;

[0024] S37, the output feature map sizes of the semantic segmentation branch, C2F_3, and C2F_4 modules are 80*80*128, 40*40*256, and 20*20*512 respectively, which are input into the Segment detection head to detect objects of different sizes;

[0025] Among them, C2F_1 module to C2F_4 module are all C2F modules.

[0026] The loss functions of the instance segmentation detection algorithm include category confidence loss, bounding box regression loss, target confidence loss and mask loss.

[0027] Furthermore, the four sizes of convolution kernels are: 3*3, 5*5, 7*7, and 9*9.

[0028] Furthermore, the semantic segmentation branch is a prototype mask coding branch TPMCB, and the features of level 1 in C2F_3, C2F_2, and Column2 are fused through the prototype mask coding branch TPMCB, including:

[0029] First, two convolutional layers (CBS) are used to downsample the Superficial with a size of 160*160*32 by two times, so that the number of channels c is expanded to 64, and the first-level feature map with a scale of 80*80*128 is obtained;

[0030] Then, the high-resolution reconstruction of the emantic feature map is performed through the data-dependent upsampling module Dupsample, and then the second-level feature map with a scale of 80*80*128 is obtained through the convolutional layer (CBS);

[0031] Secondly, the convolution module (CBS) further extracts the Detail to obtain the second-level feature map with a scale of 80*80*128;

[0032] Finally, the feature maps of three different levels are combined by addition and further fused by the convolutional layer;

[0033] Among them, level1 in C2F_3, C2F_2, and Column2 respectively represents high-level semantic information Semantic, intermediate detail information Detail, and primary feature information Superficial.

[0034] Furthermore, digital image processing is performed based on pixel-level segmentation information of the target, thereby realizing detection of processing quality; including:

[0035] S300, target region binarization: cutting the region containing the target according to the pixel-level segmentation information obtained by the instance segmentation algorithm, and then binarizing the cutting result to obtain target information;

[0036] S301, morphological processing: in order to ensure the integrity and connectivity of the target area, morphological processing is applied to the target area after binarization to fill tiny holes, connect scattered areas, and smooth boundaries;

[0037] S302, edge detection: using the Canny algorithm to perform edge detection on the target area after morphological processing, and obtain the edge contour of the target;

[0038] S303, skeleton extraction: use the medial axis transformation algorithm to extract the skeleton of the target area after morphological processing, and obtain the center line (the center line is the skeleton) as a basis for target size calculation;

[0039] S304, width calculation: First, the normal vector of the center line is calculated by the kd Tree algorithm and the SVD algorithm, and then the two intersection points with the edge contour line in the direction of the normal vector are obtained, and the target pixel width here is calculated according to the coordinates of the two points. If the processing method is closed area cutting, the width calculation is the distance between the two intersection points of the closed area on the normal vector of the center line of the closed area; if the processing method is laser cutting, the width calculation is the width between the two edge contours of the plate (seam).

[0040] Furthermore, the image captured by the line array camera is captured by an image acquisition platform, and the image acquisition platform includes a line array camera, a linear light source and an image acquisition platform bracket;

[0041] The image acquisition platform bracket includes a vertical cylindrical metal rod, and at least three inclined rods are arranged at the bottom of the vertical cylindrical metal rod to achieve stable support for the image acquisition platform bracket;

[0042] A round tube fixing fixture is clamped in the middle of the vertical cylindrical metal rod, a light source fixing cross bar is arranged on the round tube fixing fixture, and the linear light source is fixedly installed on the light source fixing cross bar; the irradiation position of the light source can be adjusted by adjusting the round tube fixing fixture;

[0043] A moving mechanism for the image acquisition module to move up and down is arranged at the top of the vertical cylindrical metal rod, and the image acquisition module includes a linear camera;

[0044] It also includes a photoelectric switch and / or an encoder. When the photoelectric switch detects the drag board, the photoelectric switch sends a control signal to the linear light source to make the linear light source emit light; and sends a control signal to the linear camera to make the linear camera start working; the encoder is used to provide the camera with a shooting frequency according to the line frequency signal output by the encoder;

[0045] The linear array camera and the photoelectric switch are located on the same vertical line, and the photoelectric switch is placed in a position parallel to the conveyor belt carriage. When the front end of the carriage moves to the position of the photoelectric switch, the photoelectric switch is triggered, and a frame trigger signal is sent to the camera, and the linear light source is turned on;

[0046] The image acquisition platform also includes an encoder, which is fixed on the processing machine by a T-shaped fixing bracket and wrapped by a rubber wheel on the outside. The edge of the rubber wheel is slightly higher than the carriage. When the carriage passes by, the encoder is pressed down to ensure that the carriage is in close contact with the outer rubber wheel.

[0047] The encoder provides the camera with a line frequency signal. When the encoder rotates a certain angle, it will trigger the camera to collect a line of image data. Its function is to provide the camera with a shooting frequency so that the collected image is not distorted.

[0048] In summary, due to the adoption of the above technical solutions, the quality inspection method proposed in the present invention is specially designed for large-area plates and can achieve fast, accurate and real-time defect detection. This method adopts an optimized lightweight instance segmentation algorithm, combined with advanced digital image processing technology, which not only significantly improves the image processing speed, but also effectively identifies and quantifies the original defects of the plate and the new defects generated during the processing process through precise target area identification and quantification. In addition, this technology ensures real-time quality control in the production process by continuously monitoring and evaluating the processing quality of the plate, thereby preventing and reducing the defect rate of the finished product. Ultimately, the application of these technologies ensures that the processing quality of the workpiece strictly complies with industrial standards, significantly improves production efficiency and product quality, and is of great significance to promoting the development of industrial automation and intelligent manufacturing.

[0049] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0051] Figure 1 Schematic diagram of the image acquisition platform of the present invention Figure 1 .

[0052] Figure 2 Schematic diagram of the image acquisition platform of the present invention Figure 2 .

[0053] In the figure, 1 is a dust cover; 2 is a lead screw; 3 is a linear array camera; 4 is an image acquisition platform bracket; 5 is a light source fixing cross bar; 6 is a linear light source; 7 is a plate; 8 is a drag plate; 9 is a camera module fixing profile; 10 is an outer frame of a tape feeding platform; 11 is a driving roller; 12 is an encoder; and 13 is a photoelectric switch.

[0054] Figure 3 It is a schematic diagram of the image acquisition platform bracket encoder of the present invention.

[0055] Figure 4 It is the overall flow chart of the present invention.

[0056] Figure 5 It is the quality detection sub-flow chart of the present invention.

[0057] Figure 6 It is a schematic diagram of the structure of the LiteRevNet instance segmentation algorithm of the present invention.

[0058] Figure 7 It is a schematic diagram of the lightweight three-sided prototype mask coding structure (TPMCB) of the present invention.

[0059] Figure 8 Schematic diagram of the PSCA attention mechanism of the present invention.

[0060] Fig. 9 It is a schematic diagram of the C2F module structure of the present invention.

[0061] Fig.10 It is a flow chart of a specific embodiment of the present invention.

[0062] Fig.11 It is a schematic diagram of the results of binarization and opening operation processing in a specific embodiment of the present invention.

[0063] Fig.12 It is a weight diagram of the filter in a specific embodiment of the present invention.

[0064] Fig.13 It is a schematic diagram of edge detection results of a specific embodiment of the present invention.

[0065] Fig.14 It is a schematic diagram of pixel scanning encoding according to a specific embodiment of the present invention.

[0066] Fig.15 It is a schematic diagram of the skeleton extraction result of a specific embodiment of the present invention.

[0067] Fig.16 It is a schematic diagram of quantization of the widths of two points corresponding to the edge of a specific embodiment of the present invention. DETAILED DESCRIPTION

[0068] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0069] For large-area plates of two meters, the camera has a fixed focal length of 50mm and uses a 16384*1 CMOS sensor. A long, flicker-free, high-brightness lamp tube is used as the light source with adjustable brightness. The image acquisition system will be deployed between the processing station and the sorting station on the machine tool, using the visual system to detect the plates after assembly line processing, and the results will be sent to the machine tool system to realize the detection of processing quality.

[0070] Figure 1-2 It is a schematic diagram of the image acquisition platform of the present invention. Figure 1Above the bracket in the middle is the camera bracket, and in the middle is the linear light source, which is fixed by a circular hollow steel tube, and the irradiation position of the light source can be rotated and adjusted. A long strip lamp with high brightness without flicker can be used as the light source, and the brightness is adjustable. The light source is powered by a matching digital power supply, and the brightness can be infinitely adjusted by the power controller. There is a camera platform in the camera bracket, and the camera is fixed on the camera platform by four screws; the camera platform is fixed on the vertical screw rod and moves up and down with the screw rod; the screw rod is controlled by a stepper motor, the motor working voltage is 24V, and the movable range of the screw rod is 30cm up and down. In addition, the camera bracket is a metal bracket, and a dust cover can be attached to its surface.

[0071] Figure 3 It is a schematic diagram of the encoder of the image acquisition platform of the present invention. Figure 3 The photoelectric switch is placed parallel to the carriage, about 5cm away from the carriage. The photoelectric switch is powered by a 24V power supply, and the output signal is connected to the light source power supply and the camera. When the front end of the carriage moves to the position of the photoelectric switch, the photoelectric switch is triggered, and a frame trigger signal is sent to the camera, and the linear light source is turned on. The encoder is fixed to the processing machine by a T-shaped fixing bracket, and the outside is wrapped by a rubber wheel with a circumference of 20cm and an edge slightly higher than the carriage. When the carriage passes by, the encoder will be pressed down to ensure that the carriage is in close contact with the outer rubber wheel.

[0072] The detailed description of the hardware platform is as follows:

[0073] 1. Image acquisition platform bracket. The image acquisition platform bracket consists of an image acquisition module, a light source fixing module and a bracket body. The overall height of the image acquisition platform bracket is 2.8m. The main body of the bracket is made of profiles with a width of 4cm, and two oblique profiles are added at the bottom to ensure the stability of the entire bracket. The bottom of the bracket is fixed to the ground with screws. There are two vertically adjustable profiles on the top of the bracket body for fixing the image acquisition module. There is a round horizontal cylindrical metal rod at a height of about 1.5m in the middle for fixing the linear light source. When acquiring images, the light source needs to be rotated to align it with the field of view of the linear array camera to achieve suitable lighting conditions.

[0074] 2. Camera fine-tuning module, fix the screw on the vertical column of the camera object distance fine-tuning module. The screw uses a stepper motor to control the movement. The motor working voltage is 24V, and the movable range of the screw is within 30cm up and down. A platform for placing the camera is fixed on the shelf where the screw moves. The platform can move up and down with the rotation of the screw to fine-tune the distance between the camera and the processing platform.

[0075] 3. Image acquisition module. The image acquisition module consists of a linear array camera, a camera fine-tuning module, and a dust cover. The linear array camera is fixed on the camera platform of the camera object distance fine-tuning module. The linear array camera and the camera fine-tuning module are located in the dust cover.

[0076] 4. Light source: use a high-brightness line light source, which is fixed on a light source fixing metal rod on the image acquisition platform bracket. It can be rotated and adjusted as needed to illuminate the target area, and the brightness can be adjusted by the power supply.

[0077] 5. Power supply: The stepper motor of the camera object distance fine-tuning module is powered by 24V. The light source uses a digital power supply with an input of AC95-265V and an output of 24V / 12V.

[0078] 6. Camera and lens: Use a 16K industrial line array camera with a CMOS sensor, connected to the acquisition card via dual Camera Link cables, with a maximum line rate of 40kHz. Use a 50MM focal length FA lens. The camera and lens are connected via a lens adapter.

[0079] 7. The rotary encoder contacts the drag plate of the assembly line and provides an external line trigger signal for the camera, which is powered by a 24V power supply.

[0080] 8. Infrared diffuse reflection photoelectric switch, fixed in a position parallel to the moving direction of the carriage and with an interval of 15cm, provides frame trigger signal for the linear array camera and is powered by a 24V power supply.

[0081] 9. The capture card, combined with two CameraLink lines, forms a high-speed data transmission channel.

[0082] 10. The visual workstation uses an Intel(R) Xeon(R) CPU E3-1275 v5@3.60GHz processor, 32G memory, a graphics card of NVIDIA Geforce GTX 1080, and an operating system of Windows 10.

[0083] The image of a large area of ​​highly reflective metal sheet is obtained through the image acquisition platform. Due to the limitation of the acquisition card cache space, a three-segment image acquisition and splicing method is adopted. Generally, the image size acquired according to the width of the drag plate is 16384*30000, and the resolution of a single segment image is 16380*10000. The image is temporarily stored in the cache. After the three-segment image acquisition is completed, the complete image is stored in the memory.

[0084] Figure 4This is the overall flow chart of the present invention. When the photoelectric switch is triggered, the light source is turned on and the camera receives a frame trigger signal, waiting for the encoder to send a line trigger signal. When the encoder is triggered, the camera starts to collect images according to the trigger signal sent by the encoder, and the image data is sent to the acquisition card through the Camera Link cable. After the acquisition card processes the image data line by line, it sends the spliced ​​image data to the visual workstation through the PCIe interface.

[0085] Figure 5 This is the quality detection sub-flow chart of the present invention. The complete image is cut into small images of 640*640 size in sequence through the SAHI algorithm, and there is partial overlap between the small images to ensure that the small target will not be divided into two parts and affect the detection accuracy. The small images after cutting will be sent to the instance segmentation detection algorithm and the digital image processing algorithm to detect the processing quality. It is judged whether the test result is qualified. If it is unqualified, the position coordinates of the unqualified area are output. Finally, both the qualified and unqualified results are displayed on the host computer, and the test results are output to the sorting system.

[0086] The instance segmentation detection algorithm of the present invention is based on the YOLO V8S-SEG algorithm model, and a single-stage real-time instance segmentation algorithm (LiteRevNet) is designed. First, coordinate information is added to the convolution operation, and a multi-scale convolution kernel is used to form an efficient spatial perception convolution (ESSC), so that the convolution operation obtains multiple receptive fields and spatial perception capabilities, thereby improving the model detection accuracy. On this basis, ESSC is used to construct a lightweight reversible column backbone network (RevCol-ESSC), while maintaining the detection accuracy of the network model, reducing its calculation amount and parameter amount. Secondly, the pyramid coordinate channel attention (PSCA) with both spatial and channel perception capabilities is introduced to strengthen the network model's attention to the target area, and improve the detection accuracy with a small computational cost. Finally, in order to solve the problem of high computational complexity of the full convolutional network (FCN), a lightweight three-sided prototype mask coding structure (TPMCB) is reconstructed, which greatly reduces the computational amount of the prototype mask branch.

[0087] The structure of the single-stage real-time instance segmentation algorithm (LiteRevNet) is as follows Figure 6 shown.

[0088] (1) The input image is uniformly resized to an appropriate size, such as 640*640, with a fixed aspect ratio.

[0089] (2) The image is input into the backbone network based on the reversible column network. The backbone network consists of a stem and two columns. The stem uses a convolutional layer to extract a feature map of shape 320*320*16 from the input image. Each column contains four levels (level1, level2, level3, level4) for feature extraction and fusion.

[0090] The first column downsamples the feature map step by step, and obtains feature maps of sizes 160*160*32, 80*80*64, 40*40*128, and 20*20*256 at four levels. In the second column, level1 to level3 fuse the output feature map of the next layer of the previous column by upsampling, and downsample the fused feature map to the next level, finally obtaining output feature maps of four sizes, which are the same as the feature maps output by the four levels in the first column.

[0091] (3) The four feature maps extracted by the backbone network will be sent to the neck fusion network for multi-scale feature map fusion. The network has two paths: bottom-up and top-down.

[0092] (4) The output feature map of level 4 will be sent to the bottom-up path. First, it will pass through the fast feature pyramid SPPF and use pooling kernels of different sizes to extract multi-scale features to enhance the algorithm's detection ability for objects of different sizes, and obtain a feature map of size 20*20*512.

[0093] The feature map after SPPF is then input into the pyramid space channel attention mechanism PSCA. The PSCA attention mechanism first extracts multi-scale feature maps through four sizes of convolution kernels (3*3, 5*5, 7*7, 9*9), and then extracts multi-scale attention weights by the channel attention branch and the coordinate attention branch, and recalibrates the extracted attention weights using softmax. Finally, the calibrated attention weights are weighted on the input feature map to obtain a feature map of size 20*20*512.

[0094] The feature map recalibrated by PSCA is upsampled by 2 times using Upsample_1 to obtain a feature map of size 40*40*256, and then concatenated with the feature map of size 40*40*128 output by level3 through Concat_1 to obtain a feature map of size 40*40*640.

[0095] The feature map output by Concat_1 will obtain rich gradient flow information through C2F_1, and then the output feature map of C2F_1 will be further upsampled (that is, upsampled by 2 times using Upsample_2) to obtain a feature map with a shape of 80*80*256. This feature map is concatenated with the size of 80*80*64 output by level2 to obtain a feature map with a size of 80*80*320. The concatenated feature map is then fused through C2F_2 and a feature map with a size of 80*80*128 is output.

[0096] (5) After the upsampling path is calculated, the multi-scale features will be further fused through the downsampling path. For the feature map output by C2F_2, the convolution layer Conv_1 with a step size of 2 will downsample it to obtain a feature map of size 40*40*128.

[0097] Concat_3 concatenates the outputs of Conv_1 and C2F_1 to obtain a feature map of size 80*80*384, and fuses this feature map using C2F_3 to obtain a feature map of size 80*80*256. This feature map is then downsampled using the convolutional layer Conv_2 with a stride of 2 to obtain a feature map of size 40*40*256.

[0098] Conv_4 concatenates the output of Conv_2 with the output of the PSCA attention mechanism to obtain a feature map of size 20*20*768, and fuses this feature map through C2F_4 to obtain a feature map of size 20*20*512.

[0099] (6) After the neck network completes the fusion of multi-scale features, the level 1 in C2F_3, C2F_2, and Column2 is input into the prototype mask encoding branch TPMCB for feature fusion. The processing capability of multi-scale targets is improved by integrating features of three different scales. Level 1 in C2F_3, C2F_2, and Column2 represents feature information of three scales, namely, high-level semantic information, such as complete objects, object categories, etc.; mid-level detail information, such as local geometric shapes, texture patterns, simple object parts, etc.; primary feature information, such as edges, textures, etc. The prototype mask (TPMCB) generation branch improves the algorithm's processing capability of multi-scale targets and enhances the accuracy of the prototype mask by integrating features of three different scales. In order to reduce the computational complexity of the model while ensuring the accuracy of the prototype mask, the output result scale of this branch is 80*80*128.

[0100] In particular, this chapter adopts a data-dependent upsampling module Dupsample to achieve high-resolution reconstruction of feature maps, aiming to enhance the segmentation effect. As the core component of the prototype mask generation branch, TPMCB not only greatly reduces the amount of calculation and parameters, but also improves the inference speed.

[0101] (7) The output feature map sizes of the three modules TPMCB, C2F_3, and C2F_4 are 80*80*128, 40*40*256, and 20*20*512, respectively. They are input into the Segment detection head to detect small, medium, and large objects.

[0102] The loss function of the LiteRevNet algorithm is divided into category confidence loss, bounding box regression loss, target confidence loss, and mask loss.

[0103] The class confidence loss and target confidence loss are calculated using the binary cross entropy function:

[0104]

[0105] The bounding box regression loss is calculated using CIOU loss:

[0106]

[0107] Where α is the weight function and ν is used to measure the similarity of aspect ratios:

[0108]

[0109] The mask loss uses the Mask IoU metric to measure the segmentation accuracy of the model by the intersection ratio between the predicted mask and the true mask:

[0110]

[0111] Where mask(C) represents the predicted box area, and mask(G) represents the real box area.

[0112] The loss function is used as a parameter and the gradient descent method is used to train the model. Specifically, the directional derivative of the function at this point is calculated to achieve the maximum value along this direction, that is, the function changes fastest along this direction (the direction of this gradient) at this point and has the largest rate of change (the modulus of the gradient). The optimal weights and biases of each layer are calculated to complete the model training.

[0113] Use the above instance segmentation model for detection and use the detection results as the basis for size quantization.

[0114] Figure 7This is a schematic diagram of the lightweight three-sided prototype mask coding structure (TPMCB) of the present invention. First, the Superficial primary features with a size of 160*160*32 are expanded to 64 channels and doubled downsampled through two convolutional layers (CBS) to make their scale 80*80*128; then, the Semantic feature map is reconstructed with high resolution through the data-dependent upsampling module Dupsample, and then a feature map with a scale of 80*80*128 is obtained through the convolutional layer (CBS); secondly, the mid-level detail feature Detail is further extracted by the convolutional module (CBS), and the obtained feature map scale is also 80*80*128. Finally, the feature information of three different levels is combined by simple addition, and further fused by the convolutional layer.

[0115] Figure 7 The batchsize in the formula represents the number of images processed by the algorithm in each batch, c represents the number of channels, h represents the feature map height, and w represents the feature map width. In this paper, hw is 80, c is 128, and b is unspecified.

[0116] Figure 8 Schematic diagram of the PSCA attention mechanism of the present invention.

[0117] Modules C2F_1 to C2F_4 are all C2F modules. The structural diagram is as follows: Fig. 9 shown.

[0118] The specific embodiments are as follows:

[0119] Digital image processing technology has the advantages of mature technology, high computational efficiency, and strong interpretability. As the foundation of computer vision, it occupies an important position in science, industry, and social applications. Before the emergence of deep learning technology, digital image processing technology was the main means to deal with image and video related problems. However, there are problems such as weak feature extraction ability, poor generalization and robustness. The powerful feature extraction ability, excellent versatility and practicality of deep learning technology just make up for the shortcomings of digital image processing technology. To this end, this chapter combines the instance segmentation algorithm with the digital image processing algorithm to quantify the target size, and combines the self-built steel plate laser cutting data set for quantification experiments.

[0120] The process of the specific embodiment is as follows Fig.10 As shown, the following steps are included:

[0121] (1) Image cropping: Since some images are large in size and the objects to be detected contained in them are relatively small in size, in order to input the images into the instance segmentation model for training or prediction without losing the target detail information, the images are first cropped so that the size of a single image is 640×640, and there is partial overlap between adjacent images to ensure the integrity of the target in the cropped image.

[0122] (2) Instance segmentation algorithm: The cropped image is input into the trained instance segmentation model to obtain the pixel-level segmentation information of the target.

[0123] (3) Binarization of target area: The area containing the target is cut according to the pixel-level segmentation information provided by the instance segmentation algorithm, and then the cut result is binarized to obtain accurate target information.

[0124] (4) Morphological processing: In order to ensure the integrity and connectivity of the target area, morphological processing is used on the target area obtained by binarization to fill holes, connect scattered areas, and smooth boundaries.

[0125] (5) Edge detection: Use the Canny algorithm to perform edge detection on the target area that has been morphologically processed in the previous step to obtain the outer edge contour of the target.

[0126] (6) Skeleton extraction: Use the medial axis transformation algorithm to extract the skeleton of the morphological processing results to provide a basis for target size calculation.

[0127] (7) Width calculation: First, the normal vector of the center line is calculated using the kd Tree algorithm and the SVD algorithm. Then, the two intersection points of this normal vector with the edge contour line are obtained. The target pixel width is calculated based on the coordinates of these two points.

[0128] (3) Binarization of target area

[0129] The purpose is to separate the foreground from the background remaining in the instance segmentation detection results and convert each image into a binary image. The instance segmentation detection results before binarization are as follows Fig.11 (a) shows an example of the binarization result. Fig.11 As shown in (b), pixel 0 represents the background and pixel 255 represents the target.

[0130] (4) Morphological processing

[0131] Morphological processing is an important image processing method used for noise removal, shape extraction, image enhancement, edge detection, skeleton extraction, etc. It mainly includes four operations: dilation, erosion, opening operation, and closing operation:

[0132] Dilation: Used to eliminate noise on the image boundaries, or to reduce the number of pixels contained in an area.

[0133] Erosion: can fill holes in an area or expand its boundaries.

[0134] Opening operation: first erosion and then dilation, used to remove small objects or separate adhering objects.

[0135] Closing operation: dilation followed by erosion, used to fill holes in objects or connect adjacent objects.

[0136] To avoid the influence of impurities around the target on the width measurement, the target area after binary processing is opened. The result is as follows: Fig.11 (c) as shown.

[0137] (5) Edge detection

[0138] The distance between the two corresponding points on the edge of the target represents the width. In order to accurately extract the edge of the target, the Canny algorithm is used for edge detection in the binary image after morphological processing. This algorithm is a multi-stage edge detection method, which includes four steps: noise removal, gradient calculation, non-maximum suppression, and edge calculation:

[0139] 1) Noise removal: In order to reduce the noise interference in the image, the Canny algorithm first uses a Gaussian filter to smooth the image. This filter is a low-pass filter that can effectively reduce the mutation effect in the image. The Gaussian function in two-dimensional space can be expressed by formula (4.11). After the Gaussian kernel is calculated by this function, the Gaussian kernel is used to slide on the image and recalculate the value of each pixel. Due to the ingenious design of the Gaussian function, the Gaussian kernel has a greater weight closer to the center, and the smoothed image can also retain important information such as the lower edge.

[0140]

[0141] Among them, x and y represent the horizontal and vertical distances from the center pixel, and σ is the standard deviation of the Gaussian kernel, which controls the degree of smoothing.

[0142] 2) Gradient calculation: The edges of the image often exist where the gradient changes greatly. The Canny algorithm uses the Sobel operator to calculate the magnitude and direction of the gradient. It usually has two filters in the x-direction and y-direction to calculate the gradients in the two directions respectively. The weights of the two filters are as follows: Fig.12 As shown, the left side is the filter in the x direction, and the right side is the filter in the y direction. The gradient of each pixel is calculated by formula (4.12):

[0143]

[0144] Among them G x Indicates the gradient value in the x direction, G y represents the gradient value in the y direction, and G represents the gradient result value of the pixel point. The gradient direction is calculated using formula (4.13):

[0145]

[0146] 3) Non-maximum suppression: In order to get the thinnest possible edges, the algorithm applies the non-maximum suppression technique. This step goes through each pixel of the image and removes those pixels that are not the local maximum in the gradient direction. This means that if a pixel is not the maximum value in its gradient direction in the neighborhood on both sides, it will be set to 0 (suppressed).

[0147] 4) Edge calculation: Finally, Canny edge detection uses a double threshold method to decide which edges are real edges. Set two thresholds: a high threshold and a low threshold. If the gradient strength at a pixel position is higher than the high threshold, it will be marked as a strong edge; if the gradient strength is lower than the low threshold, it will be suppressed; if the gradient strength is between the two, it will only be considered part of the edge if it is connected to a strong edge. This step helps remove some false positives and makes the edge detection results more robust. The edge detection results in the binary image after the opening operation are shown in Figure 2. Fig.13 As shown:

[0148] (6) Skeleton extraction

[0149] The skeleton is not only the center line of the target, but also represents its direction. By calculating the intersection of the vertical line of the skeleton line and the edge, the width of the target can be obtained. In order to quickly and accurately obtain the skeleton of the image, this chapter uses the iterative deletion of edge pixels method. The algorithm process includes the following steps:

[0150] 1) Binarization: First, the image is binarized to distinguish the object (foreground) from the background. Usually, the object pixel value is 1 and the background pixel value is 0.

[0151] 2) Scanning and marking: In this step, the algorithm scans each foreground pixel in the image in parallel (usually a pixel with a value of 1, indicating an object). 1 , the algorithm will check its 8 neighboring pixels, which are marked as P 2 To P 9 , forming a 3x3 area, P 1 In the center, the schematic diagram is as Fig.14 shown.

[0152] 3) Delete pixels by condition:

[0153] The foreground pixel P will be determined by the following two steps1 Delete:

[0154] Step 1 Conditions:

[0155] ①2≤N(P1)≤6

[0156] ②S(P1)=1

[0157] ③At least one of P2, P4, and P6 is 1

[0158] ④At least one of P4, P6, and P8 is 1

[0159] Step 2 Conditions:

[0160] ①2≤N(P1)≤6

[0161] ②S(P1)=1

[0162] ③At least one of P2, P4, and P8 is 1

[0163] ④At least one of P2, P6, and P8 is 1

[0164] Where N(P1) represents P 1 The number of surrounding foreground pixels is calculated by P 2 To P 9 The number of pixels with a median value of 1 is obtained, and S(P1) represents the number of pixels from P 2 To P 9 The number of transitions in the algorithm is the number of times the pixel value changes from 0 to 1. When executing step 1, if the foreground pixel meets all its conditions, the pixel is marked for deletion, and then it is determined whether the conditions in step 2 are met. If so, the pixel is deleted. The algorithm alternates between the two steps and iterates continuously until no pixels are marked for deletion. This algorithm flow ensures that the important structural features of the object are not destroyed and can better maintain the connectivity of the shape. Skeleton extraction example is as follows Fig.15 shown.

[0165] (6) Width calculation

[0166] In order to accurately calculate the size of each part of the target, this chapter uses a combination of the kd tree algorithm and the SVD algorithm to solve the normal vector of the skeleton line, and then finds the corresponding two points on the edge according to the direction of the normal vector to calculate the width.

[0167] ①k-d tree

[0168] The normal vector reflects the local geometric characteristics of a curve or surface at a specific point. In order to accurately calculate the normal vector of a point, it is necessary to consider the geometric structure near the point, so it is necessary to consider the neighboring points of the point to construct a point set that can reflect the local geometric characteristics. In addition, in the actual calculation process, there may be outliers, and there is often noise in the data. Using a point set composed of nearest neighbor points to represent a single point can effectively improve the robustness of the normal vector calculation results. This chapter uses the kd tree algorithm to find the nearest neighbor points of each point in the skeleton line to improve the overall efficiency of the algorithm. The following will explain the construction of k-dtree and the method of finding the nearest neighbor points:

[0169] 1) kd tree construction:

[0170] a. Select axis: First calculate the variance of each dimension, and then select the dimension with the largest variance as the split axis to make the partition more balanced.

[0171] b. Select the split point: After completing the selection of the split axis, you need to select a point as the split point to split the data set into two subsets. Usually, this point is the median point on the selected axis, which can make the constructed tree more balanced.

[0172] c. Recursive construction: Divide the data set into two parts according to the split point in b, and then iteratively execute steps a and b in each subset until the subset divided in b contains only one point or no point.

[0173] 2) Find the nearest neighbor:

[0174] a. Search downward: Starting from the root node, compare the value of the current node with the query point, decide whether to move to the left branch or the right branch based on the comparison result, and then recursively traverse the tree in the branch until a leaf node is found, and use this leaf node as the candidate value of the nearest neighbor point.

[0175] b. Backtracking: Start the backtracking process and check if the subtree on the other side may contain a closer point. This step is done by calculating the distance from the query point to the split plane and comparing it with the current closest distance. If the distance to the split plane is less than the current closest distance, there may be a closer point in the subtree on the other side, which needs to be searched. If there is a closer point in the tree on the other side, the closest point found in a is updated.

[0176] c. Termination condition: When the backtracking process is completed and all possible closer areas have been checked, the nearest neighbor search ends.

[0177] ②SVD solves the normal vector

[0178] Singular value decomposition (SVD) is a powerful matrix decomposition method, which is often used for data dimension reduction, noise removal, feature extraction and other tasks. This method has strong robustness, stability and versatility. The direction corresponding to the minimum singular value is also the direction where the point data changes the least, that is, the direction of the normal vector. In order to enhance the robustness of solving the normal vector, this chapter uses 9 nearest neighbor points to solve the skeleton line normal vector through SVD. The solution steps are as follows:

[0179] 1) Construct the matrix: First, calculate the mean x of the two dimensions of the 9 points respectively 0 ,y 0 , and then the coordinates of the 9 points are subtracted from the mean to perform centralization. The calculation process can be expressed by formula (4.14):

[0180]

[0181] 2) SVD: Perform singular value decomposition on matrix A. The singular value decomposition calculation process is expressed by formula (4.15):

[0182] A=U∑V T (4.15)

[0183] The last column of the V matrix obtained by solving is the normal vector. The direction information provided by the normal vector can be used to obtain the two corresponding intersection points on the edge. The example result is as follows: Fig.16 , the blue line segment in the figure represents the pixel width of some areas.

[0184] 6.5 Linear Array Camera Accuracy Calculation

[0185] To convert the pixel distance in the image into the actual distance, it is necessary to calculate the accuracy of the camera in combination with the imaging principle of the linear array camera. The lateral accuracy and longitudinal accuracy of the linear array camera are calculated by formula (4.16) and formula (4.17) respectively:

[0186]

[0187] Where P H and P L Represents the lateral accuracy and longitudinal accuracy respectively, L o Indicates the horizontal field of view width, H c Indicates the number of pixels of the line array camera, V o and V c Represent the running speed of the target and the line frequency of the linear array camera respectively. In order to ensure that the acquired image is not distorted, it is necessary to adjust the running speed of the target or the line frequency of the camera so that P H and P L The horizontal field of view width is calculated as follows:

[0188]

[0189] Where W s represents the physical width of the linear array camera sensor, D represents the distance between the camera lens and the object being photographed, and f represents the focal length of the camera.

[0190] In order to illustrate the effectiveness of this technical solution, an ablation experiment was conducted, as follows:

[0191] The experiment was conducted under the Ubuntu 20.04 system, using the PyTorch 1.11.1 deep learning framework and the CUDA 11.3 toolkit for GPU acceleration. The computer configuration used included: NVIDIA GeForce 3060Ti 8GB (GPU) and Intel Core i5-13490F (CPU).

[0192] The initial learning rate during model training is set to 0.01, and the final learning rate coefficient is set to 0.01, so the final learning rate is 0.00001. The number of iterations is set to 1000, and in order to save computing resources during training, the number of early stopping waiting rounds is set to 100. Based on stochastic gradient descent (SGD) training, the momentum is set to 0.937 and the weight decay coefficient is set to 0.0005.

[0193] The laser cutting plates are mainly cold-rolled coils, galvanized coils, aluminum alloy coils, etc. with a thickness of 0.6mm to 3.0mm. The surface of the plates before cutting is flat, clean, and free of particulate dirt. During image acquisition, the camera is fixed at a height of about 2.4m from the conveyor belt, and a 16K linear array camera is used in conjunction with a rotary encoder to capture the original image of the steel plate after laser cutting. The resolution is 16384×30000. Due to the large size of a single image, in order to facilitate the annotation of the real laser-cut steel plate image. First, the original image is threshold segmented and cropped to delete the redundant image without the steel plate area, and then it is randomly cropped to generate an image of 640×640 size.

[0194] The images without slits after cropping were removed to save computing resources, and more than 6,400 images were generated as the self-built steel plate laser slit dataset. Subsequently, the image annotation toolbox "labelme" was used to manually annotate these 6,400 images, and the slit area was represented by a closed polygon surrounded by a series of points around the slit. The maximum and minimum coordinates in the width and height directions of the image were used to determine the rectangular bounding box. Finally, the dataset was divided into training set, test set, and validation set in a ratio of 6:2:2.

[0195] In order to evaluate the effectiveness of each module in the proposed method, six groups of experiments were designed to test different backbone networks, different prototype mask generation networks, and models with attention mechanisms. The results are shown in Table 1. ① Model 1 is the benchmark YOLOv8s-Seg; ② Model 2 replaces RevCol as the backbone network based on Model 1. Although the detection accuracy is slightly reduced, the number of parameters and the amount of calculation are greatly reduced by 25.2% and 24.3%, respectively, and the inference speed is greatly improved. This shows that RevCol as the backbone network maintains good feature extraction capabilities and effectively reduces the demand for computing resources. ③ Model 3 uses RevCol-ESSC as the backbone network. All its accuracy test indicators exceed the baseline model, which fully demonstrates that the ESSC convolution module with multi-scale receptive field and spatial perception can better extract feature information and maintain a low number of parameters and calculations. However, due to the increase in the number of stacked layers in the model, the calculation speed has decreased slightly. ④ In model 4, the PCCA attention mechanism is added only on the basis of the baseline model. By comparing model 1 and model 4, it is found that after the introduction of the PCCA module, the border AP50 is increased by 0.4 percentage points, the border AP50-95 is increased by 0.9 percentage points; the mask AP50 is increased by 1.4 percentage points, and the mask AP50-95 is increased by 1.6 percentage points. Figure 7 As shown, Grad-CAM

[19] is used to visualize the feature maps before and after PCCA, where (a) is the original image, (b) and (c) are the feature maps before PCCA, and (d) and (e) are the feature maps after PCCA. It can be seen from the figure that the feature map after PCCA pays more attention to the target, indicating that PCCA can effectively perceive channel information and coordinate information. ⑤ Since the TPMB prototype mask generation branch requires the output content of the reversible column backbone network, RevCol-ESSC and TPMB are combined as model 5. Compared with model 3, model 5 does not show much fluctuation in accuracy, but the computing resource usage and FPS have better performance. This fully demonstrates that using TPMB as the prototype mask branch is more suitable for lightweight device deployment. ⑥ Model 6 is the proposed lightweight instance segmentation network, which uses RevCol-ESSC, PCCA, and TPMB at the same time. Figure 8The following is a graph of the change in mAP and loss during training. It can be seen that when the epoch is between 500-600, the border loss and mask loss tend to be stable, the border mAP and mask mAP change range decreases, and the network gradually converges. Compared with the original YOLOV8s-Seg network model, the border AP50 of model 6 increased by 0.3 percentage points, the border AP50-95 increased by 0.6 percentage points, the mask mAP50 increased by 1.2 percentage points, and the mask AP50-95 increased by 1 percentage point. The number of parameters decreased, and the amount of calculation and parameters were far less than those of several other mainstream instance segmentation networks.

[0196] Table 1. Comparison of ablation experiment indicators

[0197]

[0198] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A large-area plate processing quality inspection method based on line array camera and computer vision, characterized in that: include: S1, stitching the images collected by the line array camera to obtain a complete image; S2, the complete image is cut into small images in sequence through the SAHI algorithm, and there is partial overlap between the small images; S3, the small image after cutting is sent to the instance segmentation detection algorithm to obtain the pixel-level segmentation information of the target; then digital image processing is performed based on the pixel-level segmentation information of the target, so as to realize the detection of processing quality; finally, whether the detection result is qualified is judged according to the detection result.

2. According to claim 1, a large-area plate processing quality inspection method based on line array camera and computer vision is characterized in that: The step of sending the cut small image to the instance segmentation detection algorithm to obtain pixel-level segmentation information of the target comprises: S31, input the image into a backbone network built based on a reversible column network, wherein the backbone network includes a stem and two columns. The stem uses a convolutional layer to extract feature maps from the input image. Each column includes four levels, namely level 1, level 2, level 3, and level 4, for feature extraction and fusion. The first column downsamples the feature map step by step to obtain feature maps at four levels. In the second column, level 1 to level 3 fuse the output feature map of the next layer of the previous column by upsampling, and downsample the fused feature map to the next level. Finally, four sizes of output feature maps are obtained, which are the same as the feature maps output by the four levels in the first column. S32, the four feature maps extracted by the backbone network will be sent to the neck fusion network for fusion of multi-scale feature maps. The network has two paths: bottom-up and top-down. S33, the output feature map of level 4 will be sent to the bottom-up path. First, it will be extracted by the fast feature pyramid SPPF using pooling kernels of different sizes to enhance the algorithm's detection ability for objects of different sizes and obtain the feature map. The feature map after SPPF is then input into the pyramid spatial channel attention mechanism PSCA. The PSCA attention mechanism first extracts multi-scale feature maps through four sizes of convolution kernels, then extracts multi-scale attention weights through the channel attention branch and the coordinate attention branch, and recalibrates the extracted attention weights using softmax. Finally, the calibrated attention weights are weighted on the input feature map to obtain the feature map. S34, the feature map recalibrated by PSCA is upsampled by 2 times using Upsample_1 to obtain a feature map, and then concatenated with the feature map output by level3 through Concat_1 to obtain a feature map; The feature map output by Concat_1 will obtain the gradient flow information through C2F_1, and then upsample the output feature map of C2F_1 to obtain a feature map, which is then concatenated with the feature map output by level2 to obtain a feature map, and then the concatenated feature map is fused through C2F_2 and output; S35, after the upsampling path is calculated, the multi-scale features will be further fused through the downsampling path. For the feature map output by C2F_2, the convolution layer Conv_1 with a step size of 2 will downsample it to obtain the feature map; Concat_3 concatenates the outputs of Conv_1 and C2F_1 to obtain a feature map, fuses this feature map using C2F_3 to obtain a feature map, and then downsamples this feature map using the convolutional layer Conv_2 with a stride of 2 to obtain a feature map; Conv_4 concatenates the output of Conv_2 with the output of the PSCA attention mechanism to obtain a feature map, and fuses this feature map through C2F_4 to obtain a feature map; S36, after the neck network completes the fusion of multi-scale features, level 1 in C2F_3, C2F_2, and Column2 is input into the semantic segmentation branch for feature fusion. By integrating features of three different scales, the processing capability of multi-scale targets is improved; S37, the output feature maps of the semantic segmentation branch, C2F_3, and C2F_4 are input into the Segment detection head to detect objects of different sizes; Among them, C2F_1 module to C2F_4 module are all C2F modules.

3. The large-area plate processing quality inspection method based on line array camera and computer vision according to claim 2 is characterized in that: The four sizes of convolution kernels are: 3*3, 5*5, 7*7, and 9*9.

4. The large-area plate processing quality inspection method based on line array camera and computer vision according to claim 1 is characterized in that: The semantic segmentation branch is a prototype mask coding branch TPMCB, which fuses the features of level 1 in C2F_3, C2F_2, and Column2 through the prototype mask coding branch TPMCB, including: First, the Superficial is downsampled by two times through two convolutional layers to obtain the first-level feature map; Then, the high-resolution reconstruction of the emantic feature map is performed through the data-dependent upsampling module Dupsample, and then the second-level feature map is obtained through the convolution layer; Secondly, the convolution module further extracts the details to obtain the second-level feature map; Finally, the feature maps of three different levels are combined by addition and further fused by the convolutional layer; Among them, level1 in C2F_3, C2F_2, and Column2 respectively represents high-level semantic information Semantic, intermediate detail information Detail, and primary feature information Superficial.

5. The large-area plate processing quality inspection method based on line array camera and computer vision according to claim 1 is characterized in that: Digital image processing is performed based on the pixel-level segmentation information of the target to detect the processing quality; including: S300, target region binarization: cutting the region containing the target according to the pixel-level segmentation information obtained by the instance segmentation algorithm, and then binarizing the cutting result to obtain target information; S301, morphological processing: applying morphological processing to the target area after binarization; S302, edge detection: using the Canny algorithm to perform edge detection on the target area after morphological processing, and obtain the edge contour of the target; S303, skeleton extraction: using the medial axis transformation algorithm to extract the skeleton of the target area after morphological processing to obtain the center line; S304, width calculation: firstly, the normal vector of the center line is calculated by the kd Tree algorithm and the SVD algorithm, and then two intersection points with the edge contour line in the direction of the normal vector are obtained, and the target pixel width here is calculated according to the coordinates of the two points.

6. The large-area plate processing quality inspection method based on line array camera and computer vision according to claim 1 is characterized in that: The image captured by the linear array camera is captured by an image acquisition platform, which includes a linear array camera, a linear light source and an image acquisition platform bracket; The image acquisition platform bracket includes a vertical cylindrical metal rod, and at least three inclined rods are arranged at the bottom of the vertical cylindrical metal rod to achieve stable support for the image acquisition platform bracket; A round tube fixing fixture is clamped in the middle of the vertical cylindrical metal rod, a light source fixing cross bar is arranged on the round tube fixing fixture, and the linear light source is fixedly installed on the light source fixing cross bar; the irradiation position of the light source can be adjusted by adjusting the round tube fixing fixture; A moving mechanism for the image acquisition module to move up and down is arranged at the top of the vertical cylindrical metal rod, and the image acquisition module includes a linear camera; It also includes a photoelectric switch and / or an encoder. When the drag board is detected by the photoelectric switch, the photoelectric switch sends a control signal to the linear light source to make it emit light; and sends a control signal to the linear camera to make the linear camera start working; the encoder is used to provide the camera with a shooting frequency according to the line frequency signal output by the encoder.

7. An electronic device, characterized in that: include: at least one processor; and a memory storing instructions, wherein when the instructions are executed by the at least one processor, the at least one processor executes the large-area plate processing quality inspection method based on a line array camera and computer vision as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the large-area plate processing quality inspection method based on line array camera and computer vision are implemented as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Intelligent mineral sorting method based on lightweight target detection architecture

    CN115953568A

  • Mask R-CNN-based gear defect segmentation model, training method and detection method

    CN116758036A

  • Deep learning technology-based protists image analysis method

    CN117809042A

  • Boundary-optimized remote sensing image semantic segmentation method and apparatus, and device and medium

    WO2023077816A1

Cited By

  • Image detection method, system and device, storage medium and program product

    CN121329953A

  • Multi-sensor collaborative fabric defect detection method and device

    CN121685537A