A method for multi-angle imaging and image fusion of area array cameras

Abstract

The present invention relates to the field of defect detection imaging technology, and in particular to a method for multi-angle imaging and image fusion of an area array camera. The present invention can obtain a large amount of source image data in a short time by using an area array camera and a light source combination, and improves the quality of the source image by preprocessing the source image data. Then, an image is created based on a frame image and a specific formula, and data of a specified row is extracted from each frame image. Then, the row data extracted from all frame images are fused until the movement ends, and a preliminary fused image is generated, which integrates information at different time points, thereby more comprehensively describing the characteristics and changes in the movement process. Finally, the preliminary fused image is decomposed and weightedly fused by wavelet transform to obtain an enhancement coefficient of the preliminary fused image, and the preliminary fused image is enhanced by the enhancement coefficient to obtain a fused image, which highlights the detail information in the image and makes the image clearer and sharper.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection imaging, and in particular to a method for multi-angle imaging and image fusion of an area array camera. Background Art

[0002] Defect detection imaging is an important technology. It can accurately detect tiny defects on the surface or inside of a product, greatly improving the accuracy and reliability of detection. It can detect the entire surface or inside of an object to avoid missing defects. For large mechanical parts, it can comprehensively detect defects in various parts. The multi-angle imaging of area array cameras is closely combined with defect detection imaging. The current imaging method is to synchronously move a line scan camera and a light source to scan the product line by line to obtain product imaging or to scan the entire product area through an area array camera and a light source to obtain a product image. Line scan camera imaging has the problems of single image illumination angle and small depth of field, and defect features may not be highlighted or the contrast is not high. Area array camera imaging has the problems of uneven image illumination and large image distortion, and defect features may be difficult to segment. In order to solve the above technical problems, we provide a method of multi-angle imaging and image fusion of area array cameras. Summary of the invention

[0003] The object of the present invention is to provide a method for multi-angle imaging and image fusion of an area array camera to solve the problems raised in the above background technology.

[0004] To achieve the above object, the present invention provides a method for multi-angle imaging and image fusion of an area array camera, comprising the following steps:

[0005] S1. Use a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the product to be inspected, and take fixed-frame photos at the same time. The source image is cached in an image queue, and the source image queue is traversed to take out the source image frame by frame and perform preprocessing to obtain a frame image.

[0006] S2, determine the width of the fused image according to the frame image, calculate the height of the fused image by a specific formula, create an image according to the width and height of the image, take out the cached frame images frame by frame, extract the data of the specified row in each frame image, and then fuse the row data extracted from all frame images until the movement is completed to generate a preliminary fused image;

[0007] S3. Decompose the preliminary fused image using wavelet transform to obtain low-frequency and high-frequency coefficients in the column direction of the preliminary fused image, and perform weighted fusion operations on the low-frequency coefficients and high-frequency coefficients in the column direction to obtain enhancement coefficients of the preliminary fused image. Use the enhancement coefficients to enhance the preliminary fused image to obtain a fused image.

[0008] As a further improvement of the technical solution, the S1 uses a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the inspected product and simultaneously perform fixed-frame shooting, specifically including:

[0009] For shaft products that require 360-degree shooting to detect defects, the product is rotated for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows:

[0010]

[0011] Where, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), Cnt is the number of extracted rows, R is the product rotation radius (mm), and π is the pi;

[0012] For all products with defects detected on the upper surface, the product is moved horizontally for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows:

[0013] V=Fps*Acc*Cnt

[0014] Among them, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), and Cnt is the number of extracted rows.

[0015] As a further improvement of the technical solution, the step of taking out frame images frame by frame and performing preprocessing in S1 specifically includes:

[0016] A combination of an area array camera and a light source is used to perform relative uniform motion on the surface of the inspected product, and fixed-frame shooting is performed at the same time to obtain a source image of the inspected product. The source image is then processed and corrected using a median filtering algorithm to obtain a frame image. The correction includes geometric correction and color correction. The geometric correction is used to eliminate image deformation caused by differences in camera shooting angles and positions, and the color correction is used to adjust color differences between images taken by different cameras.

[0017] As a further improvement of the technical solution, the width of the image is determined according to the frame image in S2, and then the height of the fused image is calculated by a specific formula, which specifically includes:

[0018] When performing image fusion processing, the width w of the fused image is directly set to be the same as the width of the frame image. The specific formula for calculating the height of the fused image is:

[0019]

[0020] Among them, Rad represents the radian corresponding to a single pixel, Acc is the camera pixel accuracy (mm / pixel), and h represents the image height. The image is automatically created according to the width and height of the fused image to ensure that the fused image can accurately integrate the information of the frame image.

[0021] As a further improvement of the technical solution, the step of extracting data of a specified row in each frame of the image in S2 specifically includes:

[0022] Extract the specified row data from the i-th frame image, the specified row data is H j ~H k , the H j Indicates the starting row number for extracting row data from the frame image, H k It represents the end row number of the row data extracted from the frame image, j≤k≤h. If j=k, one row is extracted. If j≠k, multiple rows are extracted. The calculation formula of the number of rows Cnt is as follows:

[0023] Cnt=k-j+1.

[0024] As a further improvement of the technical solution, the line data extracted from all frame images are fused in S2 until the motion is completed to generate a fused image, which specifically includes:

[0025] From the frame image, select row number H j To line number H k The row data is assigned to the fused image from row number Hf (i) To line number Hg (i) The line, f (i) The calculation formula is:

[0026] f (i) =(i-1)*Cnt+1=(i-1)*(kj)+i

[0027] The g (i) The calculation formula is:

[0028] g (i) =(i-1)*Cnt+Cnt=i*(k-j+1)

[0029] Until the movement ends, a preliminary fused image is generated.

[0030] As a further improvement of the technical solution, the preliminary fused image is decomposed into sub-bands of different frequencies in S3, specifically including:

[0031] The preliminary fused image is decomposed by wavelet transform, the wavelet transform is DC wavelet, and the filter coefficient of DC wavelet is used to perform convolution operation on the image in the row direction to obtain low-frequency and high-frequency coefficients in the column direction. The calculation formula of the filter coefficient of DC wavelet is:

[0032]

[0033] α1(β)=(-1) β α0(N-1-β)

[0034] Wherein, α0(β) represents the low-pass filter coefficient, α1(β) represents the high-pass filter coefficient, N represents the order of the DC wavelet, β=0, 1, ..., N-1.

[0035] As a further improvement of the technical solution, the acquisition of the low-frequency sub-band and the high-frequency sub-band specifically includes:

[0036] The filter coefficients of the DC wavelet are used to perform a row-wise convolution operation on the image. The calculation formulas for the low-frequency coefficients and the high-frequency coefficients are as follows:

[0037]

[0038] Among them, c L (m) is the low frequency coefficient, c H (m) is a high-frequency coefficient, n represents the index of the filter coefficient, and m represents the index of the image line signal. The low-frequency coefficient contains the energy information of the image, and the high-frequency coefficient contains the edge information of the image. The enhancement coefficient of the preliminary fused image is obtained by weighted fusion of the low-frequency coefficient and the high-frequency coefficient. The enhancement coefficient is used to enhance the preliminary fused image. The calculation formula of the enhancement coefficient is:

[0039] δ=ω1*c L (m)+ω2*c H (m)

[0040] Among them, ω1 and ω2 are the influence weights of the low-frequency coefficient and the high-frequency coefficient on the enhancement coefficient.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The invention provides a method for multi-angle imaging and image fusion of an area array camera. By using an area array camera and a light source combination, a large amount of source image data can be acquired in a short time. By preprocessing the source image data, the source image quality is improved and the interference of subsequent processing is reduced. An image is created according to a frame image and a specific formula, and data of a specified row is extracted from each frame image. Then, the row data extracted from all frame images are fused until the movement ends to generate a preliminary fused image. The information at different time points is integrated to more comprehensively describe the characteristics and changes in the movement process. Finally, the preliminary fused image is decomposed by wavelet transform, and an enhancement coefficient of the preliminary fused image is obtained by weighted operation. The preliminary fused image is enhanced by the enhancement coefficient to obtain a fused image, which highlights the detail information in the image and makes the image clearer and sharper. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is the overall workflow diagram of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Example 1

[0046] See also Figure 1 As shown, this embodiment provides a method for multi-angle imaging and image fusion of an area array camera, comprising the following steps:

[0047] S1. Use a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the product to be inspected, and take fixed-frame photos at the same time. The source image is cached in an image queue, and the source image queue is traversed to take out the source image frame by frame and perform preprocessing to obtain a frame image.

[0048] S1 uses a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the product being inspected, while also taking fixed-frame photos, specifically including:

[0049] For shaft products that require 360-degree shooting to detect defects, the product is rotated for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows:

[0050]

[0051] Where, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), Cnt is the number of extracted rows, R is the product rotation radius (mm), and π is the pi;

[0052] For all products with defects detected on the upper surface, the product is moved horizontally for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows:

[0053] V=Fps*Acc*Cnt

[0054] Among them, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), and Cnt is the number of extracted rows.

[0055] S1 takes out frame images one by one and performs preprocessing, which specifically includes:

[0056] A combination of an area array camera and a light source is used to perform relative uniform motion on the surface of the inspected product and simultaneously perform fixed-frame shooting to obtain the source image of the inspected product. The source image is then processed and corrected using a median filtering algorithm to obtain a frame image. The correction includes geometric correction and color correction. The geometric correction is used to eliminate image deformation caused by differences in camera shooting angles and positions. The color correction is used to adjust the color differences between images taken by different cameras to ensure color consistency of the fused image.

[0057] S2, determine the width of the fused image according to the frame image, calculate the height of the fused image by a specific formula, create an image according to the width and height of the image, take out the cached frame images frame by frame, extract the data of the specified row in each frame image, and then fuse the row data extracted from all frame images until the movement is completed to generate a preliminary fused image;

[0058] In S2, the width of the image is determined according to the frame image, and then the height of the fused image is calculated by a specific formula, which includes:

[0059] When performing image fusion processing, directly setting the width w of the fused image to be the same as the width of the frame image can ensure that the fused image can completely contain the information of the frame image in terms of width, providing a basis for subsequent row data extraction and fusion. The specific formula for calculating the height of the fused image is:

[0060]

[0061] Among them, Rad represents the radian corresponding to a single pixel, Acc is the camera pixel accuracy (mm / pixel), and h represents the image height. The image is automatically created according to the width and height of the fused image to ensure that the fused image can accurately integrate the information of the frame image.

[0062] In S2, the data of the specified row is extracted from each frame of the image, including:

[0063] Extract the specified row data from the i-th frame image, the specified row data is H j ~H k , H j Indicates the starting row number for extracting row data from the frame image, H k It represents the end row number of the row data extracted from the frame image, j≤k≤h. If j=k, one row is extracted. If j≠k, multiple rows are extracted. The calculation formula of the number of rows Cnt is as follows:

[0064] Cnt=k-j+1.

[0065] In S2, the row data extracted from all frame images are fused until the motion ends to generate a fused image, which specifically includes:

[0066] From the frame image, select row number H j To line number H k The row data is assigned to the fused image from row number Hf (i) To line number Hg (i) The line, f (i) The calculation formula is:

[0067] f (i) =(i-1)*Cnt+1=(i-1)*(kj)+i

[0068] g (i) The calculation formula is:

[0069] g (i) =(i-1)*Cnt+Cnt=i*(k-j+1)

[0070] Among them, f (i) is the starting row number of the fused image, g (i) is the termination row number of the fused image until the motion ends and a preliminary fused image is generated.

[0071] S3, decomposing the preliminary fused image by using wavelet transform to obtain low-frequency and high-frequency coefficients in the column direction of the preliminary fused image, and performing column-wise convolution operations on the low-frequency coefficients and high-frequency coefficients in the row direction to obtain enhancement coefficients of the preliminary fused image, and enhancing the preliminary fused image by using the enhancement coefficients to obtain a fused image;

[0072] In S3, the preliminary fused image is decomposed into sub-bands of different frequencies, including:

[0073] The preliminary fusion image is decomposed by wavelet transform, and the wavelet transform is DC wavelet. The filter coefficient of DC wavelet is used to perform convolution operation on the image in the row direction to obtain the low-frequency and high-frequency coefficients in the column direction. The calculation formula of the filter coefficient of DC wavelet is:

[0074]

[0075] α1(β)=(-1) β α0(N-1-β)

[0076] Wherein, α0(β) represents the low-pass filter coefficient, α1(β) represents the high-pass filter coefficient, N represents the order of the DC wavelet, β=0, 1, ..., N-1.

[0077] The acquisition of low-frequency coefficients and high-frequency coefficients includes:

[0078] The filter coefficients of the DC wavelet are used to perform row-wise convolution operations on the image. The calculation formulas for the low-frequency coefficients and high-frequency coefficients are:

[0079]

[0080] Among them, c L (m) is the low frequency coefficient, c H (m) is the high-frequency coefficient, n represents the index of the filter coefficient, m represents the index of the image line signal, the low-frequency coefficient contains the energy information of the image, and the high-frequency coefficient contains the edge information of the image. The enhancement coefficient of the preliminary fused image is obtained by weighted fusion of the low-frequency coefficient and the high-frequency coefficient. The enhancement coefficient is used to enhance the preliminary fused image. The calculation formula of the enhancement coefficient is:

[0081] δ=ω1*c L (m)+ω2*c H (m)

[0082] Among them, ω1 and ω2 are the influence weights of the low-frequency coefficient and the high-frequency coefficient on the enhancement coefficient.

[0083] The invention provides a method for multi-angle imaging and image fusion of an area array camera. By using an area array camera and a light source combination, a large amount of source image data can be acquired in a short time. By preprocessing the source image data, the source image quality is improved and the interference of subsequent processing is reduced. An image is created according to a frame image and a specific formula, and data of a specified row is extracted from each frame image. Then, the row data extracted from all frame images are fused until the movement ends to generate a preliminary fused image. The information at different time points is integrated to more comprehensively describe the characteristics and changes in the movement process. Finally, the preliminary fused image is decomposed by wavelet transform, and an enhancement coefficient of the preliminary fused image is obtained by weighted operation. The preliminary fused image is enhanced by the enhancement coefficient to obtain a fused image, which highlights the detail information in the image and makes the image clearer and sharper.

[0084] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for multi-angle imaging and image fusion of an area array camera, characterized in that: The following steps are involved: S1. Use a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the product to be inspected, and take fixed-frame photos at the same time. The source image is cached in an image queue, and the source image queue is traversed to take out the source image frame by frame and perform preprocessing to obtain a frame image. S2, determine the width of the fused image according to the frame image, calculate the height of the fused image by a specific formula, create an image according to the width and height of the image, take out the cached frame images frame by frame, extract the data of the specified row in each frame image, and then fuse the row data extracted from all frame images until the movement is completed to generate a preliminary fused image; S3. Decompose the preliminary fused image using wavelet transform to obtain low-frequency and high-frequency coefficients in the column direction of the preliminary fused image, and perform weighted fusion operations on the low-frequency coefficients and high-frequency coefficients in the column direction to obtain enhancement coefficients of the preliminary fused image. Use the enhancement coefficients to enhance the preliminary fused image to obtain a fused image.

2. The method for multi-angle imaging and image fusion of an area array camera according to claim 1, characterized in that: S1 uses a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the product to be inspected, and simultaneously performs fixed-frame shooting, specifically including: For shaft products that require 360-degree shooting to detect defects, the product is rotated for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows: Where, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), Cnt is the number of extracted rows, R is the product rotation radius (mm), and π is the pi; For all products with defects detected on the upper surface, the product is moved horizontally for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows: V=Fps*Acc*Cnt Among them, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), and Cnt is the number of extracted rows.

3. The method for multi-angle imaging and image fusion of an area array camera according to claim 2, characterized in that: The S1 takes out frame images frame by frame and performs preprocessing, specifically including: A combination of an area array camera and a light source is used to perform relative uniform motion on the surface of the inspected product, and fixed-frame shooting is performed at the same time to obtain a source image of the inspected product. The source image is then processed and corrected using a median filtering algorithm to obtain a frame image. The correction includes geometric correction and color correction. The geometric correction is used to eliminate image deformation caused by differences in camera shooting angles and positions, and the color correction is used to adjust color differences between images taken by different cameras.

4. The method for multi-angle imaging and image fusion of an area array camera according to claim 1, characterized in that: In S2, the width of the image is determined according to the frame image, and then the height of the fused image is calculated by a specific formula, which specifically includes: When performing image fusion processing, the width w of the fused image is directly set to be the same as the width of the frame image. The specific formula for calculating the height of the fused image is: Among them, Rad represents the radian corresponding to a single pixel, Acc is the camera pixel accuracy (mm / pixel), and h represents the image height. The image is automatically created based on the width and height of the fused image.

5. The method for multi-angle imaging and image fusion of an area array camera according to claim 1, characterized in that: The step S2 extracts data of a specified row from each frame of the image, specifically including: Extract the specified row data from the i-th frame image, the specified row data is H j ~H k , the H j Indicates the starting row number for extracting row data from the frame image, H k It is expressed as the end row number of the row data extracted from the frame image, j≤k≤h. If j=k, one row is extracted, and if j≠k, multiple rows are extracted. The calculation formula of the number of rows Cnt is as follows: Cnt=k-j+1.

6. The method for multi-angle imaging and image fusion of an area array camera according to claim 5, characterized in that: In S2, the row data extracted from all frame images are fused until the motion is completed to generate a fused image, which specifically includes: From the frame image, select row number H j To line number H k The row data is assigned to the fused image from row number Hf (i) To line number Hg (i) The line, f (i) The calculation formula is: f (i) =(i-1)*Cnt+1=(i-1)*(k-j)+i The g (i) The calculation formula is: g (i) =(i-1)*Cnt+Cnt=i*(k-j+1) Among them, f (i) is the starting row number of the fused image, g (i) The ending row number of the fused image until the motion ends, generating a preliminary fused image.

7. The method for multi-angle imaging and image fusion of an area array camera according to claim 6, characterized in that: In S3, the preliminary fused image is decomposed into sub-bands of different frequencies, specifically including: The preliminary fused image is decomposed by wavelet transform, the wavelet transform is DC wavelet, and the filter coefficient of DC wavelet is used to perform convolution operation on the image in the row direction to obtain low-frequency and high-frequency coefficients in the column direction. The calculation formula of the filter coefficient of DC wavelet is: α1(β)＝(-1) β α0(N-1-β) Wherein, α0(β) represents the low-pass filter coefficient, α1(β) represents the high-pass filter coefficient, N represents the order of the DC wavelet, β=0, 1, ..., N-1.

8. The method for multi-angle imaging and image fusion of an area array camera according to claim 7, characterized in that: The acquisition of the low-frequency coefficients and the high-frequency coefficients specifically includes: The filter coefficients of the DC wavelet are used to perform a row-wise convolution operation on the image. The calculation formulas for the low-frequency coefficients and the high-frequency coefficients are as follows: Among them, c L (m) is the low frequency coefficient, c H (m) is a high-frequency coefficient, n represents the index of the filter coefficient, and m represents the index of the image line signal. The low-frequency coefficient contains the energy information of the image, and the high-frequency coefficient contains the edge information of the image. The enhancement coefficient of the preliminary fused image is obtained by weighted fusion of the low-frequency coefficient and the high-frequency coefficient. The enhancement coefficient is used to enhance the preliminary fused image. The calculation formula of the enhancement coefficient is: δ=ω1*c L (m)+ω2*c H (m) Among them, ω1 and ω2 are the influence weights of the low-frequency coefficient and the high-frequency coefficient on the enhancement coefficient.

Images