Area-array camera multi-angle imaging and image fusion method

Abstract

The invention relates to the technical field of defect detection imaging, in particular to a method for multi-angle imaging and image fusion of an area-array camera, which can acquire a large amount of source image data in a short time by using the combination of the area-array camera and a light source, and improves the quality of a source image by preprocessing the source image data. The method comprises the following steps: firstly, creating a frame image, creating an image according to the frame image and a specific formula, extracting data of a specified row from each frame image, fusing the row data extracted from all frame images until the motion is finished, generating a preliminary fused image, and integrating information at different time points, so that features and changes in the motion process are more comprehensively described, and finally, the motion speed is improved. According to the method, the preliminary fusion image is decomposed through wavelet transform, the enhancement coefficient of the preliminary fusion image is obtained through convolution operation, the preliminary fusion image is enhanced through the enhancement coefficient, the fusion image is obtained, detail information in the image is highlighted, and the image is clearer and sharper.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection imaging, and in particular to a method for multi-angle imaging and image fusion of an area array camera. Background Art

[0002] Defect detection imaging is an important technology. It can accurately detect tiny defects on the surface or inside of a product, greatly improving the accuracy and reliability of detection. It can detect the entire surface or inside of an object to avoid missing defects. For large mechanical parts, it can comprehensively detect defects in various parts. The multi-angle imaging of area array cameras is closely combined with defect detection imaging. The current imaging method is to synchronously move a line scan camera and a light source to scan the product line by line to obtain product imaging or to scan the entire product area through an area array camera and a light source to obtain a product image. Line scan camera imaging has the problems of single image illumination angle and small depth of field, and defect features may not be highlighted or the contrast is not high. Area array camera imaging has the problems of uneven image illumination and large image distortion, and defect features may be difficult to segment. In order to solve the above technical problems, we provide a method of multi-angle imaging and image fusion of area array cameras. Summary of the invention

[0003] The object of the present invention is to provide a method for multi-angle imaging and image fusion of an area array camera to solve the problems raised in the above background technology.

[0004] To achieve the above object, the present invention provides a method for multi-angle imaging and image fusion of an area array camera, comprising the following steps:

[0005] S1. Use a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the product to be inspected, and take fixed-frame photos at the same time. The source image is cached in an image queue, and the source image queue is traversed to take out the source image frame by frame and perform preprocessing to obtain a frame image.

[0006] S2, determine the width of the fused image according to the frame image, calculate the height of the fused image by a specific formula, create an image according to the width and height of the image, take out the cached frame images frame by frame, extract the data of the specified row in each frame image, and then fuse the row data extracted from all frame images until the movement is completed to generate a preliminary fused image;

[0007] S3. Decompose the preliminary fused image using wavelet transform to obtain low-frequency and high-frequency coefficients in the column direction of the preliminary fused image, and perform column-wise convolution operations on the low-frequency coefficients and high-frequency coefficients in the row direction to obtain enhancement coefficients of the preliminary fused image. Use the enhancement coefficients to enhance the preliminary fused image to obtain a fused image.

[0008] As a further improvement of the technical solution, the S1 uses a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the inspected product and simultaneously perform fixed-frame shooting, specifically including:

[0009] For shaft products that require 360-degree shooting to detect defects, the product is rotated for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows:

[0010]

[0011] Where, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), Cnt is the number of extracted rows, R is the product rotation radius (mm), and π is the pi;

[0012] For all products with defects detected on the upper surface, the product is moved horizontally for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows:

[0013] V=Fps*Acc*Cnt

[0014] Among them, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), and Cnt is the number of extracted rows.

[0015] As a further improvement of this solution, the derivation formula of the relationship between the motion speed and the camera frame rate is as follows, specifically including:

[0016] Calculate the rotation circumference C:

[0017] C=2*π*R

[0018] Calculate the radian Rad corresponding to a single pixel:

[0019]

[0020] Calculate the angular velocity W required for motion:

[0021]

[0022] Convert the angular velocity of the movement into the speed V in units of r / min (revolutions per minute). The radian corresponding to one circle of movement is 2π:

[0023]

[0024] Wherein, Rad is represented by the radian corresponding to a single pixel.

[0025] As a further improvement of the technical solution, the step of taking out frame images frame by frame and performing preprocessing in S1 specifically includes:

[0026] A combination of an area array camera and a light source is used to perform relative uniform motion on the surface of the inspected product, and fixed-frame shooting is performed at the same time to obtain a source image of the inspected product. The source image is processed and corrected by a median filter algorithm to obtain a frame image. The correction includes geometric correction and color correction. The geometric correction is used to eliminate image deformation caused by differences in camera shooting angles and positions, and the color correction is used to adjust the color differences between images taken by different cameras.

[0027] As a further improvement of the technical solution, the width of the image is determined according to the frame image in S2, and then the height of the fused image is calculated by a specific formula, which specifically includes:

[0028] When performing image fusion processing, the width w of the fused image is directly set to be the same as the width of the frame image. The specific formula for calculating the height of the fused image is:

[0029]

[0030] Among them, Rad represents the radian corresponding to a single pixel, Acc is the camera pixel accuracy (mm / pixel), and h represents the image height. The image is automatically created according to the width and height of the fused image to ensure that the fused image can accurately integrate the information of the frame image.

[0031] As a further improvement of the technical solution, the step of extracting data of a specified row in each frame of the image in S2 specifically includes:

[0032] Extract the specified row data from the i-th frame image, the specified row data is H j ~H k , the H j Indicates the starting row number for extracting row data from the frame image, H k It represents the end row number of the row data extracted from the frame image, j≤k≤h. If j=k, one row is extracted. If j≠k, multiple rows are extracted. The calculation formula of the number of rows Cnt is as follows:

[0033] Cnt=k-j+1.

[0034] As a further improvement of the technical solution, the line data extracted from all frame images are fused in S2 until the motion is completed to generate a fused image, which specifically includes:

[0035] From the frame image, select row number H j To line number H k The row data is assigned to the fused image from row number Hf (i) To line number Hg (i) The line, f (i) The calculation formula is:

[0036] f(i) =(i-1)*Cnt+1=(i-1)*(kj)+i

[0037] The g (i) The calculation formula is:

[0038] g (i) =(i-1)*Cnt+Cnt=i*(k-j+1)

[0039] Until the movement ends, a preliminary fused image is generated.

[0040] As a further improvement of the technical solution, the preliminary fused image is decomposed into sub-bands of different frequencies in S3, specifically including:

[0041] The preliminary fused image is decomposed by wavelet transform, the wavelet transform is DC wavelet, and the filter coefficient of DC wavelet is used to perform convolution operation on the image in the row direction to obtain low-frequency and high-frequency coefficients in the column direction. The calculation formula of the filter coefficient of DC wavelet is:

[0042]

[0043] α1(β)=(-1) β α0(N-1-β)

[0044] Among them, α0(β) represents the low-pass filter coefficient, α1(β) represents the high-pass filter coefficient, N represents the order of DC wavelet, β=0, 1, ..., N-1, and convolution operations are performed on the low-frequency coefficients and high-frequency coefficients in the row direction in the column direction to obtain the enhancement coefficients of the preliminary fused image.

[0045] As a further improvement of the technical solution, the acquisition of the low-frequency sub-band and the high-frequency sub-band specifically includes:

[0046] Using the filter coefficients of the selected wavelet basis function, a convolution operation is performed on the image in the column direction. The convolution operation in the column direction is to perform a convolution operation on the low-frequency coefficients and the high-frequency coefficients in the column direction. The calculation formula of the low-frequency coefficients and the high-frequency coefficients is:

[0047]

[0048]

[0049] Among them, c L (m) is the low frequency coefficient, c H(m) is a high-frequency coefficient, n represents the index of the filter coefficient, and m represents the index of the image line signal. The low-frequency coefficient contains the energy information of the image, and the high-frequency coefficient contains the edge information of the image. The enhancement coefficient of the preliminary fused image is obtained by weighted fusion of the low-frequency coefficient and the high-frequency coefficient. The enhancement coefficient is used to enhance the preliminary fused image. The calculation formula of the enhancement coefficient is:

[0050] δ=ω1*c L (m)+ω2*c H (m)

[0051] Among them, ω1 and ω2 are the influence weights of the low-frequency coefficient and the high-frequency coefficient on the enhancement coefficient.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The invention provides a method for multi-angle imaging and image fusion of an area array camera. By using an area array camera and a light source combination, a large amount of source image data can be acquired in a short time. By preprocessing the source image data, the source image quality is improved and the interference of subsequent processing is reduced. An image is created according to a frame image and a specific formula, and data of a specified row is extracted from each frame image. Then, the row data extracted from all frame images are fused until the movement ends to generate a preliminary fused image. The information at different time points is integrated to more comprehensively describe the characteristics and changes in the movement process. Finally, the preliminary fused image is decomposed by wavelet transform, and an enhancement coefficient of the preliminary fused image is obtained by convolution operation. The preliminary fused image is enhanced by the enhancement coefficient to obtain a fused image, which highlights the detail information in the image and makes the image clearer and sharper. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is the overall workflow diagram of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Example 1

[0057] See also Figure 1 As shown, this embodiment provides a method for multi-angle imaging and image fusion of an area array camera, comprising the following steps:

[0058] S1. Use a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the product to be inspected, and take fixed-frame photos at the same time. The source image is cached in an image queue, and the source image queue is traversed to take out the source image frame by frame and perform preprocessing to obtain a frame image.

[0059] S1 uses a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the product being inspected, while also taking fixed-frame photos, specifically including:

[0060] For shaft products that require 360-degree shooting to detect defects, the product is rotated for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows:

[0061]

[0062] Where, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), Cnt is the number of extracted rows, R is the product rotation radius (mm), and π is the pi;

[0063] For all products with defects detected on the upper surface, the product is moved horizontally for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows:

[0064] V=Fps*Acc*Cnt

[0065] Among them, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), and Cnt is the number of extracted rows.

[0066] The derivation formula of the relationship between motion speed and camera frame rate is as follows, including:

[0067] Calculate the rotation circumference C:

[0068] C=2*π*R

[0069] Calculate the radian Rad corresponding to a single pixel:

[0070]

[0071] Calculate the angular velocity W required for motion:

[0072]

[0073] Convert the angular velocity of the movement into the speed V in units of r / min (revolutions per minute). The radian corresponding to one circle of movement is 2π:

[0074]

[0075] Wherein, Rad is represented by the radian corresponding to a single pixel.

[0076] S1 takes out frame images one by one and performs preprocessing, which specifically includes:

[0077] A combination of an area array camera and a light source is used to perform relative uniform motion on the surface of the inspected product while capturing fixed-frame images to obtain the source image of the inspected product. The source image is processed and corrected using a median filter algorithm to obtain a frame image. The correction includes geometric correction and color correction. The geometric correction is used to eliminate image deformation caused by differences in camera shooting angles and positions. The color correction is used to adjust the color differences between images taken by different cameras to ensure color consistency of the fused image.

[0078] S2, determine the width of the fused image according to the frame image, calculate the height of the fused image by a specific formula, create an image according to the width and height of the image, take out the cached frame images frame by frame, extract the data of the specified row in each frame image, and then fuse the row data extracted from all frame images until the movement is completed to generate a preliminary fused image;

[0079] In S2, the width of the image is determined according to the frame image, and then the height of the fused image is calculated by a specific formula, which includes:

[0080] When performing image fusion processing, directly setting the width w of the fused image to be the same as the width of the frame image can ensure that the fused image can completely contain the information of the frame image in terms of width, providing a basis for subsequent row data extraction and fusion. The specific formula for calculating the height of the fused image is:

[0081]

[0082] Among them, Rad represents the radian corresponding to a single pixel, Acc is the camera pixel accuracy (mm / pixel), and h represents the image height. The image is automatically created according to the width and height of the fused image to ensure that the fused image can accurately integrate the information of the frame image.

[0083] In S2, the data of the specified row is extracted from each frame of the image, including:

[0084] Extract the specified row data from the i-th frame image, the specified row data is H j ~H k , H j Indicates the starting row number for extracting row data from the frame image, H k It represents the end row number of the row data extracted from the frame image, j≤k≤h. If j=k, one row is extracted. If j≠k, multiple rows are extracted. The calculation formula of the number of rows Cnt is as follows:

[0085] Cnt=k-j+1.

[0086] In S2, the row data extracted from all frame images are fused until the motion ends to generate a fused image, which specifically includes:

[0087] From the frame image, select row number H j To line number H k The row data is assigned to the fused image from row number Hf (i) To line number Hg (i) The line, f (i) The calculation formula is:

[0088] f (i) =(i-1)*Cnt+1=(i-1)*(kj)+i

[0089] g (i) The calculation formula is:

[0090] g (i) =(i-1)*Cnt+Cnt=i*(k-j+1)

[0091] Among them, f (i) is the starting row number of the fused image, g (i) is the termination row number of the fused image until the motion ends and a preliminary fused image is generated.

[0092] S3, decomposing the preliminary fused image by using wavelet transform to obtain low-frequency and high-frequency coefficients in the column direction of the preliminary fused image, and performing column-wise convolution operations on the low-frequency coefficients and high-frequency coefficients in the row direction to obtain enhancement coefficients of the preliminary fused image, and enhancing the preliminary fused image by using the enhancement coefficients to obtain a fused image;

[0093] In S3, the preliminary fused image is decomposed into sub-bands of different frequencies, including:

[0094] The preliminary fusion image is decomposed by wavelet transform, and the wavelet transform is DC wavelet. The filter coefficient of DC wavelet is used to perform convolution operation on the image in the row direction to obtain the low-frequency and high-frequency coefficients in the column direction. The calculation formula of the filter coefficient of DC wavelet is:

[0095]

[0096] α1(β)=(-1) β α0(N-1-β)

[0097] Among them, α0(β) represents the low-pass filter coefficient, α1(β) represents the high-pass filter coefficient, N represents the order of DC wavelet, β=0, 1, ..., N-1, and convolution operations are performed on the low-frequency coefficients and high-frequency coefficients in the row direction in the column direction to obtain the enhancement coefficients of the preliminary fused image.

[0098] The acquisition of low-frequency coefficients and high-frequency coefficients includes:

[0099] Using the filter coefficients of the selected wavelet basis function, the image is convolved in the column direction. The convolution operation in the column direction is to convolve the low-frequency coefficients and high-frequency coefficients in the column direction. The calculation formula of the low-frequency coefficients and the high-frequency coefficients is:

[0100]

[0101]

[0102] Among them, c L (m) is the low frequency coefficient, c H (m) is the high-frequency coefficient, n represents the index of the filter coefficient, m represents the index of the image line signal, the low-frequency coefficient contains the energy information of the image, and the high-frequency coefficient contains the edge information of the image. The enhancement coefficient of the preliminary fused image is obtained by weighted fusion of the low-frequency coefficient and the high-frequency coefficient. The enhancement coefficient is used to enhance the preliminary fused image. The calculation formula of the enhancement coefficient is:

[0103] δ=ω1*c L (m)+ω2*c H (m)

[0104] Among them, ω1 and ω2 are the influence weights of the low-frequency coefficient and the high-frequency coefficient on the enhancement coefficient.

[0105] The invention provides a method for multi-angle imaging and image fusion of an area array camera. By using an area array camera and a light source combination, a large amount of source image data can be acquired in a short time. By preprocessing the source image data, the source image quality is improved and the interference of subsequent processing is reduced. An image is created according to a frame image and a specific formula, and data of a specified row is extracted from each frame image. Then, the row data extracted from all frame images are fused until the movement ends to generate a preliminary fused image. The information at different time points is integrated to more comprehensively describe the characteristics and changes in the movement process. Finally, the preliminary fused image is decomposed by wavelet transform, and an enhancement coefficient of the preliminary fused image is obtained by convolution operation. The preliminary fused image is enhanced by the enhancement coefficient to obtain a fused image, which highlights the detail information in the image and makes the image clearer and sharper.

[0106] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for multi-angle imaging and image fusion of an area array camera, characterized in that: The following steps are involved: S1. Use a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the product to be inspected, and take fixed-frame photos at the same time. The source image is cached in an image queue, and the source image queue is traversed to take out the source image frame by frame and perform preprocessing to obtain a frame image. S2, determine the width of the fused image according to the frame image, calculate the height of the fused image by a specific formula, create an image according to the width and height of the image, take out the cached frame images frame by frame, extract the data of the specified row in each frame image, and then fuse the row data extracted from all frame images until the movement is completed to generate a preliminary fused image; S3. Decompose the preliminary fused image using wavelet transform to obtain low-frequency and high-frequency coefficients in the column direction of the preliminary fused image, and perform column-wise convolution operations on the low-frequency coefficients and high-frequency coefficients in the row direction to obtain enhancement coefficients of the preliminary fused image. Use the enhancement coefficients to enhance the preliminary fused image to obtain a fused image.

2. The method for multi-angle imaging and image fusion of an area array camera according to claim 1, characterized in that: S1 uses a combination of an area array camera and a light source to perform a relatively uniform motion on the surface of the product to be inspected, and simultaneously performs fixed-frame shooting, specifically including: For shaft products that require 360-degree shooting to detect defects, the product is rotated for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows: Where, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), Cnt is the number of extracted rows, R is the product rotation radius (mm), and π is the pi; For all products with defects detected on the upper surface, the product is moved horizontally for shooting. The relationship between the movement speed and the camera frame rate is calculated as follows: V=Fps*Acc*Cnt Among them, v is the motion speed (r / min), Fps is the camera frame rate, Acc is the camera pixel accuracy (mm / pixel), and Cnt is the number of extracted rows.

3. The method for multi-angle imaging and image fusion of an area array camera according to claim 2, characterized in that: The derivation formula of the relationship between the motion speed and the camera frame rate is as follows, specifically including: Calculate the rotation circumference C: C=2*π*R Calculate the radian Rad corresponding to a single pixel: Calculate the angular velocity W required for motion: Convert the angular velocity of the movement into the speed V in units of r / min (revolutions per minute). The radian corresponding to one circle of movement is 2π: Wherein, Rad is represented by the radian corresponding to a single pixel.

4. The method for multi-angle imaging and image fusion of an area array camera according to claim 3, characterized in that: The S1 takes out frame images frame by frame and performs preprocessing, specifically including: A combination of an area array camera and a light source is used to perform relative uniform motion on the surface of the inspected product, and fixed-frame shooting is performed at the same time to obtain a source image of the inspected product. The source image is processed and corrected by a median filter algorithm to obtain a frame image. The correction includes geometric correction and color correction. The geometric correction is used to eliminate image deformation caused by differences in camera shooting angles and positions, and the color correction is used to adjust the color differences between images taken by different cameras.

5. The method for multi-angle imaging and image fusion of an area array camera according to claim 1, characterized in that: In S2, the width of the image is determined according to the frame image, and then the height of the fused image is calculated by a specific formula, which specifically includes: When performing image fusion processing, the width w of the fused image is directly set to be the same as the width of the frame image. The specific formula for calculating the height of the fused image is: Among them, Rad represents the radian corresponding to a single pixel, Acc is the camera pixel accuracy (mm / pixel), and h represents the image height. The image is automatically created based on the width and height of the fused image.

6. The method for multi-angle imaging and image fusion of an area array camera according to claim 1, characterized in that: The step S2 extracts data of a specified row from each frame of the image, specifically including: Extract the specified row data from the i-th frame image, the specified row data is H j ~H k , the H j Indicates the starting row number for extracting row data from the frame image, H k It is expressed as the end row number of the row data extracted from the frame image, j≤k≤h. If j=k, one row is extracted, and if j≠k, multiple rows are extracted. The calculation formula of the number of rows Cnt is as follows: Cnt=k-j+1.

7. The method for multi-angle imaging and image fusion of an area array camera according to claim 6, characterized in that: In S2, the row data extracted from all frame images are fused until the motion is completed to generate a fused image, which specifically includes: From the frame image, select row number H j To line number H k The row data is assigned to the fused image from row number Hf (i) To line number Hg (i) The line, f (i) The calculation formula is: f (i) =(i-1)*Cnt+1=(-1)*(k-j)+i The g (i) The calculation formula is: g (i) =(i-1)*Cnt+Cnt=i*(k-j+1) Among them, f (i) is the starting row number of the fused image, g (i) The ending row number of the fused image until the motion ends, generating a preliminary fused image.

8. The method for multi-angle imaging and image fusion of an area array camera according to claim 7, characterized in that: In S3, the preliminary fused image is decomposed into sub-bands of different frequencies, specifically including: The preliminary fused image is decomposed by wavelet transform, the wavelet transform is DC wavelet, and the filter coefficient of DC wavelet is used to perform convolution operation on the image in the row direction to obtain low-frequency and high-frequency coefficients in the column direction. The calculation formula of the filter coefficient of DC wavelet is: α1(β)＝(-1) β α0(N-1-β) Among them, α0(β) represents the low-pass filter coefficient, α1(β) represents the high-pass filter coefficient, N represents the order of DC wavelet, β=0, 1, ..., N-1, and convolution operations are performed on the low-frequency coefficients and high-frequency coefficients in the row direction in the column direction to obtain the enhancement coefficients of the preliminary fused image.

9. The method for multi-angle imaging and image fusion of an area array camera according to claim 8, characterized in that: The acquisition of the low-frequency coefficients and the high-frequency coefficients specifically includes: Using the filter coefficients of the selected wavelet basis function, a convolution operation is performed on the image in the column direction. The convolution operation in the column direction is to perform a convolution operation on the low-frequency coefficients and the high-frequency coefficients in the column direction. The calculation formula of the low-frequency coefficients and the high-frequency coefficients is: Among them, c L (m) is the low frequency coefficient, c H (m) is a high-frequency coefficient, n represents the index of the filter coefficient, and m represents the index of the image line signal. The low-frequency coefficient contains the energy information of the image, and the high-frequency coefficient contains the edge information of the image. The enhancement coefficient of the preliminary fused image is obtained by weighted fusion of the low-frequency coefficient and the high-frequency coefficient. The enhancement coefficient is used to enhance the preliminary fused image. The calculation formula of the enhancement coefficient is: δ=ω1*c L (m)+2*c H (m) Among them, ω1 and ω2 are the influence weights of the low-frequency coefficient and the high-frequency coefficient on the enhancement coefficient.

Images