A method and system for measuring motion blur kernels in a visual inspection system for materials

Abstract

The present invention discloses a method and system for measuring a motion blur kernel of a material vision detection system, which relates to the technical field of industrial detection and includes: calculating the motion blur angle according to the corner points of the calibration plate before and after moving with the conveyor belt; calculating the moving distance of the conveyor belt during the camera exposure time based on the real-time speed of the conveyor belt; calculating the motion blur length corresponding to the moving distance of the conveyor belt on the image according to the internal parameters of the camera; and constructing a motion blur kernel function according to the motion blur angle and the motion blur length. The present invention uses a speed measurement encoder or other speed measurement sensors to measure the speed in real time. On the one hand, the blur length can still be solved in real time even during variable speed, and on the other hand, it is stable enough to improve the recognition accuracy of the blur kernel and ensure the stability of the image restoration quality.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial inspection, and particularly to a method and system for measuring motion blur kernels of a material vision inspection system. Background Art

[0002] During the imaging process of a camera for a moving object on a conveyor belt (during exposure), there is relative motion between the camera and the object, resulting in the captured image being blurred. This kind of blur belongs to motion blur. In the process of detecting an object during transmission on a conveyor belt, the obtained pictures often show motion blur, which affects the accuracy of vision inspection. For the restoration of motion-blurred images, one approach is to use deep learning to restore the degraded images. The pictures obtained by this restoration method can often reach a relatively high clarity and good restoration effect. However, it requires a large number of data sets for training and usually has a poor effect on unfamiliar scenes outside the data set. Moreover, the deep learning method has high requirements for restoration performance and takes a long time, making it difficult to meet the deployment requirements of high-speed real-time detection of objects on conveyor belts in industry.

[0003] Another approach is to adopt traditional methods, considering motion blur as the convolution of the original image and the motion blur kernel, estimating the motion blur kernel, and then restoring the image using inverse filtering, Wiener filtering, etc. according to the obtained blur kernel. This method has a fast operation speed and a wide application range, and can meet the requirements of high-speed real-time detection in diverse industrial scenarios. However, the acquisition of the motion blur kernel is the key to this method, and the accuracy of motion blur kernel estimation directly affects the effect of motion blur restoration. Usually, the motion blur kernel is obtained through image processing methods, and the image is transformed into the frequency domain for the identification of the motion blur kernel. However, this method for identifying the motion blur kernel depends on a relatively fixed imaging environment. When the environment changes, it is necessary to re-adjust the image processing parameters, otherwise, the recognition accuracy of the motion blur kernel will be low and the quality of the restored image will be unstable. This method is difficult to deploy, and the restoration quality is unstable, unable to meet the actual industrial deployment requirements. Summary of the Invention

[0004] Aiming at the deficiencies of the existing technology, the purpose of the present invention is to provide a method and system for measuring motion blur kernels of a material vision inspection system, which uses a speed encoder or other speed sensors to measure speed in real time. On the one hand, the blur length can still be solved in real time during variable speed, and on the other hand, it is stable enough to improve the recognition accuracy of the blur kernel and ensure the stability of the image restoration quality.

[0005] To achieve the above purpose, the present invention is implemented through the following technical solutions:

[0006] In the first aspect, an embodiment of the present invention provides a method for measuring motion blur kernels of a material vision inspection system, including:

[0007] Calculate the motion blur angle based on the corner points of the calibration board images before and after the calibration board moves with the conveyor belt;

[0008] Based on the real-time speed of the conveyor belt, calculate the moving distance of the conveyor belt during the camera exposure time;

[0009] According to the camera internal parameters, calculate the motion blur length on the image corresponding to the moving distance of the conveyor belt;

[0010] Construct a motion blur kernel function based on the motion blur angle and the motion blur length.

[0011] As a further implementation method, collect calibration board images before and after the conveyor belt moves respectively, and obtain the motion blur angle parameter of the psf blur kernel; and use the solvePnP method to obtain the relative position relationship between the camera and the conveyor belt.

[0012] As a further implementation method, before the conveyor belt moves, use the camera to take a picture of the calibration board to obtain the calibration board image before movement; use the solvePnP method to calculate the relative position relationship between the conveyor belt and the camera, and obtain the vertical distance between the conveyor belt plane and the camera optical center;

[0013] After the conveyor belt moves a set distance, use the camera to take a picture of the calibration board to obtain the calibration board image after movement; extract the corner points of the calibration board images before and after the conveyor belt moves respectively, and obtain the angle formed by the connection line of the corner points and the horizontal axis of the image in the picture, which is the motion blur angle.

[0014] As a further implementation method, the method for converting the moving distance of the conveyor belt into the motion blur length is:

[0015] Decompose the moving distance s of the conveyor belt into the displacement in the x-axis direction and the displacement in the y-axis direction through the motion blur angle θ, and the corresponding image shift distances on the image are decomposed into the displacement in the x-axis direction and the displacement in the y-axis direction , and the motion blur length L is calculated by .

[0016] As a further implementation method, the displacement of the conveyor belt during the exposure time is calculated by , ;

[0017] The corresponding image shift amount in the picture is calculated by , ;

[0018] Among them, the parameters and are obtained from the camera internal parameter matrix, is the vertical distance between the conveyor belt plane and the optical center of the camera.

[0019] As a further implementation, the motion blur kernel function is expressed as:

[0020] .

[0021] As a further implementation, the Zhang-Zhengyou calibration method is used to calibrate the internal parameters of the camera.

[0022] In a second aspect, an embodiment of the present invention further provides a motion blur kernel measurement system for a material vision detection system, including a conveyor belt, a camera is provided above the conveyor belt, and the imaging plane of the camera is parallel to the conveyor plane of the conveyor belt; a speed measurement encoder is installed on the conveyor belt, and the speed measurement encoder is used to collect the real-time speed of the conveyor belt;

[0023] The camera, the conveyor belt, and the speed measurement encoder are connected to a controller, and the controller is configured to execute the motion blur kernel measurement method of the material vision detection system.

[0024] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device includes a memory and a processor, a computer program is stored in the memory, and when the computer program is executed by the processor, the motion blur kernel measurement method of the material vision detection system is implemented.

[0025] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, and when the computer program is executed by one or more processors, the motion blur kernel measurement method of the material vision detection system is implemented.

[0026] The beneficial effects of the present invention are as follows:

[0027] (1) The present invention uses a speed measurement encoder or other speed measurement sensors to measure the speed in real time. On the one hand, the blur length can still be solved in real time during variable speed. On the other hand, it is stable enough and will not be affected by external light, noise, etc. like the pure image processing method, improving the recognition accuracy of the blur kernel and ensuring the stability of the image restoration quality.

[0028] (2) Through the imaging relationship between the camera and the conveyor belt plane obtained by calibration and the real-time speed measurement value of the speed measurement encoder, high-precision blur kernel parameters can be obtained; the blur kernel parameters are measured in real time, and the blur length is calculated in real time, which can meet the measurement requirements of blur kernel parameters in the conveyor belt variable speed scenario; no image processing parameters or visual target objects other than the calibration board need to be set for blur kernel measurement, and the deployment is simple; the blur kernel calculation is simple and time-consuming is short, which can meet the real-time requirements of visual motion blur restoration in the conveyor belt high-speed transmission scenario. Description of the Drawings

[0029] The accompanying drawings forming a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0030] Figure 1 is a schematic structural diagram of a measurement system according to one or more embodiments of the present invention;

[0031] Figure 2 is a flowchart of a measurement method according to one or more embodiments of the present invention;

[0032] Figure 3 is a comparison diagram of the restoration of concrete blocks on a conveyor belt according to the measurement method of one or more embodiments of the present invention. Among them, (a) is a blurred image of the moving concrete block, (b) is a grayscale image of the restored blurred motion of the concrete block, and (c) is a color image of the restored blurred motion of the concrete block.

[0033] Among them, 1. Conveyor belt, 2. Camera, 3. Tachometric encoder. Detailed implementation manners

[0034] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanations of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0035] Example 1:

[0036] In a typical embodiment of the present invention, as Figure 1 and Figure 2 shown, a method for measuring the motion blur kernel of a material vision detection system is given.

[0037] Currently, the method for measuring the motion blur kernel usually assumes that the conveyor belt 1 is calibrated in advance under the condition of uniform speed. When the speed of the conveyor belt 1 changes, it needs to be recalibrated, or a pure image processing method (parameters need to be adjusted according to the actual image) is used, which is unstable when the external light environment changes.

[0038] Based on this, this embodiment provides a method for measuring the motion blur kernel of a material vision detection system. According to the corner points of the calibration plate before and after moving with the conveyor belt 1, the motion blur angle is calculated; based on the real-time speed of the conveyor belt 1, the moving distance of the conveyor belt 1 within the exposure time of the camera 2 is calculated; according to the internal parameters of the camera 2, the motion blur length corresponding to the moving distance of the conveyor belt 1 on the image is calculated; according to the motion blur angle and the motion blur length, a motion blur kernel function is constructed.

[0039] Specifically, the motion blur kernel measurement method of this embodiment includes the following steps:

[0040] S1: Calibrate the internal parameters K of camera 2 using the Zhang-Zhengyou calibration method:

[0041] Among them, ;

[0042] f x and f y are the focal lengths along the X-axis and Y-axis of the image respectively, u 0 and v 0 are the principal point coordinates of the image respectively.

[0043] S2: Place the calibration board on conveyor belt 1, take pictures of the calibration board before and after the movement of conveyor belt 1, obtain the motion blur angle parameter of the psf blur kernel, and use the solvePnP method to obtain the relative position relationship between camera 2 and conveyor belt 1.

[0044] S21: Install camera 2 above conveyor belt 1 to ensure that the imaging plane of camera 2 is parallel to conveyor belt 1.

[0045] S22: Install a speed measurement encoder 3 with a pulley on conveyor belt 1 to measure the speed of conveyor belt 1 in real time.

[0046] S23: Place the calibration board on conveyor belt 1 and take pictures with camera 2. Use the solvePnP method to calculate the relative position relationship between conveyor belt 1 and camera 2, and then obtain the vertical distance between the plane of conveyor belt 1 and the optical center of camera 2 .

[0047] S24: Start conveyor belt 1, move the calibration board on conveyor belt 1 for a certain distance and then stop conveyor belt 1, and then take pictures with camera 2 to obtain the calibration board pictures. Extract the corner points of the calibration board pictures obtained before and after the movement of conveyor belt 1 respectively, and obtain the angle formed by the connection line of the corner points and the horizontal axis of the picture in the picture, that is, the transmission direction angle of conveyor belt 1, which is also the motion blur angle θ.

[0048] S3: Use the speed measurement encoder 3 to measure the speed in real time, combine the exposure time set by camera 2, and the internal parameters of camera 2 calibrated in step S1 to calculate the motion blur length parameter of the psf blur kernel.

[0049] S31: Install the speed measurement encoder 3 on conveyor belt 1 to measure the speed in real time and obtain the real-time speed v of conveyor belt 1.

[0050] S32: According to the set exposure time t of camera 2, obtain the moving distance s = vt of conveyor belt 1 during the exposure time of camera 2.

[0051] S33: Calculate the image shift distance L corresponding to the moving distance s of the conveyor belt 1 on the image based on the internal parameters of camera 2 obtained in step S1, which is the motion blur length L.

[0052] S34: Obtain the motion blur kernel function based on the motion blur angle θ calculated in step S24 and the motion blur length L calculated in S33 .

[0053] In step S33, convert the moving distance s of the conveyor belt 1 within the exposure time of camera 2 obtained in step S32 into the motion blur length L corresponding to the motion of the conveyor belt 1 in the picture taken by camera 2, as follows:

[0054] Decompose the moving distance s into the displacement in the x-axis direction (horizontal direction of the pixel coordinate system) through the motion blur angle θ and the displacement in the y-axis direction (vertical direction of the pixel coordinate system) , and the corresponding image shift distances on the picture are decomposed into the displacement in the x-axis direction and the displacement in the y-axis direction , and finally obtain the motion blur length L.

[0055] Among them, the displacement of the conveyor belt 1 within the exposure time and are obtained from . Calculate and obtain.

[0056] The corresponding image shift amounts in the photo and are obtained from and Calculate and obtain.

[0057] Finally, the motion blur length L is obtained from Calculate and obtain.

[0058] The parameters and in the above formula are obtained from the internal parameters K of camera 2 calibrated in step S1, is obtained from step S23.

[0059] In this embodiment, there is no need to process the real-time image. By using the pre-calibration method and the imaging relationship between camera 2 and the plane of conveyor belt 1, the real-time motion blur kernel parameters of conveyor belt 1 after speed change can be obtained; in this embodiment, a speed measurement encoder 3 or other speed measurement sensors are used for real-time speed measurement. On the one hand, the blur length can still be solved in real time during speed change, and on the other hand, it is stable enough and will not be affected by external light, noise, etc. like the pure image processing method.

[0060] Such as Figure 3As shown in the figure, in this embodiment, a motion-blurred kernel image is obtained by placing concrete blocks on the conveyor belt 1 and then restored. It can be seen that the measurement method of this embodiment can meet the real-time requirements of visual motion blur restoration in the scenario of high-speed conveyor belt transmission, improve the recognition accuracy of the blur kernel, and ensure the stability of image restoration quality.

[0061] Embodiment 2:

[0062] This embodiment provides a motion blur kernel measurement system for a material visual inspection system. As Figure 1 shown, it includes a conveyor belt 1, a camera 2 is provided above the conveyor belt 1, and the imaging plane of the camera 2 is parallel to the conveying plane of the conveyor belt 1; a speed measurement encoder 3 is installed on the conveyor belt 1, and the speed measurement encoder 3 is used to collect the real-time speed of the conveyor belt 1; the camera 2, the conveyor belt 1 and the speed measurement encoder 3 are connected to a controller, and the controller is configured to execute the motion blur kernel measurement method of the material visual inspection system described in Embodiment 1.

[0063] Embodiment 3:

[0064] This embodiment provides an electronic device, which includes a memory and a processor. When a computer program stored in the memory is executed by the processor, the motion blur kernel measurement method of the material visual inspection system described in Embodiment 1 is implemented.

[0065] Embodiment 4:

[0066] This embodiment provides a computer-readable storage medium, when a computer program is executed by one or more processors, the motion blur kernel measurement method of the material visual inspection system described in Embodiment 1 is implemented.

[0067] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant software through a computer program. The program can be stored in a non-volatile computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, an optical disk, a read-only memory, etc.

[0068] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for measuring motion blur kernel of a material visual inspection system, characterized in that: include: Calculate the motion blur angle based on the corner points of the images before and after the calibration plate moves with the conveyor belt; Based on the real-time speed of the conveyor belt, calculate the conveyor belt moving distance within the camera exposure time; According to the camera internal parameters, the conveyor belt moving distance corresponds to the motion blur length on the image. Construct a motion blur kernel function according to the motion blur angle and motion blur length; The calibration plate images are collected before and after the conveyor belt moves to obtain the motion blur angle parameters of the psf blur kernel; and the solvePnP method is used to obtain the relative position relationship between the camera and the conveyor belt; The pre-calibration method is used to obtain the real-time motion blur kernel parameters of the conveyor belt after speed change by using the imaging relationship between the camera and the conveyor belt plane. After the conveyor belt moves a set distance, a camera is used to take a picture of the calibration plate to obtain a picture of the calibration plate after movement; the corner points of the calibration plate pictures before and after the conveyor belt moves are extracted respectively, and the angle formed by the line connecting the corner points in the picture and the horizontal axis of the picture is obtained, which is the motion blur angle; Use a speed encoder to measure speed in real time, and calculate the motion blur length parameter of the PSF blur kernel by combining the exposure time set by the camera and the camera internal parameters obtained by calibration; install the speed encoder on the conveyor belt to measure speed in real time and obtain the real-time speed v of the conveyor belt; according to the set camera exposure time t, calculate the moving distance s=vt of the conveyor belt within the camera exposure time; According to the camera internal parameters, the conveyor belt moving distance s is calculated to correspond to the image displacement distance L on the image, which is the motion blur length L; according to the motion blur angle θ and the motion blur length L, the motion blur kernel function is obtained .

2. The method for measuring motion blur kernel of a material visual inspection system according to claim 1, characterized in that: Before the conveyor belt moves, a camera is used to take a picture of the calibration plate to obtain the image of the calibration plate before movement; the solvePnP method is used to calculate the relative position relationship between the conveyor belt and the camera, and the vertical distance between the conveyor belt plane and the camera optical center is obtained.

3. The method for measuring motion blur kernel of a material visual inspection system according to claim 1, characterized in that: The method of converting the conveyor belt moving distance into motion blur length is: The conveyor belt moving distance s is decomposed into the displacement in the x-axis direction by the motion blur angle θ and the displacement in the y-axis direction , the corresponding image shift distance on the image is decomposed into the displacement in the x-axis direction and y-axis displacement , the motion blur length L is given by Calculated.

4. The method for measuring motion blur kernel of a material visual inspection system according to claim 3 is characterized in that: The displacement of the conveyor belt during the exposure time is given by , Calculated; The image shift in the corresponding image is given by , Calculated; Among them, the parameters and Obtained from the camera intrinsic parameter matrix, is the vertical distance between the conveyor belt plane and the camera optical center.

5. The method for measuring motion blur kernel of a material visual inspection system according to claim 1 or 4, characterized in that: The Zhang Zhengyou calibration method is used to calibrate the camera intrinsic parameters.

6. A motion blur kernel measurement system for a material visual inspection system, characterized in that: It includes a conveyor belt, a camera is arranged above the conveyor belt, and the imaging plane of the camera is parallel to the conveying plane of the conveyor belt; a speed encoder is installed on the conveyor belt, and the speed encoder is used to collect the real-time speed of the conveyor belt; The camera, conveyor belt and speed encoder are connected to a controller, and the controller is configured to execute the motion blur kernel measurement method of a material vision inspection system as described in any one of claims 1-5.

7. An electronic device, characterized in that: The electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the motion blur kernel measurement method of the material vision inspection system as described in any one of claims 1-5 is implemented.

8. A computer-readable storage medium, characterized in that: When the computer program is executed by one or more processors, the motion blur kernel measurement method of the material vision inspection system as described in any one of claims 1 to 5 is implemented.

Images