Method and system for measuring motion blurring kernel of material visual inspection system

Abstract

The invention discloses a material visual inspection system motion blurring kernel measurement method and system, and relates to the technical field of industrial detection, and the method comprises the steps: calculating a motion blurring angle according to angular points of a picture before and after a calibration plate moves along with a conveyor belt; calculating the moving distance of the conveyor belt within the exposure time of the camera based on the real-time speed of the conveyor belt; according to the camera internal reference, calculating the motion blur length on the image corresponding to the moving distance of the conveyor belt; and constructing a motion blur kernel function according to the motion blur angle and the motion blur length. According to the method, the speed measurement encoder or other speed measurement sensors are adopted for real-time speed measurement, on one hand, the fuzzy length can still be solved in real time after speed change, on the other hand, stability is enough, the fuzzy kernel recognition precision can be improved, and it is guaranteed that the image restoration quality is stable.

Description

Technical Field

[0001] The present invention relates to the field of industrial detection technology, and in particular to a motion blur kernel measurement method and system for a material visual detection system. Background Art

[0002] When the camera is imaging the moving objects on the conveyor belt (during exposure), there is relative motion between the camera and the object, which makes the captured image blurred. This blur is called motion blur. In the process of detecting objects in transit on the conveyor belt, the acquired images often show motion blur, which affects the accuracy of visual detection. For the restoration of motion blurred images, one method is to use deep learning to restore the degraded images. The images obtained by this restoration method can often achieve higher clarity and better restoration effects. However, more data sets are required for training, and the effect is usually poor for unfamiliar scenes outside the data set. In addition, the deep learning method has high restoration performance requirements and takes a long time, which makes it difficult to meet the deployment requirements of high-speed real-time detection of objects on conveyor belts in industry.

[0003] The other is to adopt the traditional method, which regards motion blur as the convolution of the original image and the motion blur kernel, estimates the motion blur kernel, and then restores the image by inverse filtering, Wiener filtering, etc. based on the obtained blur kernel. This method runs fast and has a wide range of applications, and can meet the needs of high-speed real-time detection of diverse scenes in industry. However, the acquisition of the motion blur kernel is the key to this method, and the accuracy of the motion blur kernel estimation directly affects the effect of motion blur restoration. Usually, the motion blur kernel is obtained by image processing methods, and the image is converted to the frequency domain for motion blur kernel identification. However, this motion blur kernel identification method relies on a relatively fixed imaging environment. When the environment changes, the image processing parameters need to be readjusted, otherwise the motion blur kernel identification accuracy will be low and the quality of the restored image will be unstable. This method is difficult to deploy, and the restoration quality is unstable, which cannot meet the actual industrial deployment needs. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a motion blur kernel measurement method and system for a material vision inspection system, which adopts a speed encoder or other speed sensor to measure the speed in real time. On the one hand, the blur length can be solved in real time when the speed changes. On the other hand, it is stable enough to improve the recognition accuracy of the blur kernel and ensure the stability of image restoration quality.

[0005] In order to achieve the above object, the present invention is implemented through the following technical solutions: In a first aspect, an embodiment of the present invention provides a method for measuring motion blur kernel of a material visual inspection system, comprising: Calculate the motion blur angle based on the corner points of the images before and after the calibration plate moves with the conveyor belt; Based on the real-time speed of the conveyor belt, calculate the conveyor belt moving distance within the camera exposure time; According to the camera internal parameters, the conveyor belt moving distance corresponds to the motion blur length on the image. Construct a motion blur kernel function based on the motion blur angle and motion blur length. As a further implementation method, calibration plate images are collected before and after the conveyor belt moves to obtain the motion blur angle parameters of the psf blur kernel; and the solvePnP method is used to obtain the relative position relationship between the camera and the conveyor belt.

[0006] As a further implementation method, before the conveyor belt moves, a camera is used to take a picture of the calibration plate to obtain a picture of the calibration plate before the movement; the solvePnP method is used to calculate the relative position relationship between the conveyor belt and the camera, and the vertical distance between the conveyor belt plane and the camera optical center is obtained; After the conveyor belt moves a set distance, a camera is used to take a picture of the calibration plate to obtain an image of the calibration plate after movement; the corner points of the calibration plate images before and after the conveyor belt moves are extracted respectively, and the angle between the corner point connection line and the horizontal axis of the image in the image is obtained, which is the motion blur angle.

[0007] As a further implementation, the method of converting the conveyor belt moving distance into the motion blur length is: The conveyor belt moving distance s is decomposed into the displacement in the x-axis direction by the motion blur angle θ and the displacement in the y-axis direction , the corresponding image shift distance on the image is decomposed into the displacement in the x-axis direction and y-axis displacement , the motion blur length L is given by Calculated.

[0008] As a further implementation, the displacement of the conveyor belt during the exposure time is given by , Calculated; The image shift in the corresponding image is given by , Calculated; Among them, the parameters and Obtained from the camera intrinsic parameter matrix, is the vertical distance between the conveyor belt plane and the camera optical center.

[0009] As a further implementation, the motion blur kernel function is expressed as: .

[0010] As a further implementation method, the Zhang Zhengyou calibration method is used to calibrate the camera intrinsic parameters.

[0011] In a second aspect, an embodiment of the present invention further provides a motion blur kernel measurement system for a material visual inspection system, comprising a conveyor belt, a camera is provided above the conveyor belt, and the imaging plane of the camera is parallel to the conveying plane of the conveyor belt; a speed encoder is installed on the conveyor belt, and the speed encoder is used to collect the real-time speed of the conveyor belt; The camera, conveyor belt and speed encoder are connected to a controller, and the controller is configured to execute the motion blur kernel measurement method of the material vision inspection system.

[0012] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the motion blur kernel measurement method of the material vision inspection system is implemented.

[0013] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which, when a computer program is executed by one or more processors, implements the motion blur kernel measurement method of the material vision inspection system.

[0014] The beneficial effects of the present invention are as follows: (1) The present invention uses a speed encoder or other speed sensors to measure speed in real time. On the one hand, the blur length can be solved in real time when the speed changes. On the other hand, it is stable enough and will not be affected by external light, noise, etc. like pure image processing methods. It improves the recognition accuracy of the blur kernel and ensures the stability of image restoration quality.

[0015] (2) The present invention can obtain high-precision blur kernel parameters by calibrating the imaging relationship between the camera and the conveyor belt plane and the real-time speed measurement value of the speed encoder; the blur kernel parameters are measured in real time, and the blur length is calculated in real time, which can meet the measurement requirements of blur kernel parameters in the conveyor belt speed change scenario; blur kernel measurement does not require setting image processing parameters or setting visual targets outside the calibration plate, and the deployment is simple; the blur kernel calculation is simple and time-saving, which can meet the real-time requirements of visual motion blur restoration in the conveyor belt high-speed transmission scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0017] Figure 1 is a schematic diagram of the structure of a measurement system according to one or more embodiments of the present invention; Figure 2is a flow chart of a measurement method according to one or more embodiments of the present invention; Figure 3 It is a comparison diagram of the restoration of concrete blocks on a conveyor belt according to one or more embodiments of the measurement method of the present invention, wherein (a) is a motion blurred image of the concrete block, (b) is a grayscale image of the restored motion blurred concrete block, and (c) is a color image of the restored motion blurred concrete block.

[0018] Among them, 1. conveyor belt, 2. camera, 3. speed encoder. DETAILED DESCRIPTION It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0019] Embodiment 1: In a typical embodiment of the present invention, Figure 1 and Figure 2 As shown, a motion blur kernel measurement method for a material vision inspection system is given.

[0020] The current motion blur kernel measurement method is usually pre-calibrated under the assumption that the conveyor belt 1 is at a constant speed. When the speed of the conveyor belt 1 changes, it needs to be recalibrated, or a pure image processing method is used (parameters need to be adjusted according to the actual image), which is unstable when the external lighting environment changes.

[0021] Based on this, this embodiment provides a motion blur kernel measurement method for a material vision inspection system, which calculates the motion blur angle according to the corner points of the pictures of the calibration plate before and after the conveyor belt 1 moves; calculates the moving distance of the conveyor belt 1 within the exposure time of the camera 2 based on the real-time speed of the conveyor belt 1; calculates the motion blur length on the image corresponding to the moving distance of the conveyor belt 1 based on the internal parameters of the camera 2; and constructs a motion blur kernel function based on the motion blur angle and the motion blur length.

[0022] Specifically, the motion blur kernel measurement method of this embodiment includes the following steps: S1: Use Zhang Zhengyou calibration method to calibrate the intrinsic parameter K of camera 2: in, ; f x and f y are the focal lengths along the X and Y axes of the image, respectively, u 0 and v 0 are the principal point coordinates of the image.

[0023] S2: Place the calibration plate on the conveyor belt 1, take pictures of the calibration plate before and after the conveyor belt 1 moves, obtain the motion blur angle parameters of the psf blur kernel, and use the solvePnP method to obtain the relative position relationship between the camera 2 and the conveyor belt 1.

[0024] S21: Place camera 2 above conveyor belt 1 to ensure that the imaging plane of camera 2 is parallel to conveyor belt 1.

[0025] S22: A speed encoder 3 including a pulley is installed on the conveyor belt 1 to measure the speed of the conveyor belt 1 in real time.

[0026] S23: Place the calibration plate on the conveyor belt 1 and take a photo with the camera 2. Use the solvePnP method to calculate the relative position relationship between the conveyor belt 1 and the camera 2, and then obtain the vertical distance between the plane of the conveyor belt 1 and the optical center of the camera 2. .

[0027] S24: Start the conveyor belt 1, move the calibration plate on the conveyor belt 1 for a certain distance, then stop the conveyor belt 1, and then take a picture with the camera 2 to obtain the calibration plate picture. Extract the corner points of the calibration plate pictures obtained before and after the conveyor belt 1 moves, respectively, and obtain the angle between the corner point connection line and the horizontal axis of the picture, that is, the transmission direction angle of the conveyor belt 1, that is, the motion blur angle θ.

[0028] S3: Use the speed encoder 3 to perform real-time speed measurement, and combine the exposure time set by the camera 2 and the internal parameters of the camera 2 calibrated in step S1 to calculate the motion blur length parameter of the psf blur kernel.

[0029] S31: Install the speed encoder 3 on the conveyor belt 1 to perform real-time speed measurement to obtain the real-time speed v of the conveyor belt 1.

[0030] S32: According to the set exposure time t of camera 2, the moving distance s=vt of conveyor belt 1 within the exposure time of camera 2 is calculated.

[0031] S33: Based on the internal parameters of the camera 2 obtained in step S1, it is calculated that the moving distance s of the conveyor belt 1 corresponds to the image movement distance L on the image, that is, the motion blur length L.

[0032] S34: According to the motion blur angle θ calculated in step S24 and the motion blur length L calculated in step S33, the motion blur kernel function is obtained. .

[0033] In step S33, the moving distance s of the conveyor belt 1 during the exposure time of the camera 2 obtained in step S32 is converted into the motion blur length L of the conveyor belt 1 corresponding to the picture taken by the camera 2, as follows: The moving distance s is decomposed into the displacement in the x-axis direction (horizontal direction of the pixel coordinate system) by the motion blur angle θ. And the displacement in the y-axis direction (vertical direction of the pixel coordinate system) , the corresponding image shift distance on the image is decomposed into the displacement in the x-axis direction and y-axis displacement , and finally the motion blur length L is obtained.

[0034] The displacement of conveyor belt 1 during the exposure time and Depend on , Calculated.

[0035] The image shift in the corresponding photo and Depend on and Calculated.

[0036] Finally, the motion blur length L is given by Calculated.

[0037] The parameters in the above formula are and The intrinsic parameter K of camera 2 obtained by calibration in step S1 is obtained. Obtained from step S23.

[0038] This embodiment does not need to process the real-time image. A pre-calibrated method is used to utilize the imaging relationship between the camera 2 and the plane of the conveyor belt 1 to obtain the real-time motion blur kernel parameters of the conveyor belt 1 after the speed change. This embodiment uses a speed encoder 3 or other speed sensors to measure the speed in real time. On the one hand, the blur length can be solved in real time when the speed changes. On the other hand, it is stable enough and will not be affected by external light, noise, etc. like the pure image processing method.

[0039] like Figure 3 As shown, in this embodiment, a motion blur kernel image is obtained by placing a concrete block on the conveyor belt 1 and then restored. It can be seen that the measurement method of this embodiment can meet the real-time requirements of visual motion blur restoration in the high-speed transmission scenario of the conveyor belt, can improve the recognition accuracy of the blur kernel, and ensure the stability of the image restoration quality.

[0040] Embodiment 2: This embodiment provides a motion blur kernel measurement system for a material visual inspection system, such as Figure 1As shown, it includes a conveyor belt 1, a camera 2 is arranged above the conveyor belt 1, and the imaging plane of the camera 2 is parallel to the conveying plane of the conveyor belt 1; the conveyor belt 1 is installed with a speed encoder 3, and the speed encoder 3 is used to collect the real-time speed of the conveyor belt 1; the camera 2, the conveyor belt 1 and the speed encoder 3 are connected to a controller, and the controller is configured to execute the motion blur kernel measurement method of the material vision inspection system described in Example 1.

[0041] Embodiment 3: This embodiment provides an electronic device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the motion blur kernel measurement method of the material vision inspection system described in Example 1 is implemented.

[0042] Embodiment 4: This embodiment provides a computer-readable storage medium, which, when the computer program is executed by one or more processors, implements the motion blur kernel measurement method of the material vision inspection system described in Embodiment 1.

[0043] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related software through a computer program. The program can be stored in a non-volatile computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, etc.

[0044] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for measuring motion blur kernel of a material visual inspection system, characterized in that: include: Calculate the motion blur angle based on the corner points of the images before and after the calibration plate moves with the conveyor belt; Based on the real-time speed of the conveyor belt, calculate the conveyor belt moving distance within the camera exposure time; According to the camera internal parameters, the conveyor belt moving distance corresponds to the motion blur length on the image. Construct a motion blur kernel function based on the motion blur angle and motion blur length.

2. The method for measuring motion blur kernel of a material visual inspection system according to claim 1, characterized in that: Calibration plate images are collected before and after the conveyor belt moves to obtain the motion blur angle parameters of the PSF blur kernel; and the solvePnP method is used to obtain the relative position relationship between the camera and the conveyor belt.

3. The method for measuring motion blur kernel of a material visual inspection system according to claim 2, characterized in that: Before the conveyor belt moves, a camera is used to take a picture of the calibration plate to obtain the calibration plate image before movement; the solvePnP method is used to calculate the relative position relationship between the conveyor belt and the camera, and the vertical distance between the conveyor belt plane and the camera optical center is obtained; After the conveyor belt moves a set distance, a camera is used to take a picture of the calibration plate to obtain an image of the calibration plate after movement; the corner points of the calibration plate images before and after the conveyor belt moves are extracted respectively, and the angle between the corner point connection line and the horizontal axis of the image in the image is obtained, which is the motion blur angle.

4. The method for measuring motion blur kernel of a material visual inspection system according to claim 1, characterized in that: The method of converting the conveyor belt moving distance into motion blur length is: The conveyor belt moving distance s is decomposed into the displacement in the x-axis direction by the motion blur angle θ and the displacement in the y-axis direction , the corresponding image shift distance on the image is decomposed into the displacement in the x-axis direction and y-axis displacement , the motion blur length L is given by Calculated.

5. The method for measuring motion blur kernel of a material visual inspection system according to claim 4, characterized in that: The displacement of the conveyor belt during the exposure time is given by , Calculated; The image shift in the corresponding image is given by , Calculated; Among them, the parameters and Obtained from the camera intrinsic parameter matrix, is the vertical distance between the conveyor belt plane and the camera optical center.

6. A method for measuring motion blur kernel of a material visual inspection system according to claim 4 or 5, characterized in that: The motion blur kernel function is expressed as: 。 7. The method for measuring motion blur kernel of a material visual inspection system according to claim 1 or 5, characterized in that: The Zhang Zhengyou calibration method is used to calibrate the camera intrinsic parameters.

8. A motion blur kernel measurement system for a material visual inspection system, characterized in that: It includes a conveyor belt, a camera is arranged above the conveyor belt, and the imaging plane of the camera is parallel to the conveying plane of the conveyor belt; a speed encoder is installed on the conveyor belt, and the speed encoder is used to collect the real-time speed of the conveyor belt; The camera, conveyor belt and speed encoder are connected to a controller, and the controller is configured to execute the motion blur kernel measurement method of a material vision inspection system as described in any one of claims 1-7.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the motion blur kernel measurement method of the material vision inspection system as described in any one of claims 1-7 is implemented.

10. A computer-readable storage medium, characterized in that: When the computer program is executed by one or more processors, the motion blur kernel measurement method of the material vision inspection system as described in any one of claims 1 to 7 is implemented.

Images