A misregistration imaging joint calculation detection method and detection system

Abstract

The application discloses a misregistration imaging combined calculation detection method and a detection system, and is especially suitable for small field depth incompatible micro workpiece detection. A vertical camera is used to obtain a first projection image of a multi-level three-dimensional workpiece, the posture of the multi-level three-dimensional workpiece is adjusted, a second projection image is obtained by shooting again, a current center coordinate is calculated according to the second projection image, and an offset value is calculated according to the difference between a reference center coordinate and the current center coordinate. Then, a servo point of a horizontal camera is calculated, a first image with a clear upper end and a second image with a clear lower part are shot, an upper end edge is extracted, and a coordinate of an edge curve is recorded. A lower end edge is extracted in the second image, and the coordinate of the lower end edge is affinely transformed into the first image to form a misregistration imaging combined result. Whether the workpiece is depth compatible or not, the application can obtain a high-precision detection result, and solves the detection demand of small field and small depth cross verification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial detection, and in particular to a misregistration imaging combined calculation detection method for incompatible small field depth of view. BACKGROUND

[0002] In industrial product detection, when a small part is photographed, a small field of view camera needs to be used at a close distance. However, when the small part to be detected is a multi-level three-dimensional product, for example, a complex structure including an upper top surface, a lower surface having two feet, and each foot being composed of a series of cylindrical bodies of different sizes, the product to be detected is a multi-level three-dimensional product, and the distance from the upper top surface to the lower top surface or the distance from the bottom corner to the upper top surface needs to be detected. At this time, the sizes of two different levels of planes need to be cross-calculated. However, in the small field of view camera, when the upper top surface is in focus, the lower top surface will be displayed as blurred, and when the lower top surface is in focus, the upper top surface will be displayed as blurred in the field of view. Alternatively, the depth of field can be compatible when the current part is detected, but after the part to be detected is replaced, the small field of view of the equipment and the small depth of field of the matching center lens will appear incompatible, and the detection accuracy of such small parts often needs to be controlled within 2 um. Therefore, cross-calculation verification will fail. SUMMARY

[0003] Therefore, the purpose of the present application is to provide a misregistration imaging combined calculation detection method and a detection system, especially for the case when the small field of view depth of field is incompatible and cross-calculation verification cannot be performed. The present application adopts the idea of secondary imaging and misregistration merging to achieve the detection requirement and solve the problem of cross-verification calculation measurement result of small field of view and small depth of field.

[0004] In order to achieve the above-mentioned purpose, the present application provides a misregistration imaging combined calculation detection method, which comprises the following steps:

[0005] S1, a vertical camera and a movable horizontal camera are arranged in the horizontal direction and directly below the detection position of the multi-level three-dimensional workpiece to be detected;

[0006] S2, the first projection image of the multi-level three-dimensional workpiece is obtained by using the vertical camera, the posture of the multi-level three-dimensional workpiece is adjusted, the second projection image is obtained by taking a photograph again, the current center coordinate is calculated according to the second projection image, and the offset value t is calculated according to the difference between the reference center coordinate and the current center coordinate;

[0007] S3, when the horizontal camera focuses on the upper edge of the multi-level three-dimensional workpiece and displays clearly, the position of the current horizontal camera is recorded as the reference position CamJZ, and when the horizontal camera focuses on the lower edge of the multi-level three-dimensional workpiece and displays clearly, the moving distance D is recorded;

[0008] S4, calculating a first servo point CamXZ of the upper end of the horizontal camera according to the offset value t and the reference position CamJZ and S2; calculating a second servo point CamXY of the lower end of the camera according to the first servo point CamXZ and the moving distance D;

[0009] S5, obtaining a first image of the horizontal camera at the first servo point CamXZ and a second image at the second servo point CamXY; extracting the upper end edge in the first image and recording the coordinates of the edge curve, and extracting the lower end edge in the second image, and performing affine transformation on the coordinates of the lower end edge to the first image to form a misregistration imaging joint result.

[0010] Further, in S2, the reference point center coordinates are obtained by using a debugging process before detection, and the current center coordinates are obtained in the actual detection process, and the offset value t is calculated according to the difference between the reference center coordinates and the current center coordinates.

[0011] Further, the reference point center coordinates obtained by using the debugging process before detection include:

[0012] A first vertical mapping image of the multi-level three-dimensional workpiece is obtained by using the vertical camera, the posture of the multi-level three-dimensional workpiece is adjusted, and then the multi-level three-dimensional workpiece is photographed again; wherein the posture of the multi-level three-dimensional workpiece is adjusted by: calculating the deflection angle of the multi-level three-dimensional workpiece according to the first projection image of the multi-level three-dimensional workpiece, adjusting the multi-level three-dimensional workpiece by using a posture correction device according to the deflection angle, rotating the multi-level three-dimensional workpiece to an angle parallel to the optical center of the horizontal camera, and then photographing again to obtain a second vertical mapping image, and calculating the center coordinates (x0, y0) of the multi-level three-dimensional workpiece according to the second vertical mapping image as the reference center point coordinates.

[0013] Further, the current center coordinates obtained in the actual detection process include:

[0014] In the actual detection, a current first projection image of the multi-level three-dimensional workpiece is obtained by using the vertical camera, the center coordinates (x1, y1) of the multi-level three-dimensional workpiece being detected at this time are calculated according to the current second projection image obtained by the vertical camera being photographed for the second time, and the difference between (x1, y1) and (x0, y0) is taken as the offset value t.

[0015] Further, the center coordinates (x0, y0) of the multi-level three-dimensional workpiece include:

[0016] S201, extracting the edge of the first vertical mapping image, and extracting the horizontal center line L3 of the first vertical mapping image by using the edge of the first vertical mapping image;

[0017] S202, a deflection angle of the horizontal center line L3 is calculated, a standard horizontal line L4 is obtained by using the deflection angle for affine transformation, and a center point position of the standard horizontal line L4 is recorded as a center coordinate (x0, y0).

[0018] Further preferably, in S4, the offset value t obtained according to the reference position CamJZ and S2 is used to calculate a first servo point position CamXZ of the upper end of the horizontal camera focus, including the following calculation method:

[0019] CamXZ=CamJZ±t.

[0020] Further preferably, in S4, the moving distance D obtained according to the first servo point position CamXZ and S3 is used to calculate a second servo point position CamXY of the lower end of the camera focus, including the following calculation method:

[0021] CamXY=CamXZ±D.

[0022] The application also provides a misregistration imaging combined calculation detection system for implementing the misregistration imaging combined calculation detection method, including:

[0023] A vertical camera is located directly below the multi-level three-dimensional workpiece to be detected, and is used to shoot a first projection image of the multi-level three-dimensional workpiece and, after adjusting the posture of the multi-level three-dimensional workpiece, to shoot again to obtain a second projection image;

[0024] An image processing module is used to calculate a current center coordinate according to the second projection image, to calculate an offset value t according to the difference between the reference center coordinate and the current center coordinate, to record the position of the horizontal camera as a reference position CamJZ when the horizontal camera focuses on the upper edge of the multi-level three-dimensional workpiece to display clearly, to record a moving distance D when the horizontal camera is moved to the lower edge to display clearly, to calculate a first servo point position CamXZ of the upper end of the horizontal camera focus according to the obtained reference position CamJZ and the obtained offset value t, and to calculate a second servo point position CamXY of the lower end of the camera focus according to the first servo point position CamXZ and the obtained moving distance D.

[0025] A movable horizontal camera is used to focus on the multi-level three-dimensional workpiece, to shoot a first image at the first servo point position, and to shoot a second image at the second servo point position.

[0026] The image processing module further includes extracting the upper end edge in the first image and recording the coordinates of the edge curve, and extracting the lower end edge in the second image and performing affine transformation on the coordinates of the lower end edge to the first image to form a misregistration imaging combined result.

[0027] More preferably, a horizontal sliding module is installed below the horizontal camera. The horizontal sliding module is used to keep the object distance consistent when the horizontal camera moves forward and backward, with the focal length fixed, when capturing the first image and the second image.

[0028] The misalignment imaging joint calculation detection method and system disclosed in this application adopts a combination of vertical and horizontal cameras. Under the premise of ensuring the focusing state of the horizontal camera, the reference position of two shots is obtained. Then, the offset value of the workpiece is determined by combining the vertical camera, and the servo point is determined for shooting. The images of the two shots can be merged for misalignment. Thus, regardless of whether the workpiece is depth-of-field compatible, high-precision detection results can be obtained, solving the detection needs of cross-verification with small field of view and small depth of field. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating the misalignment imaging joint calculation detection method provided by the present invention.

[0030] Figure 2 This is a schematic diagram of the structure of the misalignment imaging joint calculation detection system provided by the present invention.

[0031] Figure 3 This is a schematic diagram of the image used in this invention to extract the horizontal center line from the projected image of the workpiece.

[0032] Figure 4 This is a flowchart illustrating the process of merging misaligned images of a multi-layered three-dimensional workpiece to be inspected. Detailed Implementation

[0033] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] like Figure 1 As shown, one embodiment of the present invention provides a method for joint calculation and detection of misalignment imaging, which includes the following steps:

[0035] S1. A vertical camera and a movable horizontal camera are respectively installed directly below the inspection station of the multi-layered three-dimensional workpiece to be inspected and in the horizontal direction; it should be noted that the positions of the horizontal and vertical cameras are as follows: Figure 2 As shown, the optical centers of the horizontal and vertical cameras are at a 90-degree angle to each other on the vertical plane where the multi-layered three-dimensional workpiece is located. Therefore, the vertical camera takes a projection view of the multi-layered three-dimensional workpiece, i.e., a bottom view, while the horizontal camera takes a side view.

[0036] S2. Use a vertical camera to obtain the first projection image of the multi-layered three-dimensional workpiece, adjust the posture of the multi-layered three-dimensional workpiece, take another picture to obtain the second projection image, calculate the current center coordinates based on the second projection image, and calculate the offset value t based on the difference between the reference center coordinates and the current center coordinates.

[0037] Specifically comprising:

[0038] During debugging, in S2, a first vertical mapping image of the multi-level three-dimensional workpiece is obtained by using the vertical camera, the posture of the multi-level three-dimensional workpiece is adjusted, and second shooting includes: calculating the deflection angle of the multi-level three-dimensional workpiece according to the first vertical mapping image of the multi-level three-dimensional workpiece, adjusting the multi-level three-dimensional workpiece by using the suction nozzle according to the deflection angle, rotating the multi-level three-dimensional workpiece to an angle parallel to the optical center of the horizontal camera, and then shooting again, and calculating the center coordinates (x0, y0) of the multi-level three-dimensional workpiece according to the obtained second vertical mapping image as the reference center point coordinates;

[0039] It should be noted that after obtaining the coordinates of the reference position, the angle is different and the center of mass position changes every time the product comes to the station for detection. After the product is turned, the current center of mass of the product will change from the center of mass coordinates of the reference position. Therefore, the center of mass coordinates of the current product obtained according to the second projection image is combined with the deviation result obtained by calculating the reference position to be applied to the horizontal camera moving distance.

[0040] During actual detection, according to the process of S2, the center coordinates (x1, y1) of the multi-level three-dimensional workpiece being detected at this time are calculated according to the image obtained by the vertical camera during the second shooting, and the difference between (x1, y1) and (x0, y0) is taken as the offset value t.

[0041] First, calculating the center coordinates (x0, y0) of the multi-level three-dimensional workpiece includes:

[0042] As shown in Figure 3 S201, the edges of the first vertical mapping image are extracted, and the horizontal center line L3 of the first vertical mapping image is extracted by using the edges of the first vertical mapping image. It should be noted that the midpoints P1 and P2 of the edge curves are extracted at the edge lines C1 and C2 of the first vertical mapping image, and the horizontal center line L3 is obtained according to the line connecting P1 and P2.

[0043] S202, calculating the deflection angle of the horizontal center line L3, using the deflection angle to obtain the standard horizontal line L4, and recording the center point position of the standard horizontal line L4 as the center coordinates (x0, y0).

[0044] It should be noted that the deflection angle of the horizontal center line L3 calculated by default is the deflection angle of the horizontal direction camera relative to the horizontal direction 0°. This ensures that the horizontal camera moves in the horizontal 0° direction. There will be no angle error in the horizontal direction.

[0045] Then, the center coordinates (x1, y1) at the actual detection time are calculated according to the steps S201 and S202 above;

[0046] Finally, the offset values of (x0, y0) and (x1, y1) are calculated. The offset value Δx in the x direction can be converted into the offset value t as follows. The offset value in the y direction can be used to drive the feeding mechanism of the nozzle to move forward and backward, or can not be used. The actual physical distance deviation value t is obtained by multiplying the single-phase precision by Δx. The position of the bottom servo mechanism of the horizontal camera at the current time is obtained by adding the offset value t to the reference position CamJZ of the horizontal camera servo mechanism recorded just now.

[0047] S3, when the horizontal camera focuses on the upper edge of the multi-level three-dimensional workpiece and displays clearly, record the position of the horizontal camera as the reference position CamJZ, drive the sliding module to move forward and backward to focus on the lower edge of the multi-level three-dimensional workpiece and display clearly, record the moving distance as D;

[0048] S4, according to the reference position CamJZ and the offset value t obtained in S2, calculate the first servo point CamXZ of the horizontal camera focusing on the upper end;

[0049] CamXZ = CamJZ ± t.

[0050] According to the first servo point CamXZ and the moving distance D, calculate the second servo point CamXY of the camera focusing on the lower end;

[0051] CamXY = CamXZ ± D.

[0052] It should be noted that when the reference position CamJZ of the bottom servo mechanism of the horizontal camera moves forward or backward by a distance D, the moving distance D is fixed, which is determined by the actual physical distance of the upper and lower edges of the multi-level three-dimensional workpiece. Therefore, the images captured by the camera when focusing clearly at different positions can be fused and calculated, which will not introduce new errors, and at the same time, the detection requirements can be realized.

[0053] S5, obtain the first image of the horizontal camera at the first servo point CamXZ and the second image at the second servo point CamXY; extract the upper edge in the first image and record the coordinates of the edge curve, extract the lower edge in the second image, and perform affine transformation on the coordinates of the lower edge to the first image to form a joint result of misregistration imaging. Using the joint result of misregistration imaging formed, the distance from a monitoring point on the upper top surface to the lower top surface and other cross calculations can be calculated in the same coordinate system, while meeting the detection accuracy requirements of small parts.

[0054] Wherein, the edge curve can be a straight line, a circular edge line, a straight line is two points to confirm a straight line, the coordinates are 2 coordinates, a total of 4 values, a circle is a center coordinate and a diameter, a total of 3 values, and the like.

[0055] The application also provides a misregistration imaging combined calculation detection system for implementing the misregistration imaging combined calculation detection method.

[0056] A vertical camera is located directly below the multi-level three-dimensional workpiece to be detected, and is used to shoot a first projection image of the multi-level three-dimensional workpiece and to shoot again after adjusting the posture of the multi-level three-dimensional workpiece to obtain a second projection image.

[0057] An image processing module is used to calculate a current center coordinate according to the second projection image, to calculate an offset value t according to the difference between the reference center coordinate and the current center coordinate, to record a reference position CamJZ of the horizontal camera when the horizontal camera focuses on the upper edge of the multi-level three-dimensional workpiece, to record a moving distance D when the horizontal camera is moved to focus on the lower edge, to calculate a first servo point CamXZ of the horizontal camera focusing on the upper end according to the reference position CamJZ and the offset value t, and to calculate a second servo point CamXY of the horizontal camera focusing on the lower end according to the first servo point CamXZ and the moving distance D.

[0058] A movable horizontal camera is used to focus on the multi-level three-dimensional workpiece, to shoot a first image at the first servo point, and to shoot a second image at the second servo point. Further preferably, a horizontal sliding module is installed below the horizontal camera, and the horizontal sliding module is used to keep the object distance consistent when shooting the first image and the second image under the condition that the focal length of the horizontal camera is fixed when the horizontal camera is moving forward and backward. The optical fiber is a sensor commonly used in industry, which is used to detect the presence or absence and the position. When the product comes into the field of view of the camera, the sensor senses the presence of the product, the servo mechanism for moving the product stops, and a signal is sent to indicate that the product has been positioned and the camera can be triggered to take a picture.

[0059] The image processing module further comprises extracting the upper end edge in the first image, recording the coordinates of the edge curve, extracting the lower end edge in the second image, and performing affine transformation on the coordinates of the lower end edge to the first image to form a misregistration imaging combined result. Figure 4 As shown in Figure 4 The left side is the first image, and the right side is the second image. First, the coordinates of the edge indicated by the yellow arrow on the upper left side are calculated. Figure 4 The edge L1 can be expressed in two-point form as Then, recalculate according to the above process Figure 4 The right side green arrow edge L2 (green arrow), the edge L2 is affine transformed to Figure 4 The left side image (red arrow) is recorded as L2', and after the transformation, it is merged with Figure 4 The left side yellow arrow edge L1 to obtain Figure 4 All the edges of the multi-level three-dimensional workpiece can be calculated subsequently.

[0060] The application further provides an electronic device, comprising a memory storing computer program instructions, and a processor, which implements the steps of the misregistration imaging joint calculation detection method when the computer program instructions are executed by the processor.

[0061] The application further provides a computer readable storage medium for storing instructions, which, when executed on a computer, cause the computer to perform the steps of the misregistration imaging joint calculation detection method.

[0062] Obviously, the above embodiments are only examples for clearly illustrating the application, and are not intended to limit the embodiments. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all the embodiments cannot be exhausted, and the obvious changes or variations derived therefrom are still within the protection scope of the application.

Claims

1. A misregistration imaging joint calculation detection method, characterized in that, The method comprises the following steps: S1, a vertical camera and a movable horizontal camera are arranged respectively below and horizontally to a detection position of a multi-level three-dimensional workpiece to be detected, a horizontal sliding module is arranged below the horizontal camera, and the horizontal sliding module is used to keep the object distance consistent when the horizontal camera shoots a first image and a second image in the case of fixed focal length when the horizontal camera is moving forward and backward; S2, a first projection image of the multi-level three-dimensional workpiece is obtained by using the vertical camera, the posture of the multi-level three-dimensional workpiece is adjusted, a second projection image is obtained by shooting again, a current center coordinate is calculated according to the second projection image, and an offset value t is calculated according to the difference between the reference center coordinate and the current center coordinate; The method further comprises obtaining the reference center coordinate by using a debugging process before detection, obtaining a current center coordinate in an actual detection process, and calculating the offset value t according to the difference between the reference center coordinate and the current center coordinate; The method of obtaining the reference center coordinate by using the debugging process before detection comprises: A first vertical mapping image of the multi-level three-dimensional workpiece is obtained by using the vertical camera, the posture of the multi-level three-dimensional workpiece is adjusted, a second vertical mapping image is obtained by shooting again, and the center coordinate (x0, y0) of the multi-level three-dimensional workpiece is calculated according to the second vertical mapping image, which is taken as the reference center coordinate; The method of adjusting the posture of the multi-level three-dimensional workpiece comprises: calculating a deflection angle of the multi-level three-dimensional workpiece according to the first projection image of the multi-level three-dimensional workpiece, and adjusting the multi-level three-dimensional workpiece by using a posture correction device according to the deflection angle, so that the multi-level three-dimensional workpiece is rotated to an angle parallel to the optical center of the horizontal camera; S3, when the horizontal camera focuses on the upper edge of the multi-level three-dimensional workpiece and displays clearly, the position of the horizontal camera is recorded as a reference position CamJZ, the horizontal sliding module is driven to move forward and backward, the lower edge of the multi-level three-dimensional workpiece is focused and displayed clearly, and the moving distance D is recorded; S4, according to the reference position CamJZ and the offset value t obtained in S2, a first servo point CamXZ of the horizontal camera focusing on the upper edge is calculated, and according to the first servo point CamXZ and the moving distance D, a second servo point CamXY of the camera focusing on the lower edge is calculated; S5, a first image of the horizontal camera at the first servo point CamXZ and a second image of the horizontal camera at the second servo point CamXY are obtained, the upper edge is extracted in the first image, the coordinates of the edge curve are recorded, the lower edge is extracted in the second image, and the coordinates of the lower edge are affine transformed into the first image to form a misregistration imaging joint result.

2. The misregistration imaging joint calculation detection method of claim 1, wherein, The method of obtaining the current center coordinate in the actual detection process comprises: In the actual detection, a first projection image of the multi-level three-dimensional workpiece is obtained by using the vertical camera, a center coordinate (x1, y1) of the multi-level three-dimensional workpiece being detected at this time is calculated according to a second projection image obtained by the vertical camera shooting for the second time, and the difference between (x1, y1) and (x0, y0) is taken as the offset value t.

3. The misregistration imaging joint calculation detection method of claim 1, wherein, In S4, the method of calculating the first servo point CamXZ of the horizontal camera focusing on the upper edge according to the reference position CamJZ and the offset value t obtained in S2 comprises the following calculation method: CamXZ=CamJZ±t.

4. The misregistration imaging joint calculation detection method of claim 1, wherein, In S4, the second servo point CamXY of the lower end of the camera focus is calculated according to the first servo point CamXZ and the moving distance D obtained in S3, and the following calculation method is adopted: CamXY=CamXZ±D.

5. A misregistration imaging joint computational detection system, characterized by, The method for implementing the misregistration imaging joint calculation detection method in any one of claims 1-4, comprising: a vertical camera located directly below the multi-level three-dimensional workpiece to be detected, for shooting a first projection image of the multi-level three-dimensional workpiece and, after adjusting the posture of the multi-level three-dimensional workpiece, shooting again to obtain a second projection image; an image processing module for calculating a current center coordinate according to the second projection image, calculating an offset value t according to the difference between the reference center coordinate and the current center coordinate, recording the position of the horizontal camera as a reference position CamJZ when the horizontal camera focuses on the upper edge of the multi-level three-dimensional workpiece to display clearly, recording the moving distance D when the horizontal camera is moved to the lower edge to display clearly, calculating the first servo point CamXZ of the horizontal camera focusing on the upper end according to the obtained reference position CamJZ and the obtained offset value t, and calculating the second servo point CamXY of the lower end of the camera focus according to the first servo point CamXZ and the obtained moving distance D; a movable horizontal camera for focusing on the multi-level three-dimensional workpiece, shooting a first image at the first servo point, and shooting a second image at the second servo point; a horizontal sliding module is installed below the horizontal camera, which is used to keep the object distance consistent when shooting the first image and the second image under the condition that the horizontal camera advances and retreats and the focal length is fixed; the image processing module further comprises extracting the upper end edge in the first image and recording the coordinates of the edge curve, extracting the lower end edge in the second image, and performing affine transformation on the coordinates of the lower end edge to the first image to form a misregistration imaging joint result.

Images