A camera with biased configuration

By using a bias-based camera and image fusion technology, the problem of insufficient depth of field when shooting curved object surfaces by macro cameras is solved, achieving a larger depth of field and a clear imaging surface, which is suitable for detecting glass edge defects in industrial inspection systems.

CN110536048BActive Publication Date: 2024-11-12YISHI INTELLIGENT TECH (SHANGHAI) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201810967732.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-07-12
Filing Date
2018-08-23
Publication Date
2024-11-12
Estimated Expiration
2038-08-23

AI Technical Summary

Technical Problem

Traditional macro cameras cannot obtain complete and clear images when photographing curved surfaces, especially the curved parts of car windshields, where the shallow depth of field results in a mismatch between the clear image surface and the object's surface.

Method used

Using a camera with a bias configuration, by adjusting the tilt angle between the lens and the image sensor and the camera's rotation knob, combined with the drive device and processor, the clear parts in the image are identified and the image is fused to obtain a complete image of the object's surface.

Benefits of technology

It achieves a large depth of field and a clear imaging surface, improving the shooting effect of objects with surface depth at close range. It can obtain clear images of object surfaces in industrial inspection, especially for the detection of defects on glass edges.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN110536048B_ABST
    Figure CN110536048B_ABST
Patent Text Reader

Abstract

The present invention discloses a camera with an offset structure, comprising: an imaging sensor, the imaging sensor is used to sense light to convert it into a corresponding image signal, the imaging sensor is provided with a sensor plane, the sensor plane is used to receive light; a housing is used to accommodate the imaging sensor; a lens is used to transmit light to the imaging sensor; the housing accommodates the imaging sensor, and the lens is mounted to the housing; the sensor plane of the imaging sensor faces the lens; the main optical axis of the lens intersects with the sensor plane at an angle, and the value range of the angle formed by the intersection is 25 degrees to 85 degrees. The present invention is beneficial in that it provides a camera with an offset structure, the camera has a large depth of field and a large clear imaging surface, and improves the shooting effect of close-range shooting of objects with surface depth.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an image acquisition device, in particular to a camera with a bias structure. Background Art

[0002] When using a macro camera for macro photography, it is often impossible to obtain a complete and clear image of the surface of an object with a certain depth because the depth of field of the macro camera is often small. For example, when shooting an image of the surface of a curved car windshield, the curvature of the glass gives the glass surface a certain depth, and a traditional macro camera cannot obtain a complete and clear image of the curved part of the curved car windshield in one shot.

[0003] The area that can be clearly photographed by the camera is defined as the clear imaging surface. In traditional cameras, the size of the clear imaging surface can only be changed by changing the size of the lens, the image sensor, or the distance between the lens and the image sensor. Summary of the invention

[0004] A camera with an offset structure comprises: an imaging sensor, the imaging sensor is used to sense light to convert it into a corresponding image signal, the imaging sensor is provided with a sensor plane, the sensor plane is used to receive light; a shell is used to accommodate the imaging sensor; a lens is used to transmit light to the imaging sensor; the shell accommodates the imaging sensor, and the lens is mounted to the shell; the sensor plane of the imaging sensor faces the lens; the main optical axis of the lens is inclined to intersect with the sensor plane, and the value range of the angle formed by the intersection is 25 degrees to 85 degrees.

[0005] Furthermore, the length direction of the shell is parallel to the main optical axis of the lens.

[0006] Furthermore, the length direction of the shell is perpendicular to the sensor plane.

[0007] Furthermore, the sensor plane is circular.

[0008] Furthermore, the sensor plane is rectangular.

[0009] Furthermore, the lens is a convex lens.

[0010] Furthermore, the camera with the offset structure also includes: a mounting shaft rotatably mounted to the housing, the mounting shaft being fixedly connected to the image sensor, and the mounting shaft partially exposed from the upper portion of the housing, and the mounting shaft being used to operably change the tilt angle of the image sensor.

[0011] Furthermore, the camera with the offset structure also includes: a rotating dial button, which is fixedly mounted to the portion of the mounting shaft exposed from the housing; and an angle encoder disk, which is used to indicate the rotation angle of the rotating dial button, thereby indicating the tilt angle of the image sensor.

[0012] Further, the ratio of the longest line segment in the sensor plane to the length of the shell is in a range of 0.1 to 0.3.

[0013] Furthermore, a ratio of an area of ​​a projection of the lens on a projection plane perpendicular to its optical axis to an area of ​​the sensor plane on the projection plane ranges from 1.3 to 2.1.

[0014] The invention is beneficial in that it provides a camera with an offset structure, which has a larger depth of field and a larger clear imaging surface, thereby improving the shooting effect of close-range shooting of objects with surface depth. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figures 1A-1C is a schematic diagram of various image acquisition systems of the present invention;

[0016] Figure 2 It is a schematic diagram of the working status of the scanning camera;

[0017] Figure 3A It is a system block diagram of an industrial detection system;

[0018] Figure 3B It is a schematic diagram of an industrial inspection system;

[0019] Figure 4 This is a schematic diagram of the encoder installation method;

[0020] Figure 5A is a stereoscopic schematic diagram of a camera having an offset configuration;

[0021] Figure 5B is an internal schematic diagram of a camera having a biased configuration;

[0022] Fig. 6A is a schematic diagram of an image acquisition subsystem including three cameras;

[0023] Figure 6B-6D Schematic diagram of the positional relationship of the clear imaging planes P of the cameras of the image acquisition subsystem including three cameras;

[0024] Fig. 7A It is a schematic diagram of an image of a defective portion of edge frosted glass;

[0025] Figure 7B This is another schematic diagram of the image of the edge frosted glass defect;

[0026] Fig. 8A It is a schematic diagram of a strip light source for an industrial inspection system;

[0027] Figures 8B-8DIt is a schematic diagram of a ring light source for an industrial inspection system;

[0028] Fig.9A It is a flow chart of an industrial inspection system including a position acquisition system using an image before fusion to judge edge defects of frosted glass;

[0029] Fig. 9B The invention is a flow chart of an industrial inspection system including a position acquisition system using a fused image to judge edge defects of frosted glass. DETAILED DESCRIPTION

[0030] Clear imaging surface P

[0031] For an image acquisition device, such as a camera, camcorder, scanner, etc., it can be known from optical principles that if the physical conditions such as the model and position of the optical and electronic components in the image acquisition device are not changed, the distance and range at which the camera can capture clear images are often fixed.

[0032] like Figures 1A-1C As shown, the area that can be clearly photographed by the camera 101 is defined as a clear imaging surface P. Generally speaking, the shape of the clear imaging surface P depends on the hardware of the camera 101, especially the shape of the image sensor. Figures 1A-1C In the example, it is drawn as a rectangle for illustration only. In fact, the clear imaging surface P can be of any shape. The size of the clear imaging surface P and the distance and angle between it and the camera 101 are determined by the relative relationship between the lens and the image sensor.

[0033] Image acquisition system

[0034] The image acquisition system at least includes an image acquisition device and a processor, and a camera is used as the image acquisition device here.

[0035] The camera's clear imaging surface P is parallel to the surface of the target object

[0036] When performing macro photography, the camera is often required to shoot the object 104 at a close distance.

[0037] like Figure 1A As shown, 102 is a loading device for placing a target object 104. 103 is a positioning block for fixing the position of the target object 104. If the portion of the object 104 to be photographed is a plane and is within the clear imaging plane P, a complete and clear image of the photographed portion can be obtained.

[0038] But if Figure 1BAs shown, the position of the object 104 changes (or the object itself does not have a planar structure). At this time, if the object 104 is photographed by the camera 101, the depth of field of the camera shooting at a close distance is small, so that the part of the object 104 that is not within the clear imaging plane P cannot obtain a clear image.

[0039] In this case, even by adjusting the focal length of the camera 101 or adjusting the relative distance between the camera 101 and the object 104 , a complete and clear image of the photographed portion of the object 104 cannot be obtained.

[0040] The camera's clear imaging surface P is not parallel to the surface of the target object

[0041] The present invention provides an image acquisition system comprising: a camera 101, a drive device 107 and a processor ( Figure 1A and Figure 1B not shown).

[0042] The camera 101 is used to capture an image of an object. The camera 101 can be a variable-focus camera or a fixed-focus camera.

[0043] The driving device 107 can drive the camera 101 or the object 104, or both at the same time. In this embodiment, the driving device 107 is a motor. As a specific solution, the driving device includes a carrying device 102 for contacting, holding or conveying the object 104, and the carrying device 102 is driven by the driving device to drive the movement of the object 104. This has the advantage that the camera 101 does not need to be adjusted frequently.

[0044] like Figure 1B , 1C As shown, when the clear imaging plane P of the camera is not parallel to the surface of the object, no matter how the focal length / object distance is adjusted, the clear imaging plane P of the camera cannot completely overlap with the surface of the target object. At this time, the clear imaging plane P of the camera and the surface of the target object often have only one intersection line from the geometric relationship. In this case, only part of the image in the photo taken by the camera is clear, and the clear part is the intersection line between the clear imaging plane P of the camera and the surface of the target object.

[0045] The following takes the acquisition of an image of a cube surface as an example. The surface of the cube is not parallel to the clear imaging surface P of the camera, and it is considered that the clear imaging surface P of the camera intersects with the surface of the target object.

[0046] Identify the clear part of the image by the peak of the image parameter gradient

[0047] As an optional implementation, the steps of collecting and determining the clear part of the cube image are as follows:

[0048] Collect the original image of the cube surface. In this image, the image where the camera's clear imaging surface P intersects with the object surface is clear, and the image of other parts is blurred. The image collected in this step can be a grayscale or color image.

[0049] The original image of the cube surface is denoised to obtain a denoised image. Denoising can reduce the impact of noise generated by hardware or environment on subsequent image processing.

[0050] Calculate the absolute value of the parameter value gradient between adjacent pixels / pixel groups in the denoised image. The parameters here can be grayscale values, brightness values, or other parameters such as contrast and saturation.

[0051] According to the absolute value of the gradient value calculated in the previous steps, the Gaussian function is used to determine the peak point x of the gradient max In the same picture, the "blurred" pixel can be regarded as the average value of the pixels around the blurred pixel, which reduces the contrast between the pixel and the surrounding pixels, thus producing a blurred visual effect. The contrast between the "clear" pixel and the surrounding pixels is more distinct, which can better reflect the details of the pixel and the visual effect is clearer. Excluding the influence of noise, in the same picture, the larger the absolute value of the gradient, the clearer the image at that location. As an optional solution, the method for determining the peak point of the gradient value can be other calculation methods besides the Gaussian function. Peak point X max The location is the clear part of the picture.

[0052] In the process of calculating the gradient value, the gradient value can be arranged in order according to the position of the pixel / pixel group in the image, that is, the calculation result of each gradient value corresponds to a specific position in the image. max At the same time, the position of the peak point in the image can also be obtained, that is, the specific position of the clear part of the image.

[0053] This peak value judgment can be implemented using a gradient processing unit in a processor.

[0054] The method of judging the clear part by the peak value of image parameters can achieve better results when shooting objects with rich surface textures.

[0055] Identify sharp parts of the image by calculating intersection points

[0056] Establishing the coordinate system

[0057] As an optional implementation, the line segment representing the clear imaging surface P of the camera and the surface contour of the target object are represented in the same coordinate system. The coordinate system may be a two-dimensional coordinate system, and the coordinate system plane is perpendicular to the clear imaging surface P of the camera.

[0058] The position of the camera is fixed, and the parameters of the camera lens and image sensor remain unchanged. At this time, the position of the line segment p corresponding to the clear imaging surface P of the camera in the coordinate system is also fixed. Select the x-axis, y-axis and coordinate origin, then each point on the line segment p has a fixed coordinate value. The line segment p can be represented by a linear function within a certain interval. The position of the intersection of the line segment p and the surface contour of the target object corresponds to the position of the clear part in the image.

[0059] Get the coordinate information of the surface contour of the target object

[0060] like Figure 2 As shown, a scanning camera 201 is used to obtain the coordinate information of the surface contour of an object. The scanning camera is set above the target object, usually directly above it. The motion mechanism causes relative motion between the scanning camera and the target object so that the scanning camera can traverse all parts of the surface of the object. In this schematic diagram, the scanning camera 201 is installed on the guide rail 202 through the motion mechanism. The shooting direction of the scanning camera is perpendicular to the plane of the coordinate system to obtain the projection contour of the target object in the coordinate system. During the relative motion of the two, the distance between the scanning camera 201 and the plane where the coordinate system is located remains constant. The scanning camera 201 can be a linear array camera or a planar array camera. The relative motion direction between the camera and the target object is defined as the x direction, and the direction perpendicular to the x direction is the y direction.

[0061] Linear array camera obtains the coordinate information of the surface contour of the object

[0062] A line scan camera is a camera that uses a linear array image sensor. Compared to an area scan camera, the image obtained by a line scan camera has a higher resolution, but since it can only obtain one line image at a time, subsequent processing operations are required to obtain a complete image of the object to be photographed.

[0063] The coordinate information of the surface contour of an object is obtained using a line array camera. The specific working steps are as follows:

[0064] Linear camera scanning: The motion range of the linear camera is fixed, and it continuously scans at a fixed interval and outputs images, with each image being one line.

[0065] Threshold processing: The image obtained by the line array camera is subtracted from the background image when there is no occlusion, and the difference or the absolute value of the difference is compared with the preset threshold. When the difference is within a certain range, it is judged that the row image obtained by the line array camera does not contain the target object. At this time, the grayscale value of the image at this location is adjusted to a fixed value. When the difference exceeds a certain range, it is considered that the image here is the image of the target object. At this time, the grayscale value of the image at this location is adjusted to another fixed value and the two endpoints of the target object image (row image) obtained by the line array camera are recorded. Through such an operation, an image of the area covered by the object can be obtained, and the image is a binary image. Binary images are easier to process. In addition, the image information only records the two endpoints of the row image to form the contour of the object, discarding other useless information and reducing the storage capacity of the system.

[0066] Acquisition of x-direction coordinates: The motion mechanism causes the linear array camera and the target object to move relative to each other. The displacement information of the motion mechanism at any time is obtained by using a displacement acquisition device such as an encoder, and thus the displacement information of the target object at that time is obtained. The encoder outputs the displacement information corresponding to the row of images. Since the shooting direction of the linear array camera is perpendicular to the plane of the coordinate system, the position information of the encoder corresponding to the row of images is the x-coordinate information of the row of images.

[0067] Acquisition of y-direction coordinates: The length of the scan line of the linear array camera is fixed. The y coordinate of one end of the scan line is set to 0. The y coordinates of the two endpoints in the row image are obtained by calculating the distances between the two endpoints and the scan line endpoint.

[0068] Smoothing: stitch the multiple frames of images obtained by the linear array camera, and smooth the stitched contour information to obtain the curve information of the edge of the entire object. Smoothing filtering can eliminate the errors caused by single-frame image processing, so that the contour of the target object can more accurately reflect the real information, which is conducive to subsequent processing.

[0069] The area array camera obtains the coordinate information of the surface contour of the object

[0070] The area array camera uses an area array image sensor. Its imaging area is a plane, and a complete image of the object to be photographed can be obtained through a single shot.

[0071] The coordinate information of the surface contour of an object is obtained using an array camera. The specific working steps are as follows:

[0072] Shooting with an area array camera: The target object is placed within the shooting range of the area array camera to ensure that the area array camera can obtain a complete image of the target object in one shot. If the image captured by the area array camera is not a complete image, you can refer to the working steps of the line array camera to capture a complete image of the target object.

[0073] Thresholding: The thresholding procedure is similar to that of a line scan camera.

[0074] Obtaining x and y coordinates: The position of the array camera is fixed, and its shooting range is also fixed. Define the coordinates of a boundary point of the shooting range of the array camera as (0,0), then the x and y coordinate information of the contour point represented by the binary image can be obtained by calculating the distance between the contour point and the boundary point.

[0075] Smoothing: The smoothing process is similar to that of the line scan camera.

[0076] Calculate intersection points to get sharp positions in the image

[0077] As an optional implementation, the steps of collecting and determining the clear part of the cube image are as follows:

[0078] The original image of the cube surface is collected. The collected image can be a grayscale image or a color image.

[0079] The scanning camera scans the image of the target object to obtain the coordinate information of the edge contour of the target object. The coordinate information of the line segment p projected by the clear imaging surface P of the camera in the coordinate system is pre-calibrated.

[0080] The intersection point between the line segment p and the edge contour of the target object is calculated. The image at the intersection point is the clear part of the original image. At this point, the processor obtains the relative position information of the clear imaging part in the target object.

[0081] Laser marking of the clear imaging surface to identify the clear part of the image

[0082] When the clear imaging plane of the camera is not parallel to the surface of the target object, if there is still a clear part in the image captured by the camera, then the clear imaging plane P of the camera and the surface of the target object can be approximately considered to have an intersection line from the perspective of geometric relationship.

[0083] The position of the clear imaging surface is marked by laser, and the place on the surface of the target object illuminated by the laser at any time is the intersection line between the camera's clear imaging surface P and the surface of the target object. When reflected in the image taken by the camera, the place illuminated by the laser in the image is the location of the clear part in the image.

[0084] like Figure 1B As shown, as an embodiment, the laser emitting device 108 is arranged on the side of the clear imaging surface P of the camera, and includes a plurality of laser emitters, and the laser beams emitted by the plurality of laser emitters are parallel to each other, and each laser beam is located on the same plane to form a laser area, and the laser area covers the clear imaging surface P of the camera. As another optional embodiment, Figure 1CAs shown, the laser emitting device 108 is arranged above the clear imaging surface P, and the laser emitter emits a planar laser from top to bottom. Since the laser has a certain width, the portion where the surface of the target object overlaps with the laser area will still have a laser spot on the surface of the object due to diffuse reflection of the surface itself, and will not be completely blocked by the object. The planar laser area covers the clear imaging surface P of the camera. In this embodiment, the laser uses a red laser with a wavelength of 650nm. As other optional embodiments, the color of the laser can also be green or other colors.

[0085] As a method of judging the clear part of the image based on the laser spot, the camera adopts a multi-color camera, including a first channel and a second channel. In this embodiment, the camera adopts a color camera, including three channels of RGB. In this embodiment, the laser adopts a red laser with a wavelength of 650nm, the first channel is the R channel, and the second channel is the G channel or the B channel. It is worth mentioning that the first channel and the second channel defined here can include several channels. For example, in this embodiment, the second channel can be the GB channel. The processor includes a channel selection unit for selecting the channel of the multi-color camera. The processor extracts the first channel image captured by the multi-color camera and judges the laser spot area in the first channel image. Then the second channel image is extracted, and the laser spot area in the first channel image is matched with the second channel image. The laser spot area in the first channel image is the area of ​​the clear image in the second channel image. As an optional embodiment, when the color of the laser is green, the first channel is the G channel. The use of a multi-color camera to judge the laser spot area and the clear image position respectively can avoid the interference of laser irradiation on the image information, and facilitate the use of the image for subsequent operations such as defect detection.

[0086] As an optional implementation, optical elements such as gratings are used to make the light intensity distribution of the laser area uniform. Uniform laser light intensity distribution will not cause excessive interference to the surface information of the target object. In this case, there is no need to use a multi-color camera to filter the laser.

[0087] The use of laser marking to clearly image the surface can also adapt to different shooting scenes and eliminate the need for equipment such as scanning cameras.

[0088] Image Fusion

[0089] In order to obtain a complete image of the object surface, a processor is used to fuse the clear images corresponding to different positions on the object surface to form a complete clear image of the object surface. Before image fusion, the processor has obtained the relative position relationship of each clear image portion relative to other clear image portions in the same image according to the above system and method.

[0090] As an implementation mode, the camera is mounted on a driving device and can change the shooting position with the driving device. As an implementation mode, the driving device includes a mounting seat, a stepper motor, a transmission mechanism, a wheel, and a guide rail. The camera is fixedly connected to the mounting seat. The stepper motor can be arranged inside the mounting seat or outside the mounting seat. One end of the transmission mechanism is connected to the wheel, and the other end is connected to the stepper motor. The stepper motor drives the transmission mechanism to drive the wheel to rotate. The wheel can move along the guide rail, and its motion trajectory is determined by the guide rail. During a shooting process, the driving device moves in a certain direction according to the motion trajectory of the guide rail. Driven by the stepper motor, the camera changes the same distance each time. The camera takes pictures of the surface of the object at different positions according to the walking order of the driving device. The processor receives these pictures, obtains the clear parts of these pictures by judging the gradient peak or calculating the intersection, and splices the clear parts of these pictures in order. (D in the accompanying drawings indicates the relative motion direction of the object and the camera)

[0091] As an optional implementation, the motor of the driving device may be a servo motor.

[0092] As another implementation, the driving device can drive the target object or the camera to rotate around a rotation axis, and the rotation axis is parallel to the camera's clear imaging surface P. In this way, images of different parts of the target object surface can also be obtained.

[0093] As another optional implementation, the position of the camera is fixed, and the driving device drives the photographed object to move. The camera takes multiple pictures at different positions of the target object. The processor receives these pictures, obtains the clear parts of these pictures by determining the gradient peak or calculating the intersection, and splices the clear parts of these pictures in order. At this time, the driving device can be an industrial production line, a roller, a transport vehicle, etc.

[0094] In summary, the driving device is used to drive the camera and / or the target object so that the portion of the target object to be photographed can appear in the camera's clear imaging plane P in a time-sharing manner. As an optional embodiment, the image acquisition system may further include a positioning device, which is respectively data-connected with the driving device and the processor so that the processor can match the clear portion in the image with the portion corresponding to the target object.

[0095] As an optional implementation, the camera can be connected to the processor. The camera transmits the captured image to the processor, and the processor processes the image to obtain the surface information of the object contained in the image. The surface information of the object includes the defect information of the object surface, the text and logo information on the object surface, the color, texture, pattern, etc. of the object surface. Among them, the defect information at least includes: whether there is a defect in the image; the location information of the defect in the image.

[0096] By calculating the position where the clear imaging surface intersects with the object surface, the clear part of the image is determined. This can adapt to various shooting scenes and will not be affected by lighting conditions or the surface texture of the object.

[0097] Multiple cameras shooting together

[0098] The image acquisition system may include several cameras to improve the efficiency of shooting. For example, when the rotary drive device 107 is used to drive the object to rotate, if there is only one camera, the rotary drive device usually needs to rotate 360° to capture a complete image of the surface of the target object. If two relatively arranged cameras are used, the target object is placed between the two cameras, and the clear imaging surfaces of the two cameras are perpendicular to the same plane. At this time, the drive device only needs to rotate 180° to complete the shooting, which reduces the time by half. If three cameras are used, the center line angle between the two cameras is 120°, and the planes where the clear imaging surfaces of the two cameras are located are inclined and intersected. At this time, the drive device only needs to rotate 120° to complete the shooting.

[0099] Online detection system

[0100] Industrial inspections often have higher inspection standards. If image acquisition inspection is used instead of manual visual inspection, the camera needs to shoot the object to be inspected at a close distance to obtain more detailed surface image information of the object.

[0101] like Figure 3A , 3B As shown, an industrial detection system based on image acquisition includes an industrial production line 30, a position acquisition subsystem 31, an image acquisition subsystem 32, and a processor 33.

[0102] The following takes the glass edge online detection system as an example to specifically introduce the online detection system including the image acquisition system of the present invention. In this embodiment, the glass 39 to be detected is a sheet of glass. As an optional implementation, the detection system can also detect surface defects of other objects besides glass, such as wood surface defects, steel surface defects, stone surface defects, etc.

[0103] The position acquisition subsystem 31 is used to obtain the coordinate information of the edge of the glass to be measured. When the image parameter gradient peak is calculated or the clear imaging surface is marked with a laser to identify the clear part in the image, the position acquisition subsystem can be omitted. The image acquisition subsystem 32 includes one or more cameras for acquiring images of the glass edge. The processor 33 is data-connected with at least the position acquisition subsystem 31 and the image acquisition subsystem 32 to perform various image and data processing operations.

[0104] This industrial inspection system can obtain a clear image of the surface of the object to be tested through the machine. It saves labor and improves the detection accuracy. With this industrial inspection system, the object to be tested can be placed in any position and direction on the industrial production line.

[0105] Industrial production line

[0106] The industrial production line 30 is used to place the glass to be photographed. The industrial production line can at least transport the object to be inspected on a transport surface in a transport direction along a transport straight line. In this embodiment, the industrial production line drives the glass placed on the industrial production line to move.

[0107] The industrial production line comprises a transport unit for transporting the products to be tested.

[0108] As an optional implementation, the industrial production line includes a roller frame and a plurality of conveying rollers, the plurality of conveying rollers are rotatably connected to the roller frame, and the axes of the plurality of conveying rollers are located on the same plane.

[0109] As another optional implementation, the industrial production line may also include conveying devices such as conveyor belts and conveyor vehicles.

[0110] The surface of the industrial production line can be made of anti-slip rubber material, or a suction cup can be set to increase the friction between the transported object, thereby increasing the friction between the industrial production line and the glass and improving the transportation efficiency of the industrial production line.

[0111] Location Acquisition Subsystem 31

[0112] The position acquisition subsystem is used to obtain the coordinate information of the product to be tested. When the clear part in the image is identified by calculating the peak value of the image parameter gradient or by using a laser to mark the clear imaging surface, the position acquisition subsystem can be omitted.

[0113] The coordinate system is established with the conveying surface of the industrial production line as the plane where the coordinate system is located. The coordinate system can be a two-dimensional coordinate system. The outline of the object placed on the industrial production line is projected into a closed figure in the coordinate system. The conveying direction of the industrial production line is the x direction, and the direction perpendicular to the conveying direction is the y direction. The position acquisition subsystem needs to at least know the relative position between the product to be inspected and the camera.

[0114] Front camera

[0115] like Figure 3BAs shown, the front camera 312 is arranged above the industrial production line 30, usually directly above the industrial production line 30. Of course, as an optional embodiment, the front camera 312 can be set according to actual needs. However, no matter how it is set, it is necessary to ensure that the imaging range of the front camera 312 can cover the appropriate industrial production line area. The front camera 312 is separated from the industrial production line 30 by a certain distance. The lens of the front camera faces the industrial production line 30, and its optical axis is perpendicular to the plane where the axes of several conveyor rollers are located.

[0116] The front camera can be a line scan camera (line array camera) or an area array camera. The scanning camera is used to obtain the coordinate information of the object to be measured.

[0117] As an optional implementation, the front camera 312 is a line scan camera.

[0118] Acquisition of industrial production line location information (glass outline x-coordinate value)

[0119] The glass to be tested is placed on the industrial production line and moves with the industrial production line. From the coordinate plane, at any time, as long as the position information of the industrial production line is obtained, the x-coordinate value of the glass to be tested at that time can be obtained.

[0120] As an implementation manner, the image acquired by the front camera is judged, and the x-coordinate value of the portion of the same piece of glass to be tested that is first captured by the front camera is set to 0.

[0121] As an optional implementation, a position detection device is provided in the y direction of the industrial production line. As an implementation, the position detection device is a photoelectric gate, including a signal sending device provided on one side of the industrial production line and a signal receiving device provided opposite thereto, and the line connecting the signal transmitting device and the receiving device is perpendicular to the transport line of the industrial production line. The x coordinate of any point on the line is set to 0. In the absence of obstruction, the receiving device of the photoelectric gate can always receive the light signal emitted by the light-emitting device of the photoelectric gate. In the presence of obstruction, the receiving device of the photoelectric gate cannot receive the light signal emitted by the light-emitting device of the photoelectric gate.

[0122] The photoelectric gate is connected to the processor, and the distance between the photoelectric gate and the front camera is kept constant. At this time, the x-coordinate information of the front camera scanning area can also be obtained.

[0123] Encoder to obtain coordinates

[0124] As a displacement detection device, the encoder includes an encoding disk and an encoding reading unit, which is used to convert angular displacement or linear displacement into an electrical signal.

[0125] like Figure 3B , Figure 4As shown, the industrial detection system based on image acquisition includes an encoder 311, which can be a rotary encoder, including an absolute encoder and an incremental encoder.

[0126] The encoder disc can be connected to the motor through a coupling, or it can be connected to a transmission device used to directly move the industrial production line. When the encoder is connected to the motor, the encoder directly obtains the rotation information of the motor, and the corresponding motion coordinates of the industrial production line need to be converted through the reduction ratio.

[0127] The encoder is connected to the processor and is used to send information to the processor. According to the image first acquired by the scanning camera or the start of the photoelectric gate, the motion coordinates of the industrial production line recorded by the encoder at any time are the x-coordinates of the point in front of the glass.

[0128] Since obtaining the position information of the industrial production line is the original step and the key step of the entire detection system detection process, if there is an error in this step, the output results of various subsequent steps may also be wrong. Therefore, it is necessary to ensure the accuracy of the information obtained by the encoder.

[0129] As a solution to replace the photoelectric gate, the processor calculates the real-time speed of the industrial production line based on the data transmitted by the encoder. When the real-time speed of the industrial production line suddenly drops, it is determined that glass is placed on the industrial production line. The motion coordinates of the industrial production line at the moment of the sudden drop in speed are marked as 0. At this time, the x-coordinate of the point in front of the glass is also 0.

[0130] As an optional embodiment, the glass edge detection system includes multiple encoders, which are installed at different positions or different components of the industrial production line. For example, when two encoders are used, the main encoder 311a and the auxiliary encoder 311b are connected to the motor and the industrial production line transmission mechanism respectively. If the information obtained by the auxiliary encoder 311b is consistent with the information obtained by the main encoder 311a, or the error is within a certain range, it is considered that the information obtained by the encoder is accurate. Such a setting of the main and auxiliary encoders can detect encoder failures in time and improve detection accuracy.

[0131] Using brushless motor parameters to obtain coordinates

[0132] The brushless motor is controlled by electronic commutation instructions, and its speed can be controlled by pulse width modulation signals. Due to this characteristic of the brushless motor, the brushless motor controller can obtain the rotation angle of the brushless motor by recording the number of control signals, thereby obtaining the motion information of the industrial production line.

[0133] When the photoelectric sensor outputs zero coordinates, the glass enters the detection area, and the brushless motor controller records the pulse width and number of the brushless motor's control signal, and calculates the motion coordinates of the industrial production line at any time based on the pulse width and number of the control signal, that is, the x-coordinate of the point in front of the glass.

[0134] The time interval between front camera shots

[0135] Whether using an encoder or a brushless motor, the real-time speed of the industrial production line can be obtained. Due to the different loads placed on the industrial production line, or due to the different lubrication environments of the mechanical devices of the industrial production line, the industrial production line often does not move at a uniform speed. At this time, the processor calculates the real-time movement speed of the industrial production line. Calculate the time required for the industrial production line to move at a fixed interval at this speed, and trigger the front camera to shoot at the corresponding time point. Using this triggering method at the moment of front camera shooting can ensure that the pictures taken by the front camera meet the requirements of post-processing. For example, when the front camera adopts a linear array camera, it can ensure that the movement interval of the industrial production line corresponding to each shooting of the linear array camera is constant; when the front camera adopts an area array camera, it can ensure that the area array camera can obtain a complete glass surface image each time it shoots.

[0136] When using a line scan camera, the specific shooting interval setting is based on the needs under different imaging quality requirements. For example, for larger glass to be tested, the shooting frequency of the line scan camera is relatively reduced to reduce energy consumption and reduce the heat of the line scan camera. Of course, the shooting frequency of the line scan camera can also be increased to increase the fineness of the imaging.

[0137] After the front camera completes the shooting, the coordinate information of any point on the edge of the glass at any time can be obtained by combining the industrial production line coordinate information corresponding to the encoder or brushless motor control parameters.

[0138] Image acquisition subsystem 32

[0139] The image acquisition subsystem acquires images, and the processor identifies the clear part of the acquired image by determining the peak value of the image parameter gradient, calculating the intersection position, or using a laser to mark the clear imaging surface. In this embodiment, the image acquisition subsystem 32 is used to acquire the image of the edge of the glass to be measured. The image acquisition subsystem 32 may include multiple cameras 321.

[0140] As an implementation method, the image acquisition subsystem 32 is arranged after the position acquisition subsystem 31. When the glass to be tested is transported, it first passes through the position acquisition subsystem 31 and then passes through the image acquisition subsystem 32. Such an arrangement enables the image acquisition subsystem 32 to know the coordinate information of the edge of the glass to be tested when collecting images, thereby facilitating subsequent processing operations.

[0141] Working process of a single camera in the image acquisition subsystem

[0142] When detecting glass edge defects, the position and orientation of the glass on the industrial production line are often arbitrary.

[0143] As an implementation, as shown in the figure, the position of the camera is fixed, the center of its imaging range is kept horizontal with the industrial production line, and the projection of the camera's clear imaging surface P from directly above the industrial production line at least partially covers the industrial production line. During the movement of the industrial production line, the position of the camera remains unchanged.

[0144] The glass to be tested is placed on the industrial production line in an arbitrary direction. In this case, the glass edge and the camera image are often not parallel. In the glass edge image taken by the camera, the image at the intersection of the glass edge and the camera clear imaging surface P is clear, and the other parts of the image except the intersection are unclear.

[0145] As an optional implementation, the processor controls the camera to shoot according to the real-time position of the glass to be tested obtained by the position acquisition subsystem. When the front point of the glass to be tested is about to enter the camera shooting area or has entered the camera shooting area, the processor controls the camera to start shooting.

[0146] In order to ensure that every point on the edge of the glass can intersect with a clear imaging surface P of the image acquisition subsystem 32 during the movement of the glass to be tested, as an optional implementation, a limit mechanism can be set at the edge of the industrial production line to prevent the edge of the glass to be tested from exceeding the width range of the industrial production line; as another optional implementation, an alarm device / function can be introduced: when the front camera 312 determines that the shape and position information of the glass to be tested exceeds the width range of the industrial production line, an alarm is issued.

[0147] Take multiple pictures and perform image fusion

[0148] A camera has depth of field, and the sharp part in the photo taken by the camera has a certain range.

[0149] In order to obtain a complete image of the glass edge, the camera takes multiple photos continuously at a specific moment, and the clear parts of each image are stitched together to form a complete image of the glass edge.

[0150] The horizontal center point of the clear imaging area in the i-th frame image is denoted as x i .

[0151] x i The range W of the image captured in horizontal and vertical directions is the center x and W y Among them, x i ,W x and Wy The units are all in pixels. The clear range of the image captured by the camera is: [x i -W x / 2,x i +W x / 2].

[0152] The following formulas represent the following parameters: image distance: u, object distance: v, belt speed: V, distance the object moves perpendicular to the camera optical axis: S v , the distance moved in the direction perpendicular to the camera optical axis: S u , the horizontal coordinate of the image on the sensor plane: x (corresponding to the horizontal position of the pixel at that point on the image), sensor pixel size: d, glass thickness: T, lens focal length: f, sensor tilt angle: θ, reference values ​​x0 and u0, given the horizontal coordinate x0 on the sensor, the corresponding image distance of the lens is u0.

[0153] In the method of calculating the clear part of the image by the peak point of the image gradient, it is necessary to calculate the area of ​​the clear range in the image. The moving speed V of the belt and the angle α of the camera optical axis relative to the belt; the exposure interval t of two frames of images.

[0154] In the method of calculating the clear part of the image by the peak point of the image gradient,

[0155] W x =S u / cos(θ) / d,W y =(u*T) / (v*d)

[0156] Among them, S u =u*S v / v, v=1 / (1 / f–1 / u), u=u0+Δu; S v =V*sin(α)*t; Δu=(x i -x0)*sin(θ) The camera’s shooting interval is t.

[0157] In the algorithm that obtains the clear part of the image by calculating the intersection position,

[0158] W x =S u / cos(θ) / dAt the same time W y =(u*T) / (v*d)

[0159] Among them, S u =u*S v / v, v=1 / (1 / f–1 / u), u=u0+Δu; S v =V*sin(α)*t

[0160] In this method, the rear camera selects an appropriate time to take a photo based on the glass edge position information and encoder readings provided by the front camera.

[0161] The processor controls the image acquisition subsystem to capture images based on the relative position between the product to be detected and the camera. i The timing is calculated by the following method:

[0162] Assume that the interval between the two camera encoders is C i-1 +C pb and C i +C pb , then the change in the position of the glass between the two shots is (C i -C i-1 )*S c .

[0163] Because the camera sensor is tilted at an angle θ, the pixel size along the x-axis of the image plane is d*Cos(θ), and the pixel size along the y-axis of the image plane is still d.

[0164] The distance S that the object moves perpendicular to the camera optical axis v =(C i -C i-1 )*S c *Sin(α).

[0165] Then according to the position information of the edge point provided by the front camera (X i , Y i +S pb ) and D, we can get the object distance v of the camera = (D + X i ) / sin(α). The image distance u can also be obtained based on the lens focal length formula.

[0166] Then we get x i =(u-u0) / sin(θ)+x0.

[0167] The controller controls the camera to shoot based on the above calculation results and the information obtained by the position acquisition subsystem, so that the images captured by the camera twice at an interval time / distance can meet the requirements of image stitching.

[0168] By fusing multiple clear images corresponding to different positions of the glass edge, a complete glass edge image within the shooting range of a single camera is obtained.

[0169] If the current glass movement direction is from right to left: collect the [xD / 2, x+D / 2] image of the current frame and place it on the right side of the output image.

[0170] If the current glass movement direction is from left to right: collect the [xD / 2, x+D / 2] image of the current frame and place it on the left side of the output image.

[0171] Repeat the above two steps until a complete and clear image is obtained.

[0172] Sensor / Lens Tilt

[0173] When detecting surface defects of an object, the object to be detected can be placed on the industrial production line in any direction. The image acquisition and detection system of the present invention acquires the surface image of the object by obtaining a clear image of the overlap between the surface of the object and the clear imaging surface P of the camera. The image acquisition device includes a camera, a video camera, a scanner, etc., and its clear imaging surface P has a certain length. However, the surface defect detection system of the object requires a large depth of field range of the camera so as to cover a sufficiently wide area on the industrial production line.

[0174] In order to obtain a larger depth of field, the lens and / or image sensor can be replaced, but a larger depth of field means more cost.

[0175] As an optional implementation, the image acquisition subsystem in the industrial inspection system uses a camera with a bias structure.

[0176] As shown in FIG5 , the camera with an offset structure includes a housing 501, a lens 502 and an image sensor 503, wherein the image sensor is connected to a mounting shaft 504, and the mounting shaft is rotatably mounted to the housing 501. The mounting shaft partially exposes the upper part of the housing 501, and a rotary dial 505 is fixedly connected to the portion of the mounting shaft exposed from the housing. As an optional embodiment, an angle code disk 506 is arranged around the rotary dial to indicate the rotation angle of the image sensor. The lens 502 defines a main optical axis, and the image sensor 503 defines a sensing plane. A straight line passing through the mounting shaft 504 and perpendicular to the main optical axis of the lens forms an angle α with the sensing plane. The rotary dial 505 can be operated to be rotated, so that the mounting shaft 504 rotates, and the mounting shaft 504 drives the image sensor 503 to rotate. Through the angle code disk 506 arranged around the rotary dial, the current tilt angle of the image sensor can be intuitively known.

[0177] As an optional implementation, the lens 502 may also be rotated to change the tilt relationship between the image sensor and the lens.

[0178] Taking the tilt of the image sensor as an example, the focal range of the sensor is expressed by the following formula:

[0179] [(u+sin(α)*x) / (u+sin(α)*x-1),u / (u-1)]

[0180] Wherein, u is the image distance, α is the rotation angle of the image sensor, x is the normalized size of the focal length of the image sensor in the radial direction of the rotation axis (x=X / f, where X is the actual size), and sin(α)*x is the projection distance of the sensor in the direction of the optical axis.

[0181] It can be seen from the formula that for given x and u, the larger α is, the smaller (u+sin(α)*x) / (u+sin(α)*x-1) is, which means the depth of field is larger. When the α angle range is between 5° and 65° (it can also be expressed as the angle formed by the intersection of the main optical axis of the lens and the sensing plane is between 25° and 85°), a better imaging effect can be obtained.

[0182] According to the imaging principle of the lens, the camera with an offset structure has a longer clear imaging surface P than an ordinary camera using the same lens and image sensor, which corresponds to a larger depth of field.

[0183] The longer length of the clear imaging surface P and the depth of field range enable the imaging range of the camera with an offset structure to cover as many areas as possible on the industrial production line. Since the position of the glass to be tested on the industrial production line is uncertain, the use of a camera with an offset structure can ensure that any point on the industrial production line can coincide with a point on the camera's clear imaging surface P at a certain moment during the movement at the same hardware cost, thus ensuring that any position on the edge of the glass can be clearly imaged in the camera with an offset structure at a certain moment.

[0184] At the same time, a larger depth of field range can obtain more clear images when shooting images of the surface of an object with a certain depth. For example, when shooting curved glass, a camera with an offset structure can capture a clear macro image of the entire glass surface at a certain shooting angle at one time. However, traditional macro cameras may not be able to simultaneously obtain clear images of different positions of the curved glass due to their relatively small depth of field.

[0185] Camera with biased configuration Camera with biased configuration Camera with biased configuration Camera with biased configuration Camera with biased configuration

[0186] Multiple cameras shooting together

[0187] Objects often have multiple surfaces, and multiple surfaces of an object or different positions on a surface may block each other. In this case, a single camera cannot obtain images of multiple surfaces of an object at the same time.

[0188] The image acquisition subsystem 32 is used to acquire a complete image of the glass edge. The image acquisition subsystem 32 includes multiple cameras. The multiple cameras can be set on the same horizontal plane. Usually, the multiple cameras are distributed around the periphery of the industrial production line and are set on both sides of the industrial production line transportation line. The clear imaging planes P of any two cameras do not overlap.

[0189] like Fig. 6A As shown, as an optional implementation, multiple identical cameras are evenly distributed around a point on the midline of the width direction of the industrial production line (this point is called the center point), and the angle formed by the line between any two adjacent cameras and the center point remains consistent. It can also be described as the multiple cameras being evenly distributed on the circumference of a circle with the center point as the center. This setting method makes each camera centrally symmetrical with respect to the center point. The centrally symmetrical relationship makes the installation and replacement of the entire image acquisition subsystem 32 easier and more convenient.

[0190] As an improvement of the above embodiment, the number of the multiple cameras is an even number. The even number of cameras is evenly distributed on both sides of the transport line of the industrial production line. At this time, the cameras are not only centrally symmetrical but also axially symmetrical. The setting of an even number of cameras further reduces the difficulty of assembling and replacing the image acquisition subsystem.

[0191] Using multiple cameras to shoot at the same time can obtain images of different edges of the glass at the same time. If the number of cameras used is too small, it cannot meet the requirements of obtaining images of different edges of the glass. If too many cameras are used, the installation method of these units relative to the industrial production line will become complicated, and the corresponding cost will also increase due to the increase in the number of cameras.

[0192] Each camera has a certain shooting angle β when shooting the glass to be tested at different positions. Only when β>0° can the camera work normally. When multiple cameras are used, the shooting angle β of each camera is required to be greater than 0°. Considering the uniform distribution of multiple cameras, the number of cameras n and the limit shooting angle β max The relationship between is expressed by the following formula:

[0193] β max =90°-180° / n

[0194] Only β max >0°, the camera can work normally. At the same time, since n must be an integer, it can be seen from the formula that n≥3, that is, at least 3 cameras are needed to meet the shooting requirements. At this time, at least two cameras are located on the same side of the transport line.

[0195] β max It can also be regarded as the incident angle of the camera, β maxThe larger it is, the more perpendicular the camera's shooting direction is to the object's surface, and the better the shooting effect will be.

[0196] As can be seen from the table below, the larger n is, the greater β max The larger the number of cameras, the higher the average image quality. In other words, the more cameras there are, the better the image quality.

[0197] Number of cameras n <![CDATA[β max ]]> Angle Improvement 3 30 - 4 45 50% 5 54 25% 6 60 11% 7 64.3 7%

[0198] But the more cameras there are, the higher the cost and the more complicated the assembly process will be.

[0199] According to the results of multiple experiments, the absolute value of the imaging quality, the imaging quality improvement value and the hardware cost of the entire system when using different numbers of cameras are comprehensively considered. The optimal state b of the entire system is expressed by the following formula:

[0200] b=8n-n 2

[0201] It can be seen from the formula that when n=4, b reaches its maximum value.

[0202] When n is 4, the cameras can be set on both sides of the industrial production line to ensure that they are on the same horizontal plane as the glass to be tested. In addition, since the distance between each camera set on both sides of the industrial production line and the industrial production line is the same, the difficulty of assembly can be reduced.

[0203] Using 4 cameras, the angle is improved by 50% compared to 3 cameras. According to the experimental results, the image quality is improved the most when the number of cameras in the image acquisition subsystem increases from 3 to 4.

[0204] Camera installation location

[0205] like Fig. 6A As shown, taking 3 cameras as an example, Figure 6B-6D The relationship between the clear imaging surfaces of each camera in different image acquisition subsystems is given.

[0206] As an implementation method, the main optical axes of the three cameras are parallel to the same plane. Such an arrangement can prevent the distortion of the collected pictures caused by the longitudinal tilt of the cameras.

[0207] The clear imaging surface P of each camera must satisfy the following conditions: in the process of relative motion between the target object and the camera driven by the industrial production line, each point on the surface of the target object can intersect with a certain clear imaging surface P, that is, the plane perpendicular to the clear imaging surface P is taken as the projection plane (the transport plane in this embodiment can be used as the projection plane), and the projection of the clear imaging surface P on the projection plane is approximately regarded as a projection line segment, then the motion trajectory of any point on the surface of the target object on the projection plane has an intersection with the projection line segment of the clear imaging surface P.

[0208] The target is placed on the industrial production line, and the edge of the target will not exceed the edge of the industrial production line when viewed from above. This setting can ensure that no matter what direction and position the target is placed on the industrial production line, the edge of the target will not exceed the plane of the industrial production line, that is to say: there will be no part on the surface of the target that cannot be imaged (for example, the part that exceeds the plane of the industrial production line cannot be imaged at all).

[0209] In addition, no matter it is human eye observation or device imaging, there will be the problem of front and back occlusion of the observed object itself. In the scene of shooting the glass edge image, such as Figure 6B-6C As shown in FIG. 1 , when the glass is placed in a specific position, it will be blocked by the front and back, so that it is impossible to obtain a complete image of the edge of the glass to be measured. Figure 6B As shown, in this scenario, the image acquisition subsystem cannot obtain the image of the right edge of the rectangular glass 39 to be viewed; Figure 6C In the scenario, the image acquisition subsystem cannot obtain the image of the left edge of the rectangular glass 39 to be tested.

[0210] According to multiple experiments, Fig.6D As shown, in the case where a plurality of clear imaging surfaces P form a closed area between line segments on the projection plane, as long as any point in the projection pattern of the inspected product transported on the industrial production line on the transport plane can pass through the closed area, that is, the inspected product can pass through the closed area completely during the transport process, then a complete and clear image of the inspected product can be obtained.

[0211] Multiple cameras that meet the above conditions can obtain a complete image of the glass edge without considering the direction and position of the glass on the industrial production line.

[0212] Best Camera Choices

[0213] The positions of multiple cameras are fixed, that is, the shooting angles of each camera are fixed. Because the edges of the glass may be blocked, and the glass is placed at any angle on the industrial production line, usually not every camera can fully capture the image of a certain edge of the glass.

[0214] After obtaining the glass edge image and the corresponding position information through the position acquisition subsystem, the glass edge image is processed to obtain the normal vector of any point on the glass edge image. The position pointed by the normal vector is the best shooting angle for the point.

[0215] If the glass has multiple edges, the average value of the normal vectors on each edge of the glass is calculated respectively, and the best shooting angle is determined according to the average value of the normal vectors corresponding to each edge, so as to select the best camera.

[0216] If the shape of the glass edge is a closed curve, the normal vectors at fixed intervals of the glass edge can be sorted according to the angles between them and a reference direction, and the best camera can be selected according to the angles of the normal vectors at that location.

[0217] As an optional implementation, the image of the glass edge is a closed figure, and the normal vector of each point on the closed figure is obtained. The angle range of the normal vector corresponding to each camera can be the same or different, but the sum of the angle ranges of the normal vectors corresponding to all cameras should be no less than 360°, so as to ensure that there is at least one corresponding camera at any point on any glass edge image. When collecting the glass edge image, the image collected by the camera corresponding to the normal vector at the glass edge is the optimal image.

[0218] As an optional embodiment, multiple cameras are evenly distributed around a point on the midline of the width direction of the industrial production line (this point is called the center point), and the angle formed by the line between two adjacent cameras and the center point remains consistent. It can also be described as the multiple cameras being evenly distributed on the circumference of a circle with the center point as the center. This setting makes each camera centrally symmetric with respect to the center point. In the case of n cameras, each camera selects the corresponding optimal normal vector within the range of 360° / n. This setting can make the normal vector angle range corresponding to each camera the same, that is, the workload of each camera is the same, which is beneficial to the stable operation of the image acquisition subsystem. Figure 3B As shown, as an implementation method, the industrial inspection system based on image acquisition adopts a centrally symmetrical arrangement of four cameras, which are evenly distributed on both sides of the industrial production line transport line, and the angle between the central axes of the two cameras is 90°.

[0219] The corresponding camera is selected according to the normal vector of the glass edge, and the glass edge image with the best imaging quality can be obtained while controlling the working timing of the camera to reduce the camera workload.

[0220] Glass edge defect judgment

[0221] like Figure 7A-7B The image of the glass edge is shown in the figure. The defect may take different forms for different types of glass.

[0222] As an example, in this embodiment, the edge of the glass to be tested should be covered with frosting under normal circumstances. At this time, the main types of glass edge defects include "uncovered frosting", and the defective part produces mirror reflection. The brightness or darkness of the image obtained by the camera shooting the defective part not covered with frosting depends on the angle of the mirror reflection surface and the incident direction of the light source.

[0223] If the light source, the defective part and the camera cannot form a light reflection relationship, then the non-defective part will undergo diffuse reflection, reflecting part of the light source to the camera; the defective part will undergo specular reflection, and since the reflection angle is not aligned with the camera, the light source will not enter the camera through specular reflection. Fig. 7A As shown, in this case, the visual effect of the defective part is darker than that of the non-defective part, so the unfrosted defective part of the glass appears black.

[0224] Figure 7B The image shown is another case where there is a defect on the edge of the glass to be tested. At this time, a geometric reflection relationship is formed between the light source, the defective part and the camera, and all the light from the light source is reflected into the camera. The non-defective part still undergoes diffuse reflection, reflecting part of the light from the light source to the camera. In this case, the visual effect of the defective part is brighter than that of the non-defective part. Therefore, with this type of light source, the defective part that is "not covered with frosting" may have two conditions: "high light" and "dark".

[0225] As an optional solution. When judging the defects on the glass edge, two different judgment thresholds are set, including a dark threshold and a bright threshold. When the brightness value of a certain position on the glass edge is less than the dark threshold, the position is judged to be a defect; or when the brightness value of a certain position on the glass edge is greater than the bright threshold, the position is also judged to be a defect. As another optional solution. First, calculate the average brightness value of the entire glass edge. Then subtract the brightness values ​​at different positions on the glass edge from the average brightness value, and calculate the absolute value of the subtraction result. Judge the absolute value: if the absolute value is greater than a certain threshold, the brightness of the position is too bright or too dark, that is, the position is a mirror reflection position. At this time, it is judged that there is a defect at the position. The brightness value mentioned here includes not only the brightness value in the color image, but also the grayscale value in the black and white image.

[0226] light source

[0227] When the above determination method is used to determine defects, if the difference between the brightness of the defect and the brightness of the non-defect is too small, it is difficult to select a threshold value and the determination of the defect is not accurate enough.

[0228] In order to solve this problem, as an optional implementation method, refer to Fig. 8A, a strip light source 81 is arranged above the clear imaging surface P of the camera. From a bird's-eye view, each strip light source can cover the clear imaging surface P of one or more image acquisition units in the image acquisition subsystem 32. The strip light source may include a point light source or a surface light source evenly distributed on the strip light source. With such a setting, it can be ensured that when the image acquisition subsystem acquires an image, the clear parts in the image can be illuminated by light from the light source of the same brightness and angle, so that the absolute value of the brightness difference between each defect and non-defect can be increased, thereby improving the brightness contrast between the defect and non-defect, so as to make the judgment of the defect more accurate. The strip light source can be as close to the industrial production line as possible without interfering with the movement of objects on the industrial production line to reduce the loss of brightness.

[0229] There may also be flaws where the edge of frosted glass meets the glass surface. This flaw is called "edge bursting."

[0230] As another optional implementation, the light emitted by the light source is parallel light. The parallel light covers the clear imaging surface P of the camera. The light source can be set next to the camera and on the same side of the industrial production line as the camera. It can also be set on both sides of the industrial production line. With such a light source setting, it is possible to illuminate the intersection of the glass edge and the glass surface, so that the glass "burst edge" can be detected, which increases the application scope of the online detection system.

[0231] As another optional implementation, refer to Figure 8B , the light source 82 is an annular surface light source, and the shape of the surface light source is similar to the side of a cylinder or the side of a cuboid. The light source surrounds the glass to be tested, and the inner wall of the entire light source emits uniform light. Such a light source setting method can ensure that a mirror reflection relationship can always be formed between the light emitted by the inner wall of the light source, the defective part of the glass edge, and a camera in the image acquisition subsystem. At this time, no matter in what direction the glass to be tested is placed on the industrial production line, the defective part of the edge of the glass can always reflect the light from somewhere on the light source into a camera in the image acquisition subsystem through mirror reflection, so that the visual effect of the image of the defective part of the glass edge can always be highlighted.

[0232] As an optional implementation, the light source is a ring-shaped surface light source plus a cover surface light source. Compared with the case of only including a ring-shaped light source, this light source structure can improve the detection effect of "burst edge".

[0233] When setting the light source, the lower edge of the light source should at least be consistent with the lower end of the glass edge or lower than the lower end of the glass edge to ensure the light source effect. However, at the same time, such a light source may hinder the movement of the glass on the industrial production line.

[0234] As an optional implementation, Figure 8CAs shown, the side view of the industrial production line 30 is in the shape of a trapezoid with a small top and a large bottom, and the industrial production line 30 has a highest plane during operation. The annular light source or the covered annular light source is arranged around the highest plane of the industrial production line. The position of the lower end of the annular light source 83 can be flush with the surface where the highest plane of the industrial production line 30 is located or slightly lower than the surface where the highest plane of the industrial production line is located. At the same time, the lower end of the annular light source maintains a certain distance from the surface of the industrial production line 30, and the distance is greater than the maximum thickness of the glass 39 to be tested. When the glass 39 to be tested is transported to the highest plane of the industrial production line, the annular light source or the covered annular light source surrounds the glass to be tested. Since the distance maintained between the lower end of the annular light source 83 and the industrial production line 30 is greater than the maximum thickness of the glass, it can be ensured that the position of the annular light source does not need to be changed when the glass moves on the industrial production line, and the movement of the glass will not be interfered with by the light source. It can also ensure that the lower edge of the light emitting part of the light source is flush with or lower than the lower end of the glass edge, ensuring the integrity of the angle of the incident light of the light source.

[0235] like Fig.8D As shown, as another optional embodiment, the annular light source 84 can move up and down. A telescopic rod 841 that can move up and down is provided at the upper end of the annular light source, and the telescopic rod can be driven by a cylinder, hydraulic pressure or a motor. When the glass to be tested on the industrial production line 30 moves to the bottom of the light source 84, the movement of the industrial production line stops. At this time, the telescopic rod 841 descends so that the lower edge of the light source emitting part is flush with the lower end of the glass to be tested 39. After completing the acquisition of the glass edge image, the telescopic rod 841 rises and the industrial production line resumes movement. Correspondingly, the industrial production line 30 can be driven by a servo motor, and a fixed area for placing the glass to be tested 39 is set on the industrial production line. Each time the servo motor is operated, the industrial production line 30 moves a fixed distance. Such a setting can make the light source 84 rise and fall at the same time interval, thereby simplifying the procedure and reducing the probability of error. At the same time, a pressure feedback device or a distance measuring device can be set on the light source to ensure that the light source does not crush the glass to be tested 39 when it descends.

[0236] Since the light source itself is opaque, in order to obtain the image of the glass to be tested, the image acquisition subsystem 32 usually needs to be installed between the light source and the glass to be tested. At the same time, since the camera in the image acquisition system 32 is usually opaque, the camera often blocks a part of the light from the light source. If there are multiple cameras, and two cameras form a reflection relationship with the defects on the edge of the glass, the image of the defective part of the edge of the glass will appear "dark".

[0237] In order to solve the above problem, as an optional solution, a light source is provided on the camera 321. The light source can be provided on the side of the camera lens, and the light source can be provided in the form of a surface light source, so that the angle of the emitted light can at least make up for the light of the ring light source portion blocked by the camera. The light source can also be annular and installed around the camera lens. Such a setting can further ensure that the light emitted by the light source can make up for the light blocked by the camera.

[0238] Defect Detection

[0239] After obtaining a clear image of the glass edge, the processor 33 detects defects from the image.

[0240] Including front camera defect detection method

[0241] 1. Defect detection method without image stitching

[0242] Without performing defect detection on image stitching, defect detection is performed on the clear image acquired by the image acquisition subsystem at each shooting moment to determine whether the image at a certain position of the glass edge acquired at the shooting moment has defects. At the same time, since the detection system including the front camera 312 can directly obtain the position of the clear image at the glass edge at any time, when a defect is detected at the clear image of the glass edge at any time, the spatial position of the defect can also be known. In this way, whether there is a defect on the glass edge and the position of the glass edge where the defect is located can be known at the same time.

[0243] The specific judgment process is as follows: Fig.9A shown.

[0244] In step S911, a reference value of brightness or grayscale is determined. The reference value can be determined, as an implementation, by the average brightness or grayscale of the edge of the previous glass. As another implementation, the average brightness or grayscale can be determined by flawless glass before the test.

[0245] In step S912, the absolute value of the difference between the average value of the brightness or grayscale of the current picture and the reference value is calculated.

[0246] In step S913, it is determined whether the absolute value exceeds a reasonable error range. If the absolute value of the calculated result does not exceed a reasonable error range, it is considered that there is no defect at the position of the glass edge. If the absolute value of the calculated result exceeds a reasonable error range, it is considered that there is a defect at the position of the glass edge, and the process proceeds to step S914.

[0247] In step S914, the spatial information of the glass edge corresponding to the defect image is obtained. The spatial information can be obtained by the position acquisition subsystem 31.

[0248] In step S915, feedback is provided. The feedback may be provided in the form of sound or light alarm, or may be provided to the work computer of the quality inspector through a wired connection, or may be provided to an external mobile terminal, such as a smart watch or a smart phone, through a wireless connection.

[0249] Directly judge whether the clear image obtained at any time has defects, eliminating the image stitching step, which is efficient and fast.

[0250] 2. Defect detection method after image stitching

[0251] In the defect detection method without image stitching, each detection is relatively independent. Therefore, the detection result is greatly affected by the previous steps. For example, the reference value depends on the brightness and grayscale information of the glass before the detection, which will be different under the influence of different ambient lighting. Therefore, as the natural lighting changes, the accuracy of defect detection will be affected. And the position information of the clear imaging point at any time is completely dependent on the front camera 312 and the encoder 311. If the information obtained by the front camera 312 or the encoder 311 itself is biased, it will cause the obtained position information to be biased.

[0252] As another way to detect glass edge defects, Fig. 9B As shown, the steps are as follows:

[0253] In step S921, the images are stitched to form a clear image of the complete glass edge.

[0254] In step S922, the average value of the brightness or grayscale of the complete image obtained in step S921 is calculated. When the acquired image is a color image, the average value is the brightness average value; when the acquired image is a black and white image, the average value is the grayscale average value.

[0255] In step S923, the brightness or grayscale of each pixel / pixel group in the complete image obtained in step S921 is calculated according to a specific calculation order, and compared with the average value of the brightness or grayscale obtained in step S921. The comparison method can be to calculate the absolute value of the difference between the brightness or grayscale of the pixel and the average value. It can also be to calculate the absolute value of the difference between the brightness or grayscale of the pixel and the average value divided by the average value, and divide the difference by the average value, which can effectively eliminate the influence of lighting conditions on the judgment of defective parts. If the result of comparing the brightness or grayscale of the pixel with the reference value is greater than a certain threshold, the pixel position is considered to be a defective position.

[0256] In step S924, the relative position of the defective part at the edge of the glass is obtained according to the order of calculating the defect obtained in step S923. At this time, the spatial information obtained by the position acquisition subsystem 31 can also be received and integrated with the relative position obtained previously.

[0257] In step S925, the relative position is fed back to the quality inspector. The feedback can be fed back to the quality inspector's work computer through a wired connection, or sent to an external mobile terminal, such as a smart watch or a smart phone, through a wireless connection.

[0258] Defect detection method without front camera 312

[0259] 1. Defect detection method without image stitching

[0260] Without performing defect detection on image stitching, defect detection is performed on the clear image acquired by the image acquisition subsystem at each shooting moment to determine whether the image at a certain position on the edge of the glass acquired at the shooting moment has defects. Since it is difficult to obtain the position of the clear image at any time in a defect detection method that does not include the front camera 312, such a detection method is usually used to determine whether the edge of the glass to be tested has defects.

[0261] Directly judge whether the clear image obtained at any time has defects, eliminating the image stitching step, which is efficient and fast.

[0262] 2. Defect detection method after image stitching

[0263] Defect detection after image stitching can improve detection accuracy on the one hand, and on the other hand, can obtain the position information of the defective part by obtaining the position relationship between the defective part and other parts of the object to be detected. In this way, the position of the defective part can be obtained even without the position information provided by the position acquisition system.

[0264] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A camera with a biased structure, comprising: An imaging sensor, the imaging sensor is used to sense light to convert it into a corresponding image signal, and the imaging sensor is provided with a sensor plane, the sensor plane is used to receive light; Features: The camera with the biased configuration also includes: A housing for accommodating the imaging sensor; A lens is used to transmit light to the imaging sensor; The housing accommodates the imaging sensor, and the lens is mounted on the housing; the sensor plane of the imaging sensor faces the lens; the main optical axis of the lens intersects the sensor plane at an angle, and the value range of the angle formed by the intersection is 25 degrees to 85 degrees; The camera with the biased configuration also includes: a mounting shaft rotatably mounted to the housing, the mounting shaft being fixedly connected to the imaging sensor and the mounting shaft partially exposed from the upper portion of the housing, The mounting shaft is operable to change the tilt angle of the imaging sensor; A rotating dial button, the rotating dial button is fixedly mounted to the portion of the mounting shaft exposed from the housing; An angle code disk, used to indicate the rotation angle of the rotary dial, thereby indicating the tilt angle of the imaging sensor; When the imaging sensor is tilted, the focal length range of the sensor is expressed by the following formula: [(u+sin(α)*x) / (u+sin(α)*x-1),u / (u-1)], where u is the image distance, α is the rotation angle of the imaging sensor, x is the normalized size of the focal length of the imaging sensor in the radial direction of the rotation axis (x=X / f, where X is the actual size), and sin(α)*x is the projection distance of the sensor in the optical axis direction. That is, for given x and u, the larger α is, the smaller (u+sin(α)*x) / (u+sin(α)*x-1) is, and the larger the depth of field range is.

2. The camera with an offset structure according to claim 1, characterized in that: The length direction of the shell is parallel to the main optical axis of the lens.

3. The camera with an offset structure according to claim 1, characterized in that: The length direction of the shell is perpendicular to the sensor plane.

4. The camera with an offset structure according to claim 1, characterized in that: The sensor plane is circular.

5. The camera with an offset structure according to claim 1, characterized in that: The sensor plane is rectangular.

6. The camera with an offset structure according to claim 1, characterized in that: The lens is a convex lens.

7. The camera with an offset structure according to claim 1, characterized in that: The ratio of the longest line segment in the sensor plane to the length of the housing is in a range of 0.1 to 0.

3.

8. The camera with an offset structure according to claim 1, characterized in that: The ratio of the area of ​​the projection of the lens on a projection plane perpendicular to the optical axis thereof to the area of ​​the sensor plane on the projection plane ranges from 1.3 to 2.1.