A specular, quasi-specular object surface image acquisition system

Abstract

The application discloses a mirror surface, a kind of mirror surface object surface image acquisition system, including visual sensor being installed in detection station, detection station is isolated natural light and presents dark state;Visual sensor includes light source and camera, light source is strip or arc and is used to produce uniform strip light band on the surface of the object to be measured;Camera acquires strip light band;Translation visual sensor or object to be measured, make camera and object to be measured occur relative position change, camera obtains multiple images;Controller is according to the acquisition sequence of multiple images and the position change of region of interest under pixel coordinate system, sequentially splices the multiple images collected, completes image acquisition.The system cooperates with camera through strip light source, makes relative motion between visual sensor and object to be measured, completes scanning image acquisition on the surface of the object to be measured, realizes the acquisition of complete image on the surface of the object to be measured through image splicing, with the characteristics of convenient installation, low equipment cost and high detection beat.

Description

Technical Field

[0001] This invention relates to the field of visual inspection, and more specifically to an image acquisition system for the surface of mirrored or mirror-like objects. Background Technology

[0002] Existing visual inspection methods for mirror and near-mirror surfaces typically use an external light source for illumination, with a camera capturing images of the light reflected from the surface under test. The morphology of the object is obtained by analyzing the degree of distortion of the light source modulated by the surface under test in the image. However, due to the characteristic that the specular reflection of the surface under test is higher than that of diffuse reflection, the captured image cannot directly capture the true surface under test; instead, it is submerged in the surrounding environment of its reflection, making it difficult to identify the true morphology of the surface under test from the image. In particular, when the surface to be measured is large, such as the painted surface of a car body, multiple cameras are usually required, each capturing multiple images. For example, patent document CN110146507 discloses a system and method for detecting appearance defects on the surface of a car paintwork. In this scheme, multiple cameras are arranged around the front, rear, left, and right sides of the car body. In order to capture information about the entire car body surface, the number of cameras can be as high as thirty or forty, and the total number of images captured can reach tens of thousands. In addition, many light sources are also set up in parallel, resulting in high equipment costs and low detection efficiency. Due to the large size of the light sources, this scheme requires moving the light sources, which places many requirements on the measurement site. At the same time, although this scheme uses multiple cameras and captures multiple images, it does not check whether the field of view of multiple cameras has covered the entire object being measured, which can easily lead to missed detections. In order to avoid missed detections, simply increasing the number of cameras will also cause system redundancy, increase the number of image processing operations, and increase equipment costs.

[0003] In visual measurement applications, the surface of the object to be measured is usually not a smooth plane / curved surface, but has surface features such as edges and grooves. When the pose of the camera and light source is not set properly, these features are easy to miss. After the visual sensor is installed, it is very important to verify whether the area to be measured has been fully collected and without any omissions. The existing verification process is mostly manual, which is inefficient and the specular reflection can easily cause eye fatigue, resulting in a poor user experience and easy misjudgment. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an image acquisition system for mirror and mirror-like object surfaces. This system utilizes a bar light source and a camera to create relative motion between the visual sensor and the object under test, thereby completing the scanning image acquisition of the object's surface. Through image stitching, a complete image of the object's surface is obtained. This system features convenient installation, low equipment cost, and high detection cycle time.

[0005] Therefore, the technical solution of the present invention is as follows:

[0006] A system for acquiring images of mirrored or mirror-like object surfaces includes a vision sensor installed in a detection station, which is isolated from natural light and presents a dark state.

[0007] The visual sensor includes a light source and a camera with fixed relative positions, both of which are controlled by a controller;

[0008] The light source is strip-shaped or arc-shaped, and is used to generate a uniform strip-shaped light band on the surface of the object to be tested, which is a mirror or a mirror-like surface. The reflected light from the strip-shaped light band on the surface of the object to be tested can meet the imaging requirements of the camera.

[0009] The camera is used to capture strip-shaped light bands. In the captured image, the extension direction of the strip-shaped light bands is approximately parallel to any coordinate axis of the pixel coordinate system. The region of the strip-shaped light bands in the image is called the region of interest.

[0010] The controller is installed inside or outside the testing station;

[0011] When the object to be tested enters the detection station, the visual sensor or the object to be tested is moved to cause a change in the relative position between the camera and the object to be tested. The amount of displacement change in a single instance is recorded as L. L < 1 / 2 of the short side width of the strip light source. After multiple relative position changes, the strip light band appears in the detection area of ​​all objects to be tested.

[0012] The camera and the object under test change relative to each other, and the camera captures an image. This process is repeated to acquire multiple images.

[0013] The controller stitches together the acquired images sequentially according to the acquisition order of the multiple images and the positional change of the region of interest in the pixel coordinate system, thus completing the acquisition of images of mirror and mirror-like object surfaces.

[0014] Furthermore, the method for sequentially stitching multiple images is as follows:

[0015] The coordinate axis in the pixel coordinate system that makes a larger angle with the extension direction of the region of interest is denoted as the feature axis;

[0016] When the object under test is translated and the vision sensor remains stationary, it is determined whether the direction of the feature axis is consistent with the translation direction of the object under test. If they are consistent, the opposite direction of the feature axis is recorded as the splicing direction; if they are inconsistent, the direction of the feature axis is recorded as the splicing direction.

[0017] When the vision sensor moves while the object being measured remains stationary, it is determined whether the direction of the feature axis is consistent with the direction of the vision sensor's translation. If they are consistent, the direction of the feature axis is recorded as the stitching direction; if they are inconsistent, the opposite direction of the feature axis is recorded as the stitching direction.

[0018] Keeping the gray values ​​of each pixel in the first image unchanged, the coordinates of each pixel in the i-th image are translated by (i-1)×D pixels along the splicing direction. The translated pixels and their gray values ​​are spliced ​​to the first image, and the gray values ​​of pixels with the same coordinates are accumulated to form the spliced ​​image.

[0019] Where i = 2, 3, 4...N, and N is the total number of images captured by the camera;

[0020] P is the pixel equivalent of the camera at the current working distance.

[0021] Preferably, the short side width of the strip light source is ≤ ×K×P, where K is the total number of pixels in the image along the feature axis.

[0022] To facilitate the identification of target features in the image, preferably, the gray values ​​of each pixel (u, v) in the stitched image are analyzed using the following method. Normalization processing yields the normalized grayscale values ​​of each pixel. :

[0023]

[0024] Where min and max represent the minimum and maximum gray values ​​in the stitched image, respectively.

[0025] As one application of the image acquisition system in this system:

[0026] Using the stitched image, determine whether there are any missed detection areas in the image acquired by the image acquisition system:

[0027] Check if there are any regions in the stitched image with grayscale values ​​lower than a preset value:

[0028] If such a region exists, and the location of the region is not a hole, then it is considered that there is a missed area in the image acquired by the image acquisition system. It is necessary to add a camera and / or a strip light source, or adjust the installation angle and position of the camera and / or the strip light source.

[0029] If no such area exists, it is assumed that there are no missed areas in the image acquired by the image acquisition system, and the current image acquisition system does not need to be adjusted.

[0030] Furthermore, threshold segmentation and feature recognition are performed on the normalized image. Based on the identified features, it is determined whether there are any missed detection areas, or whether there are any defects on the surface of the object to be tested.

[0031] In order to maximize the utilization of the strip light source by extending the strip light area through the image, preferably, the angle between the long side mounting direction of the strip light source and the translation direction of the vision sensor / object under test is within the range of 90°±30°.

[0032] The angle between the extension direction of the region of interest and the u-axis or v-axis of the pixel coordinate system is less than 30°.

[0033] Furthermore, a robot or linear slide is installed in the inspection station; the robot / linear slide is used to translate the vision sensor.

[0034] Alternatively, a roller bed can be installed in the testing station to move the object to be tested.

[0035] Furthermore, when the object to be measured is large or there are multiple areas to be measured, multiple vision sensors are set up. The multiple vision sensors are distributed on a cutting plane, which is a spatial plane perpendicular to the translation direction of the vision sensor / object to be measured.

[0036] To reduce the impact of noise points at the edge of the striped light band on image stitching, the preferred value of L is: The width of the short side of the strip light source.

[0037] The present invention has the following beneficial effects:

[0038] This method enables simultaneous detection while the object under test is moving, without requiring the object to stop or disrupting the production line cycle; it is time-efficient and highly effective.

[0039] The images stitched by this method can highlight the surface features of the object under test, thus allowing for a more intuitive observation of the features within the test area, which facilitates improved accuracy in subsequent feature recognition and defect detection. In addition, the images acquired and stitched by this system can demonstrate the effective detection area of ​​each camera, thereby quickly identifying whether there are any missed detections and adjusting the installation angles of the cameras and light sources in a timely manner to avoid missed detections. Attached Figure Description

[0040] Figure 1 This refers to a single image captured by the camera in a specific implementation method;

[0041] Figure 2 In this specific implementation, N images are captured by the camera.

[0042] Figure 3 a is the image obtained by stitching together N images captured by camera one in a specific implementation;

[0043] Figure 3 b is the image obtained by stitching together N images captured by camera two in a specific implementation;

[0044] Figure 3 c represents the image obtained by stitching together N images captured by camera three in a specific implementation method;

[0045] Figure 3 d represents the image obtained by stitching together N images captured by camera four in a specific implementation embodiment;

[0046] Figure 3 e represents the image obtained by stitching together N images captured by camera five in a specific implementation;

[0047] Figure 4 This is a schematic diagram showing the translation direction of the object under test and the direction of the feature axis in a specific implementation method;

[0048] Figure 5 This is a schematic diagram of the overall structure of the image acquisition system in a specific implementation embodiment;

[0049] Figure 6 This is a schematic diagram of the image stitching process in a specific implementation method. Detailed Implementation

[0050] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0051] A system for acquiring images of mirrored or mirror-like object surfaces includes a vision sensor installed in a detection station, which is isolated from natural light and is in a dark state.

[0052] A vision sensor consists of a light source and a camera that are fixed in relative positions, both of which are controlled by a controller;

[0053] The light source is strip-shaped or arc-shaped, and is used to generate a uniform strip-shaped light band on the surface of the object to be measured, which is a mirror or a mirror-like surface. The reflected light from the strip-shaped light band on the surface of the object to be measured can meet the imaging requirements of the camera.

[0054] The camera is used to capture stripe-shaped light bands. In the captured image, the extension direction of the stripe-shaped light band is approximately parallel to any coordinate axis (u-axis or v-axis) of the pixel coordinate system. The region of the stripe-shaped light band in the image is called the region of interest. The gray value of the pixels in the region of interest is higher than the gray value of the surrounding pixels.

[0055] The controller can be installed inside or outside the testing station;

[0056] When the object to be tested enters the detection station, the visual sensor or the object to be tested is moved to cause a change in the relative position between the camera and the object to be tested. The amount of displacement change in a single instance is recorded as L. L < 1 / 2 of the short side width of the strip light source. After multiple relative position changes, the strip light band appears in the detection area of ​​all objects to be tested.

[0057] The camera and the object under test change relative to each other, and the camera captures an image. This process is repeated to acquire multiple images.

[0058] That is: at each interval L, the camera acquires one image. As the relative positions of the camera and the object under test continuously change, the camera continuously acquires images, thus obtaining multiple images (e.g., Figure 2 (as shown)

[0059] The controller stitches together the acquired images sequentially according to the acquisition order of the multiple images and the positional change of the region of interest in the pixel coordinate system, thus completing the acquisition of images of mirror and mirror-like object surfaces.

[0060] The image captured by the reflection camera is a virtual image of the light band. The effective area of ​​the light band projected onto the surface of the object under test will be halved. In order to balance image stitching accuracy and efficiency, the preferred implementation is as follows:

[0061] The value of L is: The width of the short side of the strip light source.

[0062] Specifically, the method for sequentially stitching multiple images is as follows:

[0063] The coordinate axis in the pixel coordinate system that makes a larger angle with the extension direction of the region of interest is denoted as the feature axis;

[0064] like Figure 4 As shown, when the object to be measured is translated and the vision sensor remains stationary, it is determined whether the direction of the feature axis is consistent with the translation direction of the object to be measured. If they are consistent, the opposite direction of the feature axis is recorded as the splicing direction; if they are inconsistent, the direction of the feature axis is recorded as the splicing direction.

[0065] When the vision sensor moves while the object being measured remains stationary, it is determined whether the direction of the feature axis is consistent with the direction of the vision sensor's translation. If they are consistent, the direction of the feature axis is recorded as the stitching direction; if they are inconsistent, the opposite direction of the feature axis is recorded as the stitching direction.

[0066] like Figure 6 As shown, keeping the gray values ​​of each pixel in the first image unchanged, the coordinates of each pixel in the i-th image are translated by (i-1)×D pixels along the splicing direction. The translated pixels and their gray values ​​are spliced ​​to the first image, and the gray values ​​of pixels with the same coordinates are accumulated to form the spliced ​​image.

[0067] Where i = 2, 3, 4...N, and N is the total number of images captured by the camera;

[0068] P is the pixel equivalent of the camera at the current working distance.

[0069] When selecting a light source, the shorter side width of the strip light source should be ≤ ×K×P, where K is the total number of pixels in the image along the feature axis.

[0070] Since image stitching involves accumulating the gray values ​​of pixels with the same coordinates, meaning that the gray value of some pixels may exceed 255, a preferred implementation method is adopted to facilitate observation and identification of target features in the image:

[0071] The gray values ​​of each pixel (u, v) in the stitched image Normalization processing yields the normalized grayscale values ​​of each pixel. :

[0072]

[0073] Where min and max represent the minimum and maximum gray values ​​in the stitched image, respectively.

[0074] As one application of the image acquisition system in this system:

[0075] Using the stitched image, determine whether there are any missed detection areas in the image acquired by the image acquisition system:

[0076] Check if there are any regions in the stitched image with grayscale values ​​lower than a preset value:

[0077] If such a region exists, and the location of the region is not a hole, then it is considered that there is a missed area in the image acquired by the image acquisition system. It is necessary to add a camera and / or a strip light source, or adjust the installation angle and position of the camera and / or the strip light source.

[0078] If no such area exists, it is assumed that there are no missed areas in the image acquired by the image acquisition system, and the current image acquisition system does not need to be adjusted.

[0079] More specifically, threshold segmentation and feature recognition are performed on the normalized image, and the identified features are used to determine whether there are any missed detection areas, or to determine whether there are any defects on the surface of the object under test.

[0080] In order to maximize the utilization of the strip light source by ensuring that the strip light area extends through the image as much as possible, the angle between the long side of the strip light source and the translation direction of the vision sensor / object under test should be within the range of 90°±30° during equipment installation.

[0081] The extension direction of the strip light band is approximately parallel to any coordinate axis (u-axis or v-axis) of the pixel coordinate system. In practice, the angle between the extension direction of the region of interest (strip light band) and the u-axis or v-axis of the pixel coordinate system is less than 30°.

[0082] The relative positional changes between the camera and the object under test are as follows:

[0083] Install a robot or linear slide in the inspection station; the robot / linear slide is used to translate the vision sensor.

[0084] Alternatively, a roller bed can be installed in the testing station to move the object to be tested.

[0085] When the object to be measured is large or there are multiple areas to be measured, multiple vision sensors are set up and distributed on the cutting plane, which is a spatial plane perpendicular to the translation direction of the vision sensor / object to be measured.

[0086] like Figure 5 As shown, taking vehicle body inspection as an example, the following is an illustrative explanation:

[0087] Multiple vision sensors are mounted on a gantry, below which a roller bed is installed to move the object under test (a car body). An encoder is mounted on the motor rotor to record the distance the roller bed travels. For every L movement of the roller bed, multiple cameras simultaneously acquire a strip-shaped light band image. The roller bed moves continuously, and the multiple cameras acquire images continuously. Once the test area of ​​the car body (such as the roof, door, or the entire body) has completely passed the inspection station, the images acquired by each camera are stitched together, such as... Figure 3 a~ Figure 3 e represents the vehicle body surface image obtained by simultaneously acquiring images from five cameras and then stitching them together. During installation, the five cameras are distributed on the cutting plane, acquiring images of different parts of the vehicle body respectively. The five stitched images can reflect the overall appearance of the roof.

[0088] based on Figure 3 a~ Figure 3 The stitched image in e (already normalized) is used to determine whether there are any missed detection areas in the images acquired by the image acquisition system:

[0089] Check if there are any regions in the stitched image with grayscale values ​​lower than a preset value:

[0090] If such a region exists, and the location of the region is not a hole, then it is considered that there is a missed area in the image acquired by the image acquisition system. It is necessary to add a camera and / or a strip light source, or adjust the installation angle and position of the camera and / or the strip light source.

[0091] If no such area exists, it is assumed that there are no missed areas in the image acquired by the image acquisition system, and the current image acquisition system does not need to be adjusted.

[0092] Determine whether there are any missed detection areas for cameras one through five. If so, adjust the installation angle and position of the corresponding camera and light source. If not, the current system position does not need to be adjusted and can be used directly for actual detection.

[0093] This method can replace manual screening of missed areas, assist in the installation of cameras and light sources, verify whether there are any missed areas, and adjust the installation positions of cameras and light sources in a timely manner.

[0094] For ease of interpretation and precise definition of the appended claims, the terms “upper,” “lower,” “inner,” and “outer” are used to describe features of exemplary embodiments with reference to the positions of these features shown in the accompanying drawings.

[0095] The foregoing description of specific exemplary embodiments of the present invention is for illustrative and descriptive purposes. It is not intended to be exhaustive, nor to limit the invention to the precise forms disclosed; obviously, many changes and variations are possible in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to implement and utilize various exemplary embodiments of the invention, as well as their different alternatives and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A system for acquiring images of mirrored or mirror-like object surfaces, characterized in that: This includes a vision sensor installed in a testing station that is isolated from natural light and is in a dark state; The visual sensor includes a bar light source and a camera with fixed relative positions, both of which are controlled by a controller; The strip light source is used to generate a uniform strip light band on the surface of the object to be tested, which is a mirror or a mirror-like surface. The reflected light from the strip light band on the surface of the object to be tested can meet the imaging requirements of the camera. The camera is used to capture strip-shaped light bands. In the captured image, the extension direction of the strip-shaped light bands is approximately parallel to any coordinate axis of the pixel coordinate system. The region of the strip-shaped light bands in the image is called the region of interest. The controller is installed inside or outside the testing station; When the object to be tested enters the detection station, the visual sensor or the object to be tested is moved to cause a change in the relative position between the camera and the object to be tested. The amount of displacement change in a single instance is recorded as L. L < 1 / 2 of the short side width of the strip light source. After multiple relative position changes, the strip light band appears in the detection area of ​​all objects to be tested. The camera and the object under test change relative to each other, and the camera captures an image. This process is repeated to acquire multiple images. The controller stitches together the acquired images sequentially according to the acquisition order of multiple images and the positional change of the region of interest in the pixel coordinate system to complete the acquisition of images of mirror and mirror-like object surfaces. The method for sequentially stitching multiple images is as follows: The coordinate axis in the pixel coordinate system that makes a larger angle with the extension direction of the region of interest is denoted as the feature axis; When the object under test is translated and the vision sensor remains stationary, it is determined whether the direction of the feature axis is consistent with the translation direction of the object under test. If they are consistent, the opposite direction of the feature axis is recorded as the splicing direction; if they are inconsistent, the direction of the feature axis is recorded as the splicing direction. When the vision sensor moves while the object under test remains stationary, it is determined whether the direction of the feature axis is consistent with the direction of the vision sensor's translation. If they are consistent, the direction of the feature axis is recorded as the stitching direction; if they are inconsistent, the opposite direction of the feature axis is recorded as the stitching direction. Keeping the gray values ​​of each pixel in the first image unchanged, the coordinates of each pixel in the i-th image are translated by (i-1)×D pixels along the splicing direction. The translated pixels and their gray values ​​are spliced ​​to the first image, and the gray values ​​of pixels with the same coordinates are accumulated to form the spliced ​​image. Where i = 2, 3, 4...N, and N is the total number of images captured by the camera; P is the pixel equivalent of the camera at the current working distance; Using the stitched image, determine whether there are any missed detection areas in the image acquired by the image acquisition system: Check if there are any regions in the stitched image with grayscale values ​​lower than a preset value: If such a region exists, and the location of the region is not a hole, then it is considered that there is a missed area in the image acquired by the image acquisition system. It is necessary to add a camera and / or a strip light source, or adjust the installation angle and position of the camera and / or the strip light source. If no such region exists, it is assumed that there are no missed areas in the image acquired by the image acquisition system, and the current image acquisition system does not require adjustment.

2. The system as described in claim 1, characterized in that: The short side width of the strip light source is ≤ ×K×P, where K is the total number of pixels in the image along the feature axis.

3. The system as described in claim 1, characterized in that: The following method is used to analyze the grayscale values ​​of each pixel (u, v) in the stitched image. Normalization processing yields the normalized grayscale values ​​of each pixel. : Where min and max represent the minimum and maximum gray values ​​in the stitched image, respectively.

4. The system as described in claim 3, characterized in that: Threshold segmentation and feature recognition are performed on the normalized image. Based on the identified features, it is determined whether there are any missed detection areas, or whether there are any defects on the surface of the object to be tested.

5. The system as described in claim 1, characterized in that: The angle between the long side mounting direction of the strip light source and the translation direction of the vision sensor / object under test is within the range of 90°±30°; The angle between the extension direction of the region of interest and the u-axis or v-axis of the pixel coordinate system is less than 30°.

6. The system as described in claim 1, characterized in that: A robot or linear slide is installed in the inspection station; the robot / linear slide is used to translate the vision sensor. Alternatively, a roller bed can be installed in the testing station to move the object to be tested.

7. The system as described in claim 1, characterized in that: When the object to be measured is large or there are multiple areas to be measured, multiple vision sensors are set up. The multiple vision sensors are distributed on a cutting plane, which is a spatial plane perpendicular to the translation direction of the vision sensor / object to be measured.

8. The system as described in claim 1, characterized in that: The value of L is: The width of the short side of the strip light source.

Images