Method and apparatus for focusing an industrial camera

Abstract

A method (200) for focusing an industrial camera (102) fixed on a mobile robot (101) includes: capturing a first target image of a target object (105) with a first step length mobile robot (101) (S201); determining a first ROI image (S202); evaluating the first ROI image to generate a first plurality of sharpness values ​​(S203); determining a single-peak search direction based on the first plurality of sharpness values ​​(S204); capturing a second target image of the target object (105) with a second step length mobile robot (101) according to the single-peak search direction (S205); determining a second ROI image (S206); evaluating the second ROI image to generate a second plurality of sharpness values ​​(S207); and estimating the sharpest focus position based on a portion of the first plurality of sharpness values ​​and a portion of the second plurality of sharpness values ​​(S208). It enables fast and accurate autofocus of industrial cameras (102) without human intervention, effectively improving on-site work efficiency. It is also flexible and applicable to various industrial cameras (102) and lenses, thereby effectively reducing on-site costs.

Description

Technical Field

[0001] This disclosure relates to the field of machine vision, and more specifically, to methods, apparatus, computing devices, computer-readable storage media, and program products for achieving focusing of industrial cameras. Background Technology

[0002] Robots are increasingly used in industry, medicine, and other fields, but most are used in relatively simple applications. There is a growing demand for robots with more intelligence, such as vision, to adapt to uncertainty and changes in the working environment. This type of system is called a hand-eye system. Based on the mounting location of the camera and the robot, hand-eye systems are divided into eye-in-hand systems and eye-to-hand systems. In an eye-in-hand system, the camera is mounted on the robot's end effector and can move with the robot, while in an eye-to-hand system, the camera is mounted outside the robot and does not move with it during robot movement.

[0003] When a hand-eye system senses environmental information, acquiring a clear image is crucial. If the image captured by the camera is blurry due to out-of-focus objects, it may lead to misinterpretations and incorrect image analysis. Therefore, fast and accurate autofocus technology is particularly important in machine vision systems. In the field of machine vision, industrial cameras and lenses are independent, and different cameras and lenses can be selected and matched for different scenarios based on requirements such as field of view and resolution, typically using fixed-focus lenses. Currently, hand-eye systems mainly rely on manual focusing of the camera and lens, depending on the human's subjective judgment of image sharpness to determine whether the image is in focus. This method is inefficient and complex.

[0004] Existing autofocus technologies used in ordinary cameras have various shortcomings in the field of machine vision. For example, they are not accurate enough; the focus area is difficult to select; the focus stability is poor and is susceptible to environmental factors (e.g., dust and vibration in the environment); and the focusing speed is low, which cannot meet the requirements of machine vision systems (e.g., the focusing time is too long, which cannot ensure the completion of industrial tasks). Summary of the Invention

[0005] Current focusing methods for industrial cameras require manual focusing for various application environments. However, manual focusing is inefficient, complex, and highly dependent on the operator's subjective judgment and experience. Besides the various shortcomings of existing autofocus technologies for conventional cameras in machine vision, industrial cameras lack focusing motors (e.g., voice coil motors) compared to conventional cameras. This makes focusing methods for conventional cameras unsuitable for industrial cameras, and they struggle to adapt to environmental influences (e.g., dust and vibration) and meet the requirements of machine vision systems (e.g., excessively long focusing times that cannot guarantee the completion of industrial tasks).

[0006] A first embodiment of this disclosure provides a method for focusing an industrial camera, the camera being fixed on a movable robot and having a fixed-focus lens. The method includes the following steps: S1, moving the robot by a first step length to capture first target images of a target object at first plurality of locations using the industrial camera; S2, determining a first ROI image in the first target image corresponding to a ROI in the focus area of ​​the industrial camera based on a reference pattern; S3, evaluating the first ROI images determined at the first plurality of locations to generate first plurality of sharpness values; S4, determining a single-peak search direction based on the first plurality of sharpness values; S5, moving the robot by a second step length according to the single-peak search direction to capture second target images of the target object at second plurality of locations using the industrial camera, the second step length being less than the first step length; S6, determining a second ROI image in the second target image corresponding to a ROI in the focus area of ​​the industrial camera based on the reference pattern; S7, evaluating the second ROI images determined at the second plurality of locations to generate second plurality of sharpness values; S8, estimating the sharpest focus position based on a portion of the first plurality of sharpness values ​​and a portion of the second plurality of sharpness values.

[0007] In this embodiment, the industrial camera can achieve fast and accurate autofocus with high focusing accuracy, easy selection of the focusing area, high focusing stability, and is not affected by environmental factors. It does not require manual intervention, effectively improving on-site work efficiency. Furthermore, it can be flexibly applied to various industrial cameras and lenses, thereby effectively reducing on-site costs.

[0008] A second embodiment of this disclosure provides a computing device, the computing device including: a processor; and a memory for storing computer-executable instructions, which, when executed, cause the processor to perform the following steps: S1, moving the robot by a first step length to capture first target images of a target object using the industrial camera at first plurality of locations; S2, determining a first ROI image in the first target image corresponding to a ROI in the focus area of ​​the industrial camera based on a reference pattern; S3, evaluating the first ROI images determined at the first plurality of locations to generate first plurality of sharpness values; S4, based on the first... S5. Based on the single-peak search direction, the robot is moved with a second step size according to the single-peak search direction to capture second target images of the target object at a second plurality of locations using the industrial camera, the second step size being smaller than the first step size; S6. Based on the reference pattern, a second ROI image corresponding to the ROI in the focus area of ​​the industrial camera is determined in the second target image; S7. The second ROI images determined at the second plurality of locations are evaluated to generate a second plurality of sharpness values; S8. Based on a portion of the first plurality of sharpness values ​​and a portion of the second plurality of sharpness values, the sharpest focus position is estimated.

[0009] A third embodiment of this disclosure provides an apparatus for focusing an industrial camera, the industrial camera being fixed on a movable robot and having a fixed-focus lens. The apparatus includes: an image capture unit configured to move the robot by a step length and acquire first target images captured by the industrial camera at first plurality of locations for a target object; an image determination unit configured to determine a first ROI image in the first target image corresponding to a ROI in the focusing area of ​​the industrial camera based on a reference pattern; an image evaluation unit configured to evaluate the first ROI images determined at the first plurality of locations to generate first plurality of sharpness values; and a direction determination unit configured to determine a single-peak search direction based on the first plurality of sharpness values; wherein, when the direction unit determines... After the single-peak search direction: the image capture unit is further configured to: move the robot with a second step size according to the single-peak search direction, and acquire second target images captured by the industrial camera at a second plurality of positions for the target object, wherein the second step size is less than the first step size; the image determination unit is further configured to: determine a second ROI image in the second target image corresponding to the ROI in the focus area of ​​the industrial camera according to the reference pattern; the image evaluation unit is further configured to evaluate the second ROI images determined at the second plurality of positions to generate a second plurality of sharpness values; and the position estimation unit is configured to estimate the sharpest focus position based on a portion of the first plurality of sharpness values ​​and a portion of the second plurality of sharpness values.

[0010] A fourth embodiment of this disclosure provides a computer-readable storage medium having computer-executable instructions stored thereon for performing the method described in the first embodiment.

[0011] A fifth embodiment of this disclosure provides a computer program product tangibly stored on a computer-readable storage medium and including computer-executable instructions that, when executed, cause at least one processor to perform the method described in the first embodiment. Attached Figure Description

[0012] Features, advantages, and other aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description, in which several embodiments of the disclosure are illustrated by way of example and not limitation, in the drawings:

[0013] Figure 1 Exemplary scenarios in which embodiments of this disclosure can be applied are shown.

[0014] Figure 2 A flowchart of a method for focusing an industrial camera according to an embodiment of the present disclosure is shown.

[0015] Figure 3 An exemplary reference pattern for focusing an industrial camera according to an embodiment of the present disclosure is shown.

[0016] Figure 4 Another exemplary reference pattern for focusing an industrial camera according to an embodiment of the present disclosure is shown.

[0017] Figure 5 A schematic diagram of a focus evaluation curve according to an embodiment of the present disclosure is shown.

[0018] Figure 6 An exemplary system for focusing an industrial camera is shown according to an embodiment of the present disclosure.

[0019] Figure 7 A block diagram of an exemplary apparatus for implementing focusing of an industrial camera according to embodiments of the present disclosure is shown.

[0020] Figure 8 A block diagram of an exemplary computing device for implementing focusing of an industrial camera according to an embodiment of the present disclosure is shown. Detailed Implementation

[0021] Various exemplary embodiments of this disclosure are described in detail below with reference to the accompanying drawings. While the exemplary methods and apparatuses described below include software and / or firmware executed on hardware among other components, it should be noted that these examples are merely illustrative and should not be considered limiting. For example, it is conceivable that any or all hardware, software, and firmware components may be implemented exclusively in hardware, exclusively in software, or in any combination of hardware and software. Therefore, although exemplary methods and apparatuses have been described below, those skilled in the art will readily understand that the examples provided are not intended to limit the ways in which these methods and apparatuses may be implemented.

[0022] Furthermore, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of the methods and systems according to various embodiments of this disclosure. It should be noted that the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0023] The terms “comprising,” “including,” and similar terms as used herein are open-ended, meaning “including / including but not limited to,” implying that other content may also be included. The term “based on” means “at least partially based on.” The term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment,” and so on.

[0024] Figure 1An exemplary scenario 100 in which embodiments of the present disclosure can be applied is shown. Scenario 100 includes a robot 101 and an industrial camera 102 fixed to the robot 101. For example, the robot 101 may be a multi-jointed manipulator or a multi-degree-of-freedom machine device for industrial applications. The robot 101 includes a movable end effector 103, and the camera 102 may be fixed to the end effector 103 and move with the end effector 103 of the robot 101. The camera 102 carries a fixed-focus lens. The end effector 103 of the robot 101 may also be equipped with a tool 104 for manipulating actual objects in scenario 100. Scenario 100 also includes a target object 105. The target object 105 may be an actual object to be manipulated, with its surface having a specific reference pattern (e.g., by attaching a label with a reference pattern to a flat surface portion of the object) for focusing by the camera 102. Alternatively, the target object 105 may be a reference object for the camera 102 to observe the actual object. The reference object is placed at the position where the actual object is to be processed and has the same appearance as the actual object (e.g., the same height) and has a specific reference pattern on its surface for the camera 102 to focus on. After focusing on the reference object, the reference object can be replaced with the actual object.

[0025] As an example and not a limitation, robot 101 could also be, for example, an AGV (Automated Guided Vehicle) transport robot. When industrial camera 102 moves toward target object 105 along with robot 101 or its end effector 103 in a certain direction (e.g., perpendicular to the surface of target object 105), the working distance WD (i.e., the distance from the lowest mechanical surface of the lens to the object) and field of view of camera 102 also change accordingly. (See the following...) Figures 2-5 The described focusing method for industrial cameras allows adjustment of the working distance WD of the lens of camera 102 (e.g., driven by an actuator such as a robot) to bring camera 102 into a focusing position.

[0026] Figure 2 A flowchart of a method 200 for focusing an industrial camera according to an embodiment of the present disclosure is shown, wherein the industrial camera is fixed to a movable robot and has a fixed-focus lens. Method 200 can be applied to, for example... Figure 1 The exemplary scenario 100 shown includes a robot 101 and an industrial camera 102, and as shown in the example scenario 100. Figure 6 The exemplary system 600 is shown. For example, method 200 can be derived from... Figure 6The exemplary system 600 is implemented in a computing device 603 that is communicatively coupled to a robot 601 and an industrial camera 602, rather than by the industrial camera itself. Therefore, it does not occupy the camera's storage space and does not introduce camera instability, thus making it flexible to be used in various combinations of cameras and lenses.

[0027] refer to Figure 2 Method 200 begins with step 201. In step 201, a first-step mobile robot is used to capture first target images of a target object at first plurality of locations using an industrial camera. For example, camera 102 can be moved at a first rate (e.g., via a mobile end effector 103) to the first plurality of locations (e.g., from away from the target object to closer to the target object, or vice versa) by sending a movement command to robot 101, and camera 102 can capture first target images of target object 105 at the first plurality of locations. Camera 102 may include an image sensor (e.g., CCD / CMOS) for capturing images and a storage area or buffer for storing images.

[0028] Next, method 200 proceeds to step 202. In step 202, a first ROI image corresponding to the ROI (region of interest) in the focus area of ​​the industrial camera is determined based on a reference pattern. For example, the target object 105 may have a reference pattern. When the mobile robot 101, i.e., the moving camera 102, moves to different positions, a portion or all of the reference pattern can be selected as the focus window or area to capture the image of the target object 105, so that a portion or all of the reference pattern falls within the ROI. After capturing the image, the first ROI image corresponding to the ROI can be determined based on the reference pattern. For example, the reference pattern may include pattern features that are easily recognizable by the camera to locate the target object in the camera's field of view, while avoiding interference from the background, for example. Unlike the use of reference patterns (e.g., checkerboard texture patterns, etc.) to obtain the camera's intrinsic and extrinsic parameters during camera calibration, the embodiments of this disclosure use reference patterns to achieve camera focusing.

[0029] In some embodiments, the reference pattern includes a group of patterns arranged in a first direction, each group of patterns including a first region and a second region, the first region having a first color and the second region having a second color different from the first color. Because the first and second regions have different colors, abrupt changes occur between the regions, thereby providing rich image edge information while advantageously reducing, for example, the impact of noise on the image.

[0030] Figure 3An exemplary reference pattern 300 for use in focusing an industrial camera according to embodiments of the present disclosure is shown. The reference pattern 300 includes pattern groups 301, 302, and 303, each pattern group including a first region 3001 of a first color (e.g., black) and a second region 3002 of a second color (e.g., white), with the first direction being from left to right. The reference pattern 300 is a checkerboard texture pattern and also has pattern groups arranged in a second direction at an angle (e.g., 90 degrees) to the first direction.

[0031] Figure 4 An exemplary reference pattern 400 for use in focusing an industrial camera according to an embodiment of the present disclosure is shown. The reference pattern 400 includes a pattern group 401, which includes a first region 4001 of a first color (e.g., white) and a second region 4002 of a second color (e.g., white), with a first direction from the inside out. The reference pattern 400 is a nested arrangement of triangular texture patterns.

[0032] It should be understood that Figure 3 and Figure 4 The reference pattern is merely an example and not intended to limit the scope of this disclosure. Pattern groups arranged in various shapes and orientations can also be used. For example, the first and second regions can be arranged alternately. Furthermore, each pattern group may include other regions in addition to the first and second regions, which is not limited by the embodiments of this disclosure.

[0033] In the example using reference pattern 300, p×q grids of reference pattern 300 can be made to fall within the focus window or region, and the portion of the first target image corresponding to the p×q grids is selected as the first ROI image, where Figure 3 The checkerboard texture pattern can have squares of the same size or different sizes. In the example using reference pattern 400, 2×m triangles of reference pattern 400 can be placed in the focus window or area, and the portion of the first target image corresponding to the 2×m triangles can be selected as the first ROI image. Compared to reference pattern 400, reference pattern 300 is more convenient for quickly evaluating image sharpness.

[0034] Next, method 200 proceeds to step 203. In step 203, the first ROI image determined at the first plurality of locations is evaluated to generate first plurality of sharpness values. Sharpness is an important indicator of digital image quality and whether they are in focus. Image sharpness approximates the fidelity of image detail. The more image details are preserved, the higher the contrast, the sharper the image, and the higher its resolvability. Thus, the sharpness can be evaluated to determine whether the camera is in focus.

[0035] In some embodiments, step 203 may include: evaluating the first ROI image using a first image evaluation function to generate a first plurality of sharpness values ​​based on the grayscale of each pixel of the first ROI image determined at a first plurality of locations, the first image evaluation function including a gradient function. Since the edge pixels of the reference pattern have large grayscale value variations and thus larger gradient values, sharpness can be evaluated by the grayscale variations of the image (e.g., the grayscale weighted values ​​of neighboring pixels). For example, the first and second image evaluation functions may include gradient functions, and gradient functions can be used to evaluate grayscale variations. Gradient functions may include, but are not limited to: the Energy Gradient (EOG) function, the Roberts function, the Tenengrad function, the Brenner function, the Variance function, the Laplace function, or combinations thereof. Ideally, various gradient functions are unimodal and unbiased, and obtain the maximum evaluation value at the focus location. In some examples, the first and / or second image evaluation functions may also include entropy functions, spectral functions, statistical functions, or combinations of the above functions, or combinations of the above functions and gradient functions, to comprehensively evaluate sharpness.

[0036] In a focused image, the edge regions tend to be small. For most non-edge pixels, the gradient values ​​are small. These non-edge regions are not suitable for characterizing image sharpness. Conventional methods of focusing using image gradients tend to simply characterize image sharpness using the average gradient of the entire image. However, this conventional method has two problems. First, for focused images with few textured edges, if the focus region is not selected correctly, a small number of image edge points with large gradient values ​​will often be associated with many non-edge points with small gradient values, making them indistinguishable. Second, irregular background noise appears at a certain distance from the focus position. These noisy images have many textured edges, which can cause multiple peaks in the sharpness curve. As mentioned above, this disclosure provides rich image edge information by using a reference pattern to determine the ROI image corresponding to the ROI in the focus region, while advantageously reducing the impact of noise on the image.

[0037] In some embodiments, the gradient function may include a gradient function based on the Sobel operator. For example, the gradient function may be the Tenengrad function based on the Sobel operator, see equations (1)-(4) below:

[0038] f(I)=∑ x ∑ y S(x,y),S(x,y)>T(1)

[0039]

[0040]

[0041]

[0042] Among them, the Tenengrad function f(I) is defined as the sum of the squares of the pixel point gradients, and a threshold T is set for the gradient to adjust the sensitivity of the function; S(x, y) is the gradient at the pixel point (x, y), and G x and G y are the gradient values in the horizontal and vertical directions of the pixel point respectively. The Tenengrad function is simple to calculate, and its implementation speed can meet the real-time requirements of machine vision systems (especially hand-eye systems).

[0043] In one example, an absolute value Tenengrad function based on the Sobel operator can be further adopted, that is, by defining S(x, y) = |G x (x, y)| + |G y (x, y)| to further reduce the calculation amount and improve the implementation speed.

[0044] Next, method 200 proceeds to step 204. In step 204, based on the first plurality of sharpness values, a unimodal search direction is determined. Due to various focusing scenarios, it is difficult for the focusing curve to meet the unimodal characteristic, and the actual curve often has local peaks, which sometimes causes the focus to fall into the local peak and lead to focusing errors. Therefore, it is necessary to find the unimodal peak of the sharpness curve. For example, by determining the unimodal search direction, it is further convenient to estimate the focusing position in the unimodal region.

[0045] In some embodiments, step 204 may include: identifying the longest increasing subsequence of the first plurality of sharpness values; determining the position movement direction corresponding to the longest increasing subsequence as the unimodal search direction. For example, for the first plurality of sharpness values a1, a2, a3, a4... a n , the longest increasing subsequence a i <= a j <= a k ... <= a m , i < j < k... < m can be found, for example, by dynamic programming, and the position movement direction corresponding to the longest increasing subsequence is determined as the unimodal search direction.

[0046] Figure 5 FIG. shows a schematic diagram of a focus evaluation curve (or sharpness curve) 500 according to an embodiment of the present disclosure. The X-axis represents the working distance WD of the camera, and the Y-axis represents the sharpness evaluation value. The solid dots on the curve 500 represent moving the robot to different positions with the first step length and the corresponding sharpness of the determined first ROI image. Figure 5The actual focus position 501 is shown in the figure, and the longest rising subsequence identified is indicated by the arrow direction. As shown in the figure, by identifying the longest rising subsequence and determining the single-peak search direction, it is possible to effectively prevent the focus position from falling on the first local peak in curve 500.

[0047] Next, method 200 proceeds to step 205. In step 205, based on the single-peak search direction, the robot moves with a second step size to capture second target images of the target object at multiple second locations using an industrial camera, the second step size being smaller than the first step size. In this step, after coarsely searching for the focus area with a larger step size (i.e., the first step size), the actual focus position can be finely searched with a smaller step size (i.e., the second step size).

[0048] Next, method 200 proceeds to step 206. In step 206, a second ROI image corresponding to the ROI in the focus area of ​​the industrial camera is determined based on the reference pattern. This step is similar to the aforementioned step 202 and will not be described in detail again.

[0049] Next, method 200 proceeds to step 207. In step 207, the second ROI image determined at the second plurality of locations is evaluated to generate a second plurality of sharpness values. In some embodiments, step 207 may include: evaluating the second ROI image using a second image evaluation function to generate the second plurality of sharpness values ​​based on the grayscale of each pixel of the second ROI image determined at the second plurality of locations, the second image evaluation function including a gradient function. In some embodiments, the gradient function may include a gradient function based on the Sobel operator. This step is similar to the aforementioned step 203 and will not be described in detail again.

[0050] Next, method 200 proceeds to step 208. In step 208, the sharpest focus position is estimated based on a portion of the first plurality of sharpness values ​​and a portion of the second plurality of sharpness values. In this step, the actual focus position can be estimated by searching a portion of the sharpness values ​​obtained by traversing the entire sharpness curve with a first step length and a portion of the sharpness values ​​obtained by searching with a second step length, without having to traverse the sharpness curve with a second step length, which reduces computation and improves focusing speed. For example, when moving the robot to different positions with a second step length yields multiple consecutively decreasing sharpness values, moving with a second step length can be stopped. Reference Figure 5 The hollow dots on curve 500 represent the corresponding sharpness of the second ROI image as the robot moves to different positions with the second step size.

[0051] In some embodiments, step 208 may include: using curve fitting to estimate the sharpest focus position based on the last plurality of sharpness values ​​in the longest ascending subsequence of a first plurality of sharpness values ​​and a plurality of consecutively decreasing sharpness values ​​in the second plurality of sharpness values. In this step, considering that the corresponding position of the last point in the longest ascending subsequence may fall before or after the actual focus position, and considering the possible influence of noise on the sharpness value, the last plurality of sharpness values ​​in the longest ascending subsequence and the plurality of consecutively decreasing sharpness values ​​in the second plurality of sharpness values ​​are selected, and curve fitting is used to quickly estimate the actual focus position.

[0052] For example, such as Figure 5 As shown, starting from the previous position of the maximum position in the previous search (i.e., with the first step size), a smaller step size (i.e., the second step size) is set, and one sharpness and position data is recorded each time until the second consecutive decreasing point of the sharpness value is found, then the movement stops. Next, using four points close to the curve peak—the last two points in the longest ascending subsequence of the first plurality of sharpness values ​​and the two consecutive decreasing points of the second plurality of sharpness values—a cubic spline curve fitting function is used to estimate the location of the focus extremum (i.e., the sharpest focus position). In other examples, different numbers of points and curve fitting methods can also be used to estimate the location of the focus extremum.

[0053] According to embodiments of this disclosure, in addition to the aforementioned advantages, fast and accurate autofocus of industrial cameras can be achieved without manual intervention, effectively improving on-site work efficiency. Furthermore, it can be flexibly applied to various industrial cameras and lenses, thereby effectively reducing on-site costs.

[0054] Figure 6 An exemplary system 600 for focusing an industrial camera is illustrated according to an embodiment of the present disclosure. The exemplary system 600 includes a robot 601, an industrial camera 602, and a computing device 603. The robot 601 may be similar to... Figure 1 Robot 101, camera 602 can be similar to Figure 1Camera 102, fixed to a movable robot 601, has a fixed-focus lens. A computing device 603 can be communicatively coupled to the robot 601 and the industrial camera 602 for information exchange. For example, the computing device 603 can communicate with the robot 601 and the camera 602 via a wired or wireless data link to send instructions (e.g., movement instructions, shooting instructions, etc.) to the robot 601 and the camera 602 and to acquire data (e.g., captured images, etc.) from them. The computing device 603 can be a computer (PC), a programmable logic controller (PLC), and / or any suitable control device to implement the method for focusing an industrial camera as described in embodiments of this disclosure (e.g., any one or more steps of the aforementioned method 200). Since autofocus is not implemented by the industrial camera itself, the focusing algorithm avoids occupying the camera's storage space and does not introduce camera instability, thus allowing for flexible application to various combinations of cameras and lenses.

[0055] Figure 7 A block diagram of an exemplary apparatus 700 for implementing focusing of an industrial camera according to an embodiment of the present disclosure is shown. For example, apparatus 700 may be... Figure 6 The system 600 includes a computing device 603. The device 700 includes an image capture unit 701, an image determination unit 702, an image evaluation unit 703, an orientation determination unit 704, and a position estimation unit 705. To enable information transmission between the device 700 and the industrial camera and robot, the device 700 also includes a communication unit 706, which can be configured to send instructions to the robot and the industrial camera and acquire data from the robot and the industrial camera.

[0056] The image capture unit 701 is configured to move the robot by a step length and acquire first target images captured by an industrial camera at first plurality of locations for the target object.

[0057] The image determination unit 702 is configured to determine a first ROI image in the first target image that corresponds to the ROI in the focus area of ​​the industrial camera, based on a reference pattern.

[0058] The image evaluation unit 703 is configured to evaluate the first ROI image determined at the first plurality of locations to generate the first plurality of sharpness values.

[0059] The direction determination unit 704 is configured to determine the single-peak search direction based on a first plurality of resolution values.

[0060] After the direction determination unit 704 determines the single-peak search direction, the image acquisition unit 701 is further configured to: move the robot with a second step length according to the single-peak search direction, and acquire second target images captured by the industrial camera at the target object at the second and multiple positions respectively, wherein the second step length is less than the first step length.

[0061] After the direction determination unit 704 determines the single-peak search direction, the image determination unit 702 is further configured to: determine the second ROI image in the second target image that corresponds to the ROI in the focus area of ​​the industrial camera according to the reference pattern.

[0062] After the direction determination unit 704 determines the single-peak search direction, the image evaluation unit 703 is further configured to evaluate the second ROI image determined at the second plurality of locations to generate a second plurality of sharpness values.

[0063] The position estimation unit 705 is configured to estimate the sharpest focus position based on a portion of a first plurality of sharpness values ​​and a portion of a second plurality of sharpness values.

[0064] In some embodiments, the reference pattern includes a group of patterns arranged in a first direction, each group of patterns including a first region and a second region, the first region having a first color and the second region having a second color different from the first color. In some embodiments, the reference pattern may be a checkerboard texture pattern.

[0065] In some embodiments, the image evaluation unit 703 may be further configured to: evaluate the first ROI image using a first image evaluation function based on the grayscale of each pixel of the first ROI image determined at a first plurality of locations to generate a first plurality of sharpness values; and evaluate the second ROI image using a second image evaluation function based on the grayscale of each pixel of the second ROI image determined at a second plurality of locations to generate a second plurality of sharpness values; wherein the first image evaluation function and the second image evaluation function include gradient functions.

[0066] In some embodiments, the gradient function may include a gradient function based on the Sobel operator.

[0067] In some embodiments, the direction determination unit 704 may be further configured to: identify the longest rising subsequence of the first plurality of sharpness values; and determine the position movement direction corresponding to the longest rising subsequence as the single-peak search direction.

[0068] In some embodiments, the position estimation unit 705 may be further configured to: use curve fitting to estimate the sharpest focus position based on the last plurality of sharpness values ​​in the longest rising subsequence of the first plurality of sharpness values ​​and the plurality of consecutively decreasing sharpness values ​​in the second plurality of sharpness values.

[0069] In some examples, the image capture unit 701 of the device 700 may also be implemented as two separate first image capture units and second image capture units to implement... Figure 2Method steps 201 and 205. In some examples, the image determination unit 702 of the device 700 may also be implemented as two separate first image determination units and second image determination units to implement... Figure 2 Method steps 202 and 206. In some examples, the image evaluation unit 703 of the device 700 may also be implemented as two separate first image evaluation units and second image evaluation units to implement... Figure 2 Method steps 203 and 207.

[0070] Figure 8 A block diagram of an exemplary computing device 800 for implementing focusing of an industrial camera according to an embodiment of the present disclosure is shown. The computing device 800 includes a processor 801 and a memory 802 coupled to the processor 801. The memory 802 is used to store computer-executable instructions that, when executed, cause the processor 801 to perform the methods described above (e.g., any one or more steps of the aforementioned method 200).

[0071] Alternatively, the above methods can be implemented using a computer-readable storage medium. The computer-readable storage medium carries computer-readable program instructions for executing the various embodiments of this disclosure. The computer-readable storage medium can be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combinations thereof. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0072] Therefore, in another embodiment, this disclosure provides a computer-readable storage medium having computer-executable instructions stored thereon for performing the methods of various embodiments of this disclosure.

[0073] In another embodiment, this disclosure provides a computer program product tangibly stored on a computer-readable storage medium and including computer-executable instructions that, when executed, cause at least one processor to perform the methods of various embodiments of this disclosure.

[0074] Generally, the various example embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of the embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0075] Computer-readable program instructions or computer program products for executing the various embodiments of this disclosure can also be stored in the cloud. When needed, users can access the computer-readable program instructions stored in the cloud for executing an embodiment of this disclosure via mobile internet, fixed network or other networks, thereby implementing the technical solutions disclosed in the various embodiments of this disclosure.

[0076] While embodiments of this disclosure have been described with reference to several specific examples, it should be understood that the embodiments of this disclosure are not limited to the specific embodiments disclosed. The embodiments of this disclosure are intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the claims is to be interpreted in the broadest sense, thereby encompassing all such modifications and equivalent structures and functions.

Claims

1. A method for focusing an industrial camera, the industrial camera being fixed on a movable robot and having a fixed-focus lens, the method... Includes the following steps: S1, move the robot by a step length to capture first target images of the target object at first multiple locations using the industrial camera; S2, determine the first ROI image in the first target image that corresponds to the ROI in the focus area of ​​the industrial camera based on the reference pattern; S3, Evaluate the first ROI image determined at the first plurality of locations to generate the first plurality of sharpness values; S4, Based on the first plurality of sharpness values, determine the single-peak search direction; S5, according to the single-peak search direction, the robot moves with a second step length to use the industrial camera to capture second target images of the target object at a second plurality of positions respectively, the second step length being less than the first step length; S6, determine the second ROI image in the second target image that corresponds to the ROI in the focus area of ​​the industrial camera based on the reference pattern; S7, Evaluate the second ROI image determined at the second plurality of locations to generate a second plurality of sharpness values; S8, based on a portion of the first plurality of sharpness values ​​and a portion of the second plurality of sharpness values, estimate the sharpest focus position. Step S4 includes: S41, identify the longest rising subsequence of the first plurality of sharpness values; S42, the position movement direction corresponding to the longest rising subsequence is determined as the single-peak search direction.

2. The method according to claim 1, wherein, The reference pattern includes a group of patterns arranged in a first direction, each group of patterns including a first region and a second region, the first region having a first color and the second region having a second color different from the first color.

3. The method according to claim 2, wherein, The reference pattern is a checkerboard texture pattern.

4. The method according to claim 1, wherein, Step S3 includes: Based on the grayscale of each pixel of the first ROI image determined at the first plurality of locations, the first ROI image is evaluated using a first image evaluation function to generate a first plurality of sharpness values, the first image evaluation function including a gradient function.

5. The method according to claim 1, wherein, Step S6 includes: Based on the grayscale of each pixel of the second ROI image determined at the second plurality of locations, the second ROI image is evaluated using a second image evaluation function to generate a second plurality of sharpness values, the second image evaluation function including a gradient function.

6. The method according to claim 4 or 5, wherein, The gradient function includes a gradient function based on the Sobel operator.

7. The method according to claim 6, wherein, Step S8 includes: Based on the last multiple sharpness values ​​in the longest rising subsequence of the first plurality of sharpness values ​​and the multiple consecutively decreasing sharpness values ​​in the second plurality of sharpness values, curve fitting is used to estimate the sharpest focus position.

8. A computing device, the computing device comprising: processor; as well as A memory for storing computer-executable instructions that, when executed, cause the processor to perform the following steps: S1, using an industrial camera to capture first target images of the target object at multiple locations using a first long mobile robot; S2, determine the first ROI image in the first target image that corresponds to the ROI in the focus area of ​​the industrial camera based on the reference pattern; S3, Evaluate the first ROI image determined at the first plurality of locations to generate the first plurality of sharpness values; S4, Based on the first plurality of sharpness values, determine the single-peak search direction; S5, according to the single-peak search direction, the robot moves with a second step length to use the industrial camera to capture second target images of the target object at a second plurality of positions respectively, the second step length being less than the first step length; S6, determine the second ROI image in the second target image that corresponds to the ROI in the focus area of ​​the industrial camera based on the reference pattern; S7, Evaluate the second ROI image determined at the second plurality of locations to generate a second plurality of sharpness values; S8, based on a portion of the first plurality of sharpness values ​​and a portion of the second plurality of sharpness values, estimate the sharpest focus position. Step S4 includes: S41, identify the longest rising subsequence of the first plurality of sharpness values; S42, the position movement direction corresponding to the longest rising subsequence is determined as the single-peak search direction.

9. The computing device according to claim 8, wherein, The reference pattern includes a group of patterns arranged in a first direction, each group of patterns including a first region and a second region, the first region having a first color and the second region having a second color different from the first color.

10. The computing device according to claim 9, wherein, The reference pattern is a checkerboard texture pattern.

11. The computing device according to claim 8, wherein, Step S3 includes: Based on the grayscale of each pixel of the first ROI image determined at the first plurality of locations, the first ROI image is evaluated using a first image evaluation function to generate a first plurality of sharpness values, the first image evaluation function including a gradient function.

12. The computing device according to claim 8, wherein, Step S6 includes: Based on the grayscale of each pixel of the second ROI image determined at the second plurality of locations, the second ROI image is evaluated using a second image evaluation function to generate a second plurality of sharpness values, the second image evaluation function including a gradient function.

13. The computing device according to claim 11 or 12, wherein, The gradient function includes a gradient function based on the Sobel operator.

14. The computing device according to claim 8, wherein, Step S8 includes: Based on the last multiple sharpness values ​​in the longest rising subsequence of the first plurality of sharpness values ​​and the multiple consecutively decreasing sharpness values ​​in the second plurality of sharpness values, curve fitting is used to estimate the sharpest focus position.

15. An apparatus for focusing an industrial camera, the industrial camera being mounted on a movable robot and having a fixed-focus lens, the apparatus comprising: An image capture unit is configured to move the robot by a step length and acquire first target images captured by the industrial camera at first plurality of locations for the target object. The image determination unit is configured to determine a first ROI image in the first target image that corresponds to the ROI in the focus area of ​​the industrial camera, based on a reference pattern; The image evaluation unit is configured to evaluate a first ROI image determined at a first plurality of locations to generate a first plurality of sharpness values; The direction determination unit is configured to determine the single-peak search direction based on the first plurality of sharpness values; Wherein, after the direction determination unit determines the single-peak search direction: The image capture unit is further configured to: move the robot with a second step length according to the single-peak search direction, and acquire second target images captured by the industrial camera at a second plurality of positions for the target object, wherein the second step length is less than the first step length; The image determination unit is further configured to: determine a second ROI image in the second target image that corresponds to the ROI in the focus area of ​​the industrial camera, based on the reference pattern; The image evaluation unit is further configured to evaluate the second ROI image determined at the second plurality of locations to generate a second plurality of sharpness values; and The position estimation unit is configured to estimate the sharpest focus position based on a portion of the first plurality of sharpness values ​​and a portion of the second plurality of sharpness values. The direction determination unit is further configured as follows: Identify the longest rising subsequence of the first plurality of sharpness values; The direction of position movement corresponding to the longest increasing subsequence is determined as the single-peak search direction.

16. A computer-readable storage medium having computer-executable instructions stored thereon for performing the method according to any one of claims 1-7.

17. A computer program product tangibly stored on a computer-readable storage medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform the method according to any one of claims 1-7.

Images