Automatic focusing control circuit and control method

Abstract

The invention discloses an automatic focusing control circuit and a control method, and relates to the field of automatic focusing. The circuit comprises a ZYNQ chip which comprises a programmable logic unit and is used for calculating the coordinates of the center of mass according to an image of an object to be measured; the first ARM core unit is used for calculating driver focusing control information according to the center-of-mass coordinates and sending the driver focusing control information to the programmable logic unit, so that the transmission efficiency among the units in the ZYNQ chip is improved; the driver is connected with the programmable logic unit and is used for acquiring focusing control information of the driver, so that the driver is used for adjusting the distance from the image sensor to the to-be-detected object, and the to-be-detected object is focused; the ZYNQ chip further comprises a second ARM core unit, and the second ARM core unit is connected with the upper computer through the serial port communication module. Wherein the serial port communication module is connected with the upper computer through a 485 chip and an RS-485 bus, and data transmission is based on a Modbus communication protocol.

Description

Technical Field

[0001] The invention belongs to the field of automatic focusing, and in particular relates to an automatic focusing control circuit and a control method. Background Art

[0002] Existing autofocus technologies are mainly divided into two types: single sensor and dual sensor + laser. Single sensor means that only one sensor is used to shoot the detected object, and different image clarity indicators are used to evaluate whether the image of the detected object is in focus. For example, the image clarity indicator can be the EAV point sharpness algorithm function, the Robert gradient function, etc. The process of autofocus is to move the sensor position until the evaluation function reaches the peak. Among them, the selection of clarity indicators determines the accuracy of focus, and the focus peak search determines the speed of focus.

[0003] Dual sensors + laser are represented as follows: sensor 1 is used to capture the actual object being detected, and sensor 2 is used to capture the laser image. The information of the laser spot in the laser image is extracted to obtain the defocus value, thereby controlling the movement of the motor to drive the positions of the two sensors to change, and finally achieving autofocus.

[0004] In the field of machine vision, dual sensors + lasers are usually used for autofocus. PCB, display panels, semiconductors and other industries require high-precision detection in the automatic production process. Not only does the surface of the object being detected often have a certain height difference, but the depth of field of the high-magnification lens that shoots the actual object being detected is at the micron level, which puts higher requirements on the autofocus rate. It is very easy for the clarity of the detection image to fail to fully meet the requirements.

[0005] At the same time, as the demand for autofocus in the field of machine vision continues to increase, the application scenarios of autofocus systems are also increasing. Therefore, it is necessary to reduce the structural volume of the autofocus system to meet the needs of use in different scenarios.

[0006] Therefore, an automatic focusing control circuit (ie, an automatic focusing system) is urgently needed to solve the above problems. Summary of the invention

[0007] The present application proposes an autofocus control circuit and a control method, which further improve the autofocus rate on the basis of reducing the volume of the autofocus system.

[0008] To achieve the above objectives, this application proposes the following technical solutions: In a first aspect of the present application, an automatic focus control circuit is provided, comprising: The ZYNQ chip includes a programmable logic unit, which is used to calculate the centroid coordinates according to the image of the object to be measured containing the laser spot for focusing, so that the calculation efficiency of the centroid coordinates is improved by the image parallel processing capability of the programmable logic unit; and also includes a first ARM core unit, which is used to calculate the driver focus control information according to the centroid coordinates, and send the driver focus control information to the programmable logic unit, so that the transmission efficiency between the units in the ZYNQ chip is improved; A driver, connected to the programmable logic unit, obtains driver focus control information so that the driver adjusts the distance from the image sensor to the object to be measured, thereby focusing on the object to be measured; The ZYNQ chip also includes a second ARM core unit, which is connected to the host computer through a serial communication module; wherein the serial communication module and the host computer are connected using a 485 chip and an RS-485 bus to enable data transmission based on differential signals, and the data transmission is based on the Modbus communication protocol to reduce the bit error rate of data transmission.

[0009] Optionally, the second ARM core unit receives an operation instruction sent by the host computer to adjust the parameters of each unit in the ZYNQ chip or read the data of each unit in the ZYNQ chip.

[0010] Optionally, the first ARM core unit and the second ARM core unit share the same storage space; wherein the storage space contains the center of mass coordinates, driver focus control information and operation instructions.

[0011] Optionally, the cache function is disabled during data transmission between the first ARM core unit or the second ARM core unit and the storage space.

[0012] Optionally, the first ARM core unit and the programmable logic unit are connected via an AXI bus.

[0013] In a second aspect of the present application, an automatic focus control method is provided, which is applied to any automatic focus control circuit described in the first aspect, and the control method includes: Calculate the pixel mean of the first pixel point in each pixel row in the image of the object to be tested as the pixel row mean; wherein the first pixel point represents a pixel point in the image of the object to be tested whose pixel value is higher than a preset first pixel threshold; the image of the object to be tested represents the image obtained by selecting an area of ​​interest in the original image generated by the image sensor; Counting a set of pixel rows whose average values ​​of the pixel rows are higher than a preset second pixel threshold; wherein the elements in the set of pixel rows are the row sequences of the pixel rows; The top and bottom boundaries of the region of interest are adjusted based on the minimum and maximum values ​​of the pixels within the set of pixel rows.

[0014] Optionally, adjusting the top boundary and the bottom boundary of the region of interest based on the minimum value and the maximum value of the pixels in the pixel row set includes: Determine whether the minimum value of the elements in the pixel row set is 1, if so, move the top boundary of the region of interest upward by a preset number of pixel rows; if not, keep the top boundary of the region of interest unchanged; Determine whether the maximum value of the elements in the pixel row set is n; if so, lower the bottom boundary of the region of interest by a preset number of pixel rows; if not, keep the top boundary of the region of interest unchanged; wherein n represents the total number of pixel rows in the region of interest.

[0015] In a third aspect of the present application, another autofocus control method is provided, which is applied to any autofocus control circuit described in the first aspect, and the control method includes: Obtaining the average value of pixels in the image of the object to be tested as the pixel average value, and determining whether the pixel average value exceeds a preset pixel range; if not, terminating the exposure time adjustment process and maintaining the working state of the autofocus control circuit unchanged; If yes, then calculate the new exposure time, and determine whether the new exposure time exceeds the preset exposure time threshold; if no, set the exposure time of the autofocus control circuit to the new exposure time, and end the exposure time adjustment process; If yes, setting the exposure time of the autofocus control circuit to the exposure time threshold, and determining whether the pixel mean under the exposure time threshold exceeds a preset pixel range; if no, ending the exposure time adjustment process; If so, the laser energy of the laser is adjusted based on a preset step size until the pixel mean is within a preset pixel range.

[0016] Optionally, the preset step size is determined based on the lens magnification and the operating current value of the laser.

[0017] In a fourth aspect of the present application, another autofocus control method is provided, which is applied to any autofocus control circuit described in the first aspect, and the control method includes: Determining a defocus state of the image sensor based on the defocus amount; wherein a plurality of defocus states under different defocus amounts are divided using a plurality of preset defocus amount thresholds; Different control modes of the driver are selected based on the defocus state; wherein the control modes include: direct drive, S-curve planning and closed-loop PID algorithm.

[0018] The beneficial effects of this application are as follows: The present application provides an automatic focus control circuit, comprising: a ZYNQ chip, comprising a programmable logic unit, for calculating the centroid coordinates according to an image of an object to be measured containing a laser spot for focusing, so that the calculation efficiency of the centroid coordinates is improved by the image parallel processing capability of the programmable logic unit; also comprising a first ARM core unit, for calculating driver focus control information according to the centroid coordinates, and sending the driver focus control information to the programmable logic unit, so that the transmission efficiency between the units in the ZYNQ chip is improved; a driver, connected to the programmable logic unit, obtains the driver focus control information, so that the driver adjusts the distance from the image sensor to the object to be measured, thereby focusing on the object to be measured; the ZYNQ chip also comprises a second ARM core unit, and the second ARM core unit is connected to a host computer via a serial communication module; wherein the serial communication module and the host computer are connected using a 485 chip and an RS-485 bus, so that data transmission is based on differential signals, and the data transmission is based on the Modbus communication protocol, so as to reduce the bit error rate of data transmission.

[0019] Based on the above settings, the autofocus control circuit provided by the present application not only effectively reduces the overall structure of the autofocus system by utilizing the high integration of the ZYNQ chip, but also has a higher data transmission rate than the existing FPGA and processor system. At the same time, in the autofocus system provided by the present application, the FPGA is used to calculate the image signal, which can make full use of the feature points with fast parallel processing speed on the hardware side to improve the working efficiency of the autofocus.

[0020] At the same time, the serial communication module is used in the autofocus control circuit to realize the information interaction between the ZYNQ chip and the host computer, which can further reduce the overall volume of the control circuit compared to the existing network port communication module. And the 485 chip and RS-485 bus are used for data transmission, so that the data transmission between the first ARM core unit and the host computer adopts the form of differential signal, which effectively reduces the interference of external noise. Data transmission based on the Modbus communication protocol also improves the anti-interference ability of the data and effectively avoids the risk of bit errors.

[0021] In addition, in the autofocus control circuit provided by the present application, the first ARM core unit is used to calculate the defocus amount (i.e., the driver focus control information), the second ARM core unit is used to exchange information with the host computer, and the first ARM core unit and the second ARM core unit share the same storage space. As a result, the first ARM core unit and the second ARM core unit process different work contents respectively, and can realize information sharing, so that not only the normal operation of autofocus and external information interaction in the autofocus control circuit can be realized, but also the information interaction between the host computer and the second ARM core unit is avoided. The influence on the autofocus process is avoided, and the high-speed operation of the autofocus is ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a structural diagram of an autofocus system provided by the present application; Figure 2 is a schematic diagram of a laser image provided by the present application; Figure 3 This is a flowchart of a storage space reading process provided by this application; Figure 4 is a schematic diagram of another laser image provided by the present application; Figure 5 is a schematic diagram of laser profiles under different defocus conditions provided by the present application; Figure 6 It is a schematic diagram of a flow chart of exposure time adjustment provided by the present application; FIG. 7 is a schematic diagram of a defocus state transformation provided by the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0024] The existing autofocus solution consisting of MCU (MicroController Unit) plus FPGA (Field-Programmable Gate Array) has two defects: on the one hand, the lack of proprietary communication lines between FPGA and MCU will lead to reduced performance and high design costs; on the other hand, it will cause the entire system module to be bloated and not suitable for industrial use scenarios. In order to solve the above problems, this application builds an autofocus control circuit based on laser imaging based on the ZYNQ chip (a chip developed by Xilinx, called a fully programmable system on chip). The circuit includes: The ZYNQ chip includes a programmable logic unit for calculating the centroid coordinates according to the image of the object to be measured containing the laser spot for focusing, so that the calculation efficiency of the centroid coordinates is improved by the image parallel processing capability of the programmable logic unit. It also includes a first ARM core unit for calculating the driver focus control information according to the centroid coordinates and sending the driver focus control information to the programmable logic unit, so that the transmission efficiency between the units in the ZYNQ chip is improved.

[0025] The driver is connected to the programmable logic unit and obtains the driver focus control information so that the driver adjusts the distance from the image sensor to the object to be measured, thereby focusing on the object to be measured.

[0026] The ZYNQ chip also includes a second ARM core unit, which is connected to the host computer through a serial communication module; wherein the serial communication module and the host computer are connected using a 485 chip and an RS-485 bus to enable data transmission based on differential signals, and the data transmission is based on the Modbus communication protocol to reduce the bit error rate of data transmission.

[0027] The Zynq chip is divided into two parts: the PS side and the PL side. Among them, the PS side contains two ARM cores, namely the first ARM core unit and the second ARM core unit. In the technical solution provided in this application, core 1 is represented as the first ARM core unit, and core 0 is represented as the second ARM core unit. The PL side mainly includes an FPGA logic part, namely the programmable logic unit in this application, referred to as FPGA in the following content.

[0028] The image of the object to be tested refers to an image obtained by cutting out a region of interest (ROI) containing a laser spot from the original image generated by the image sensor, and is also referred to as a laser image.

[0029] The calculation formula of the centroid coordinates is as follows:

[0030] Among them, Q represents the centroid, m represents the total number of pixel rows in the laser image, n represents the total number of pixel columns in the laser image, i represents the number of pixel rows in the laser image, j represents the number of pixel columns in the laser image, and f(i,j) represents the grayscale value of the pixel in the i-th row and j-th column. For each frame of laser image, the PL end uses the advantages of FPGA parallel processing to calculate the centroid coordinates of the laser image frame by frame, shortening the calculation time of the centroid in each frame of laser image and ensuring high-speed autofocus.

[0031] Core 1 is connected to FPGA via AXI (Advanced eXtensible Interface) bus, and can receive the centroid coordinates generated by FPGA, and use them to generate driver focus control information. The driver focus control information is represented by the control information of the moving distance and moving direction of the driver moving image sensor, which can correspond to the defocus amount in the autofocus solution.

[0032] In actual work, for each frame of laser image, core 1 will read the centroid Q of the laser image from the PL end. now (in pixels), the current center of mass coordinates and the reference center of mass Q ref The difference is converted into defocus value .

[0033]

[0034] Among them, the reference center of mass Q ref Both the image stabilization speed and lens sensitivity s (in pixels / micron) are calibrated before autofocus.

[0035] Core 1 on the PS side calculates the defocus value After being converted into driver focus control information, it is sent to the FPGA on the PL side, and the FPGA converts the driver focus control information into pulses and sends them to the driver, thereby driving the motor of the image sensor to work, so that the image sensor moves up and down along the Z axis. The high integration inside the ZYNQ chip can effectively improve the data transmission rate between different units.

[0036] In some embodiments, the second ARM core unit (ie, core 0) receives an operation instruction sent by a host computer to adjust parameters of each unit in the ZYNQ chip or read data of each unit in the ZYNQ chip.

[0037] At the same time, the first ARM core unit (ie, core 1) and the second ARM core unit (ie, core 0) share the same storage space; wherein the storage space contains the centroid coordinates, the driver focus control information and the operation instructions.

[0038] Specifically, the operation instructions include instruction information for adjusting parameters in the ZYNQ chip, wherein the parameters of each unit include the resolution of the laser image acquired in the FPGA, the frame rate of the laser image, and also include adjusting the parameters in the laser or image sensor through the core 1 or the FPGA.

[0039] In addition, the operation instructions also include data reading instructions for reading data information of each unit in the ZYNQ chip. Among them, the data information includes relevant information of the laser image (for example, centroid, defocus, pixel mean and motion distance, etc.), as well as parameter information in the laser or image sensor. Specifically, the data connection line formed by the storage space shared between core 0 and core 1, and the AXI bus between core 1 and FPGA, realizes the overall control of the ZYNQ chip by the host computer through the connection with core 0. That is, the host computer can adjust the parameters of each unit in the ZYNQ chip (also including the laser controlled by core 1 and the image sensor controlled by FPGA) or read the data of each unit in the ZYNQ chip (also including the laser controlled by core 1 and the image sensor controlled by FPGA).

[0040] In actual work, the cache function is turned off during data transmission between the first ARM core unit or the second ARM core unit and the storage space, thereby avoiding data read and write errors in the first ARM core unit or the second ARM core unit during data update.

[0041] like Figure 1 As shown, the present application provides a ZYNQ-based autofocus system, in which a ZYNQ chip is provided, and the ZYNQ chip includes: The first ARM core unit (ie, Figure 1 The core 1 in the image is connected to the FPGA and is used to calculate the defocus amount corresponding to the image information, and convert the defocus amount into movement information, and transmit it to the FPGA; wherein the image information is obtained by the FPGA calculating the laser image and sent to the first ARM core unit; the movement information includes the moving direction and the moving distance, which is the driver focus control information in the aforementioned content.

[0042] The FPGA (ie, a programmable logic unit) is configured to generate a control signal according to the movement information, and send the control signal to a driver, so that the driver moves the image sensor.

[0043] The second ARM core unit (i.e., Figure 1 Core 2 in the ARM core unit shares the same storage space with the first ARM core unit, and the second ARM core unit is connected to the host computer for data transmission with the host computer.

[0044] Based on the above settings, the autofocus system provided by this application not only effectively reduces the overall structure of the autofocus system by utilizing the high integration of the ZYNQ chip, but also has a higher data transmission rate than the existing FPGA and processor system. At the same time, in the autofocus system provided by this application, the FPGA is used to calculate the image signal, which can make full use of the feature points with fast parallel processing speed on the hardware side to improve the working efficiency of the autofocus.

[0045] In addition, in the autofocus system provided by the present application, the first ARM core unit is used to calculate the defocus amount, the second ARM core unit is used to exchange information with the host computer, and the first ARM core unit and the second ARM core unit share the same storage space. Thus, the first ARM core unit and the second ARM core unit process different work contents respectively, and can realize information sharing, so that not only can the autofocus and the interaction with external information in the autofocus system be carried out normally, but also the information interaction between the host computer and the second ARM core unit is avoided. The influence on the autofocus process is avoided, and the high-speed operation of the autofocus is ensured.

[0046] In actual work, core 0 and core 1 correspond to two independent Cortex-A9 processors, and core 0 is connected to the host computer to handle upper-level tasks such as parameter setting in the automatic system and image information transmission to the host computer, while real-time tasks such as defocus calculation and motion control are handled by core 1. Since core 0 and core 1 are isolated from each other, the information exchange between core 0 and the host computer will not affect the real-time control of core 1, and vice versa, thus ensuring the stability of the operation of the autofocus system.

[0047] The host computer represents a computer or single-chip microcomputer that can directly send operating instructions, which is usually used to display or feedback data to the user. It includes electronic devices such as computers, mobile phones, tablets and panels. In the autofocus system provided by this application, core 0 is connected to the host computer through a serial communication module. Compared with the network port communication module with a faster transmission rate, the serial communication module is smaller in size, which can effectively reduce the overall structure of the autofocus system.

[0048] The host computer can obtain relevant information of the laser image in the autofocus system (for example, the center of mass, defocus amount, pixel mean, and movement distance, etc.) through the serial communication module, and can also set the system parameters in the autofocus system according to actual conditions (for example, system switch instructions, laser parameters, and image sensor parameters, etc.). When the autofocus is in a high-speed working state, the serial communication module also needs to be in a high baud rate working state to avoid bit errors during its data transmission. In the present application, a 485 chip and an RS-485 bus are set in the connection circuit between the serial communication module and the host computer, and data transmission is performed based on the existing Modbus communication protocol.

[0049] Based on the above settings, the 485 chip and RS-485 bus are set in the peripheral circuit of core 0, so that the data transmission between core 0 and the host computer can be in the form of differential signals, which effectively reduces the interference of external noise. On the other hand, data transmission based on the existing Modbus communication protocol also improves the anti-interference ability of the data, effectively avoids the risk of bit errors, and ensures the accuracy of parameter settings.

[0050] In addition, the above serial communication module can be used as a data transmission channel for remote firmware upgrade of the ZYNQ chip, and is used to update and iterate the programs in core 0, core 1 and FPGA in the ZYNQ chip. In the autofocus system provided by this application, core 0 and core 1 are both connected to the FPGA. On the other hand, the serial communication module can also be used for interaction with the host computer, such as the host computer conforming to the instructions of a specific protocol format to change multiple parameters in the ZYNQ chip, thereby improving the stability of the entire autofocus system.

[0051] Core 0 and core 1 share the same storage space, and core 0 can transmit relevant control instructions of the image sensor and laser to core 1 through the shared storage space. Similarly, core 0 can also obtain relevant setting parameters of the sensor and laser, as well as relevant information of the laser image, etc. in real time through the storage space.

[0052] Core 1 on the PS side can be used to initialize and control the image sensor and the laser. The image sensor is used to collect the laser image of the surface of the object, and the laser controls the brightness of the laser image.

[0053] In addition, core 1 calculates the corresponding movement distance (i.e., defocus amount) based on the centroid sent by the FPGA in the PL end, and sends the movement distance to the PL end. The absolute value and positive and negative properties of the defocus amount are used to determine the movement distance and movement direction of the image sensor.

[0054] Due to the complexity of the material of the object being detected, some areas of the laser image may be blurred, resulting in errors in the centroid calculation based on the entire laser image, which cannot meet the focus requirements. In order to solve the above problem, the PL-side FPGA divides the light spot in the laser image into multiple segments during the centroid calculation process, and calculates the average of the centroid of one or more segments as the final centroid coordinates. Figure 2 As shown in the figure, the centroids of the seven segments of the image are calculated, and different strategies are used to calculate the defocus amount. The strategies include: selecting the average of the centroids of the three middle segments, selecting the minimum centroid, etc. In actual work, in the preprocessing stage of autofocus, the height difference, material type and other information of the surface of the object to be measured are first detected to determine the centroid calculation strategy, and the same centroid calculation strategy is always maintained in the subsequent actual autofocus process.

[0055] During operation, the PS side of the Zynq chip accesses part of the space in the DDR memory (i.e., double-rate synchronous dynamic random access memory) (i.e., the storage space shared by core 0 and core 1) through the DDR controller. In order to speed up the access speed, some data is often cached in the cache (i.e., cache memory), such as Figure 3 As shown in the figure, DMA stands for Direct Memory Access.

[0056] However, when core 0 modifies the data in the DDR memory, core 1 may still read the original data in the DDR memory from the cache, resulting in an error in the data reading process of core 1. Similarly, when core 1 modifies the data in the DDR memory, core 0 still reads the original data in the DDR memory from the cache.

[0057] In order to avoid data read and write errors between core 0 and core 1 during data update, the cache function is disabled during data transmission between the dual cores and the same storage space, so that after the data in the storage space is updated, both cores can read the latest data to ensure the accuracy of data transmission in the autofocus system. The dual cores include core 0 and core 1.

[0058] The maximum format of an image sensor is usually 2048×2048 pixels. In order to increase the sampling rate of the laser image, the resolution of the laser image needs to be reduced. Therefore, in the autofocus solution provided in this application, the laser image is usually represented as the ROI area where the image sensor generates the original image, that is, the image of the object to be tested in the aforementioned content.

[0059] In actual work, in the preprocessing stage of autofocus, the original image containing the laser spot generated by the image sensor is used to set the interval pixel column (i.e., the starting column) between the edge of the ROI area and the edge of the image area and the width of the ROI area in advance, such as Figure 4 shown.

[0060] During the operation of the autofocus system, as the height between the image sensor and the surface of the object being measured changes, the height of the laser spot increases or decreases accordingly. In order to prevent the laser spot from exceeding the ROI area and affecting the normal calculation of the defocus amount, the present application provides an autofocus control method, which is applied to the autofocus control circuit of the present application and is executed by the FPGA on the PS side to automatically adjust the ROI area. The method includes the following steps: S101, using a contour detection algorithm to detect the contour of the light spot in the laser image.

[0061] The laser image refers to the image obtained by selecting a region of interest from an original image generated by the image sensor.

[0062] S102: Based on the relationship between the contour and the region of interest, adjust the height of the region of interest so that all contours are located within the boundary of the region of interest and the minimum distance between the contour and the boundary is lower than a preset distance threshold.

[0063] Based on the above processing, the ROI area is adjusted so that it completely includes the laser spot. On the basis of improving the laser image frame rate, the accuracy of the defocus result calculation is guaranteed.

[0064] For step S1, the coordinates of the four vertices in the ROI area are obtained in advance as (x1, y1), (x2, y1), (x1, y2), (x2, y2), corresponding to the lower left, lower right, upper left and upper right vertices. Then, for the acquired laser image, the contour of the laser spot is detected using the contour detection algorithm. Among them, x represents the column coordinate of the pixel point, and y represents the row coordinate of the pixel point.

[0065] For step S2, it is determined whether the coordinates of all points in the contour are within the ROI boundary. If so, the minimum value z of the distance between all points in the contour and the upper and lower boundaries of the ROI is calculated.

[0066]

[0067] Among them, I represents the set of coordinate points within the contour, i y Represents the row coordinates of the coordinate points in the contour, abs(i y -y1) means i y The distance value between y1 and abs(i y -y2) means i y The distance value from y2.

[0068] When z is less than a preset pixel row threshold, the existing ROI area is kept unchanged; when z is greater than, the ROI height becomes narrower. The pixel row threshold can be 5.

[0069] When the coordinates of some points in the contour are just above the ROI boundary, the ROI area is expanded upward and downward with a step size of 5 pixel rows.

[0070] Considering that the contour detection algorithm requires a higher amount of computing resources and a longer computing time, in order to shorten the time of ROI area adjustment and be more suitable for FPGA, the present application also provides another ROI area adjustment method, including the following steps: S201, calculating the pixel mean of the first pixel point in each pixel row in the laser image as the pixel row mean.

[0071] The first pixel point represents a pixel point in the laser image whose pixel value is higher than a preset first pixel threshold.

[0072] S202: Count a set of pixel rows whose pixel row averages are higher than a preset second pixel threshold.

[0073] The elements in the pixel row set are the row order of the pixel rows.

[0074] S203: Adjust the top boundary and the bottom boundary of the region of interest based on the minimum value and the maximum value of the pixels in the pixel row set.

[0075] For step S201, the pixel value represents the brightness value or grayscale value of the pixel point. The preset first pixel threshold can be understood as the pixel value caused by non-laser (for example, ambient light). By screening the first pixel point, the influence of external ambient light can be avoided, thereby improving the accuracy of the ROI area adjustment result.

[0076] With respect to step S202 , the row sequence of the pixel rows may be understood as the sequence number of each pixel row from top to bottom in the laser image (ie, ROI) region.

[0077] In some embodiments, step S203 includes the following: Step a: determine whether the minimum value of the elements in the pixel row set is 1. If so, the top boundary of the region of interest is raised by a preset number of pixel rows; if not, the top boundary of the region of interest is kept unchanged.

[0078] Step b, determine whether the maximum value of the elements in the pixel row set is n; if so, lower the bottom boundary of the region of interest by a preset number of pixel rows; if not, keep the top boundary of the region of interest unchanged; wherein n represents the total number of pixel rows in the region of interest.

[0079] In actual work, the above ROI area adjustment method includes the following: first, the resolution of the current laser image is obtained to be m×n, and the row offset is p. After removing the pixels that are less than the preset first pixel threshold, the pixel mean fi of all pixel rows is calculated. Among them, m represents the total number of pixel columns in the laser image; i∈[1,n] represents the row order of the pixel row, and the first pixel threshold can be 20. The row offset represents the number of pixel rows between the top boundary of the ROI area in the original image and the top of the original image.

[0080] Afterwards, all pixel row sets with values ​​higher than a preset second pixel threshold are counted. The minimum value of the elements in set I is a, the maximum value is b, and the second pixel threshold can be 30.

[0081] Finally, if a=1, the row bias is set to p-5, that is, the image is expanded upward by 5 rows; if b=n, the image is expanded downward by 5 rows, and if a, b are in the range of (1, n), they are unchanged. It can be understood that if the pixel row mean is greater than the preset second pixel threshold, it means that there is a laser spot in the pixel row.

[0082] When the object moves up or down from the focal plane, the laser spot will Figure 5As shown in the figure. At this time, in order to include the laser spot, the ROI area will adjust the height and starting line of the laser image to make the laser image higher. Since the laser energy is constant, the brightness of the spot will decrease as the spot becomes larger. When the object to be measured is far away from the focal plane, the laser spot will become extremely dark and without any effective information, resulting in subsequent focus failure. Therefore, it is necessary to automatically adjust the brightness of the laser spot in the laser image to avoid the inability to obtain image information of the laser image.

[0083] In some embodiments, the present application also provides another autofocus control method for automatically adjusting the exposure time and the laser energy of the laser, comprising the following steps: S301, obtaining the mean value of pixels in the laser image as the pixel mean value, and determining whether the pixel mean value exceeds a preset pixel range; if not, terminating the exposure time adjustment process and maintaining the working state of the autofocus system unchanged.

[0084] S302: If yes, calculate the new exposure time, and determine whether the new exposure time exceeds the preset exposure time threshold; if not, set the exposure time of the autofocus system to the new exposure time, and end the exposure time adjustment process.

[0085] S303: If yes, then set the exposure time of the autofocus system to the exposure time threshold, and determine whether the pixel mean under the exposure time threshold exceeds a preset pixel range; if not, then end the exposure time adjustment process.

[0086] S304: If yes, adjust the laser energy of the laser based on a preset step size until the pixel mean is within a preset pixel range.

[0087] Based on the above processing, the brightness of the laser spot can be adjusted by automatically adjusting the brightness of the laser spot and the exposure time of the sensor (i.e., image sensor), which can effectively increase the focus range. When the detected object is farther away from the focal plane, the laser spot has an effective signal, which effectively ensures the stable operation of the autofocus system.

[0088] In actual work, the exposure time adjustment process is performed by core 1, and the processing process is as follows: Figure 6 For each frame of laser image (i.e., laser image in this application), core 1 will read the pixel mean of the laser image from the PL end. When the pixel mean is within the set pixel range, it indicates that the brightness of the laser image is appropriate and no operation is required. When the pixel mean is outside the set pixel range, the new exposure time is calculated based on the proportional relationship between the pixel value and the exposure time. .

[0089]

[0090] Among them, E now Indicates the current exposure time, T v Represents the target value of the pixel mean, M v Represents the existing pixel mean of the laser image.

[0091] Determine whether the new exposure time exceeds the preset exposure time threshold; if not, set the exposure time of the autofocus system to the new exposure time, and end the exposure time adjustment process. If the calculated exposure time exceeds the exposure time range of the sensor, it means that even if the exposure time of the image sensor is adjusted to the limit, the brightness of the laser image cannot reach the target value. Therefore, set the exposure time of the autofocus system to the exposure time threshold, and re-determine whether the pixel mean under the exposure time threshold exceeds the preset pixel range; if not, end the exposure time adjustment process. If so, then adjust the laser energy value of the laser based on the preset step size, and return to determine whether the pixel mean is within the threshold range (i.e., pixel range) based on the adjusted laser energy value to ensure that the laser spot in the laser image is identifiable.

[0092] Among them, the step size of adjusting the laser energy is determined by the working current value of the laser. The appropriate adjustment step size ensures that the brightness of the laser image can be adjusted to the right position within a few frames, which is not only conducive to rapid focusing, but also can effectively avoid overexposure of the laser spot.

[0093] Specifically, the process of determining the step length is as follows: determine whether the operating current of the laser is greater than the preset first current value. If so, it indicates that the current image sensor has a large defocus distance and a low laser spot brightness. The laser energy needs to be adjusted quickly to enhance the laser spot brightness. Therefore, the adjustment step length of the laser is larger at this time and is set to the first step length value.

[0094] If the working current is less than the preset first current value, then determine whether the working current is greater than the preset second current value and less than the first current value. If so, then it is necessary to consider that the lens with different magnifications has different rates of change of the spot grayscale value caused by the change in laser brightness, because the sensitivity of the high-magnification lens is lower. Therefore, when the working current is in this range, the higher the lens magnification, the larger the step value, but it will not exceed the first step value.

[0095] If the working current is less than the preset second current value, it means that the defocus distance of the existing image sensor is small, the laser spot is basically in a linear shape, and the laser's ability is concentrated. If the laser's adjustment step is set to a large value, it is very easy to cause the laser spot to be overexposed, affecting normal autofocus. Therefore, at this time, the laser's adjustment step is set to the second step value, which is lower than the aforementioned step value.

[0096] In actual working process, the working current value of the laser is usually converted into DAC value (for example, the maximum working current is 160mA, and the corresponding DAC value is 4095), and then the specific adjustment step is determined based on the DAC value and the lens magnification. When the laser dac value is less than 200, the adjustment step of the laser is 2; when the laser dac value is greater than 200 and less than 2000, the lens magnification is less than 20 times, the laser adjustment step is 10, when the lens magnification is 50 times, the adjustment step is 20, when the lens magnification is 50 times, the adjustment step is 40; when the laser dac value is greater than 2000, the adjustment step is 100.

[0097] During the motion focusing process of the image sensor, the stability of the image sensor is affected to a certain extent by inertia, especially when the defocus distance is far. In addition, when the autofocus working rate is faster, the stability requirement of the autofocus system is higher. In order to solve the above problems, the present application also provides another autofocus control method, comprising the following steps: S401, determining a defocus state of an image sensor based on a defocus amount, wherein a plurality of preset defocus amount thresholds are used to divide a plurality of defocus states under different defocus amounts.

[0098] S402, selecting different control modes of the driver based on the defocus state, wherein the control modes include: direct drive, S-curve planning and closed-loop PID algorithm.

[0099] For step S401, there are usually two preset defocus thresholds, such as threshold 1 and threshold 2, and threshold 1 is greater than threshold 2. When the autofocus system is not working, the corresponding defocus state is state 0. After the autofocus system starts working, when the defocus is greater than threshold 1, the corresponding defocus state is state 1; when the defocus is greater than threshold 2 and less than threshold 1, the corresponding defocus state is state 2; when the defocus is less than threshold 2, the corresponding defocus state is state 3.

[0100] For step S401, the corresponding controller moves the image sensor in different ways under different defocus states. Specifically, state 1 indicates that the defocus amount of the object under test is too large, that is, the object under test is far away from the depth of field range, then the register in the FPGA is configured to directly drive the motor (i.e., the driver) to move the lens to a position where the defocus amount is less than the threshold value 2, so that the defocus amount is within the range corresponding to state 2. State 2 indicates that the defocus amount is within the effective linear range of the current image sensor. At this time, the driver is S-curved based on the S-curve planning algorithm combined with the defocus amount and speed to drive the motor to achieve precise focusing; state 3 indicates that the defocus amount is within the focus setting threshold, and the closed-loop PID algorithm can be used to control the motor to focus in real time. Based on the above processing, the control method under different defocus states can not only ensure fast focusing and focusing, but also improve the stability of the image sensor movement, avoid abrupt changes in the defocus amount, prevent the image sensor from shaking, and ensure the normal operation of the autofocus system.

[0101] In actual working process, the adjustment process of defocus state is as follows: Figure 7 As shown. By comparing the defocus amount with threshold 1 and threshold 2, the current defocus state is determined, and the corresponding movement distance is calculated to ensure that the image sensor can move to the focus position stably and quickly. It is worth noting that when the autofocus system is working, the comparison result of the defocus amount and the defocus amount threshold under 10 consecutive frames of laser images meets the defocus state before the conversion of different defocus states can be performed. If not, the autofocus system continues to maintain the original defocus state unchanged. Based on this, the adverse effects caused by the defocus amount calculation error can be avoided.

[0102] Since autofocus requires the detected object to be in the depth of field at all times, and since the movement rate of the detected object is high, the autofocus operating frequency is required to be faster, but the existing motion control scheme is difficult to meet this application scenario. The above ROI area adjustment process can effectively increase the sampling frequency in motion control, thereby improving the performance of the control method. At the same time, the control method of the driver under the above different defocus states can effectively improve the focusing performance and solve the problem of fast focusing and following focus under various zooms.

[0103] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An automatic focus control circuit, characterized in that: include: The ZYNQ chip includes a programmable logic unit, which is used to calculate the centroid coordinates according to the image of the object to be measured containing the laser spot for focusing, so that the calculation efficiency of the centroid coordinates is improved by the image parallel processing capability of the programmable logic unit; and also includes a first ARM core unit, which is used to calculate the driver focus control information according to the centroid coordinates, and send the driver focus control information to the programmable logic unit, so that the transmission efficiency between the units in the ZYNQ chip is improved; A driver, connected to the programmable logic unit, obtains driver focus control information so that the driver adjusts the distance from the image sensor to the object to be measured, thereby focusing on the object to be measured; The ZYNQ chip also includes a second ARM core unit, which is connected to the host computer through a serial communication module; wherein the serial communication module and the host computer are connected using a 485 chip and an RS-485 bus to enable data transmission based on differential signals, and the data transmission is based on the Modbus communication protocol to reduce the bit error rate of data transmission.

2. The control circuit according to claim 1, characterized in that: The second ARM core unit receives the operation instructions sent by the host computer to adjust the parameters of each unit in the ZYNQ chip or read the data of each unit in the ZYNQ chip.

3. The control circuit according to claim 1, characterized in that: The first ARM core unit and the second ARM core unit share the same storage space; wherein the storage space contains centroid coordinates, driver focus control information and operation instructions.

4. The control circuit according to claim 1, characterized in that: During the data transmission between the first ARM core unit or the second ARM core unit and the storage space, the cache function is turned off.

5. The control circuit according to claim 1, characterized in that: The first ARM core unit is connected to the programmable logic unit via an AXI bus.

6. An automatic focus control method, applied to the automatic focus control circuit according to any one of claims 1 to 5, characterized in that: The control method comprises: Calculate the pixel mean of the first pixel point in each pixel row in the image of the object to be tested as the pixel row mean; wherein the first pixel point represents a pixel point in the image of the object to be tested whose pixel value is higher than a preset first pixel threshold; the image of the object to be tested represents the image obtained by selecting an area of ​​interest in the original image generated by the image sensor; Counting a set of pixel rows whose average values ​​of the pixel rows are higher than a preset second pixel threshold; wherein the elements in the set of pixel rows are the row sequences of the pixel rows; The top and bottom boundaries of the region of interest are adjusted based on the minimum and maximum values ​​of the pixels within the set of pixel rows.

7. The control method according to claim 6, characterized in that: The adjusting the top boundary and the bottom boundary of the region of interest based on the minimum value and the maximum value of the pixels in the pixel row set includes: Determine whether the minimum value of the elements in the pixel row set is 1, if so, move the top boundary of the region of interest upward by a preset number of pixel rows; if not, keep the top boundary of the region of interest unchanged; Determine whether the maximum value of the elements in the pixel row set is n; if so, lower the bottom boundary of the region of interest by a preset number of pixel rows; if not, keep the top boundary of the region of interest unchanged; wherein n represents the total number of pixel rows in the region of interest.

8. An automatic focus control method, applied to the automatic focus control circuit according to any one of claims 1 to 5, characterized in that: The control method comprises: Obtaining the average value of pixels in the image of the object to be tested as the pixel average value, and determining whether the pixel average value exceeds a preset pixel range; if not, terminating the exposure time adjustment process and maintaining the working state of the autofocus control circuit unchanged; If yes, then calculate the new exposure time, and determine whether the new exposure time exceeds the preset exposure time threshold; if no, set the exposure time of the autofocus control circuit to the new exposure time, and end the exposure time adjustment process; If yes, setting the exposure time of the autofocus control circuit to the exposure time threshold, and determining whether the pixel mean under the exposure time threshold exceeds a preset pixel range; if no, ending the exposure time adjustment process; If so, the laser energy of the laser is adjusted based on a preset step size until the pixel mean is within a preset pixel range.

9. The control method according to claim 8, characterized in that: The preset step length is determined based on the lens magnification and the operating current value of the laser.

10. An automatic focus control method, applied to the automatic focus control circuit according to any one of claims 1 to 5, characterized in that: The control method comprises: Determining a defocus state of the image sensor based on the defocus amount; wherein a plurality of defocus states under different defocus amounts are divided using a plurality of preset defocus amount thresholds; Different control modes of the driver are selected based on the defocus state; wherein the control modes include: direct drive, S-curve planning and closed-loop PID algorithm.

Images