An array camera auto focus method, apparatus, and media

Abstract

The application provides an array camera automatic focusing method, device and medium, a micro stepping motor is arranged behind each camera module in the array camera, the method comprises the following steps: the array camera and the micro stepping motor are connected with a terminal device respectively; the micro stepping motor is controlled to rotate by α degrees from an initial position, a key frame of a video stream is obtained, the sharpness score is calculated by using an image evaluation function, N pictures with the highest score are selected, and the score value is recorded as a current highest score G K The micro stepping motor is rotated to the angle corresponding to G K , as a current position; then, the step is reduced, and the score is calculated by taking one step to the left and right of the current position; the highest score point is taken as the current position, the step is continuously reduced, the key frame is taken according to the current position in a pendulum or progressive mode, the current best position is continuously adjusted, and the focusing is completed until the micro stepping motor cannot be rotated. The application can realize efficient, automatic and accurate focusing of the array camera and support manual fine adjustment.

Description

Technical Field

[0001] The present invention relates to the field of array camera focusing technology, and in particular to an array camera automatic focusing method, device and medium. Background Art

[0002] An array camera is a type of camera composed of multiple cameras arranged in a geometric pattern (for example, three rows and six columns). This approach uses multiple small lenses to replace the effects of a single large lens. Its principle is similar to that of array astronomical telescopes and the compound eyes of insects. Its design and implementation comprehensively consider camera synchronization, image fusion algorithms, computing resources, and system stability. With the development of computational photography and computer vision technologies, array cameras are increasingly being used in various fields.

[0003] As an emerging device in the computational imaging field, array cameras offer a promising solution for real-time monitoring of large scenes. Adjusting the focus of a camera module affects the clarity of individual video streams, directly impacting the effectiveness of video fusion and machine recognition.

[0004] Currently, manual focus is the primary method for focusing array cameras. Array cameras typically use fixed-focus lens modules, which are arranged compactly. Each lens module has a manual focus ring for the operator to manually focus. Video from each lens module is directly connected to a computer via a network cable for the operator to view. Relying solely on manual focus for array cameras can lead to the following problems:

[0005] 1. Easy to accidentally touch and inaccurate. After the camera is set up and the angle is adjusted, the operator must both view the video stream of a single camera module on a computer and manually adjust the focus ring to achieve clear video from that module. However, the equipment's internal space is small, making it difficult to fully penetrate the lens module. Manually adjusting the focus of a single module inevitably involves touching other modules, causing the focus ring of another module to be accidentally touched. This can easily lead to excessive rotation, resulting in suboptimal results.

[0006] 2. High construction difficulty. Because array cameras are typically used for large-scale surveillance, they are often installed at high altitudes, typically on rooftops, mountaintops, signal towers, and other locations. This requires qualified operators for high-altitude operations, while debugging and focusing require computer-related expertise. If two people are involved in the installation, with one focusing at a high altitude and the other checking video clarity on a computer at a lower level, communication difficulties and delays often occur, impacting the final result. When installed outdoors at high altitude, direct sunlight can also blur laptop screens. Furthermore, prolonged exposure to high temperatures can reduce the damping of the focus ring, significantly increasing the difficulty of adjustment.

[0007] 3. Inefficiency. Manual focus requires adjusting the camera modules one by one, and also requires checking the video on the PC to confirm whether the focus is accurate. This makes the entire adjustment process inefficient and time-consuming. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide an array camera automatic focusing method, device and medium to overcome the problems of existing array camera manual focusing operation such as difficulty, low precision and low efficiency.

[0009] In a first aspect, the present invention provides an array camera autofocus method, wherein each camera module in the array camera is provided with a micro stepping motor, the method comprising the following steps:

[0010] Step S1: The array camera and the micro-stepping motor are respectively connected to a terminal device, and each micro-stepping motor number is bound to the corresponding camera module IP address through the terminal device;

[0011] Step S2: Adjust all micro-stepping motors to their initial positions. When the micro-stepping motor is used to adjust the focus ring from one end to the other, record the total rotation angle and divide it into N equal parts, with each part serving as the initial step α.

[0012] Step S3: performing an autofocus operation on each camera module until all camera modules complete autofocusing. The autofocus operation specifically includes:

[0013] Step S31: Rotate the micro-stepping motor α degrees from its initial position, and use the terminal device to pull the stream to obtain a key frame, recorded as Pi. Calculate the clarity score Gi for Pi using the image evaluation function. Rotate it N-1 times in total to obtain N images with scores.

[0014] Step S32: Select the picture with the highest score among the N pictures and record its score as the current highest score G. K, rotate the micro stepper motor to G K The corresponding angle is used as the current position;

[0015] Step S33: Set the current step β to half of the original value, and determine whether the current step is less than the minimum step length of the stepper motor. If so, the focus is successful and the debugging of the camera module is terminated. Otherwise, execute step S34.

[0016] Step S34: Rotate β degrees to both sides of the current position, pull out a key frame respectively, and calculate the evaluation value G of each image by the image evaluation function. K1 , G K2 , G K , G K1 and G K2 For comparison, if G K1 <=G K And G K2 <=G K , then return to step S33; otherwise, go to step S35;

[0017] Step S35: G with the maximum value K1 or G K2 As G K , rotate the micro stepper motor to the maximum G K1 or G K2 The corresponding angle is used as the current position, and the step direction is adjusted to the maximum G K1 or G K2 direction;

[0018] Step S36: Reduce the current step β by half and determine whether the current step is less than the minimum step length of the stepping motor. If so, the focus is successful and the debugging of the camera module is terminated. Otherwise, execute step S37.

[0019] Step S37: Rotate β degrees according to the adjusted step direction and take its key frame P M And the evaluation value G is obtained by calculating the image evaluation function M , if G M >G K , then G M As the current highest rated G K , the stepping direction remains unchanged and returns to step S37; otherwise, the stepping direction is reversed and returns to step S36.

[0020] Furthermore, the value of N is 20.

[0021] Furthermore, the image evaluation function is specifically as follows: a gradient matrix is ​​obtained by convolving the grayscale value of each pixel of the image with the Laplace operator, and the sum of the squares of the gradients of each pixel is taken as the image evaluation function value.

[0022] Furthermore, before step S37, it also includes: judging whether the corresponding direction is rotatable; if the corresponding direction is rotatable but not enough to rotate β degrees, the maximum rotatable angle is used as the current β value, and step S37 is continued; if the corresponding direction is not rotatable, the focus is considered to be successful, and the debugging of the camera is ended.

[0023] Furthermore, the method further includes step S4: after each camera component completes the autofocus operation, the terminal device displays a multi-stream playback interface for the user to view and review.

[0024] Furthermore, the method also includes the user zooming in on the video screen of the video stream and performing manual remote focus, and fine-tuning the current camera module through the "←" and "→" keys on the keyboard, each press representing the stepper motor rotating counterclockwise / clockwise by the corresponding step, and the corresponding step is fine-tuned through the "↑" and "↓" keys, each press representing the corresponding step increasing / decreasing by one fold, and the minimum is the minimum rotation angle of the stepper motor.

[0025] Furthermore, a Bluetooth communication module is added to the micro stepper motor, the micro stepper motor is connected to the terminal device via Bluetooth and is controlled by the terminal device, the array camera is connected to the device terminal via a network cable, and the micro stepper motor is connected to the focus ring on the camera module via gears.

[0026] In a second aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0027] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0028] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:

[0029] 1. A pendulum-like calculation with halved steps gradually approaches the final focus position, retaining only the current data for each calculation. This cycle of stepping, detecting, comparing, stepping, detecting, comparing, etc. is widely applicable to embedded camera devices, while also improving focus accuracy.

[0030] 2. By incorporating a stepper motor into the array camera, construction workers can focus on construction operations. Commissioning personnel can connect directly to the array camera via a computer network cable and to the stepper motor via Bluetooth, enabling simultaneous viewing and focusing. This effectively avoids accidental contact between camera modules caused by operating the focus ring in a narrow area.

[0031] 3. Support viewing on the terminal device and support individual review and focus of each sub-lens module to achieve the best focusing effect;

[0032] 4. It can realize full automatic focusing of multiple sub-lenses at the same time, which is highly efficient and can realize the extraction and calculation of real-time video streams. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0034] Figure 1 This is a flowchart of an automatic focusing method for an array camera according to the present invention.

[0035] Figure 2 The figure is a schematic diagram of the auto-focus process of each camera module of the present invention.

[0036] Figure 3 This is the hardware topology diagram of a single camera module of the present invention.

[0037] Figure 4 This is a schematic diagram of the overall process of the present invention combined with manual focus.

[0038] Figure 5 Schematic diagram of the initial operation activities of the present invention.

[0039] Figure 6 This is a schematic diagram of the specific calculation control flow of the auto-focus of the present invention.

[0040] Figure 7 This is a schematic structural diagram of an electronic device in a second embodiment of the present invention;

[0041] Figure 8 Schematic diagram of the structure of the medium in embodiment 3 of the present invention DETAILED DESCRIPTION

[0042] The present invention provides an array camera autofocus method, device, and medium to address the existing issues of difficult and unsatisfactory focusing in existing array cameras. The technical solution in the present invention is based on the following general principles: a micro-stepping motor is built into each sub-lens of the array camera, connected to the focus ring on the sub-lens. The array camera and micro-stepping motor are then connected to a debugging device terminal (such as a computer or mobile phone). This allows for fully automated streaming, analysis, and focusing on the device terminal. Specifically, each sub-lens is controlled to perform autofocus operations independently. The autofocus operation uses an initial large step position, followed by a gradually smaller step size, performing a pendulum-like search. The current position is continuously adjusted until no further rotation is possible, resulting in the final focus result. After all cameras have been autofocused, the user can simultaneously review and audit multiple streams on a computer. If unsatisfactory results are encountered, the corresponding video stream can be zoomed in and fine-tuned using the "←" and "→" keys on the keyboard. Each press rotates the stepper motor counterclockwise / clockwise by the current step value β degrees. You can also fine-tune β by pressing the "↑" and "↓" buttons. Each press doubles β, with the minimum being the minimum rotation angle of the stepping motor. This allows for optimal focusing.

[0043] To make the present invention more clearly understood, a preferred embodiment is now described in detail below with reference to the accompanying drawings.

[0044] Example 1

[0045] This embodiment provides an array camera auto-focusing method, wherein each camera module in the array camera is provided with a micro stepping motor. Figures 1 to 6 As shown, the method includes the following steps:

[0046] Step S1: The array camera and the micro-stepping motor are respectively connected to a terminal device, and each micro-stepping motor number is bound to the corresponding camera module IP address through the terminal device;

[0047] Step S2: Adjust all micro-stepping motors to their initial positions (i.e., rotate the focus ring to one end). When the micro-stepping motor is used to adjust the focus ring from one end to the other, the total rotation angle is recorded and divided into N equal parts, for example, N is 20, and each equal part is used as the initial step α;

[0048] Step S3: performing an autofocus operation on each camera module until all camera modules complete autofocusing. The autofocus operation specifically includes:

[0049] Step S31: Rotate the micro-stepping motor α degrees from its initial position, and use the terminal device to pull the stream to obtain a key frame, recorded as Pi. Calculate the clarity score Gi for Pi using the image evaluation function. Rotate it N-1 times in total to obtain N images with scores.

[0050] Step S32: Select the picture with the highest score among the N pictures and record its score as the current highest score G. K , rotate the micro stepper motor to G K The corresponding angle is used as the current position; only the current information is recorded to avoid taking up too much storage space on the embedded device.

[0051] Step S33: Set the current step β to half of the original value (the initial value is β = α / 2), and determine whether the current step is less than the minimum step length of the stepper motor. If so, the focus is successful and the debugging of the camera module is terminated. Otherwise, execute step S34;

[0052] Step S34: Rotate β degrees to both sides of the current position (first rotate β degrees clockwise, then rotate 2β degrees counterclockwise), pull out a key frame from each, and calculate the evaluation value G of each image through the image evaluation function. K1 , G K2 , G K , G K1 and G K2 For comparison, if G K1 <=G K And G K2 <=G K , then return to step S33; otherwise, if G K1 and G K2 There is a ratio of G K When the value is larger (such as G K2 <=G K <=G K1 , G K1 <=G K <=G K2 ) proceed to step S35;

[0053] Step S35: G with the maximum value K1 or G K2 As G K , rotate the micro stepper motor to the maximum G K1 or G K2 The corresponding angle is used as the current position, and the step direction is adjusted to the maximum G K1 or G K2 Direction, that is, stepping towards greater clarity;

[0054] Step S36: Reduce the current step β by half and determine whether the current step is less than the minimum step length of the stepping motor. If so, the focus is successful and the debugging of the camera module is terminated. Otherwise, execute step S37.

[0055] Step S37: Rotate β degrees according to the adjusted step direction and take its key frame P M And the evaluation value G is obtained by calculating the image evaluation function M , if G M >G K , indicating that it is currently turning in a clear direction, then G M As the current highest rated G K , the stepping direction remains unchanged and the process returns to step S37; otherwise, an over-rotation has occurred, the stepping direction is reversed and the process returns to step S36. This process may require recalculation of certain positions. Since the image is dynamic, the earlier resolution may not be the current resolution, so recalculation is required to achieve real-time updates.

[0056] Preferably, the image evaluation function is specifically as follows: a gradient matrix is ​​obtained by convolving the grayscale value of each pixel of the image with the Laplace operator, and the sum of the squares of the gradients of each pixel is taken as the image evaluation function value. In the evaluation of image clarity, a clear image has sharper edges and a significant change in grayscale value than a blurred image, and therefore has a larger gradient value. When processing an image, we regard it as a two-dimensional discrete matrix and extract grayscale information through a gradient function to evaluate the clarity of the image. In discrete signals, the gradient is usually expressed in differential form. Common gradient functions include energy gradient function (EOG), Roberts operator, Tenengrad operator, Brenner operator, variance (Variance) and Laplace operator. Here, a gradient matrix is ​​obtained by convolving the grayscale value of each pixel of the image with the Laplace operator, and the sum of the squares of the gradients of each pixel is taken as the evaluation function. The whole process is simple to calculate and has high accuracy.

[0057] Preferably, the step S37 further includes: determining whether the corresponding direction is rotatable; if the corresponding direction is rotatable but not enough to rotate β degrees, the maximum rotatable angle is used as the current β value, and step S37 is continued; if the corresponding direction is not rotatable (i.e., the maximum rotatable angle at the end is less than the minimum step length of the stepper motor, making it impossible to rotate), the focus is considered successful, and the debugging of the camera is ended.

[0058] Preferably, the method further comprises step S4, after each camera component completes the autofocus operation, the terminal device displays a multi-stream playback interface for the user to view and review.

[0059] Preferably, the method also includes the user zooming in on the video screen of the video stream and performing manual remote focus, and fine-tuning the current camera module through the "←" and "→" keys on the keyboard, each press representing the stepper motor rotating counterclockwise / clockwise by the corresponding step, and the corresponding step is fine-tuned through the "↑" and "↓" keys, each press representing the corresponding step increasing / decreasing by one time, and the minimum is the minimum rotation angle of the stepper motor.

[0060] Preferably, a Bluetooth communication module is added to the micro stepping motor, the micro stepping motor is connected to the terminal device via Bluetooth and is controlled by the terminal device, the array camera is connected to the device terminal via a network cable, and the micro stepping motor is connected to the focus ring on the camera module via gears, thereby realizing remote focusing.

[0061] Based on the same inventive concept, this application provides an electronic device embodiment corresponding to the first embodiment, see the second embodiment for details.

[0062] Example 2

[0063] This embodiment provides an electronic device, such as Figure 7 As shown, it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any implementation method in the first embodiment can be implemented.

[0064] Since the electronic device described in this embodiment is the device used to implement the method in Example 1 of this application, based on the method described in Example 1 of this application, those skilled in the art will be able to understand the specific implementation of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of this application will not be described in detail here. As long as the device used by those skilled in the art to implement the method in the embodiment of this application falls within the scope of protection to be provided by this application.

[0065] Based on the same inventive concept, this application provides a storage medium corresponding to Example 1, see Example 3 for details.

[0066] Example 3

[0067] This embodiment provides a computer-readable storage medium, such as Figure 8 As shown, a computer program is stored thereon, and when the computer program is executed by a processor, any implementation method in Example 1 can be implemented.

[0068] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages: by adding a stepper motor design to the array camera, construction workers can focus on construction operations, and debugging personnel can directly connect to the array camera via a computer network cable at a low location and connect to the stepper motor via Bluetooth, so that viewing and focusing can be carried out simultaneously in one place. By first locating the optimal position in the large step, a more accurate focus position is calculated through continuous pendulum-like real-time search, thereby improving the focus accuracy, and supporting remote manual focus operations on the computer for individual sub-lenses with poor focus effects, thereby achieving the best focus effect. The entire process can achieve fully automatic focus with high efficiency.

[0069] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. An array camera autofocus method, characterized by: Each camera module in the array camera is provided with a micro stepping motor, and the method comprises the following steps: Step S1: The array camera and the micro-stepping motor are respectively connected to a terminal device, and each micro-stepping motor number is bound to the corresponding camera module IP address through the terminal device; Step S2: Adjust all micro-stepping motors to their initial positions. When the micro-stepping motor is used to adjust the focus ring from one end to the other, record the total rotation angle and divide it into N equal parts, with each part serving as the initial step α. Step S3: performing an autofocus operation on each camera module until all camera modules complete autofocusing. The autofocus operation specifically includes: Step S31: Rotate the micro-stepping motor α degrees from its initial position, and use the terminal device to pull the stream to obtain a key frame, recorded as Pi. Calculate the clarity score Gi for Pi using the image evaluation function. Rotate it N-1 times in total to obtain N images with scores. Step S32: Select the picture with the highest score among the N pictures and record its score as the current highest score G. K , rotate the micro stepper motor to G K The corresponding angle is used as the current position; Step S33: Set the current step β to half of the original value, and determine whether the current step is less than the minimum step length of the stepper motor. If so, the focus is successful and the debugging of the camera module is terminated. Otherwise, execute step S34. Step S34: Rotate β degrees to both sides of the current position, pull out a key frame respectively, and calculate the evaluation value G of each image by the image evaluation function. K1 , G K2 , G K , G K1 and G K2 For comparison, if G K1 <=G K And G K2 <=G K , then return to step S33; Otherwise, proceed to step S35; Step S35: G with the maximum value K1 or G K2 As G K , rotate the micro stepper motor to the maximum G K1 or G K2 The corresponding angle is used as the current position, and the step direction is adjusted to the maximum G K1 or G K2 direction; Step S36: Reduce the current step β by half and determine whether the current step is less than the minimum step length of the stepping motor. If so, the focus is successful and the debugging of the camera module is terminated. Otherwise, execute step S37. Step S37: Rotate β degrees according to the adjusted step direction and take its key frame P M And the evaluation value G is obtained by calculating the image evaluation function M , if G M >G K , then G M As the current highest rated G K , the stepping direction remains unchanged and returns to step S37; otherwise, the stepping direction is reversed and returns to step S36.

2. The automatic focusing method for an array camera according to claim 1, wherein: The value of N is 20.

3. The automatic focusing method for an array camera according to claim 1, wherein: The image evaluation function is specifically as follows: a gradient matrix is ​​obtained by convolving the Laplace operator with the grayscale value of each pixel in the image, and the square sum of the gradients of each pixel is taken as the image evaluation function value.

4. The automatic focusing method for an array camera according to claim 1, wherein: Before step S37, the process also includes: determining whether the camera is rotatable in the corresponding direction; if the camera is rotatable in the corresponding direction but not by β degrees, taking the maximum rotatable angle as the current β value, and proceeding to step S37; if the camera is not rotatable in the corresponding direction, the focusing is deemed successful, and the debugging of the camera is terminated.

5. The automatic focusing method for an array camera according to claim 1, wherein: The method further includes step S4: after each camera component completes the autofocus operation, the terminal device displays a multi-stream playback interface for the user to view and review.

6. The automatic focusing method for an array camera according to claim 1, wherein: The method also includes the user zooming in on the video screen of the video stream and performing manual remote focus, and fine-tuning the current camera module using the "←" and "→" keys on the keyboard, each press representing a counterclockwise / clockwise rotation of the stepper motor by a corresponding step, and the corresponding step being fine-tuned using the "↑" and "↓" keys, each press representing a halving of the corresponding step, with the minimum being the minimum rotation angle of the stepper motor.

7. The automatic focusing method for an array camera according to claim 1, wherein: A Bluetooth communication module is added to the micro stepping motor. The micro stepping motor is connected to the terminal device via Bluetooth and is controlled by the terminal device. The array camera is connected to the device terminal via a network cable. The micro stepping motor is connected to the focus ring on the camera module via a gear.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images