Focal length adjustment method and device, equipment and storage medium

Abstract

The invention discloses a focal length adjustment method and device, equipment and a storage medium. The method is applied to a tracking system, the tracking system comprises a laser radar and an image acquisition device, and the method comprises the following steps: acquiring point cloud data acquired by the laser radar and an image acquired by the image acquisition device; determining a target distance between an image acquisition device and a target object according to the point cloud data; determining a to-be-selected focal length set according to the target distance between the image acquisition equipment and the target object, the parameters of the image acquisition equipment, the size of the target object and the proportion of the target object in the image; and determining the target focal length according to the definition of the image in the image set corresponding to the to-be-selected focal length set. According to the technical scheme of the invention, the focal length adjustment efficiency and accuracy can be improved, and the definition requirement of the target object moving at a high speed can be met.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a focal length adjustment method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of image processing and computer technologies, autofocus algorithms have entered the digital era based on image analysis and have begun to be applied in various fields, playing an increasingly important role.

[0003] Existing autofocus algorithms find the sharpest photo within the range of the image acquisition device's maximum and minimum focal lengths to determine the focal length. This means that a lot of image sharpness data needs to be acquired, resulting in a large amount of computation.

[0004] When tracking a fast-moving target, different focal lengths need to be used depending on the required sharpness of the target object. However, existing focal length adjustment methods, due to their high computational complexity, cannot quickly obtain the focal length corresponding to high sharpness in such situations. Summary of the Invention

[0005] This application provides a focal length adjustment method, apparatus, device, and storage medium, which can improve the efficiency and accuracy of focal length adjustment, thereby meeting the sharpness requirements of high-speed moving target objects.

[0006] According to one aspect of this application, a focal length adjustment method is provided, comprising:

[0007] Acquire point cloud data collected by lidar and images acquired by image acquisition devices;

[0008] Determine the target distance between the image acquisition device and the target object based on point cloud data;

[0009] The set of candidate focal lengths is determined based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image.

[0010] The target focal length is determined based on the sharpness of the images in the image set corresponding to the set of candidate focal lengths.

[0011] According to another aspect of this application, a focus adjustment device is provided, the focus adjustment device comprising:

[0012] The acquisition module is used to acquire point cloud data collected by the lidar and images acquired by the image acquisition device;

[0013] The target distance determination module is used to determine the target distance between the image acquisition device and the target object based on point cloud data;

[0014] The module for determining the set of candidate focal lengths is used to determine the set of candidate focal lengths based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image.

[0015] The target focal length determination module is used to determine the target focal length based on the sharpness of the images in the image set corresponding to the candidate focal length set.

[0016] According to another aspect of this application, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory that is communicatively connected to at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the focus adjustment method described in any embodiment of this application.

[0020] According to another aspect of this application, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the focus adjustment method described in any embodiment of this application.

[0021] According to another aspect of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the focus adjustment method described in any embodiment of this application.

[0022] This application embodiment acquires point cloud data from a LiDAR scanner and images from an image acquisition device; determines the target distance between the image acquisition device and the target object based on the point cloud data; determines a set of candidate focal lengths based on the target distance, image acquisition device parameters, target object size, and the target object's proportion in the image; and determines the target focal length based on the image sharpness of the images in the corresponding image set. This solves the problem of high computational complexity caused by searching for the sharpest image within the maximum and minimum focal length range of the image acquisition device to determine the focal length, thus narrowing the focal length adjustment range and reducing computational load. Furthermore, when tracking high-speed moving targets, frequent focal length switching is required. If the method of finding the sharpest image within the maximum and minimum focal length range of the image acquisition device is used to determine the focal length, it will lead to the problem of not being able to quickly obtain the focal length corresponding to high sharpness. Since the technical solution provided by this invention can narrow the focal length adjustment range, reduce computational load, and improve focal length adjustment efficiency, it can quickly obtain the focal length corresponding to high sharpness to meet the sharpness requirements of high-speed moving targets.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a focal length adjustment method provided in Embodiment 1 of this application;

[0026] Figure 2 This is a flowchart of a focal length adjustment method provided in Embodiment 2 of this application;

[0027] Figure 3 This is a flowchart of a focal length adjustment method provided in Embodiment 3 of this application;

[0028] Figure 4 This is a flowchart of a focal length adjustment method provided in Embodiment 4 of this application;

[0029] Figure 5 This is a flowchart of a focal length adjustment method provided in Embodiment 5 of this application;

[0030] Figure 6 This is a flowchart of a focal length adjustment method provided in Embodiment Six of this application;

[0031] Figure 7 This is a schematic diagram of the structure of a focal length adjustment device provided in Embodiment 7 of this application;

[0032] Figure 8 This is a schematic diagram of the structure of an electronic device according to Embodiment 8 of this application. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0036] To facilitate understanding, the background technology will now be explained in detail.

[0037] In multi-sensor hybrid sensing scenarios, because the position of the target object is constantly changing and the target object is not unique, the image acquisition device needs to continuously adjust its focal length. Especially when the target object occupies a small portion of the acquisition area, in order to make the target object relatively clear, it is usually necessary to "close the focal length," that is, change the focal length of the image acquisition device. In particular, for high-speed moving targets (such as drones / spacecraft moving rapidly in the sky; motor vehicles traveling on highways), because the position of the high-speed moving target object is constantly changing, in order to ensure the sharpness of the target object, the focal length of the image acquisition device needs to be constantly and rapidly adjusted; otherwise, the focal length issue will directly affect the sharpness of the target object.

[0038] However, existing focus adjustment methods evaluate image sharpness by searching for the sharpest image within the range of maximum and minimum focal lengths to determine the focal length. This means acquiring a large amount of image sharpness data, resulting in significant computational complexity. Furthermore, simple image acquisition devices typically evaluate sharpness based on the overall image (image acquisition devices generally cannot identify individual target objects). When tracking fast-moving targets, the sharpness requirement for the target object is higher than the overall image sharpness requirement. Moreover, if there are multiple target objects in different locations within the acquisition area, different focal lengths need to be used based on the sharpness requirements of each target object. In these situations, existing focus adjustment methods struggle to quickly obtain the focal length corresponding to high sharpness.

[0039] Example 1

[0040] Figure 1 This is a flowchart of a focus adjustment method provided in Embodiment 1 of this application. This embodiment is applicable to focus adjustment in scenarios involving tracking high-speed moving target objects. The method can be executed by the focus adjustment device in this embodiment, which can be implemented in software and / or hardware, such as... Figure 1 As shown, the method specifically includes the following steps:

[0041] S110 acquires point cloud data collected by the lidar and images acquired by the image acquisition device.

[0042] The image acquisition device can be a camera.

[0043] It should be noted that this application provides a tracking system, which includes a LiDAR, an image acquisition device, and a computing platform. The acquisition areas of the LiDAR and the image acquisition device may not be exactly the same, but their acquisition areas must overlap, and the target object must be located within the overlapping area. Preferably, the acquisition areas of the LiDAR and the image acquisition device overlap as much as possible, i.e., the percentage of overlapping area is as high as possible.

[0044] The focal length adjustment method provided in this embodiment is executed by a computing platform. A LiDAR and an image acquisition device are used for target object tracking. The LiDAR and image acquisition device are connected to the computing platform through corresponding communication interfaces. After the LiDAR acquires point cloud data, it sends the point cloud data to the computing platform through the communication interface. Similarly, after the image acquisition device acquires an image, it sends the image to the computing platform through the communication interface. The computing platform uses the point cloud data from the LiDAR and the image / video stream data from the image acquisition device to perform automatic focusing and image tracking of the image acquisition device's lens.

[0045] Among them, the lidar is generally a multi-line lidar, which scans at a fixed frequency to obtain obstacle angle and distance data and transmits it to the computing platform via protocol. The computing platform parses the protocol and processes it.

[0046] Among them, the image acquisition device is used for image and video stream acquisition. It can automatically focus on dynamic target objects through a variable zoom lens and maintain the tracking of target objects through high-definition resolution and high frame rate data streams.

[0047] Specifically, the lidar and image acquisition equipment are jointly calibrated in advance, and then the point cloud data collected by the lidar and the images collected by the image acquisition equipment are acquired.

[0048] S120, determine the target distance between the image acquisition device and the target object based on point cloud data.

[0049] Specifically, determining the target distance between the image acquisition device and the target object based on point cloud data can be achieved as follows: First, determine the distance between the LiDAR and the target object based on the point cloud data; second, determine the target distance between the image acquisition device and the target object based on the positional relationship between the LiDAR and the image acquisition device and the distance between the LiDAR and the target object. Alternatively, the method can be as follows: First, determine the distance between the LiDAR and the target object based on the point cloud data; third, determine a first distance between the image acquisition device and the target object based on the positional relationship between the LiDAR and the image acquisition device and the distance between the LiDAR and the target object; fourth, perform image recognition to obtain a second distance between the image acquisition device and the target object; fifth, determine the target distance between the image acquisition device and the target object based on the first distance and the second distance.

[0050] Optionally, the target distance between the image acquisition device and the target object is determined based on the point cloud data, including:

[0051] The distance between the lidar and the target object is determined based on point cloud data;

[0052] The target distance between the image acquisition device and the target object is determined based on the positional relationship between the lidar and the image acquisition device, and the distance between the lidar and the target object.

[0053] It should be noted that the lidar and image acquisition equipment are calibrated to obtain their positional relationship.

[0054] Specifically, the method for determining the distance between the LiDAR and the target object based on point cloud data can be as follows: determine the position coordinates of the target object based on the point cloud data, and then determine the distance between the LiDAR and the target object based on the position coordinates of the target object.

[0055] Optionally, the target distance between the image acquisition device and the target object is determined based on the point cloud data, including:

[0056] The distance between the lidar and the target object is determined based on point cloud data;

[0057] The first distance between the image acquisition device and the target object is determined based on the positional relationship between the lidar and the image acquisition device and the distance between the lidar and the target object;

[0058] Image recognition is used to obtain a second distance between the image acquisition device and the target object;

[0059] The target distance between the image acquisition device and the target object is determined based on the first distance between the image acquisition device and the target object and the second distance between the image acquisition device and the target object.

[0060] Specifically, determining the distance between the LiDAR and the target object based on point cloud data can be achieved by: determining the target object's position coordinates based on the point cloud data, and then determining the distance between the LiDAR and the target object based on the target object's position coordinates. One method for determining the target object's position coordinates based on point cloud data is to cluster the point cloud data to obtain the target object's position coordinates. Another method is to input the point cloud data into a trained target object detection model to obtain detection results, where the detection results include the target object's position coordinates.

[0061] It should be noted that if the image acquisition device is a stereo camera, the second distance between the image acquisition device and the target object can be determined based on the images acquired by the stereo camera.

[0062] The positional relationship between the lidar and the image acquisition device includes: the distance between the lidar and the image acquisition device, the installation angle of the lidar, and the installation angle of the image acquisition device.

[0063] Specifically, the method for determining the first distance between the image acquisition device and the target object based on the positional relationship between the lidar and the image acquisition device and the distance between the lidar and the target object can be as follows: the first distance between the image acquisition device and the target object is determined based on the distance between the lidar and the image acquisition device, the installation angle of the lidar, the installation angle of the image acquisition device, and the distance between the lidar and the target object.

[0064] Specifically, the target distance between the image acquisition device and the target object can be determined by two methods: 1) using the first distance between the image acquisition device and the target object and 2) using the second distance between the image acquisition device and the target object. 3) Using the maximum value between the first distance and the second distance. 4) Using the minimum value between the first distance and the second distance.

[0065] S130, determine the set of candidate focal lengths based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image.

[0066] The focal length of an image acquisition device refers to the distance from the optical center of the lens to the focal point where the light converges when parallel light is incident.

[0067] The image acquisition device parameters may include: the target surface imaging width of the image acquisition device. The target object size can be the actual width of the target object. The proportion of the target object in the image can be obtained by: identifying the target object in the image to obtain the detection box corresponding to the target object, and determining the proportion of the target object in the image as the ratio of the area of ​​the detection box corresponding to the target object to the total area of ​​the image.

[0068] Specifically, the method for determining the set of candidate focal lengths based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image can be as follows: determine the initial focal length based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image, and determine the set of candidate focal lengths based on the initial focal length and preset values.

[0069] S140, determine the target focal length based on the sharpness of the images in the image set corresponding to the set of candidate focal lengths.

[0070] Specifically, determining the target focal length based on the image sharpness in the image set corresponding to the candidate focal length set can be done by: determining the focal length corresponding to the image with the highest sharpness in the image set corresponding to the candidate focal length set as the target focal length. Alternatively, determining the target focal length based on the image sharpness in the image set corresponding to the candidate focal length set can be done by: obtaining at least three reference focal lengths per unit length from the candidate focal length set, and determining the target focal length based on the sharpness of each reference focal length and the corresponding image.

[0071] The technical solution of this embodiment determines a set of candidate focal lengths based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image. The target focal length is then determined based on the sharpness of the images in the corresponding image set. This solves the problem of excessive computation caused by searching for the sharpest image within the maximum and minimum focal length range of the image acquisition device to determine the focal length. It narrows the focal length adjustment range, thereby reducing computational load. Furthermore, when tracking a high-speed moving target object, frequent focal length switching is required. If the method of finding the sharpest image within the maximum and minimum focal length range of the image acquisition device is used to determine the focal length, it will lead to the problem of not being able to quickly obtain the focal length corresponding to high sharpness. Since the technical solution provided by this embodiment can narrow the focal length adjustment range, reduce computational load, and improve focal length adjustment efficiency, it can quickly obtain the focal length corresponding to high sharpness to meet the sharpness requirements of high-speed moving target objects.

[0072] Example 2

[0073] Figure 2 This is a flowchart illustrating a focal length adjustment method provided in Embodiment 2 of this application, based on the above embodiments. For example... Figure 2 As shown, the method includes:

[0074] S210 acquires point cloud data collected by the lidar and images acquired by the image acquisition device.

[0075] S220 determines the target distance between the image acquisition device and the target object based on point cloud data.

[0076] S230, determine the initial focal length based on the distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, the proportion of the target object in the image, and the first formula.

[0077] Among them, the image acquisition device parameters are the target surface imaging parameters of the image acquisition device, for example, the image acquisition device parameters can be the target surface imaging width of the image acquisition device.

[0078] Specifically, the method for determining the initial focal length based on the distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, the proportion of the target object in the image, and the first formula can be as follows: obtain the first product of the distance between the image acquisition device and the target object, the parameters of the image acquisition device, and the proportion of the target object in the image, and determine the initial focal length by the ratio of the first product to the size of the target object.

[0079] Optional parameters for the image acquisition device include: the target surface imaging width of the image acquisition device;

[0080] The first formula is:

[0081]

[0082] Where f is the initial focal length, d is the distance between the image acquisition device and the target object, w is the target surface imaging width of the image acquisition device, u is the size of the target object, and k is the proportion of the target object in the image.

[0083] In this embodiment, when tracking a moving target object, the target distance is determined based on the point cloud data collected by the lidar and the image collected by the image acquisition device. The computing platform determines a set of candidate focal lengths based on the target distance, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image. The motor drives the lens to complete focusing. The target focal length is found based on the image clarity corresponding to each candidate focal length in the set of candidate focal lengths to complete automatic focusing. The computing platform continuously performs cluster detection based on the currently obtained image of the target object to maintain the tracking and recognition of the moving target object.

[0084] S240 determines the set of selectable focal lengths based on the initial focal length and preset parameters.

[0085] Specifically, the method for determining the set of candidate focal lengths based on the initial focal length and preset parameters can be as follows: If the preset parameter is h and the initial focal length is f, then the set of candidate focal lengths consists of focal lengths fh, f, and f+h. Another method is to determine the set of candidate focal lengths based on the initial focal length and preset parameters by determining the minimum and maximum candidate focal lengths in the set, and then generating the final set of candidate focal lengths based on these minimum and maximum focal lengths. For example, if the preset parameter is h and the initial focal length is f, then the set of candidate focal lengths is determined to be [fh, f+h], where fh is the lower limit of the candidate focal lengths and f+h is the upper limit of the candidate focal lengths. Another way to determine the set of candidate focal lengths based on the initial focal length and preset parameters is as follows: If the preset parameters include a first preset parameter h1 and a second preset parameter h2, then the minimum candidate focal length in the set is determined to be f-h1, and the maximum candidate focal length is determined to be f+h2, thus determining the set of candidate focal lengths as [f-h1, f+h2]. Alternatively, if the preset parameters include a first preset parameter h1 and a second preset parameter h2, then the set of candidate focal lengths includes only the candidate focal lengths: f-h1, f, and f+h2.

[0086] S250 determines the target focal length based on the sharpness of the images in the image set corresponding to the set of candidate focal lengths.

[0087] Optionally, the preset parameters include: a first preset parameter h1 and a second preset parameter h2, and the set of selectable focal lengths is [f-h1, f+h2], where f is the initial focal length;

[0088] The target focal length is determined based on the sharpness of the images in the image set corresponding to the candidate focal length set, including:

[0089] According to the unit length, obtain no less than 3 reference focal lengths from the set of candidate focal lengths, and the reference focal lengths shall include at least f+h2 or f-h1;

[0090] Obtain the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths;

[0091] The target focal length is determined based on the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths.

[0092] The first preset parameter and the second preset parameter can be the same or different, and this application embodiment does not impose any restrictions on this. The unit length can be a preset length.

[0093] Specifically, obtaining at least three reference focal lengths from the set of candidate focal lengths based on a unit length, where the reference focal lengths must include at least f+h2 or f-h1, can be achieved by: using f-h1 as the first reference focal length, and selecting at least two reference focal lengths from the set of candidate focal lengths based on f-h1 and the unit length. Alternatively, obtaining at least three reference focal lengths from the set of candidate focal lengths based on a unit length, where the reference focal lengths must include at least f+h2 or f-h1, can also be achieved by: using f+h2 as the first reference focal length, and selecting at least two reference focal lengths from the set of candidate focal lengths based on f+h2 and the unit length.

[0094] In a specific example, the set of potential focal lengths is [9.5-10.5]. If the unit length is 0.1, the reference focal lengths include: 9.5, 9.6, 9.7, ..., 10.5. If the unit length is 0.4, the reference focal lengths can be: 9.5, 9.9, 10.3; or 9.7, 10.1, 10.5.

[0095] Specifically, determining the target focal length based on the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths can be achieved by: determining the reference focal length corresponding to the reference image with the highest sharpness as the target focal length. Alternatively, this can be done by: obtaining a functional relationship between focal lengths and the sharpness of their corresponding images in the set of candidate focal lengths through linear fitting; determining the focal length corresponding to the image with the highest sharpness in the set of candidate focal lengths based on this functional relationship; and then determining the focal length corresponding to the image with the highest sharpness in the set of candidate focal lengths as the target focal length.

[0096] The technical solution of this embodiment can more accurately locate the set of selectable focal lengths and improve the efficiency of focal length adjustment by determining the initial focal length based on the distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, the proportion of the target object in the image, and the first formula.

[0097] The technical solution of this embodiment acquires point cloud data collected by a lidar and images collected by an image acquisition device; determines the target distance between the image acquisition device and the target object based on the point cloud data; determines the initial focal length based on the distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, the proportion of the target object in the image, and a first formula; determines a set of candidate focal lengths based on the initial focal length and preset parameters; and determines the target focal length based on the image sharpness in the image set corresponding to the candidate focal length set. This solves the problem of high computational load caused by searching for the sharpest photo within the maximum and minimum focal length range of the image acquisition device to determine the focal length, thus narrowing the focal length adjustment range and reducing computational load. Furthermore, when tracking a high-speed moving target object, frequent focal length switching is required. If the method of finding the sharpest photo within the maximum and minimum focal length range of the image acquisition device is used to determine the focal length, it will lead to the problem of not being able to quickly obtain the focal length corresponding to high sharpness. Since the technical solution provided by this embodiment can narrow the focal length adjustment range, reduce computational load, and improve focal length adjustment efficiency, it can quickly obtain the focal length corresponding to high sharpness to meet the sharpness requirements of high-speed moving target objects.

[0098] Example 3

[0099] Figure 3 This is a flowchart illustrating a focus adjustment method provided in Embodiment 3 of this application, specifically explaining how image sharpness is obtained through an overall evaluation method based on Embodiments 1 and 2 described above. Figure 3 As shown, the method includes:

[0100] S310 acquires point cloud data collected by lidar and images acquired by image acquisition devices.

[0101] S320 determines the target distance between the image acquisition device and the target object based on point cloud data.

[0102] S330 determines the set of candidate focal lengths based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image.

[0103] S340, obtain no less than 3 reference focal lengths from the set of candidate focal lengths according to unit length.

[0104] The reference focal length includes at least f+h2 or f-h1.

[0105] It should be noted that the preset parameters include: the first preset parameter h1 and the second preset parameter h2, and the set of selectable focal lengths is [f-h1, f+h2], where f is the initial focal length.

[0106] S350: Divide the reference image into several sub-images according to pixels, and define a set of sub-images, which includes all sub-images.

[0107] It should be noted that the embodiments of this application use a holistic evaluation method to determine the sharpness of the reference image. That is, the sharpness of the reference image is determined based on all pixels in the reference image.

[0108] In this sub-image set, the sub-images do not overlap.

[0109] Specifically, dividing a reference image into several sub-images based on pixels can be done by dividing the reference image into several sub-images according to a preset window size. For example, if the window size is 3×3, then the reference image can be divided into several sub-images according to a 3×3 window, with each sub-image containing 9 pixels.

[0110] S360: Obtain the image sharpness evaluation operator corresponding to the sub-image based on the gray value of the center point in the sub-image and the gray values ​​of the adjacent pixels of the center point in the sub-image.

[0111] Among them, the image sharpness evaluation operator is used to quantify the sharpness of the corresponding image.

[0112] Specifically, the method for obtaining the image sharpness evaluation operator corresponding to the sub-image based on the gray value of the center point in the sub-image and the gray values ​​of the adjacent pixels of the center point in the sub-image can be as follows: filter the adjacent pixels of the center point in the sub-image to obtain a set of reference points, and obtain the image sharpness evaluation operator corresponding to the sub-image based on the gray value of the center point in the sub-image and the gray value of each reference point in the set of reference points.

[0113] Optionally, an image sharpness evaluation operator is determined based on the gray value of the center point of the sub-image and the gray values ​​of the neighboring pixels of the center point of the sub-image, including:

[0114] Define a reference point in the sub-image, which includes at least some of the neighboring pixels of the center point of the sub-image;

[0115] The image sharpness evaluation operator corresponding to the sub-image is determined based on the gray value of the center point of the sub-image, the gray value of the reference point in the sub-image, and the second formula; the second formula is:

[0116]

[0117] Among them, C t This represents the grayscale value of a single reference point in the sub-image. C represents the grayscale values ​​of all reference points in the sub-image, q represents the number of reference points in the sub-image, and C represents the grayscale values ​​of all reference points in the sub-image. oR represents the gray value of the center point of the sub-image, and R is a preset value set based on the value of q.

[0118] The value of R can be [q-1, q+1]. Preferably, R = q.

[0119] It should be noted that the reference points in the sub-image include at least some of the adjacent pixels of the center point of the sub-image. For example, if the sub-image size is 3×3 and includes 9 pixels, namely: the center point, the pixel above the center point, the pixel below the center point, the pixel to the left of the center point, the pixel to the right of the center point, the pixel to the upper left of the center point, the pixel to the lower right of the center point, the pixel to the lower left of the center point, and the pixel to the upper right of the center point. The pixels above the center point, the pixels below the center point, the pixels to the left of the center point, the pixels to the right of the center point, the pixels to the lower left of the center point, and the pixels to the upper right of the center point are determined as the reference points in the sub-image.

[0120] Optionally, the reference points include: the pixel adjacent to the center point above, the pixel adjacent to the center point below, the pixel adjacent to the center point to the left, the pixel adjacent to the center point to the right, the pixel adjacent to the center point to the lower left, and the pixel adjacent to the center point to the upper right.

[0121] The second formula is:

[0122] J i =f(x-1,y)+f(x+1,y)+f(x,y-1)+f(x+1,y-1)+f(x-1,y+1), f(x,y+1)-P*f(x,y);

[0123] Where f(x, y) is the gray value of the center point of sub-image i, f(x-1, y) is the gray value of the pixel adjacent to the center point above it, f(x+1, y) is the gray value of the pixel adjacent to the center point below it, f(x, y-1) is the gray value of the pixel adjacent to the center point to its left, f(x+1, y-1) is the gray value of the pixel adjacent to the center point to its lower left, f(x-1, y+1) is the gray value of the pixel adjacent to the center point to its upper right, and f(x, y+1) is the gray value of the pixel adjacent to the center point to its right. The coordinates of the center point of sub-image i are (x, y). i Let P be the image sharpness evaluation operator for sub-image i, where the value of P ranges from [5,7].

[0124] Where P is the number of reference points in the sub-image. Preferably, P = 6.

[0125] The second formula can be expressed as shown in Table 1:

[0126] Table 1

[0127] 0 1 1 1 -6 1 1 1 0

[0128] As shown in Table 2, Table 2 is a sub-image, including the 9 pixels in the upper left corner of the image. The grayscale values ​​of each pixel in the sub-image are arranged according to the position of each pixel in the sub-image:

[0129] Table 2

[0130] R1 R2 R3 R4 R5 R6 R7 R8 R9

[0131] Wherein, R5 is the gray value of the center point of the sub-image, R2 is the gray value of the pixel adjacent to the center point above the center point of the sub-image, R8 is the gray value of the pixel adjacent to the center point below the center point of the sub-image, R4 is the gray value of the pixel adjacent to the left of the center point of the sub-image, R6 is the gray value of the pixel adjacent to the right of the center point of the sub-image, R1 is the gray value of the pixel adjacent to the upper left of the center point of the sub-image, R7 is the gray value of the pixel adjacent to the lower left of the center point of the sub-image, R3 is the gray value of the pixel adjacent to the upper right of the center point of the sub-image, and R9 is the gray value of the pixel adjacent to the lower right of the center point of the sub-image.

[0132] The image sharpness evaluation operator corresponding to the sub-image shown in Table 2 is = R2 + R3 + R4 + R6 + R7 + R8 – 6 * R5.

[0133] S370, determine the sharpness of the reference image based on the image sharpness evaluation operator corresponding to each sub-image in the sub-image set.

[0134] Optionally, the sharpness of the reference image is determined based on the image sharpness evaluation operator corresponding to each sub-image in the sub-image set, including:

[0135] The sharpness of the target image is determined based on the image sharpness evaluation operator for each sub-image in the sub-image set and a third formula, wherein the third formula is:

[0136]

[0137] Where i is sub-image i, J i Here, n is the image sharpness evaluation operator for sub-image i, n is the number of sub-images, and S is a preset value set based on the overall evaluation method or the local evaluation method.

[0138] It should be noted that the value of S differs when determining the sharpness of the reference image using a holistic evaluation method and when using a local evaluation method. Since this embodiment uses a holistic evaluation method, the focal length does not affect the final number of sub-images. This is because the number of pixels in the image remains constant regardless of the focal length. Therefore, subdividing an image with a constant number of pixels results in a constant number of sub-images. Thus, in the above evaluation method, comparisons can be made using absolute values. Therefore, the value of S in this embodiment can be a preset positive integer; preferably, the value of S can be 1.

[0139] S380 determines the target focal length based on the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths.

[0140] The technical solution of this embodiment uses an overall evaluation method to determine the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths, which can improve the accuracy of the target focal length.

[0141] Example 4

[0142] Figure 4 This is a flowchart illustrating a focal length adjustment method provided in Embodiment 4 of this application. Based on Embodiments 1 and 2 above, it specifically explains how image sharpness is obtained through local evaluation. Figure 4 As shown, the method includes:

[0143] S410 acquires point cloud data collected by lidar and images acquired by image acquisition devices.

[0144] S420 determines the target distance between the image acquisition device and the target object based on point cloud data.

[0145] S430 determines the set of candidate focal lengths based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image.

[0146] S440, obtain no less than 3 reference focal lengths from the set of candidate focal lengths according to unit length.

[0147] The reference focal length includes at least f+h2 or f-h1.

[0148] It should be noted that the preset parameters include: the first preset parameter h1 and the second preset parameter h2, and the set of selectable focal lengths is [f-h1, f+h2], where f is the initial focal length.

[0149] S450, the reference image is divided to obtain the target image, which includes the image of the target object.

[0150] Specifically, the method for dividing a reference image to obtain a target image can be as follows: Identify the reference image to obtain the region to which the target object belongs; then divide the reference image based on the region to which the target object belongs to obtain the target image. For example, it could be to extract the target image from the reference image based on the region to which the target object belongs.

[0151] Specifically, the method for dividing the reference image to obtain the target image can be as follows: determine the position information of the target object based on the laser point cloud, and then crop the reference image based on the position of the target object to obtain the target image that includes the target object in the reference image.

[0152] It should be noted that the longer the focal length, the larger the target object in the reference image, meaning the target image contains more pixels. Conversely, the shorter the focal length, the smaller the target object in the reference image, meaning the target image contains fewer pixels.

[0153] It should be noted that calculating the sharpness of only the target image containing the target object aligns better with the principle of prioritizing target object sharpness, effectively improving the sharpness of the target object. Compared to overall evaluation, this increases the calculation speed of image sharpness while reducing the computational load.

[0154] S460: Divide the target image into several sub-images according to pixels, and define a set of sub-images, which includes all sub-images.

[0155] The sub-images do not overlap.

[0156] Specifically, the method of dividing the target image into several sub-images according to pixels can be as follows: divide the pixels in the target image according to a preset window size to obtain multiple sub-images.

[0157] It should be noted that if the focal length is closer, the target object in the reference image is larger, the target image contains more pixels, and the sub-image set contains more sub-images. If the focal length is farther, the target object in the reference image is smaller, the target image contains fewer pixels, and the sub-image set contains fewer sub-images.

[0158] S470, the image sharpness evaluation operator corresponding to the sub-image is obtained based on the gray value of the center point in the sub-image and the gray value of the adjacent pixels of the center point in the sub-image.

[0159] Among them, the image sharpness evaluation operator is used to quantify the sharpness of the corresponding image.

[0160] Optionally, an image sharpness evaluation operator is determined based on the gray value of the center point of the sub-image and the gray values ​​of the neighboring pixels of the center point of the sub-image, including:

[0161] Define a reference point in the sub-image, which includes at least some of the neighboring pixels of the center point of the sub-image;

[0162] The image sharpness evaluation operator corresponding to the sub-image is determined based on the gray value of the center point of the sub-image, the gray value of the reference point in the sub-image, and the second formula; the second formula is:

[0163]

[0164] Among them, C t This represents the grayscale value of a single reference point in the sub-image. C represents the grayscale values ​​of all reference points in the sub-image, q represents the number of reference points in the sub-image, and C represents the grayscale values ​​of all reference points in the sub-image. o R represents the gray value of the center point of the sub-image, and R is a preset value set based on the value of q.

[0165] The value of R can be [q-1, q+1]. Preferably, R = q.

[0166] Optionally, the reference points include the pixels adjacent to the center point above, the pixels adjacent to the center point below, the pixels adjacent to the center point to the left, the pixels adjacent to the center point to the right, the pixels adjacent to the center point to the lower left, and the pixels adjacent to the center point to the upper right.

[0167] The second formula is:

[0168] J i =f(x-1,y)+f(x+1,y)+f(x,y-1)+f(x+1,y-1)+f(x-1,y+1)+f(x,y+1)-P*f(x,y);

[0169] Where f(x, y) is the gray value of the center point of sub-image i, f(x-1, y) is the gray value of the pixel adjacent to the center point above it, f(x+1, y) is the gray value of the pixel adjacent to the center point below it, f(x, y-1) is the gray value of the pixel adjacent to the center point to its left, f(x+1, y-1) is the gray value of the pixel adjacent to the center point to its lower left, f(x-1, y+1) is the gray value of the pixel adjacent to the center point to its upper right, and f(x, y+1) is the gray value of the pixel adjacent to the center point to its right. The coordinates of the center point of sub-image i are (x, y). i Let P be the image sharpness evaluation operator for sub-image i, where the value of P ranges from [5,7].

[0170] S480, determine the sharpness of the reference image based on the image sharpness evaluation operator corresponding to each sub-image in the sub-image set.

[0171] Optionally, the sharpness of the reference image is determined based on the image sharpness evaluation operator corresponding to each sub-image in the sub-image set, including:

[0172] The sharpness of the target image is determined based on the image sharpness evaluation operator for each sub-image in the sub-image set and a third formula, wherein the third formula is:

[0173]

[0174] Where i is sub-image i, J i Here, n is the image sharpness evaluation operator for sub-image i, n is the number of sub-images, and S is a preset value set based on the overall evaluation method or the local evaluation method.

[0175] It should be noted that the value of S differs when determining the sharpness of the reference image using a holistic evaluation method and when using a local evaluation method. Since this embodiment uses a local evaluation method, the focal length directly affects the number of pixels in the target image, thus directly affecting the number of sub-images. Therefore, using a relative value is more reasonable in the above evaluation method.

[0176] Optionally, if a local evaluation method is selected, then

[0177] S490, determine the target focal length based on the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths.

[0178] The technical solution of this embodiment uses a local evaluation method to determine the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths, which can reduce the amount of calculation and improve the efficiency of focal length adjustment.

[0179] This application proposes a focusing method for tracking fast-moving target objects, enabling rapid focusing. Because the target object is fast-moving, rapid focusing is necessary; slow focusing will prevent tracking. The fast-moving target object can be a drone or a high-speed vehicle.

[0180] Example 5

[0181] Figure 5 This is a flowchart illustrating a focal length adjustment method provided in Embodiment 5 of this application, specifically explaining how to determine the target focal length through a defined comparison method based on Embodiments 1 to 4 described above. Figure 5 As shown, the method includes:

[0182] S510 acquires point cloud data collected by lidar and images acquired by image acquisition devices;

[0183] S520 determines the target distance between the image acquisition device and the target object based on point cloud data;

[0184] S530, determine the set of candidate focal lengths based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image;

[0185] S540 obtains no fewer than three reference focal lengths from the set of candidate focal lengths according to unit length.

[0186] The reference focal length includes at least f+h2 or f-h1;

[0187] S550: Obtain the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths.

[0188] S560 defines the reference focal length corresponding to the highest sharpness of the reference image as the target focal length.

[0189] Specifically, the method to determine the target focal length by the reference focal length corresponding to the reference image with the highest sharpness can be as follows: compare the sharpness of each reference image among all reference images, and determine the target focal length by the reference focal length corresponding to the reference image with the highest sharpness.

[0190] In a specific example, the set of candidate focal lengths is [9.5-10.5], with a unit length of 0.1. Reference focal lengths include: 9.5, 9.6, 9.7, ..., 10.4, 10.5. The sharpness of the reference image corresponding to focal length 9.5, 9.6, 9.7, ..., 10.4, and 10.5 is obtained. If the sharpness of the reference image corresponding to focal length 9.6 is greater than the sharpness of the reference images corresponding to other reference focal lengths, then focal length 9.6 is determined as the target focal length.

[0191] The technical solution of this embodiment determines the target focal length by comparing the sharpness of the reference image corresponding to the reference focal length, which can improve the accuracy of the target focal length.

[0192] Example 6

[0193] Figure 6 This is a flowchart illustrating a focal length adjustment method provided in Embodiment Six of this application, specifically explaining how to determine the target focal length by defining a fitting method based on Embodiments One to Four described above. Figure 6As shown, the method includes:

[0194] S610 acquires point cloud data collected by lidar and images acquired by image acquisition devices;

[0195] S620 determines the target distance between the image acquisition device and the target object based on point cloud data;

[0196] S630 determines the set of candidate focal lengths based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image.

[0197] S640, obtain no less than 3 reference focal lengths from the set of candidate focal lengths according to the unit length, and the reference focal lengths include at least f+h2 or f-h1;

[0198] S650, obtain the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths;

[0199] S660 uses all reference focal lengths and the sharpness of the corresponding reference images as parameters, and obtains the functional relationship between the focal length and the sharpness of the corresponding images in the set of candidate focal lengths through linear fitting. The focal length corresponding to the maximum sharpness in the set of candidate focal lengths is obtained through the functional relationship, and the focal length is defined as the target focal length.

[0200] Specifically, the method of obtaining the functional relationship between focal length and corresponding image sharpness in the set of candidate focal lengths by using all reference focal lengths and the sharpness of the corresponding reference images as parameters and through linear fitting can be as follows: Input all reference focal lengths and corresponding image sharpness as parameters into the fourth formula to obtain the functional relationship between focal length and corresponding image sharpness in the set of candidate focal lengths.

[0201] Optionally, taking three reference focal lengths as an example, the fitting method includes the following:

[0202] The three reference focal lengths of the candidate focal length set [f-h1, f+h2] are f-h1, f, and f+h2, respectively.

[0203] Obtain the image sharpness corresponding to three reference focal lengths;

[0204] The target focal length is determined based on three reference focal lengths, the image sharpness corresponding to each reference focal length, and a fourth formula, which is:

[0205]

[0206] Among them, f tLet f be the target focal length, D(f) be the sharpness of the reference image corresponding to the reference focal length f, D(f-h1) be the sharpness of the reference image corresponding to the reference focal length f-h1, D(f+h2) be the sharpness of the reference image corresponding to the reference focal length f+h2, k be the first fitting coefficient, and M be the second fitting coefficient.

[0207] It should be noted that if the first preset parameter h1 and the second preset parameter h2 are equal, both being h, then the fourth formula can also be:

[0208]

[0209] The three reference focal lengths are: f, fh, f+h, and f t Let f be the target focal length, D(f) be the sharpness of the reference image corresponding to the reference focal length f, D(fh) be the sharpness of the reference image corresponding to the reference focal length fh, D(f+h) be the sharpness of the reference image corresponding to the reference focal length f+h, k be the first fitting coefficient, and M be the second fitting coefficient.

[0210] It should be noted that the target focal length f is calculated by substituting the sharpness of the reference image corresponding to each of the three reference focal lengths into the above formula. t Based on the calculated target focal length f t The drive motor moves to complete the automatic focusing.

[0211] In a specific example, the three reference focal lengths are f, fh, and f+h. Given the reference images corresponding to these three focal lengths, the sharpness D(f) of the reference image corresponding to focal length f, the sharpness D(fh) of the reference image corresponding to focal length fh, and the sharpness D(f+h) of the reference image corresponding to focal length f+h can be calculated using the sharpness calculation formula. The following three points can be obtained:

[0212] (f, D(f)), (fh, D(fh)), (f+h, D(f+h));

[0213] If a two-dimensional coordinate system is established, the horizontal axis represents the focal length, and the vertical axis represents the corresponding image sharpness value.

[0214] Substituting the above three points into the following formula, the target focal length f can be calculated. t :

[0215]

[0216] Based on the calculated target focal length f t The drive motor moves to complete the automatic focusing.

[0217] In a specific example, after the LiDAR and image acquisition equipment are installed, joint calibration is performed. The LiDAR collects point cloud data, performs clustering on the point cloud data to identify target objects, and calculates the first distance between the image acquisition equipment and the target object based on the target object's position coordinates. For moving target objects, the calculation platform provides continuous distances corresponding to the LiDAR frame rate. The image acquisition equipment collects image data, identifies the target object's position coordinates, and calculates the second distance between the image acquisition equipment and the target object based on these coordinates. The first and second distances measured by the LiDAR and image acquisition equipment are fused to obtain the target distance. The initial focal length is determined based on the target distance between the image acquisition equipment and the target object, the image acquisition equipment parameters, the target object size, and the target object's proportion in the image. The PID controller moves the lens motor to the initial focal length. The formula for determining the initial focal length based on the target distance between the image acquisition equipment and the target object, the image acquisition equipment parameters, the target object size, and the target object's proportion in the image is as follows:

[0218]

[0219] Where f is the initial focal length, d is the distance between the image acquisition device and the target object, w is the target surface imaging width of the image acquisition device, u is the size of the target object, and k is the proportion of the target object in the image. A PID algorithm controls the lens motor to move from the current focal length to the initial focal length. Based on the initial focal length and preset parameters, a set of candidate focal lengths is determined. Three reference focal lengths are obtained from the candidate focal length set according to unit length: f-h1, f, and f+h2. The image sharpness evaluation operator is used to calculate the sharpness of the image frames corresponding to the three focal lengths. A linear fitting method is used to obtain the functional relationship between the focal length and the corresponding image sharpness in the candidate focal length set. This functional relationship yields the focal length corresponding to the maximum sharpness in the candidate focal length set, which is the target focal length.

[0220] The process of determining the sharpness of the reference images corresponding to the three reference focal lengths {f, fh, f+h} using the image sharpness evaluation operator includes the following steps:

[0221] The lens motor is controlled to move to a reference focal length f, and the image captured when the lens motor moves to the reference focal length f is obtained. The acquired image is determined as the reference image, and the reference image is divided to obtain a set of sub-images.

[0222] The sharpness of the reference image is determined based on the image sharpness evaluation operator for each sub-image in the sub-image set and the following formula:

[0223]

[0224] Where i is sub-image i, J i Here, n is the image sharpness evaluation operator for sub-image i, n is the number of sub-images, and S is a preset value set based on the overall evaluation method or the local evaluation method.

[0225] It should be noted that the larger the D(f) value, the clearer the reference image.

[0226] It should be noted that during drone image tracking, the focal length of the image acquisition device needs to be adjusted in real time to ensure the tracking of the drone's image. The quality of the autofocus algorithm directly affects the focusing efficiency and has a significant impact on drone tracking.

[0227] To address the aforementioned issues, this application embodiment determines a set of candidate focal lengths in real time based on the target distance fused from the visual ranging of the LiDAR and the image acquisition device, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image. At least three reference focal lengths are obtained from the set of candidate focal lengths per unit length, and the target focal length is determined by fitting, thereby maintaining the tracking of the moving target object.

[0228] Example 7

[0229] Figure 7 This is a schematic diagram of a focus adjustment device provided in Embodiment Seven of this application. This embodiment is applicable to focus adjustment applications. The device can be implemented using software and / or hardware, and can be integrated into any device that provides focus adjustment functionality, such as… Figure 7 As shown, the focal length adjustment device specifically includes: an acquisition module 710, a target distance determination module 720, a candidate focal length set determination module 730, and a target focal length determination module 740.

[0230] The acquisition module is used to acquire point cloud data collected by the lidar and images collected by the image acquisition device.

[0231] The target distance determination module is used to determine the target distance between the image acquisition device and the target object based on point cloud data;

[0232] The module for determining the set of candidate focal lengths is used to determine the set of candidate focal lengths based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image.

[0233] The target focal length determination module is used to determine the target focal length based on the sharpness of the images in the image set corresponding to the candidate focal length set.

[0234] The above-described products can perform the methods provided in any embodiment of this application, and have the corresponding functional modules and beneficial effects for performing the methods.

[0235] The technical solution of this embodiment acquires point cloud data collected by a lidar and images collected by an image acquisition device; determines the target distance between the image acquisition device and the target object based on the point cloud data; determines a set of candidate focal lengths based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image; and determines the target focal length based on the image clarity in the image set corresponding to the set of candidate focal lengths. This solves the problem that existing focal length adjustment methods are inefficient and easily affected by the image quality of the image acquisition device, resulting in inaccurate final focal lengths. It achieves the effect of improving the efficiency of focal length adjustment while improving the accuracy of the final focal length.

[0236] Example 8

[0237] Figure 8 A schematic diagram of an electronic device 10, which can be used to implement Embodiment 8 of this application, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0238] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0239] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0240] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the focus adjustment method.

[0241] In some embodiments, the focus adjustment method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the focus adjustment method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the focus adjustment method by any other suitable means (e.g., by means of firmware).

[0242] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0243] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0244] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0245] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0246] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0247] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0248] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0249] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the focus adjustment method according to any embodiment of the invention.

[0250] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A focal length adjustment method, characterized in that, The method is applied to a tracking system, which includes a lidar and an image acquisition device, and the focal length adjustment method includes: Acquire point cloud data collected by lidar and images acquired by image acquisition devices; The target distance between the image acquisition device and the target object is determined based on the point cloud data. The set of candidate focal lengths is determined based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image. The target focal length is determined based on the sharpness of the images in the image set corresponding to the set of candidate focal lengths; Define the following parameters: First preset parameter h 1 and second preset parameters h 2; The set of candidate focal lengths is [f-h1, f+h2]; f is the initial focal length; Determining the target focal length based on the sharpness of images in the image set corresponding to the candidate focal length set includes: Based on a unit length, obtain no fewer than three reference focal lengths from the set of candidate focal lengths, wherein the reference focal lengths include at least f+ h 2 or f- h 1; Obtain the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths; The target focal length is determined based on the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths.

2. The method according to claim 1, characterized in that, Obtaining the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths includes: defining an overall evaluation method, including the following: The reference image is divided into several sub-images according to pixels, and a set of sub-images is defined, which includes all sub-images; The image sharpness evaluation operator corresponding to the sub-image is obtained based on the gray value of the center point in the sub-image and the gray value of the adjacent pixels of the center point in the sub-image. The image sharpness evaluation operator is used to quantify the sharpness of the corresponding image. The sharpness of the reference image is determined based on the image sharpness evaluation operator corresponding to each sub-image in the sub-image set.

3. The method according to claim 1, characterized in that, Obtaining the sharpness of the image corresponding to each reference focal length in the set of candidate focal lengths includes: defining a local evaluation method, including the following: The reference image is divided to obtain a target image, wherein the target image includes an image of the target object; The target image is divided into several sub-images according to pixels, and a set of sub-images is defined, which includes all sub-images; The image sharpness evaluation operator corresponding to the sub-image is obtained based on the gray value of the center point in the sub-image and the gray value of the adjacent pixels of the center point in the sub-image. The image sharpness evaluation operator is used to quantify the sharpness of the corresponding image. The sharpness of the reference image is determined based on the image sharpness evaluation operator corresponding to each sub-image in the sub-image set.

4. The method according to claim 2 or 3, characterized in that, The image sharpness evaluation operator for a sub-image is determined based on the gray value of its center point and the gray values ​​of its neighboring pixels, including: Define reference points in a sub-image, wherein the reference points in the sub-image include at least a portion of the adjacent pixels of the center point of the sub-image; The image sharpness evaluation operator corresponding to the sub-image is determined based on the gray value of the center point of the sub-image, the gray value of the reference point in the sub-image, and the second formula; the second formula is: Among them, C t Represents the grayscale value of a single reference point in a sub-image; q represents the grayscale values ​​of all reference points in the sub-image; q represents the number of reference points in the sub-image. C o represents the gray value of the center point of the sub-image; R is a preset value set based on the value of q; J i The image sharpness evaluation operator for sub-image i.

5. The method according to claim 4, characterized in that, The reference points include the pixels adjacent to the center point above, below, to the left, to the right, to the lower left, and to the upper right of the center point. The second formula is: J i =f(x-1,y)+f(x+1,y)+f(x,y-1)+f(x+1,y-1)+f(x-1,y+1)+f(x,y+1)-P*f(x,y); Where f(x, y) is the gray value of the center point of sub-image i; f(x-1, y) is the gray value of the pixel adjacent to the center point above it; f(x+1, y) is the gray value of the pixel adjacent to the center point below it; f(x, y-1) is the gray value of the pixel adjacent to the left of the center point; f(x+1, y-1) is the gray value of the pixel adjacent to the lower left of the center point; f(x-1, y+1) is the gray value of the pixel adjacent to the upper right of the center point; f(x, y+1) is the gray value of the pixel adjacent to the right of the center point; the coordinates of the center point of sub-image i are (x, y); J i Let P be the image sharpness evaluation operator for sub-image i; the value range of P is [5, 7].

6. The method according to claim 2, 3 or 5, characterized in that, The sharpness of the reference image is determined based on the image sharpness evaluation operator corresponding to each sub-image in the sub-image set, including: The sharpness of the reference image is determined based on the image sharpness evaluation operator for each sub-image in the sub-image set and a third formula, wherein the third formula is: Where i is sub-image i; J i is the image sharpness evaluation operator for sub-image i; n is the number of sub-images; S is a preset value set based on the overall evaluation method or the local evaluation method; D(f) is the sharpness of the reference image corresponding to the reference focal length f. If the local evaluation method is used, D(f) is a relative value.

7. The method according to claim 1, 2, 3 or 5, characterized in that, Determining the target focal length based on the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths includes: The comparison method is defined as follows: the reference focal length corresponding to the highest sharpness of the reference image is defined as the target focal length; or Define the fitting method, including: The three reference focal lengths of the candidate focal length set [f-h1, f+h2] are f-h1, f, and f+h2, respectively. Obtain the image sharpness corresponding to the three reference focal lengths; The target focal length is determined based on the three reference focal lengths, the image sharpness corresponding to each reference focal length, and a fourth formula, wherein the fourth formula is: Among them, f t Let f be the target focal length, D(f) be the sharpness of the reference image corresponding to the reference focal length f, D(f-h1) be the sharpness of the reference image corresponding to the reference focal length f-h1, D(f+h2) be the sharpness of the reference image corresponding to the reference focal length f+h2, k be the first fitting coefficient, and M be the second fitting coefficient.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the focus adjustment method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the focal length adjustment method according to any one of claims 1-7.

10. A focus adjustment device, characterized in that, The system is applied to a tracking system, which includes a lidar and an image acquisition device, wherein the focus adjustment device includes: The acquisition module is used to acquire point cloud data collected by the lidar and images acquired by the image acquisition device; The target distance determination module is used to determine the target distance between the image acquisition device and the target object based on the point cloud data. The candidate focal length set determination module is used to determine the candidate focal length set based on the target distance between the image acquisition device and the target object, the parameters of the image acquisition device, the size of the target object, and the proportion of the target object in the image. The target focal length determination module is used to determine the target focal length based on the sharpness of the images in the image set corresponding to the candidate focal length set. The target focal length determination module is specifically used for: Define the following parameters: First preset parameter h 1 and second preset parameters h 2; The set of candidate focal lengths is [f-h1, f+h2]; f is the initial focal length; Determining the target focal length based on the sharpness of images in the image set corresponding to the candidate focal length set includes: According to the unit length, obtain no less than 3 reference focal lengths from the set of candidate focal lengths, and the reference focal lengths include at least f+h2 or f-h1; Obtain the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths; The target focal length is determined based on the sharpness of the reference image corresponding to each reference focal length in the set of candidate focal lengths.

Images