Focusing method and device, image acquisition equipment, storage medium and product

Abstract

The invention relates to a focusing method and device, image acquisition equipment, a storage medium and a product. The method comprises the steps that a plurality of objects in a view finding range are measured based on an ultra wide band (UWB) radar on image acquisition equipment, a plurality of first distances are obtained, and the first distances are the distances between the measured objects and the image acquisition equipment; based on the pixel size of each object on a first preview image, determining a corresponding relationship between the plurality of first distances and the plurality of objects, the first preview image being an image within a view finding range acquired by an image acquisition device; determining a to-be-focused target object from the plurality of objects, and determining a first distance corresponding to the target object based on the corresponding relationship; and based on the first distance corresponding to the target object, controlling the image acquisition device to focus the target object. Through the focusing method disclosed by the invention, the application scene of the image acquisition equipment can be expanded, and the focusing effect of the image acquisition equipment can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of image acquisition, and more particularly to a focusing method, apparatus, image acquisition device, storage medium, and product. Background Technology

[0002] During image acquisition, in order to improve the clarity of the acquired image, the camera's focal length can be adjusted to focus on the target of interest within the field of view, thereby obtaining a clear image including the target.

[0003] Currently, the focusing method used in cameras is mainly based on Time of Flight (TOF) autofocus. However, current focusing methods have limitations due to the shooting environment and shooting distance. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a focusing method, apparatus, image acquisition device, storage medium, and product, which can expand the applicable scenarios of image acquisition devices and improve the focusing effect of image acquisition devices.

[0005] According to a first aspect of the present disclosure, a focusing method is provided, the method comprising: measuring multiple objects within a field of view based on ultra-wideband (UWB) radar measurement on an image acquisition device to obtain multiple first distances, wherein the first distances are the measured distances between the objects and the image acquisition device; determining a correspondence between the multiple first distances and the multiple objects based on image information of each of the objects on a first preview image, wherein the first preview image is an image within the field of view acquired by the image acquisition device; determining a target object to be focused from the multiple objects, and determining a first distance corresponding to the target object based on the correspondence; and controlling the image acquisition device to focus on the target object based on the first distance corresponding to the target object.

[0006] According to a second aspect of the present disclosure, a focusing device is provided, comprising: a first measurement module configured to measure multiple objects within a field of view based on an ultra-wideband (UWB) radar on an image acquisition device, obtaining multiple first distances, wherein the first distances are the measured distances between the objects and the image acquisition device; a first determination module configured to determine a correspondence between the multiple first distances and the multiple objects based on image information of each object on a first preview image, wherein the first preview image is an image within the field of view acquired by the image acquisition device; a second determination module configured to determine a target object to be focused from the multiple objects, and determine the first distance corresponding to the target object based on the correspondence; and a control module configured to control the image acquisition device to focus on the target object based on the first distance corresponding to the target object.

[0007] According to a third aspect of the present disclosure, an image acquisition device is provided, comprising:

[0008] processor;

[0009] Memory used to store computer programs or instructions;

[0010] The processor executes the computer program or instructions to implement the steps of any of the methods described above.

[0011] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, the storage medium storing a computer program or instructions that, when executed by a processor, implement the steps of the method described in any of the preceding claims.

[0012] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implement the steps of the method described in any of the preceding claims.

[0013] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0014] In this disclosure, multiple objects within the field of view are measured using UWB radar to obtain multiple first distances. Based on image information of these objects in a first preview image, the correspondence between the multiple first distances and the multiple objects is determined. Since the first distance is the measured distance between the object within the field of view and the image acquisition device, after identifying the target object from the multiple objects, the image acquisition device can focus on the target object based on the corresponding first distance, i.e., the measured distance between the target object and the image acquisition device, to obtain a focused image of the target object. Because UWB radar has advantages such as long measurement range, strong penetration, low cost, and minimal susceptibility to external environmental influences, using UWB radar on the image acquisition device to measure the distance between the target object and the image acquisition device, and then controlling the image acquisition device to focus based on the measured distance, can expand the applicable scenarios of the image acquisition device and improve its focusing effect.

[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0017] Figure 1 This is a flowchart illustrating a focusing method according to an exemplary embodiment.

[0018] Figure 2 This is a schematic diagram illustrating an imaging principle according to an exemplary embodiment.

[0019] Figure 3 This is a schematic diagram illustrating a matching relationship between a first part and a second part according to an exemplary embodiment.

[0020] Figure 4 This is a flowchart illustrating a focusing method according to another exemplary embodiment.

[0021] Figure 5 This is a block diagram illustrating a focusing device according to an exemplary embodiment.

[0022] Figure 6 This is a structural block diagram of an image acquisition device according to an exemplary embodiment. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0024] Currently, in Time-of-Flight (TOF) based autofocus methods, the camera measures the distance between the target and the camera using an infrared sensor. However, the process of using an infrared sensor for distance measurement is easily affected by the environment, such as glass or lighting conditions in the shooting environment. Furthermore, infrared sensor-based distance measurement also has disadvantages such as short measurement distance and high cost.

[0025] To address the aforementioned problems, this disclosure provides a focusing method, referring to... Figure 1 , Figure 1 This is a schematic flowchart illustrating a focusing method according to an exemplary embodiment. In the following text, reference will be made to... Figure 1 The steps shown herein, with an image acquisition device as the executing entity, describe the focusing method provided in the embodiments of this disclosure.

[0026] In step 101, multiple objects within the field of view are measured using the UWB radar on the image acquisition device to obtain multiple first distances, wherein the first distance is the measured distance between the object and the image acquisition device.

[0027] In step 102, based on the image information of each object on the first preview image, the correspondence between multiple first distances and multiple objects is determined, wherein the first preview image is an image within the field of view captured by the image acquisition device;

[0028] In step 103, the target object to be focused is determined from multiple objects, and the first distance corresponding to the target object is determined based on the correspondence.

[0029] In step 104, based on the first distance corresponding to the target object, the image acquisition device is controlled to focus on the target object.

[0030] Understandably, when an image acquisition device is equipped with an Ultrawideband (UWB) radar, it can measure the distances between itself and multiple objects within its field of view. For ease of explanation, these measured distances are referred to as "first distances." Based on this, the image acquisition device can obtain multiple first distances. Furthermore, the image acquisition device can display a first preview image based on the current field of view. Therefore, after obtaining multiple first distances, the image acquisition device can determine the correspondence between these first distances and the multiple objects based on the image information of each object in the first preview image. That is, it determines the first distance corresponding to each object. Then, the image acquisition device can identify the target object to be focused from among the multiple objects and, based on the first distance corresponding to the target object, control the image acquisition device to focus on the target object.

[0031] In some embodiments, the image acquisition device described above can be any electronic device capable of image acquisition. Exemplarily, the image acquisition device performing the focusing method of this disclosure embodiment may include a terminal device with image acquisition functionality. This terminal device may include a mobile terminal and a fixed terminal. The mobile terminal may include: a camera, smartphone, tablet computer, laptop computer, digital camera, and wearable electronic device, etc. The fixed terminal may include: a monitoring device, a desktop computer, and an all-in-one computer, etc.

[0032] In some embodiments, the first distance can be understood as the distance obtained by measuring the actual distance between objects within the field of view and the image acquisition device using UWB radar. The UWB radar installed on the image acquisition device may include a transmitting antenna and a receiving antenna, wherein the transmitting antenna is used to transmit radar waves, and the receiving antenna is used to receive radar waves reflected from multiple objects within the field of view. Based on this, the image acquisition device can determine the distance between each object and the image acquisition device according to the duration between the transmission and reception times of the radar waves, thereby obtaining multiple first distances.

[0033] In some embodiments, the object described above can be a living person or a moving object within the field of view of the image acquisition device. For example, the object described above can be a person or an animal.

[0034] In some embodiments, the image information of each object on the first preview image may include, but is not limited to, the pixel size of each object on the first preview image and the position of each object on the first preview image.

[0035] In some embodiments, since the same object has different pixel sizes in the first preview image when it is at different distances from the image acquisition device, the image acquisition device can also obtain the pixel size of each object in the first preview image after obtaining multiple first distances, and determine the correspondence between the multiple first distances and the multiple objects based on the pixel size of each object in the first preview image.

[0036] For example, an image acquisition device can sort the pixel sizes of multiple objects and assign multiple first distances to each object in ascending order of pixel size to obtain a correspondence between the multiple first distances and the multiple objects. For instance, the image acquisition device can first determine the smallest first distance among the multiple first distances as the first distance corresponding to the first object with the largest pixel size. Then, the image acquisition device can determine the second object with the largest pixel size from the objects other than the first object, and determine the smallest of the remaining first distances as the first distance corresponding to the second object. This process is repeated until the first distance corresponding to each object is determined, thereby obtaining a correspondence between the multiple first distances and the multiple objects.

[0037] In some embodiments, since objects located in different directions within the field of view have different positions in the first preview image, the UWB radar on the image acquisition device can also acquire the relative direction between each object and the image acquisition device when acquiring the first distance between each object within the field of view and the image acquisition device, thereby obtaining the correspondence between direction and first distance. Based on this, the image acquisition device can also determine the direction of each object based on its position in the first preview image, and determine the first distance corresponding to each object from the correspondence between direction and first distance based on the direction of each object, thereby obtaining the correspondence between multiple first distances and multiple objects.

[0038] In some embodiments, the image acquisition device can detect a user's selection operation on multiple objects in the first preview image while displaying the first preview image, and determine the selected object as the target object. Then, based on a first distance corresponding to the target object, the image acquisition device can control itself to focus on the target object to obtain a focused image of the target object.

[0039] In other embodiments, the image acquisition device may also use multiple objects as target objects and control the image acquisition device to focus on the multiple objects sequentially at a first distance corresponding to each object, so as to obtain a focused image of each of the multiple objects.

[0040] In some other embodiments, the image acquisition device may also select any one of the multiple objects as the target object, and control the image acquisition device to focus on the target object based on a first distance corresponding to the target object, so as to obtain a focused image of the target object.

[0041] In this embodiment, multiple objects within the field of view are measured using UWB radar to obtain multiple first distances. Based on image information of these objects in a first preview image, the correspondence between the multiple first distances and the multiple objects is determined. Since the first distance is the measured distance between the object within the field of view and the image acquisition device, after identifying the target object from among the multiple objects, the image acquisition device can focus on the target object based on the corresponding first distance—that is, the measured distance between the target object and the image acquisition device—to obtain a focused image of the target object. Because UWB radar has advantages such as long measurement range, strong penetration, low cost, and minimal susceptibility to external environmental influences, using UWB radar on the image acquisition device to measure the distance between the target object and the image acquisition device, and then controlling the image acquisition device to focus based on the measured distance, can expand the applicable scenarios of the image acquisition device and improve its focusing effect.

[0042] In some embodiments, step 102 includes:

[0043] Based on the pixel size of each object in the first preview image, a second distance between each object and the image acquisition device is determined, wherein the second distance is the distance between the object and the image acquisition device estimated based on the pixel size of the object in the first preview image;

[0044] Based on the absolute value of the difference between the second distance of each object and multiple first distances, the first distance corresponding to each object is determined from the multiple first distances.

[0045] Understandably, after obtaining multiple first distances based on UWB radar, the image acquisition device, unaware of the correspondence between these first distances and the multiple objects, can estimate the distance between each object and the device based on the pixel size of each object in the first preview image, thus obtaining a second distance for each object. After obtaining the second distance for each object, for each of the multiple objects, the image acquisition device can calculate the absolute value of the difference between the second distance and the multiple first distances. Based on this absolute value, it can determine the first distance for that object from the multiple first distances, thereby establishing the correspondence between the multiple first distances and the multiple objects.

[0046] In some embodiments, the image acquisition device may pre-store a second preset mapping relationship between pixel size and distance. Based on this, after obtaining the pixel size of each object in the first preview image, the image acquisition device can determine a second distance between each object and the image acquisition device from the second preset mapping relationship. The second distance of each object is an estimate of the distance between each object and the image acquisition device based on the size of each object in the first preview image. Therefore, the second distance may not be the actual distance between each object and the image acquisition device, but it can reflect the distance between each object and the image acquisition device to a certain extent.

[0047] In some embodiments, when the object within the field of view is a living body, the image acquisition device can acquire the pixel dimensions of a first part of each living body in the first preview image, and determine a second distance for each living body based on a second preset mapping relationship between the dimensions and distances of the first parts of the living body stored in the image acquisition device. The first part can be a part of the living body with a small size variation. For example, the first part of each object can be the face or head of the object.

[0048] For example, when the first part is a face, the image acquisition device can acquire the pixel size of the face of each object within the field of view on the first preview image, and determine the second distance between each object within the field of view and the image acquisition device from the second preset mapping relationship between face size and distance stored in the image acquisition device.

[0049] In some embodiments, when the object within the frame is a person, the image acquisition device can perform face detection on the first preview image and display a first detection box including the face on the first preview image. Based on this, the image acquisition device can determine the face size according to the size of the first detection box including the face.

[0050] In some embodiments, after obtaining the second distance corresponding to each of the plurality of objects, the image acquisition device can calculate the absolute value of the difference between the second distance corresponding to the object and each of the plurality of first distances for each of the plurality of objects, and determine the first distance with the smallest absolute value of the difference with the second distance as the first distance corresponding to the object, thereby obtaining the correspondence between the plurality of first distances and the plurality of objects.

[0051] In other embodiments, after obtaining the absolute value of the difference between the second distance corresponding to each object and each of the plurality of first distances, the image acquisition device can use a Hungarian algorithm to find the first distance that matches the second distance of each object from the plurality of first distances based on the absolute value of the difference between each second distance and the plurality of first distances, and take the first distance that matches the second distance of each object as the first distance corresponding to that object, thereby obtaining the correspondence between the plurality of first distances and the plurality of objects.

[0052] For example, for one of multiple objects, the image acquisition device can calculate the absolute value of the difference between the second distance corresponding to that object and each of the multiple first distances. The first distance whose absolute value of the difference with the second distance corresponding to that object is less than a first preset threshold is selected as a candidate first distance for that object. Repeating this process, the image acquisition device can obtain candidate first distances for each object. Then, based on the Hungarian algorithm, the image acquisition device can sequentially determine one candidate first distance from the candidate first distances corresponding to each object as the first distance corresponding to that object, thereby obtaining the correspondence between multiple first distances and multiple objects.

[0053] For example, suppose there are multiple objects, A, B, and C, and the candidate first distances for object A are the first and second first distances, for object B they are the second and third first distances, and for object C they are the first and second first distances. The image acquisition device can determine the first distance corresponding to each object from the candidate first distances using the Hungarian algorithm. In this case, the image acquisition device can obtain the following results: the first distance corresponding to object A is the first first distance, the first distance corresponding to object B is the third first distance, and the first distance corresponding to object C is the second first distance.

[0054] In this embodiment of the disclosure, since the same object appears in different sizes on the image acquisition device when it is at different distances from the image acquisition device, the image acquisition device can determine a second distance between each object and the image acquisition device based on the image sizes of multiple objects on the image acquisition device. This second distance can reflect the true distance between each object and the image acquisition device to a certain extent. Therefore, the smaller the absolute value of the difference between the second distances corresponding to an object among multiple first distances, the greater the likelihood that the first distance is the first distance of that object. Therefore, determining the first distance corresponding to each object based on the absolute value of the difference between the second distances of each object and the multiple first distances can increase the probability that the identified first distances corresponding to each object are the true distances between the object and the image acquisition device.

[0055] In some embodiments, determining a second distance between each object and the image acquisition device based on the pixel size of each object in the first preview image includes:

[0056] Based on the first coordinates of at least two key points on the first part of each object on the first preview image, determine the pixel distance of at least two key points of each object on the first preview image;

[0057] A second distance between each object and the image acquisition device is determined based on the pixel distance between at least two key points on each object in the first preview image, the actual distance between at least two key points on each object, and the current focal length of the image acquisition device.

[0058] Understandably, when displaying the first preview image, the image acquisition device can identify at least two key points on the first part of each object in the first preview image to obtain the first coordinates of each of the at least two key points on the first part of each object in the first preview image. Then, based on the first coordinates of the at least two key points on the first part of each object in the first preview image, the image acquisition device can determine the pixel distance between the at least two key points of each object in the first preview image.

[0059] After obtaining the pixel distance between at least two key points on the first part of each object in the first preview image, for each of the multiple objects, the image acquisition device can determine the second distance between the object and the image acquisition device based on the pixel distance between the at least two key points on the first part of the object in the first preview image, the actual distance between the at least two key points on the first part of the object, and the current focal length of the image acquisition device, thereby obtaining the second distance between each object and the image acquisition device.

[0060] In some embodiments, when the first part is the head, key points on the first part include, but are not limited to, the center points of the left and right eyes and the left and right corners of the mouth.

[0061] For example, when at least two of the aforementioned key points are the center points of the left and right eyes, the image acquisition device can determine the pixel distance between the two pupils of each object based on the first coordinates of the center points of the left and right eyes on the first preview image. Additionally, the image acquisition device can also obtain the actual distance between the two pupils of each object and the current focal length of the image acquisition device.

[0062] Based on this, for each of multiple objects, the image acquisition device can determine a second distance between the object and the image acquisition device based on the pixel distance between the object's two pupils, the actual distance between the object's two pupils, and the current focal length of the image acquisition device, thus obtaining the second distance between each object and the image acquisition device. The actual distance between the two pupils of each object can be pre-stored in the image acquisition device; for example, the distance between the two pupils of an adult is 63mm.

[0063] For example, refer to Figure 2 , Figure 2 This is a schematic diagram illustrating an imaging principle according to an exemplary embodiment. (Reference) Figure 2 , Figure 2 In the image, W represents the pixel distance between two key points on the first part of the object in the first preview image, M represents the actual distance between the two key points on the first part of the object, and f represents the current focal length of the image acquisition device. Based on the properties of similar triangles, the image acquisition device can calculate the second distance d between the object and the image acquisition device according to W, M, and f.

[0064] In this embodiment, since the actual distance between two key points on the first part of different objects is substantially the same, the pixel distance between the two key points on the first part of each object on the image acquisition device is mainly affected by the actual distance between each object and the image acquisition device. Therefore, determining the second distance between each object and the image acquisition device based on the pixel distance between the two key points on the first part of each object on the image acquisition device, the actual distance between the two key points on the first part of each object, and the current focal length of the image acquisition device can make the obtained second distance closer to the true distance between each object and the image acquisition device. Determining the first distance corresponding to each object based on the absolute value of the difference between the second distance, which is closer to the true distance, and multiple first distances can further increase the probability that the identified first distance corresponding to each object is the true distance between the object and the image acquisition device.

[0065] In some embodiments, the method further includes:

[0066] During the focusing process, the first distance between multiple objects within the field of view and the image acquisition device is remeasured using UWB radar;

[0067] When a change in the first distance between any object within the field of view and the image acquisition device is detected, a second preview image is acquired;

[0068] Identify the first part of each object from the second preview image;

[0069] Based on the first part of each object in the second preview image, the second distance between each object and the image acquisition device is re-determined;

[0070] The correspondence is determined based on the absolute value of the difference between the remeasured first distance and the multiple redefined second distances.

[0071] Understandably, during the focusing process, the image acquisition device can remeasure the initial distances between itself and multiple objects within the field of view using UWB radar. Based on this, if a change in the distance between any object within the field of view and the image acquisition device is detected, the image acquisition device can identify the first part of each object from the displayed second preview image. Then, based on the pixel dimensions of the first part of each object in the second preview image, the image acquisition device can redetermine the second distances between each object and the image acquisition device. After obtaining the redetermined second distances for each object, the image acquisition device can determine the correspondence based on the absolute value of the difference between the redetermined second distances and the remeasured multiple first distances.

[0072] In some embodiments, determining the first part of each object from the second preview image includes: the image acquisition device can extract features from the first parts of each object on a preview image previously displayed before the second preview image to obtain features of the first parts of each object. Based on this, when displaying the second preview image, the image acquisition device can identify the first parts of each object from the second preview image based on the features of the first parts of each object.

[0073] For example, the preview image displayed before the second preview image can be the first preview image.

[0074] In some embodiments, after obtaining the pixel size of the first part of each object on the second preview image, the image acquisition device can determine the second distance corresponding to each object based on the pixel size of the first part of each object on the second preview image, the actual size of the first part of each object, and the focal length of the image acquisition device when displaying the second preview image. That is, the redefined second distance between each object and the image acquisition device is obtained.

[0075] In some embodiments, the image acquisition device may also determine the first distance corresponding to the target object based on the correspondence obtained based on the remeasured first distance and the redefined second distance, and control the image acquisition device to focus on the target object based on the first distance corresponding to the target object.

[0076] In this embodiment, when the distance between any object within the field of view and the image acquisition device changes, the image acquisition device can update the displayed first preview image to a second preview image, and redetermine the second distance between each object and the image acquisition device based on the pixel size of the first part of each object in the second preview image. Then, the image acquisition device can determine the correspondence based on the redetermined second distance and the remeasured multiple first distances. Based on this, real-time updating of the correspondence between each object and the multiple first distances can be achieved, further improving the focusing effect.

[0077] In some embodiments, determining a first part of each object from a second preview image includes;

[0078] Identify at least one first part included in the second preview image to obtain a first detection box including the first part;

[0079] Multiple second parts included in the second preview image are identified to obtain a second detection box including the second parts, wherein the size of the second part is larger than the size of the first part;

[0080] The matching relationship between the first part and the second part is determined based on the area of ​​the overlapping region between the first detection box and each of the second detection boxes;

[0081] Based on the features of the second part of each object, the second part of each object is determined from the second preview image;

[0082] Based on the matching relationship, the first part that matches each second part is determined from the second preview image;

[0083] The first part that matches the second part of each object is determined as the first part of each object.

[0084] Understandably, the image acquisition device can also identify a first region on the second preview image while displaying the second preview image, obtaining a first detection box including the first region. Additionally, the image acquisition device can also identify a second region on the second preview image, obtaining a second detection box including the second region. Then, for each first detection box on the second preview image, the image acquisition device can determine a second detection box matching the first detection box from among multiple second detection boxes based on the area of ​​the overlapping region between the first detection box and each of the second detection boxes, thus obtaining a second detection box matching each first detection box. At this point, for any given first detection box, the image acquisition device can use the second region within the second detection box matching the first detection box as the second region matching the first region within that first detection box.

[0085] Based on the above process, the image acquisition device can obtain the second part in the second preview image that matches each of the first parts, that is, obtain the matching relationship between the first parts and the second parts. Then, based on the features of the second parts of each object, the image acquisition device can determine the second parts of each object from the second preview image, and based on the matching relationship between the first and second parts, determine the first parts that match the second parts of each object from the second preview image. After obtaining the first parts that match the second parts of each object, the image acquisition device can determine the first parts that match the second parts of each object as the first parts of each object.

[0086] In some embodiments, when the object within the field of view is a person, the first part can be a face, and the second part can be a human body. Based on this, the image acquisition device can perform face recognition on the second preview image to obtain a first detection box including the face, and perform human body recognition on the second preview image to obtain a second detection box including the human body.

[0087] For example, the image acquisition device can use a single-model multi-task target detection method to detect human bodies, faces, and key points on faces in the preview image.

[0088] In some embodiments, after obtaining the first detection frame and the second detection frame, for each first detection frame, the image acquisition device can determine the overlap area between the first detection frame and each of the second detection frames, and determine the second detection frame with the largest overlap area with the first detection frame as the second detection frame that matches the first detection frame. Based on this, the image acquisition device can obtain the second detection frames that match each of the first detection frames, and thus obtain the second parts that match each of the first parts, that is, obtain the matching relationship between the first parts and the second parts.

[0089] In other embodiments, after obtaining the first detection frame and the second detection frame, for each first detection frame, the image acquisition device can determine the Intersection Over Union (IOU) value between the first detection frame and each second detection frame based on the overlap area between the first detection frame and each second detection frame. Then, the image acquisition device can determine the second detection frame whose IOU value with the first detection frame is greater than a second preset threshold as a candidate second detection frame corresponding to the first detection frame.

[0090] After obtaining the candidate second detection boxes corresponding to each first detection box, the image acquisition device can determine the second detection box that matches the first detection box from the candidate second detection boxes corresponding to each first detection box based on the Hungarian algorithm, and make the second detection box that matches each first detection box different.

[0091] For example, refer to Figure 3 , Figure 3 This is a schematic diagram illustrating a matching relationship between a first part and a second part according to an exemplary embodiment. (Reference) Figure 3 , Figure 3 In the figure, x represents the first detection bounding box that includes the face. Figure 3 In this context, y represents the second detection box that includes the human body. Each x is connected to a corresponding candidate second detection box (y). For example, x1 corresponds to candidate second detection boxes y1, y2, and y3, and x2 corresponds to candidate second detection boxes y2 and y5. Based on the Hungarian algorithm, the second detection boxes y that match each first detection box x can be obtained. Figure 3 (thick lines in the text).

[0092] For example, the second detection box matching the first detection box x1 is y2, the second detection box matching the first detection box x2 is y5, the second detection box matching the first detection box x3 is y1, the second detection box matching the first detection box x4 is y3, and the second detection box matching the first detection box x5 is y4. Since the candidate second detection boxes for x5 and x6 are both y4, when y4 is determined to be the second detection box matching x5, then x6 has no matching second detection box. Based on this, the image acquisition device can determine that the human body matching the face in x1 is the human body in y2, the human body matching the face in x2 is the human body in y5, the human body matching the face in x3 is the human body in y1, the human body matching the face in x4 is the human body in y3, and the human body matching the face in x5 is the human body in y4.

[0093] In some embodiments, the image acquisition device may further acquire the second coordinates of each first part on the second preview image and the third coordinates of each second part on the second preview image. Then, based on the second coordinates of each first part on the second preview image and the third coordinates of each second part on the second preview image, the image acquisition device may determine the second part corresponding to each first part to obtain the matching relationship between the first and second parts.

[0094] For example, the second coordinates mentioned above can be the coordinates of the center points of each first part on the second preview image, and the third coordinates mentioned above can be the coordinates of the center points of each second part on the second preview image. Based on this, for each first part on the second preview image, the image acquisition device can determine the second part that is closest to the center point of the first part as the second part that matches the first part.

[0095] In some embodiments, the image acquisition device may determine the center point of the first detection box including the first part in the second preview image as the center point of the first part, and determine the center point of the second detection box including the second part in the second preview image as the center point of the second part.

[0096] In this embodiment of the disclosure, since the size of the second part of each object is larger than the size of the first part, the second part of each object will include more features than the first part. Based on this, the image acquisition device determines the object to which each second part belongs from the second preview image based on the features of the second part of each object, and then determines the first part of each object based on the matching relationship between the first part and the second part, which can improve the recognition accuracy of each object on the second preview image.

[0097] In some embodiments, step 101 includes:

[0098] The first preview image is used to identify faces and bodies, and the recognition results are obtained.

[0099] If the recognition results determine that the first preview image contains a face or a human body, multiple objects within the field of view are measured using UWB radar.

[0100] Understandably, considering that UWB radar has higher ranging accuracy for human bodies than for objects, the focusing method provided in this embodiment can be applied to scenarios where the object is a person. Based on this, after obtaining a first preview image within the field of view, the image acquisition device can perform face and body recognition on the first preview image. If the image acquisition device recognizes a face or body in the first preview image, it indicates that the object within the field of view is a person. At this time, the image acquisition device can perform the aforementioned step 101 to measure the distance between multiple objects within the field of view and the image acquisition device.

[0101] In some embodiments, if the image acquisition device determines that the first preview image does not include a human face or human body, the image acquisition device may use a preset focusing method to focus.

[0102] For example, the preset focusing method can be any focusing method that the image acquisition device can perform, other than the focusing method provided in the embodiments of this disclosure.

[0103] In some embodiments, when the image acquisition device displays the second preview image, the image acquisition device can also perform face and body recognition on the second preview image to obtain recognition results. If the image acquisition device determines that the second preview image includes a face or a body based on the recognition results, it can also perform the above step 101 to measure the distance between multiple objects within the field of view and the image acquisition device.

[0104] In this embodiment of the present disclosure, when the image acquisition device determines that the field of view includes a face or a human body, it uses the focusing method provided in this embodiment to focus; when it determines that the field of view does not include a face or a human body, it uses other methods to focus, which can further improve the focusing effect of the image acquisition device.

[0105] In some embodiments, step 103 includes:

[0106] The target object is determined from a plurality of objects based on at least one of the size, position and imaging quality of each object on the first preview image or the second preview image.

[0107] Understandably, when displaying the first preview image, the image acquisition device can determine at least one of the size, position, and imaging quality of each object on the first preview image, and based on at least one of the size, position, and imaging quality of each object on the first preview image, determine the target object from among multiple objects. Similarly, when displaying the second preview image, the image acquisition device can determine at least one of the size, position, and imaging quality of each object on the second preview image, and based on at least one of the size, position, and imaging quality of each object on the second preview image, determine the target object from among multiple objects.

[0108] In some embodiments, the image acquisition device may, while displaying the first preview image, acquire the size of each object in the first preview image and determine the object with the largest size in the first preview image as the target object.

[0109] In other embodiments, the image acquisition device may also determine the distance between the center point of each object and the center point of the first preview image, and identify the object with the smallest distance between its center point and the center point of the first preview image as the target object.

[0110] In some other embodiments, the image acquisition device may also acquire the imaging quality of each object in the first preview image and identify the object with the best imaging quality in the first preview image as the target object.

[0111] For example, the imaging quality of each object in the first preview image can be determined based on the sharpness and contrast of each object in the first preview image. For instance, the image acquisition device can identify the object with the highest sharpness in the first preview image as the target object.

[0112] In some embodiments, when the object within the field of view is a person, the image acquisition device can also determine the target object from multiple objects based on the posture of each object. For example, the image acquisition device can determine the object in a standing posture from among the multiple objects included in the first preview image as the target object. If there are multiple objects in a standing posture in the first preview image, the image acquisition device can also determine the target object from among the multiple objects in a standing posture based on the size, position, or imaging quality of each object in a standing posture in the first preview image.

[0113] In some embodiments, the image acquisition device can also determine the target object from among the multiple objects included in the first preview image based on the facial orientation of each object in the first preview image. For example, the image acquisition device can determine the object in the first preview image whose face is facing the image acquisition device as the target object. Similarly, if there are multiple objects in the first preview image whose faces are facing the image acquisition device, the image acquisition device can also determine the target object from among the multiple objects whose faces are facing the image acquisition device based on the size, position, or imaging quality of the multiple objects facing the image acquisition device in the first preview image.

[0114] In some embodiments, the image acquisition device determines the target object from multiple objects based on at least one of the size, position, and imaging quality of each object on the second preview image. This can be achieved by referring to the above-described method of determining the target object from multiple objects based on at least one of the size, position, and imaging quality of each object on the first preview image.

[0115] In some embodiments, determining a target object from a plurality of objects based on at least one of the size, position, and image quality of each object on a first preview image includes:

[0116] A first score for each object is generated based on at least one of the size, position, and imaging quality of the first part of each object on the first or second preview image;

[0117] A second score is generated for each object based on at least one of the size, position, and imaging quality of the second part on the first or second preview image, wherein the size of the second part is larger than the size of the first part, and the weight of the first score is greater than the weight of the second score.

[0118] The target object is determined from multiple objects based on the first and second scores of each object, as well as the weight of the first score and the weight of the second score.

[0119] Understandably, the image acquisition device can generate a first score for each object based on at least one of the size, position, and imaging quality of a first part of each object on the first preview image while displaying the first preview image. Additionally, the image acquisition device can also generate a second score for each object based on at least one of the size, position, and imaging quality of a second part of each object on the first preview image while displaying the first preview image.

[0120] The image acquisition device can also assign weights to the first and second scores of each object, with the first score having a greater weight than the second score. Based on this, after obtaining the first and second scores of each object, as well as their weights, the image acquisition device can identify the target object from multiple objects.

[0121] In some embodiments, the size of the first score is positively correlated with the size of the first part on the first preview image.

[0122] For example, after obtaining the dimensions of the first parts of each object in the first preview image, the image acquisition device can calculate a first ratio between the dimensions of the first parts of each object in the first preview image and the dimensions of the first preview image. If the first ratio between the dimensions of the first parts of an object in the first preview image and the dimensions of the first preview image is greater than or equal to a first ratio threshold, the first score of the object is determined to be positive. For example, if the first ratio between the dimensions of the first parts of an object in the first preview image and the dimensions of the first preview image is greater than or equal to the first ratio threshold, the image acquisition device can set the first score of the object to 1. If the first ratio between the dimensions of the first parts of an object in the first preview image and the dimensions of the first preview image is less than the first ratio threshold, the image acquisition device can determine the first score of the object to be 0.

[0123] In some embodiments, the size of the first part on the first preview image includes, but is not limited to, the width and height of the first part on the first preview image.

[0124] In some embodiments, the width of a first part of an object on the first preview image can be the width of a first detection box including the first part, and the height of the first part of an object on the first preview image can be the height of the first detection box including the first part.

[0125] For example, after obtaining the width of the first part of each object in the first preview image, the image acquisition device can calculate a first ratio between the width of the first part of each object in the first preview image and the width of the first preview image. If the first ratio between the width of the first part of an object in the first preview image and the width of the first preview image is greater than or equal to a first ratio threshold, the image acquisition device can determine that the object's first score is positive. If the first ratio between the width of the first part of an object in the first preview image and the width of the first preview image is less than the first ratio threshold, the image acquisition device can determine that the object's first score is 0.

[0126] For example, when the object within the frame is a person, the image acquisition device can acquire the face width of each object in the first preview image and calculate a first ratio between the face width of each object in the first preview image and the width of the first preview image. If the first ratio between the face width of an object in the first preview image and the width of the first preview image is greater than or equal to a first ratio threshold, the image acquisition device can set the first score of that object to 1. If the first ratio between the face width of an object in the first preview image and the width of the first preview image is less than the first ratio threshold, the image acquisition device can determine that the first score of that object is 0.

[0127] In some embodiments, the first ratio threshold described above can be set as needed, and this disclosure does not limit this setting. For example, the first ratio threshold described above can be any value from 0.04 to 1.

[0128] In some embodiments, the image acquisition device can determine the aspect ratio of the first part of each object in the first preview image based on the acquired dimensions of the first part of each object in the first preview image. Then, the image acquisition device can determine whether the aspect ratio of the first part of each object in the first preview image is within a preset range. If the aspect ratio of the first part of an object in the first preview image is within the preset range, then the first score of the object is determined to be positive; for example, if the aspect ratio of the first part of an object in the first preview image is within the preset range, then the first score of the object can be determined to be 1. If the aspect ratio of the first part of an object in the first preview image is outside the preset range, then the first score of the object is determined to be 0.

[0129] In some embodiments, the preset range can also be set as needed, and this disclosure does not limit this. For example, the preset range can be from 0.8 to 1.2.

[0130] In some embodiments, the image acquisition device may further determine a third distance between the center point of the first part of each object on the first preview image and the center point of the first preview image, and determine a first score for each object based on the third distance between the center point of the first part of each object on the first preview image and the center point of the first preview image. The magnitude of the first score is negatively correlated with the third distance.

[0131] For example, the image acquisition device can calculate the first score of each object using the following formula (1).

[0132]

[0133] Wherein, S1 is the first score, x1 is the x-coordinate of the center point of the first part of an object on the first preview image, x0 is the x-coordinate of the center point of the first preview image, y1 is the y-coordinate of the center point of the first part of an object on the first preview image, y0 is the y-coordinate of the center point of the first preview image, W is the size of the first preview image in the X-axis direction, and H is the size of the first preview image in the Y-axis direction.

[0134] In some embodiments, the center point of the first part of each object on the first preview image may be the center point of the first detection box including the first part.

[0135] In some embodiments, the image acquisition device may also determine a first score for each object based on the sharpness of a first part of each object in the first preview image. For example, the image acquisition device may determine the first score of an object whose first part has a sharpness greater than or equal to a sharpness threshold in the first preview image as 1, and determine the first score of an object whose first part has a sharpness less than or equal to the sharpness threshold in the first preview image as 0.

[0136] In some embodiments, when the first part is the face of each object, the image acquisition device can also determine a first score for each object based on the facial orientation of each object. For example, the image acquisition device can determine a first score of 1 for objects whose facial orientation is within a preset angle range and a first score of 0 for objects whose facial orientation is outside the preset angle range.

[0137] In some embodiments, the preset angle range can also be set as needed, and this disclosure does not limit this. For example, the preset angle range can be -45° to +45°, wherein the relative angle between the face facing the image acquisition device and the image acquisition device is 0°.

[0138] In some embodiments, the size of the second score is positively correlated with the size of the second part on the first preview image.

[0139] For example, after obtaining the dimensions of the second parts of each object in the first preview image, the image acquisition device can calculate a second ratio between the dimensions of the second parts of each object in the first preview image and the dimensions of the first preview image. If the second ratio between the dimensions of the second parts of an object in the first preview image and the dimensions of the first preview image is greater than or equal to a second ratio threshold, then the second score of the object is positive.

[0140] For example, if a second ratio between the size of a second part of an object in the first preview image and the size of the first preview image is greater than or equal to a second ratio threshold, the image acquisition device can set the second score of the object to 1. If a second ratio between the size of a second part of an object in the first preview image and the size of the first preview image is less than the second ratio threshold, the image acquisition device can determine that the second score of the object is 0.

[0141] In some embodiments, the dimensions of the second part on the first preview image include, but are not limited to, the width and height of the second part on the first preview image.

[0142] In some embodiments, the width of the second part of an object on the first preview image can be the width of the second detection box including the second part, and the height of the second part of an object on the first preview image can be the height of the second detection box including the second part.

[0143] For example, after obtaining the width of the second part of each object in the first preview image, the image acquisition device can calculate a second ratio between the width of the second part of each object in the first preview image and the width of the first preview image. If the second ratio between the width of the second part of an object in the first preview image and the width of the first preview image is greater than or equal to a second ratio threshold, the image acquisition device can set the second score of the object to a positive value. If the second ratio between the width of the second part of an object in the first preview image and the width of the first preview image is less than the second ratio threshold, the image acquisition device can set the second score of the object to 0.

[0144] For example, when the object within the frame is a person, the image acquisition device can acquire the human body width of each object in the first preview image and calculate a second ratio between the human body width of each object in the first preview image and the width of the first preview image. If the second ratio between the human body width of an object in the first preview image and the width of the first preview image is greater than or equal to a second ratio threshold, the image acquisition device can set the second score of that object to 1. If the second ratio between the human body width of an object in the first preview image and the width of the first preview image is less than the second ratio threshold, the image acquisition device can determine that the second score of that object is 0.

[0145] In some embodiments, the second ratio threshold described above can be set as needed, and this disclosure does not limit this. For example, the second ratio threshold described above can be any value from 0.08 to 1.

[0146] In some embodiments, the image acquisition device may further determine a fourth distance between the center point of the second part of each object on the first preview image and the center point of the first preview image, and determine a second score for each object based on the fourth distance between the center point of the second part of each object on the first preview image and the center point of the first preview image. The magnitude of the second score is negatively correlated with the fourth distance.

[0147] For example, the image acquisition device can calculate the second score of each object using the following formula (2).

[0148]

[0149] Wherein, S2 is the second score, x2 is the x-coordinate of the center point of the second part of an object on the first preview image, x0 is the x-coordinate of the center point of the first preview image, y2 is the y-coordinate of the center point of the second part of an object on the first preview image, y0 is the y-coordinate of the center point of the first preview image, W is the size of the first preview image in the X-axis direction, and H is the size of the first preview image in the Y-axis direction.

[0150] In some embodiments, the center point of the second part of each object on the first preview image may be the center point of the second detection box including the second part.

[0151] In some embodiments, the image acquisition device may also determine a second score for each object based on the sharpness of the second part of each object in the first preview image. For example, the image acquisition device may determine the second score of an object whose second part has a sharpness greater than or equal to a sharpness threshold in the first preview image as 1, and determine the second score of an object whose second part has a sharpness less than a sharpness threshold in the first preview image as 0.

[0152] In some embodiments, when the second part is a human body, the image acquisition device can also determine a second score for each object based on the human posture of each object. For example, the image acquisition device can determine a second score of 1 for an object in a standing posture and a second score of 0 for objects in postures other than standing.

[0153] In some embodiments, the image acquisition device determines the target object from multiple objects based on at least one of the size, position, and imaging quality of each object on the second preview image. This can be achieved by referring to the above-described method of determining the target object from multiple objects based on at least one of the size, position, and imaging quality of each object on the first preview image.

[0154] It should be noted that for any object in the first preview image and the second preview image, if the first part of the object is not included in the first preview image or the second preview image, then the first score of the first part of the object can be determined to be 0.

[0155] In some embodiments, since the weight of the first score is greater than the weight of the second score, after obtaining the first score and the second score of each object, the image acquisition device can identify the object with the highest first score among the multiple objects as the target object. If the first scores of multiple objects are all 0, the image acquisition device can identify the object with the highest second score among the multiple objects as the target object.

[0156] In some embodiments, the image acquisition device may also set priorities for the first part and the second part, wherein the priority of the first part is higher than the priority of the second part.

[0157] For example, when the first part is a face and the second part is a human body, the image acquisition device can prioritize determining the face over the human body.

[0158] In some embodiments, when the priority of the first part is higher than the priority of the second part, the image acquisition device can determine the object with the highest first score of the first part in the first preview image or the second preview image as the target object. If the first scores of the first parts of multiple objects are all 0, the image acquisition device can determine the object with the highest second score of the second part among the multiple objects as the target object.

[0159] In some embodiments, the image acquisition device may also identify the object with the highest sum of the first and second scores among the various objects as the target object.

[0160] In some embodiments, step 103 includes:

[0161] Based on a first preset mapping relationship between distance and shooting focal length, the target shooting focal length corresponding to the first distance between the target object and the image acquisition device is determined.

[0162] The image acquisition device is controlled to focus on the target object based on the target shooting focal length.

[0163] Understandably, the image acquisition device can also pre-store a first preset mapping relationship between distance and shooting focal length. Based on this, after determining the first distance corresponding to the target object from multiple distances, the image acquisition device can look up the shooting focal length corresponding to the first distance from the first preset mapping relationship to obtain the target shooting distance. Then, the image acquisition device can control itself to focus on the target object based on the target shooting focal length.

[0164] In some embodiments, after obtaining the target shooting focal length, the image acquisition device can adjust the lens to the position indicated by the target shooting focal length.

[0165] In some embodiments, the first preset mapping relationship stored in the image acquisition device can be a first preset mapping relationship between distance and code value. Based on this, after obtaining the first distance corresponding to the target object, the image acquisition device can look up the target code value corresponding to the first distance between the image acquisition device and the target object from the first preset mapping relationship, and adjust the lens of the image acquisition device to the position indicated by the target code value. Here, the code value can be understood as the lens adjustment parameter input by the focusing system in the image acquisition device when adjusting the lens position, and the focusing system can adjust the lens to different positions based on different code values.

[0166] In some embodiments, the image acquisition device can perform multiple image acquisitions on the preset object when there is a first preset distance between the preset object and the image acquisition device, and continuously adjust the focal length during the image acquisition process to obtain first images of the preset object at different shooting focal lengths. Then, the image acquisition device can determine the first image with the highest clarity and use the shooting focal length at which the first image with the highest clarity was captured as the shooting focal length corresponding to the first preset distance.

[0167] The image acquisition device can also perform multiple image acquisitions of the preset object at a second preset distance, continuously adjusting the focal length during the acquisition process to obtain second images of the preset object at different shooting focal lengths. Then, the image acquisition device can determine the second image with the highest clarity and use the shooting focal length at which the highest clarity second image was acquired as the shooting focal length corresponding to the second preset distance. By repeating this process multiple times, the image acquisition device can obtain multiple shooting focal lengths corresponding to different distances, that is, obtain a first preset mapping relationship between distance and shooting focal length.

[0168] For example, the image acquisition device can photograph a checkerboard textured board at a distance of 0.5m from the device, and adjust the code value of the device during the photographing process to obtain checkerboard textured board images at different code values. Then, based on the sharpness of the checkerboard textured board images photographed at different code values, the image acquisition device can determine the code value required to obtain the checkerboard textured board image with the highest sharpness, and set the code value corresponding to the checkerboard textured board image with the highest sharpness as the code value corresponding to 0.5m.

[0169] The image acquisition device can also photograph a checkerboard textured board 1 meter away from it, adjusting the code value during the photographing process to obtain checkerboard textured board images captured at different code values. Then, based on the sharpness of the checkerboard textured board images captured at different code values, the image acquisition device can determine the code value required to obtain the checkerboard textured board image with the highest sharpness, and define the code value corresponding to the checkerboard textured board image with the highest sharpness as the code value corresponding to 1 meter.

[0170] The image acquisition device can further adjust the distance between the checkerboard texture board and the image acquisition device, repeating the above process to obtain the code values ​​corresponding to different distances between the checkerboard texture board and the image acquisition device. Then, by fitting the obtained multiple distances with the corresponding multiple code values, a first preset mapping relationship between distance and code values ​​can be obtained.

[0171] refer to Figure 4 , Figure 4 This is a flowchart illustrating another focusing method according to another exemplary embodiment. In the following text, ... Figure 4 Taking the steps shown as an example, the focusing method provided in this disclosure embodiment will be described in detail.

[0172] In step 401, a first preview image is displayed on the image acquisition device based on the current framing range.

[0173] In step 402, a face or body in the first preview image is identified.

[0174] In step 403, a first detection box including a face and a second detection box including a human body are displayed on the first preview image.

[0175] In step 404, a first score for each face is determined based on the width of each face in the first preview image, the aspect ratio of each face in the first preview image, the distance between the center point of each face and the center point of the first preview image, the angle of each face, and the imaging quality of each face in the first preview image.

[0176] In step 405, a second score for each human body is determined based on the width of each human body in the first preview image, the distance between the center point of each human body and the center point of the first preview image, the posture of each human body, and the imaging quality of each human body in the first preview image.

[0177] In step 406, using the IOU value of each first detection box and each second detection box as the cost parameter, a face in the first preview image is matched with a human body using the Hungarian algorithm to obtain the matching relationship between the face and the human body.

[0178] In step 407, when the first preview image displayed on the image acquisition device is switched to the second preview image, each human body in the second preview image is identified.

[0179] In step 408, based on the matching relationship between faces and human bodies, faces that match each human body in the second preview image are determined, and faces that match the human body are identified as the same person's human body and face.

[0180] In step 409, an identification ID number is assigned to each person.

[0181] In step 410, multiple first distances are obtained by measuring the distance between each person in the field of view and the image acquisition device based on the UWB radar on the image acquisition device.

[0182] In step 411, based on the pixel distance between the two pupils on each person's face in the second preview image, the actual distance between the two pupils of the person, and the focal length of the image acquisition device when displaying the second preview image, the second distance between each person and the image acquisition device is determined based on formula (1).

[0183] In step 412, the absolute value of the difference between each person's second distance and the plurality of first distances is determined.

[0184] In step 413, the absolute value of the difference is used as the cost value, and the first distance corresponding to the second distance of each person is determined from multiple first distances using the Hungarian algorithm.

[0185] In step 414, the face with the highest first score is identified from faces with different ID numbers, and the person with that face is identified as the target object.

[0186] In step 415, if the second preview image does not contain a face, the human body with the highest second score is identified from the human bodies with different ID numbers, and the person with that human body is identified as the target object.

[0187] In step 416, the target object is focused based on the first distance of the target object.

[0188] refer to Figure 5 , Figure 5 This is a block diagram illustrating a focusing device according to an exemplary embodiment. Figure 5 As shown, the focusing device 500 includes:

[0189] The first measurement module 501 is configured to measure multiple objects within the field of view based on the ultra-wideband UWB radar on the image acquisition device, and obtain multiple first distances, wherein the first distance is the distance between the measured object and the image acquisition device.

[0190] The first determining module 502 is configured to determine the correspondence between multiple first distances and multiple objects based on the image information of each object on the first preview image, wherein the first preview image is an image within the field of view acquired by the image acquisition device;

[0191] The second determining module 503 is configured to determine the target object to be focused from multiple objects, and determine the first distance corresponding to the target object based on the correspondence relationship;

[0192] The control module 504 controls the image acquisition device to focus on the target object based on the first distance corresponding to the target object.

[0193] In some embodiments, the first determining module 502 is configured as follows:

[0194] Based on the pixel size of each object in the first preview image, a second distance between each object and the image acquisition device is determined, wherein the second distance is the distance between the object and the image acquisition device estimated based on the pixel size of the object in the first preview image;

[0195] The correspondence is determined based on the absolute value of the difference between the second distance of each object and multiple first distances.

[0196] In some embodiments, the first determining module 502 is configured as follows:

[0197] Based on the first coordinates of at least two key points on the first part of each object on the first preview image, determine the pixel distance of at least two key points of each object on the first preview image;

[0198] A second distance between each object and the image acquisition device is determined based on the pixel distance between at least two key points on each object in the first preview image, the actual distance between at least two key points on each object, and the current focal length of the image acquisition device.

[0199] In some embodiments, the device further includes:

[0200] The second measurement module is configured to remeasure the first distance between multiple objects within the field of view and the image acquisition device using UWB radar during the focusing process.

[0201] The acquisition module is configured to acquire a second preview image when a change in the first distance between any object within the field of view and the image acquisition device is detected.

[0202] The third determining module is configured to determine the first part of each object from the second preview image;

[0203] The fourth determining module is configured to redetermine the second distance between each object and the image acquisition device based on the first part of each object in the second preview image;

[0204] The fifth determination module is configured to determine the correspondence based on the absolute value of the difference between multiple remeasured first distances and multiple redetermined second distances.

[0205] In some embodiments, the third determining module is configured as follows:

[0206] Identify at least one first part included in the second preview image to obtain a first detection box including the first part;

[0207] Multiple second parts included in the second preview image are identified to obtain a second detection box including the second parts, wherein the size of the second part is larger than the size of the first part;

[0208] The matching relationship between the first part and the second part is determined based on the area of ​​the overlapping region between the first detection box and each of the second detection boxes;

[0209] Based on the features of the second part of each object, the second part of each object is determined from the second preview image;

[0210] Based on the matching relationship, the first part that matches each second part is determined from the second preview image;

[0211] The first part that matches the second part of each object is determined as the first part of each object.

[0212] In some embodiments, the second determining module is configured to:

[0213] The target object is determined from a plurality of objects based on at least one of the size, position and imaging quality of each object on the first preview image or the second preview image.

[0214] In some embodiments, the second determining module is configured to:

[0215] A first score for each object is determined based on at least one of the size, position, and imaging quality of the first part of each object on the first or second preview image;

[0216] A second score is determined for each object based on at least one of the size, position, and imaging quality of the second part on the first or second preview image, wherein the size of the second part is larger than the size of the first part, and the weight of the first score is greater than the weight of the second score.

[0217] The target object is determined from multiple objects based on the first and second scores of each object, as well as the weight of the first score and the weight of the second score.

[0218] In some embodiments, the first measurement module 501 is configured as follows:

[0219] The first preview image is used to identify faces and bodies, and the recognition results are obtained.

[0220] If the recognition results determine that the first preview image contains a face or a human body, multiple objects within the field of view are measured using the ultra-wideband (UWB) radar on the image acquisition device.

[0221] In some embodiments, the control module is configured as follows:

[0222] Based on a first preset mapping relationship between distance and shooting focal length, the target shooting focal length corresponding to the first distance between the target object and the image acquisition device is determined.

[0223] The image acquisition device is controlled to focus on the target object based on the target shooting focal length.

[0224] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0225] Figure 6 This is a structural block diagram illustrating an image acquisition device 600 according to an exemplary embodiment. For example, the image acquisition device 600 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc., with image acquisition capabilities.

[0226] Reference Figure 6 The image acquisition device 600 may include one or more of the following components: processing component 602, memory 604, power supply component 606, multimedia component 608, audio component 610, input / output (I / O) interface 612, sensor component 614, and communication component 616.

[0227] Processing component 602 typically controls the overall operation of image acquisition device 600, such as operations associated with at least one of display, telephone call, data communication, camera operation, and recording operation. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0228] Memory 604 is configured to store various types of data to support operation on image acquisition device 600. Examples of such data include at least one of the following: instructions for any application or method operating on image acquisition device 600, contact data, phone book data, messages, pictures, and videos. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0229] The power supply component 606 provides power to the various components of the image acquisition device 600. The power supply component 606 may include at least one of the following: a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the image acquisition device 600.

[0230] Multimedia component 608 includes a screen that provides an output interface between image acquisition device 600 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing image acquisition device and / or a rear-facing image acquisition device. When image acquisition device 600 is in an operating mode, such as shooting mode or video mode, the front-facing image acquisition device and / or the rear-facing image acquisition device may receive external multimedia data. Each front-facing and rear-facing image acquisition device may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0231] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when image acquisition device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0232] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, and buttons. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0233] Sensor assembly 614 includes one or more sensors for providing status assessments of various aspects of image acquisition device 600. For example, sensor assembly 614 may detect the on / off state of image acquisition device 600, the relative positioning of components such as the display and keypad of image acquisition device 600, changes in position of image acquisition device 600 or one of its components, the presence or absence of user contact with image acquisition device 600, orientation or acceleration / deceleration of image acquisition device 600, and temperature changes of image acquisition device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include optical sensors, such as complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensors, for use in imaging applications. In some embodiments, sensor assembly 614 may also include, but is not limited to, at least one of the following: an accelerometer, a gyroscope, a magnetometer, a pressure sensor, and a temperature sensor.

[0234] Communication component 616 is configured to facilitate wired or wireless communication between image acquisition device 600 and other devices. Image acquisition device 600 can access wireless networks based on communication standards, such as Wi-Fi, 4G, 5G, or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), and other technologies.

[0235] In an exemplary embodiment, the image acquisition device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0236] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including executable instructions or a computer program, which can be executed by the processor 620 of the image acquisition device 600 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0237] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform any of the focusing methods described in the embodiments of this disclosure. For example, the method includes:

[0238] Based on the ultra-wideband UWB radar on the image acquisition device, multiple objects within the field of view are measured to obtain multiple first distances, where the first distance is the measured distance between the object and the image acquisition device.

[0239] Based on the pixel information of each object in the first preview image, the correspondence between multiple first distances and multiple objects is determined, wherein the first preview image is an image within the field of view captured by the image acquisition device;

[0240] Identify the target object to be focused from multiple objects, and determine the first distance corresponding to the target object based on the correspondence;

[0241] Based on the first distance corresponding to the target object, the image acquisition device is controlled to focus on the target object.

[0242] This disclosure provides a computer program product comprising a computer program or executable instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or executable instructions from the computer-readable storage medium and executes the computer program or executable instructions, causing the computer device to perform any of the focusing methods described above in this disclosure.

[0243] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0244] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A focusing method, characterized in that, The method includes: Based on the ultra-wideband UWB radar on the image acquisition device, multiple objects within the field of view are measured to obtain multiple first distances, wherein the first distance is the measured distance between the object and the image acquisition device; Based on the image information of each of the objects in the first preview image, the correspondence between the plurality of first distances and the plurality of objects is determined, wherein the first preview image is an image within the field of view captured by the image acquisition device; The target object to be focused is determined from the plurality of objects, and a first distance corresponding to the target object is determined based on the correspondence. Based on the first distance corresponding to the target object, the image acquisition device is controlled to focus on the target object.

2. The method according to claim 1, characterized in that, The step of determining the correspondence between the plurality of first distances and the plurality of objects based on the image information of each of the objects in the first preview image includes: Based on the pixel size of each object in the first preview image, a second distance between each object and the image acquisition device is determined, wherein the second distance is the distance between the object and the image acquisition device estimated based on the pixel size of the object in the first preview image; The correspondence is determined based on the absolute value of the difference between the second distance of each of the objects and the plurality of first distances.

3. The method according to claim 2, characterized in that, Determining the second distance between each of the objects and the image acquisition device based on the pixel size of each object in the first preview image includes: Based on the first coordinates of at least two key points on the first part of each of the objects in the first preview image, determine the pixel distance between at least two key points of each of the objects in the first preview image; A second distance between each object and the image acquisition device is determined based on the pixel distance between at least two key points on each object in the first preview image, the actual distance between the at least two key points on each object, and the current focal length of the image acquisition device.

4. The method according to claim 3, characterized in that, The method further includes: During the focusing process, the first distance between multiple objects within the field of view and the image acquisition device is remeasured using UWB radar; When a change in the first distance between any object within the field of view and the image acquisition device is detected, a second preview image is acquired; Identify the first part of each of the objects from the second preview image; Based on the first part of each of the objects in the second preview image, the second distance between each of the objects and the image acquisition device is re-determined; The correspondence is determined based on the absolute values ​​of the differences between the remeasured first distances and the redefined second distances.

5. The method according to claim 4, characterized in that, Determining the first part of each of the objects from the second preview image includes: Identify at least one first part included in the second preview image to obtain a first detection box including the first part; Multiple second parts included in the second preview image are identified to obtain a second detection box including the second parts, wherein the size of the second part is larger than the size of the first part; Based on the area of ​​the overlapping region between the first detection frame and each of the second detection frames, the matching relationship between the first part and the second part is determined; Based on the features of the second part of each of the objects, the second part of each of the objects is determined from the second preview image; Based on the matching relationship, a first part that matches each of the second parts is determined from the second preview image; The first part that matches the second part of each of the objects is determined as the first part of each of the objects.

6. The method according to claim 4, characterized in that, The step of determining the target object to be focused from the plurality of objects includes: The target object is determined from the plurality of objects based on at least one of the size, position, and imaging quality of each of the objects in the first preview image or the second preview image.

7. The method according to claim 6, characterized in that, The step of determining the target object from the plurality of objects based on at least one of the size, position, and image quality of each of the objects on the first preview image includes: A first score for each object is determined based on at least one of the size, position, and imaging quality of a first part of each object in the first preview image or the second preview image; A second score is determined for each of the objects based on at least one of the size, position, and imaging quality of the second part of each object on the first preview image or the second preview image, wherein the size of the second part is larger than the size of the first part, and the weight of the first score is greater than the weight of the second score. The target object is determined from the plurality of objects based on the first and second scores of each of the objects, as well as the weights of the first and second scores.

8. The method according to any one of claims 1 to 7, characterized in that, The ultra-wideband (UWB) radar on the image acquisition device measures multiple objects within the field of view, including: The first preview image is used to identify faces and bodies, and the recognition results are obtained. If, based on the recognition result, it is determined that the first preview image contains a face or a human body, measurements are performed on multiple objects within the field of view using the UWB radar.

9. The method according to any one of claims 1 to 7, characterized in that, The step of controlling the image acquisition device to focus on the target object based on a first distance corresponding to the target object includes: Based on a first preset mapping relationship between distance and shooting focal length, the target shooting focal length corresponding to the first distance between the target object and the image acquisition device is determined. The image acquisition device is controlled to focus on the target object based on the target shooting focal length.

10. A focusing device, characterized in that, The device includes: The first measurement module is configured to measure multiple objects within the field of view based on the ultra-wideband UWB radar on the image acquisition device, and obtain multiple first distances, wherein the first distance is the measured distance between the object and the image acquisition device; The first determining module is configured to determine the correspondence between the plurality of first distances and the plurality of objects based on the image information of each of the objects in the first preview image, wherein the first preview image is an image within the field of view acquired by the image acquisition device; The second determining module is configured to determine the target object to be focused from the plurality of objects, and to determine the first distance corresponding to the target object based on the correspondence relationship; The control module is configured to control the image acquisition device to focus on the target object based on a first distance corresponding to the target object.

11. An image acquisition device, characterized in that, include: processor; Memory used to store computer programs or instructions; The processor executes the computer program or instructions to implement the steps of the method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing a computer program or instructions, characterized in that, When the computer program or instructions in the storage medium are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 9.

Images