A method and apparatus for controlling automatic focusing of a camera

Abstract

The application provides a method and device and equipment for controlling automatic focusing of a camera. The method comprises: in response to a focusing instruction, collecting a picture containing a target object by using the camera; calculating a distance between the target object and the camera according to the collected picture to obtain a focusing distance of the camera; obtaining a current background picture and determining a depth of field value of the background picture by using an image depth estimation algorithm; determining a target camera focal length corresponding to the picture corresponding to the depth of field value taken at the focusing distance, collecting the target object by using the target camera focal length, and fusing the target object with the current background picture. Through the above method, a user does not need to adjust the position following the switching of the background picture, and the camera can adjust the focal length in real time according to the current picture and the distance of the user to adapt to the change of the scene.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to a method, apparatus and device for controlling the automatic focusing of a camera. Background Technology

[0002] With the development of technology, many manufacturers add a "user center position" feature when developing virtual video conferencing apps. As the name suggests, this feature allows users to stay centered in the video by controlling the camera's pan, tilt, and focus. Examples include video conferencing apps like WeLink. Furthermore, with increasing emphasis on user privacy, features such as background replacement are also being added to video conferencing. This feature involves enabling facial segmentation during the video conference, combining the extracted user image with a preset background image to protect user privacy.

[0003] Currently, there are many methods to achieve user-centric positioning. The mainstream method involves collecting audio and video information and then controlling the camera's pan, tilt, and focus based on the user's position and size within the video. This method ignores the background image in the virtual meeting, calculating only based on the distance and position between the user and the camera, thus neglecting the harmony between the background image and the composite image. To solve this problem, app developers need to pre-design the background image for the virtual meeting. Users then need to adjust their distance from the camera to achieve a more harmonious composite image. However, when the background image is changed, the user needs to reposition themselves to achieve the desired harmony, a cumbersome process lacking intelligence. Summary of the Invention

[0004] To address the problem in existing technologies where users need to reposition themselves to achieve harmony between the image and background after changing the background image, a process that is cumbersome and lacks intelligence, this application provides a method, apparatus, and device for controlling the automatic focusing of a camera.

[0005] In a first aspect, this application provides a method for controlling the automatic focusing of a camera, the method comprising:

[0006] In response to a focus command, the camera captures an image containing the target object.

[0007] The distance between the target object and the camera is calculated based on the captured images to obtain the camera's focusing distance;

[0008] Obtain the currently used background image and use an image depth estimation algorithm to determine the depth value of the background image;

[0009] Determine the target camera focal length corresponding to the image with the depth of field value obtained at the stated focal distance, and use the target camera focal length to capture the target object and fuse it with the current background image.

[0010] In one possible implementation, calculating the distance between the target object and the camera based on the captured images to obtain the camera's focusing distance includes:

[0011] Determine the height W of the target object within the camera's field of view;

[0012] The pixels in the collected image are analyzed to determine the pixel width P of each pixel in the image;

[0013] Based on the camera focal length F, the height W, and the pixel width P used when capturing the image, the distance between the target object and the camera is calculated to obtain the camera's focusing distance.

[0014] In one possible implementation, the target object is a human body, and determining the height W of the target object within the camera's field of view includes:

[0015] Image recognition algorithms are used to identify human bodies in images and extract age features.

[0016] The age of the identified person is determined based on the extracted human age characteristics;

[0017] Based on the pre-set height corresponding to different ages, the height corresponding to the age of the identified human body is determined, and the determined height is used as the height W of the target object within the field of view of the camera.

[0018] In one possible implementation, the depth value of the background image is determined using an image depth estimation algorithm, including:

[0019] The background image is analyzed to segment the foreground image.

[0020] The depth value of the foreground image is determined using an image depth estimation algorithm;

[0021] Based on the estimated depth value of the foreground image, a depth estimation algorithm is used to determine the depth value of the foreground image, and the depth value of the foreground image is used as the depth value of the background image.

[0022] In one possible implementation, parsing the background image and segmenting the foreground image from the background image includes:

[0023] The background image is analyzed to determine the grayscale value of each pixel in the background image;

[0024] Based on a pre-set image segmentation threshold, pixels with grayscale values ​​not less than the image segmentation threshold are identified as pixels in the foreground image.

[0025] In one possible implementation, determining the target camera focal length corresponding to the image captured at the focal distance that corresponds to the depth of field value includes:

[0026] The target camera focal length f corresponding to the depth-of-field value obtained by shooting at the stated focal distance can be derived using the following formula:

[0027]

[0028] Where ΔL1 is the depth of field, δ is the diameter of the camera's circle of confusion, F is the camera's aperture value, and L is the focusing distance.

[0029] In one possible implementation, in response to a focusing command, the following is included:

[0030] When the set time interval is reached, a focusing command is triggered; or

[0031] When the background image is changed, a focus adjustment command is triggered.

[0032] Secondly, embodiments of this application provide a device for controlling the automatic focusing of a camera, the device comprising:

[0033] The image acquisition module is used to acquire images containing the target object using the camera in response to focus commands;

[0034] The focal length determination module is used to calculate the distance between the target object and the camera based on the captured image, and to obtain the focusing distance of the camera;

[0035] The depth determination module is used to acquire the currently used background image and determine the depth value of the background image using an image depth estimation algorithm.

[0036] The focal length adjustment module is used to determine the target camera focal length corresponding to the image with the depth of field value obtained at the focal distance, and to use the target camera focal length to capture the target object and fuse it with the current background image.

[0037] Thirdly, this application provides a device for controlling the automatic focusing of a camera, the device comprising:

[0038] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect above to control the autofocus of the camera.

[0039] Fourthly, this application provides a computer storage medium storing a computer program for causing a computer to perform the method described in the first aspect above.

[0040] This application provides a method, apparatus, and device for controlling the automatic focus of a camera. Using the solution provided in this application, APP developers do not need to focus on the design of the background image, and users do not need to adjust their position according to the switching of the background image. The camera will adjust the focus in real time according to the current image and the distance to the user to adapt to the changes in the scene. Attached Figure Description

[0041] Figure 1 This is a schematic diagram illustrating an application scenario of the camera autofocus control scheme provided in the embodiments of this application;

[0042] Figure 2 A schematic diagram of a terminal device structure for applying a method for controlling automatic focus of a camera according to an embodiment of the application;

[0043] Figure 3 This is a schematic flowchart of a method for controlling the automatic focus of a camera according to an embodiment of this application;

[0044] Figure 4 This is a flowchart illustrating a method for triggering a focusing command according to an embodiment of this application;

[0045] Figure 5 This is a schematic diagram of another process for triggering a focusing command according to an embodiment of this application;

[0046] Figure 6 This is a schematic diagram illustrating the process of calculating the distance between the target object and the camera according to an embodiment of this application;

[0047] Figure 7 This is a schematic diagram illustrating the ranging principle of a monocular camera according to an embodiment of this application;

[0048] Figure 8 This is a schematic flowchart of a method for determining the height W of a target object according to an embodiment of this application;

[0049] Figure 9 This is a depth-of-field diagram of the background image provided according to an embodiment of this application;

[0050] Figure 10 This is a schematic diagram illustrating the process of determining the depth value of the background image according to an embodiment of this application;

[0051] Figure 11 This is a schematic diagram of a background image for an example embodiment of this application;

[0052] Figure 12 This is a schematic diagram of a foreground view according to an example embodiment of this application;

[0053] Figure 13 This is a schematic diagram illustrating the process of segmenting a foreground image from a background image according to an embodiment of this application;

[0054] Figure 14 This is a schematic flowchart illustrating the method steps for determining the target focal length according to an embodiment of this application;

[0055] Figure 15 This is a schematic diagram illustrating automatic focusing after background replacement according to an embodiment of this application;

[0056] Figure 16 This is a schematic diagram illustrating the output of movement prompt information on the display interface according to an embodiment of this application;

[0057] Figure 17 This is a schematic diagram of a device for controlling the automatic focus of a camera according to an embodiment of this application;

[0058] Figure 18 This is a schematic diagram of a device for controlling the automatic focus of a camera according to an embodiment of this application;

[0059] Figure 19 This is a schematic diagram of a storage medium provided according to an embodiment of this application. Detailed Implementation

[0060] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The acquisition, storage, use, and processing of data in the embodiments of this application comply with the relevant provisions of national laws and regulations.

[0061] With technological advancements, many manufacturers add a "user-center" feature to their virtual video conferencing apps. This means that during a video conference, the camera's pan, tilt, and focus are controlled to keep the user centered in the video feed. Currently, the mainstream method controls the camera's pan, tilt, and focus by detecting the person's position and size within the video, without considering the background. This approach has the drawback that app developers need to pre-design the virtual meeting's background image, and users must manually adjust their distance from the camera to achieve a more harmonious blend of person and background.

[0062] In view of the aforementioned problems in related technologies, this application proposes a method, apparatus, and device for controlling the automatic focusing of a camera. The inventive concept of this application is as follows: first, depth estimation is performed on the background image to obtain a depth value; then, using the monocular ranging principle, the distance between the target object and the camera is calculated; finally, the focal length value of the camera is derived from the depth calculation formula; and the actual focal length of the camera is set based on this focal length value.

[0063] See Figure 1 This is a schematic diagram of an application environment according to an embodiment of this application.

[0064] like Figure 1 As shown, this application environment may include, for example, a network 10, a server 20, at least one terminal device 30, and a database 40. Specifically, the terminal device 30 is equipped with a camera, used to capture images within its field of view, and sends the captured images to the server 20 via the network 10. The server 20 receives the images and transmits them to other destination terminal devices, thus achieving video data sharing.

[0065] The camera autofocus method provided in this application embodiment can be applied to the server 20 mentioned above, where each server controls the automatic focus of the camera on each connected terminal device, or it can be applied to the terminal device mentioned above, where each terminal device controls the automatic focus of the camera itself.

[0066] Figure 2 The diagram illustrates a hardware configuration block diagram of a terminal device according to an embodiment of this application. The terminal device can be a mobile terminal or other devices with a camera. It should be understood that... Figure 2 The terminal device 200 shown is merely an example, and the terminal device 200 may have more than... Figure 2 The more or fewer components shown can be combined into two or more components, or they can have different component configurations. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0067] like Figure 2 As shown, the terminal device 200 includes components such as a communication component 210, a processor 220, a memory 230, a display 240, an input component 250, an audio circuit 260, a SIM card interface 270, and a sensor 280.

[0068] The communication component 210 is used to receive or send call requests, receive and send signals during a call, and connect to a server to upload or download data. The communication component 210 may include an RF (radio frequency) circuit 211 and a Wi-Fi (Wireless Fidelity) module 212.

[0069] RF circuit 211 can be used for receiving and transmitting signals during information transmission or calls. It can receive downlink data from the base station and pass it to processor 220 for processing; it can also send uplink data to the base station. Typically, RF circuit 211 includes, but is not limited to, devices such as an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. RF circuit 211 can receive electromagnetic waves via the antenna and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. RF circuit 211 can also amplify the signal modulated by the modem processor and radiate it as electromagnetic waves via the antenna. In some embodiments, at least some functional modules of RF circuit 211 can be housed in processor 220. In some embodiments, at least some functional modules of RF circuit 211 and at least some modules of processor 220 can be housed in the same device. The RF circuit 211 and antenna of terminal device 200 are coupled, enabling terminal device 200 to communicate with networks and other devices via wireless communication technology.

[0070] Wi-Fi is a short-range wireless transmission technology. Terminal device 200 can use Wi-Fi module 212 to help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Wi-Fi module 212 can connect to a router to connect to an external network. Wi-Fi module 212 can also connect to a server to upload or download data.

[0071] The memory 230 can be used to store data or program code used during the operation of the terminal device. The processor 220 executes various functions and data processing of the terminal device 200 by running the data or program code stored in the memory 230. The memory 230 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The memory 230 stores the operating system that enables the terminal device 200 to run.

[0072] The display 240 is used to display information input by the user or information provided to the user, as well as various menus of the terminal device 200, forming a graphical user interface (GUI). Specifically, the display 240 may include a screen disposed on the front of the terminal device 200. The display may be configured as a liquid crystal display, a light-emitting diode, or the like. The display 240 can be used to display the interface of the terminal device during operation.

[0073] Input component 250 can be used to receive numeric or character information input by the user, as well as various user operations, and generate signal inputs related to user settings and function control of terminal device 200. Specifically, input component 250 may include buttons and a touch screen. The touch screen may be located on the front of terminal device 200 and can collect touch operations on or near it by the user, such as clicking buttons, dragging scroll bars, etc.

[0074] The touch screen can be placed on top of the display. In some embodiments, the touch screen and the display can be integrated to realize the input and output functions of the terminal device 200. After integration, it can be referred to as a touch display.

[0075] The terminal device 200 may also include a positioning module, such as a satellite positioning module or a mobile communication network positioning module, which can determine the geographical location of the terminal device 200 in real time.

[0076] Audio circuitry 260, speaker 261, and microphone 262 provide an audio interface between the user and terminal device 200. Audio circuitry 260 converts received audio data into electrical signals, which are then transmitted to speaker 261, where they are converted into sound signals for output. Terminal device 200 may also be equipped with volume buttons for adjusting the volume of the sound signal. Conversely, microphone 262 converts collected sound signals into electrical signals, which are received by audio circuitry 260, converted into audio data, and then output to RF circuitry 211 for transmission to, for example, another terminal, or to memory 230 for further processing.

[0077] The SIM card interface 270 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 270 to make contact with and separate from the terminal device 200. The terminal device 200 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 270 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface simultaneously. The multiple cards can be of the same or different types. The SIM card interface is also compatible with different types of SIM cards. The SIM card interface is also compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the terminal device 200 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 200 and cannot be separated from the terminal device 200. The SIM card is used to identify the user's mobile phone number.

[0078] In addition to the SIM card interface 270, the terminal device 200 may also include a USB (universal serial bus) interface. The USB interface is used to connect a charging cable or other peripherals. For example, the terminal device 200 can connect a charging cable via the USB interface. The various components or modules in the terminal device 200 are connected via a bus.

[0079] The terminal device 200 may also include at least one sensor 280, such as an accelerometer 281, a proximity sensor 282, a fingerprint sensor 283, and a temperature sensor 284. The terminal device 200 may also be equipped with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, and motion sensor. For example, the fingerprint sensor 283 can be used to sense when a user clicks an icon on the terminal device 200's user interface.

[0080] The terminal device 200 may also include a camera for capturing still images or video. There may be one or more cameras. An object is projected onto a photosensitive element through a lens, generating an optical image. The photosensitive element may be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the processor 220 for conversion into a digital image signal.

[0081] The processor 220 is the control center of the terminal device 200. It connects various parts of the terminal via various interfaces and lines, and performs various functions and processes data of the terminal device 200 by running or executing software programs stored in the memory 230 and calling data stored in the memory 230. In some embodiments, the processor 220 may include one or more processing units. In this application, the processor 220 can run an operating system, applications, user interface display and touch response, and the method for controlling the automatic focus of a camera as described in the embodiments of this application.

[0082] This application provides a method for controlling the automatic focus of a camera, such as... Figure 3 As shown, the method includes:

[0083] 301: In response to a focus command, use the camera to capture an image containing the target object.

[0084] When this application is applied to a video conferencing scenario, the target object is the human body. During a video conference, the human body's position may constantly move. In one possible embodiment, such as... Figure 4 As shown, the following triggers are used to generate focus commands, including:

[0085] Step 401; Monitor the time since the last focus command was triggered;

[0086] Step 402: Determine whether the set time interval has been reached. If yes, proceed to step 403; otherwise, continue monitoring.

[0087] Step 403: Trigger the focus adjustment command.

[0088] Therefore, the autofocus method of this application can be used to adjust the focal length of the camera in real time.

[0089] Another possible embodiment, such as Figure 5 As shown, the focus command is triggered in the following way:

[0090] Step 501; Monitor the background image;

[0091] Step 502: Determine if the background image has been changed. If yes, proceed to step 503; otherwise, continue monitoring.

[0092] Step 503: Trigger the focus adjustment command.

[0093] Real-time adjustment of the camera's focal length allows for better integration with the changed background image. During the final image compositing process, this application requires merging the captured target object with the background image. Since different background images have varying depths of field, triggering a focus adjustment command when the background image is updated allows for timely image compositing and improves the user experience.

[0094] 302: Calculate the distance between the target object and the camera based on the captured image to obtain the camera's focusing distance.

[0095] This application utilizes the principle of monocular ranging. Assuming the height of the target object is known, it calculates the distance between the target object and the camera based on the acquired image to obtain the camera's focusing distance.

[0096] 303: Obtain the currently used background image and use an image depth estimation algorithm to determine the depth value of the background image.

[0097] According to the formula for calculating depth of field, depth of field is related to the diameter of the camera's circle of confusion, the camera's aperture value, the focusing distance, and the camera's focal length. Since the focusing distance can be determined when using a camera to capture a target object, the camera's focal length can be deduced using the depth of field formula.

[0098] 304: Determine the target camera focal length corresponding to the image with the depth of field value obtained at the stated focal distance, and use the target camera focal length to capture the target object and fuse it with the current background image.

[0099] This application estimates and adjusts the camera's focal length by calculating the depth of field of the background image and the position between the target object and the camera. Using the solution provided in this application has two major advantages: First, APP developers no longer need to design background images separately for each scene; any image can be adapted by adjusting the depth of field and focal length. Second, users no longer need to manually adjust the distance and position between themselves and the camera based on the appearance of the background image; this application can perform intelligent adaptation, further improving the user experience.

[0100] The following steps provide one or more possible implementation methods for each step of this application.

[0101] The first step is to use the monocular ranging principle to calculate the distance between the target object and the camera based on the acquired images, thereby obtaining the camera's focusing distance.

[0102] like Figure 6 Calculating the distance between the target object and the camera based on the captured images mainly includes the following steps:

[0103] Step 601: Determine the height W of the target object within the field of view of the camera;

[0104] Step 602: Analyze the pixels in the acquired image to determine the pixel width P of the pixels in the image;

[0105] Step 603: Calculate the distance between the target object and the camera based on the camera focal length F, the height W of the target object, and the pixel width P used when capturing the image, to obtain the camera's focusing distance.

[0106] The following is an explanation of the principle of monocular camera ranging, which is also known as the similar triangle method and uses the following principle for distance measurement:

[0107] like Figure 7 As shown, suppose there is a target object with height W, and this target is placed at a distance D from the camera. The camera takes a picture of the target object and measures the pixel width P of the target object, and the formula for the camera focal length F is: F = (P x D) / W.

[0108] For example, suppose a standard 8.5 x 11 inch A4 sheet of paper (landscape; W = 11) is placed at a distance D = 24 inches from the camera and a photo is taken. The pixel width of the A4 sheet of paper in the photo is measured to be P = 249 pixels.

[0109] Therefore, the focal length F is determined to be: F = (248px x 24in) / 11in = 543.45.

[0110] Similarly, if the focal length of the camera is known, the distance D' from the target object to the camera can be calculated using similar triangles: D' = (W x F) / P.

[0111] Therefore, if the focal length of the camera and the size of the target object are determined, the distance D from the target object to the camera can be obtained, where P refers to the pixel distance, W is the size, and F is the focal length of the camera.

[0112] In one or more possible embodiments, the target object is the human body, such as... Figure 8 As shown, determining the height W of the target object within the camera's field of view includes:

[0113] Step 801: Use an image recognition algorithm to identify the human body in the image and extract the age features of the human body;

[0114] Step 802: Determine the age of the identified person based on the extracted human age characteristics;

[0115] Step 803: Based on the pre-set height corresponding to different ages, determine the height corresponding to the age of the identified human body, and use the determined height as the height W of the target object within the field of view of the camera.

[0116] In this embodiment of the application, a standard age and height comparison table can be predefined, as shown in Table 1, to roughly estimate the user's height and obtain the value of W. Finally, based on the current focal length value F of the camera, the distance value D between the user and the camera can be obtained.

[0117] Table 1. National Standard Age-Height Comparison Table

[0118] Age / years 1 2 3 4 5 6 7 8 9 10 Height / cm 75 87 95 103 110 116 122 128 134 140 Age / years 11 12 13 14 15 16 17 18 19 20 Height / cm 145 151 158 162 165 166 168 170 172 173

[0119] The second step is to use an image depth estimation algorithm to calculate the image depth.

[0120] When taking a photograph, theoretically only the part that is accurately in focus should be in sharp focus; objects in front of and behind the focus point will appear blurry because they are outside the focus area. However, due to factors such as the lens and shooting distance, there will still be a certain distance of objects in front of and behind the focus point that can be clearly displayed and will not fall into the blur zone. This range of sharpness is called depth of field.

[0121] like Figure 9 As shown, the depth of field is calculated from the foreground depth of field and the background depth of field, where:

[0122] Promising prospects

[0123] Depth of field

[0124] Depth of field

[0125] In the above formula, δ is the diameter of the allowable circle of confusion, F is the aperture value of the lens, f is the focal length of the lens, and L is the focusing distance between the user and the camera.

[0126] There are three factors that affect depth of field: aperture size, focusing distance, and lens focal length. In the application scenario, the aperture size of the camera is a fixed value and cannot be adjusted. The focusing distance is the distance between the user and the camera. Therefore, under the premise that the aperture size remains unchanged and the distance between the user and the camera is fixed, a shorter focal length can create a greater depth of field, highlight the characteristics of the surrounding environment, and capture more environmental elements. That is, the smaller the focal length, the greater the depth of field.

[0127] In one or more possible embodiments, an image depth estimation algorithm is used to determine the depth value of the background image, such as... Figure 10 As shown, it includes:

[0128] Step 1001: Analyze the background image and segment the foreground image from the background image;

[0129] Step 1002: Determine the depth value of the foreground image using an image depth estimation algorithm;

[0130] Step 1003: Based on the estimated depth value of the foreground image, determine the depth value of the foreground image using a depth estimation algorithm, and use the depth value of the foreground image as the depth value of the background image.

[0131] When viewing images, one is often drawn to foreground objects and strong lines, curves, and shapes. Therefore, when compositing a portrait with a background, the composite image that is adapted to the size of the foreground is the visual plane that most closely resembles the user's viewing angle. This application's embodiments can utilize an image thresholding algorithm to obtain the foreground image of the background image, such as... Figure 11 The image shown is the background image. Figure 12 To segment the foreground image from the background image, an image depth estimation algorithm is used to obtain the image's depth value based on the foreground image. Then, a depth estimation algorithm is used to obtain the image's depth value.

[0132] The basic principle of image thresholding is: by utilizing the difference in grayscale characteristics between the target and the background in the image, the image is regarded as a combination of two types of regions with different grayscale levels. A suitable threshold is selected, and all pixels with a grayscale value greater than or equal to the threshold are regarded as belonging to the object and representing the foreground; otherwise, they are regarded as the background.

[0133] In one or more possible embodiments, the background image is parsed, and a foreground image is segmented from the background image, such as... Figure 13 As shown, it includes:

[0134] Step 1301: Analyze the background image to determine the grayscale value of each pixel in the background image;

[0135] Step 1302: Based on a pre-set image segmentation threshold, pixels with gray values ​​not less than the image segmentation threshold are identified as pixels in the foreground image.

[0136] The third step is to obtain the estimated focal length of the camera based on the depth-of-field calculation formula.

[0137] In this application, since the camera is fixed, its parameters δ and F values ​​remain unchanged. Based on the distance L between the user and the camera obtained in the first step and the foreground depth value ΔL1 obtained in the second step, the target camera focal length f corresponding to the depth value of the image captured at the focal distance is derived using the following formula:

[0138]

[0139] Where ΔL1 is the depth of field value, δ is the diameter of the camera's circle of confusion, F is the camera's aperture value, and L is the focusing distance.

[0140]

[0141] The fourth step is to use the target camera's focal length to capture the target object and fuse it with the current background image.

[0142] like Figure 14 The execution flow of the method steps after determining the target focal length in the embodiments of this application specifically includes:

[0143] Step 1401: Determine the new target focal length;

[0144] Step 1402: Acquire an image including the target object using the new target focal length;

[0145] Step 1403: Use an object recognition algorithm to segment the target object from the image;

[0146] Step 1404: Merge the segmented target object with the background image.

[0147] In this embodiment of the application, the terminal device includes a display screen, which can display the focused and merged image in real time. Since this application acquires a video stream in real time, and the video stream includes multiple frames, the focus can be adjusted in real time according to the user's movement distance or the background image, making the video display effect closer to reality. Figure 15 The image shown is a composite image of a user who changed the background using the background change option on the display interface. The image is then captured using autofocus and the merged image is readjusted.

[0148] In one or more possible embodiments, after calculating the depth-of-field value of the foreground image, the optimal focusing distance and optimal camera focal length for the target object with the best imaging effect can be determined. After adjusting the camera focal length to the optimal focal length, the target object can be further prompted to move to the position corresponding to the optimal focusing distance, such as... Figure 16 As shown, the system outputs movement prompts on the display interface and continuously adjusts the prompts according to the user's movement until the user moves to the position corresponding to the optimal focus distance. The movement prompts can be superimposed on the blended image, such as being displayed in the center to enhance the prompting effect, or they can be displayed in other specified positions.

[0149] Based on the same inventive concept, embodiments of this application provide a device for controlling the automatic focusing of a camera, such as... Figure 17 As shown, the device 1700 for controlling the automatic focus of the camera includes:

[0150] Image acquisition module 1701 is used to acquire images containing target objects using a camera in response to a focus command;

[0151] The focal length determination module 1702 is used to calculate the distance between the target object and the camera based on the acquired image, and to obtain the focusing distance of the camera;

[0152] The depth determination module 1703 is used to acquire the currently used background image and determine the depth value of the background image using an image depth estimation algorithm.

[0153] The focal length adjustment module 1704 is used to determine the target camera focal length corresponding to the image with the depth of field value obtained at the focal distance, and to use the target camera focal length to acquire the target object and fuse it with the current background image.

[0154] In one possible implementation, the focal length determination module calculates the distance between the target object and the camera based on the acquired image to obtain the camera's focusing distance, including:

[0155] Determine the height W of the target object within the camera's field of view;

[0156] The pixels in the collected image are analyzed to determine the pixel width P of each pixel in the image;

[0157] Based on the camera focal length F, the height W, and the pixel width P used when capturing the image, the distance between the target object and the camera is calculated to obtain the camera's focusing distance.

[0158] In one possible implementation, the target object is a human body, and the focal length determination module determines the height W of the target object within the field of view of the camera, including:

[0159] Image recognition algorithms are used to identify human bodies in images and extract age features.

[0160] The age of the identified person is determined based on the extracted human age characteristics;

[0161] Based on the pre-set height corresponding to different ages, the height corresponding to the age of the identified human body is determined, and the determined height is used as the height W of the target object within the field of view of the camera.

[0162] In one possible implementation, the depth determination module uses an image depth estimation algorithm to determine the depth value of the background image, including:

[0163] The background image is analyzed to segment the foreground image.

[0164] The depth value of the foreground image is determined using an image depth estimation algorithm;

[0165] Based on the estimated depth value of the foreground image, a depth estimation algorithm is used to determine the depth value of the foreground image, and the depth value of the foreground image is used as the depth value of the background image.

[0166] In one possible implementation, the depth-of-field determination module parses the background image and segments the foreground image from the background image, including:

[0167] The background image is analyzed to determine the grayscale value of each pixel in the background image;

[0168] Based on a pre-set image segmentation threshold, pixels with grayscale values ​​not less than the image segmentation threshold are identified as pixels in the foreground image.

[0169] In one possible implementation, the focus adjustment module determines the target camera focal length corresponding to the image captured at the focus distance that corresponds to the depth-of-field value, including:

[0170] The target camera focal length f corresponding to the depth-of-field value obtained by shooting at the stated focal distance can be derived using the following formula:

[0171]

[0172] Where ΔL1 is the depth of field, δ is the diameter of the camera's circle of confusion, F is the camera's aperture value, and L is the focusing distance.

[0173] In one possible implementation, the image acquisition module responds to a focus command and includes:

[0174] When the set time interval is reached, a focusing command is triggered; or

[0175] When the background image is changed, a focus adjustment command is triggered.

[0176] Based on the same inventive concept, embodiments of this application provide a device for controlling the automatic focusing of a camera, the device comprising:

[0177] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method for controlling camera autofocus provided in the above embodiments to control camera autofocus.

[0178] like Figure 18 As shown, the device includes a processor 1801, a memory 1802, a communication interface 1803, and a bus 1804. The processor 1801, the memory 1802, and the communication interface 1803 are interconnected via the bus 1804.

[0179] The processor 1801 is used to read and execute instructions from the memory 1802, so that the at least one processor can execute the method for controlling the automatic focus of the camera provided in the above embodiments.

[0180] The memory 1802 is used to store various instructions and programs for the method of controlling the automatic focus of the camera provided in the above embodiments.

[0181] The 1804 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 18 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0182] The processor 1801 can be a central processing unit (CPU), a network processor (NP), a graphics processing unit (GPU), or any combination of CPU, NP, and GPU. It can also be a hardware chip. The aforementioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0183] In addition, this application also provides a computer-readable storage medium, such as Figure 19 As shown, the computer storage medium stores a computer program that causes the computer to perform any of the methods described in the above embodiments.

[0184] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.

[0185] The memory may also include a program / utility 1325 having a set (at least one) of program modules 1324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0186] These computer program instructions may be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0187] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0188] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0189] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for controlling the automatic focusing of a camera, characterized in that, The method includes: In response to a focus command, the camera captures an image containing the target object. The distance between the target object and the camera is calculated based on the captured images to obtain the camera's focusing distance; Obtain the currently used background image and use an image depth estimation algorithm to determine the depth value of the background image; Determine the target camera focal length corresponding to the image with the depth of field value obtained at the focal distance, and use the target camera focal length to capture the target object and fuse it with the current background image; The step of determining the depth value of the background image using an image depth estimation algorithm includes: The background image is analyzed to segment the foreground image. The depth value of the foreground image is determined using an image depth estimation algorithm; Based on the estimated depth value of the foreground image, a depth estimation algorithm is used to determine the depth value of the foreground image, and the depth value of the foreground image is used as the depth value of the background image.

2. The method according to claim 1, characterized in that, The distance between the target object and the camera is calculated based on the captured images to obtain the camera's focusing distance, including: Determine the height W of the target object within the camera's field of view; The pixels in the collected image are analyzed to determine the pixel width P of each pixel in the image; Based on the camera focal length F, the height W, and the pixel width P used when capturing the image, the distance between the target object and the camera is calculated to obtain the camera's focusing distance.

3. The method according to claim 2, characterized in that, The target object is a human body. Determining the height W of the target object within the camera's field of view includes: Image recognition algorithms are used to identify human bodies in images and extract age features. The age of the identified person is determined based on the extracted human age characteristics; Based on the pre-set height corresponding to different ages, the height corresponding to the age of the identified human body is determined, and the determined height is used as the height W of the target object within the field of view of the camera.

4. The method according to claim 1, characterized in that, The background image is parsed, and the foreground image is segmented from the background image, including: The background image is analyzed to determine the grayscale value of each pixel in the background image; Based on a pre-set image segmentation threshold, pixels with grayscale values ​​not less than the image segmentation threshold are identified as pixels in the foreground image.

5. The method according to claim 1, characterized in that, Determining the target camera focal length corresponding to the image with the depth of field value obtained at the stated focal distance includes: The target camera focal length f corresponding to the depth-of-field value obtained by shooting at the stated focal distance can be derived using the following formula: in, δ is the depth of field value, F is the camera's permissible circle of confusion diameter, and L is the camera's aperture value.

6. The method according to claim 1, characterized in that, In response to focus commands, including: When the set time interval is reached, a focusing command is triggered; or When the background image is changed, a focus adjustment command is triggered.

7. A device for controlling the automatic focusing of a camera, characterized in that, The device includes: The image acquisition module is used to acquire images containing the target object using the camera in response to focus commands; The focal length determination module is used to calculate the distance between the target object and the camera based on the captured image, and to obtain the focusing distance of the camera; The depth determination module is used to acquire the currently used background image and determine the depth value of the background image using an image depth estimation algorithm. The focal length adjustment module is used to determine the target camera focal length corresponding to the image with the depth of field value obtained by shooting at the focal distance, and to use the target camera focal length to capture the target object and fuse it with the current background image; The step of determining the depth value of the background image using an image depth estimation algorithm includes: The background image is analyzed to segment the foreground image. The depth value of the foreground image is determined using an image depth estimation algorithm; Based on the estimated depth value of the foreground image, a depth estimation algorithm is used to determine the depth value of the foreground image, and the depth value of the foreground image is used as the depth value of the background image.

8. A device for controlling automatic focusing of a camera, characterized in that, The device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6 to control the autofocus of the camera.

9. A computer storage medium, characterized in that, The computer storage medium stores a computer program that enables the computer to perform the method as described in any one of claims 1-6.

Images