Automatic focusing method and device, storage medium and computer program product

Abstract

The invention discloses an automatic focusing method and device, a storage medium and a computer program product, and relates to the technical field of computer vision, and the method comprises the steps: obtaining figure position information reported by a pre-constructed real-time target detection model, and determining a face image proportion based on the figure position information; when the face picture ratio is within a preset threshold range, determining a focusing weight area based on the figure position information; and dynamically allocating focusing priorities according to the focusing weight area to perform automatic focusing. Through comprehensive application of a real-time target detection model, face image proportion judgment, focusing weight area determination and dynamic focusing priority allocation, focusing can be quickly and stably carried out in a complex environment, and the focusing is kept clear.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to an automatic focusing method, device, storage medium and computer program product. Background Art

[0002] As an important function in conference scenes, face focus maintenance can ensure that the speaker in the conference remains in a locked focus state in the picture. Although traditional autofocus technology can achieve automatic focus on objects in the picture to a certain extent, in the face of complex and changing conference scenes, especially when the speaker frequently moves or swings in the picture, there are often problems such as unstable focus, excessive focus range or frequent focus, which leads to breathing effect. Traditional autofocus technology usually uses manual operation by a focus follower when facing the breathing effect, which not only increases labor costs, but also in actual operation, it is difficult to achieve the ideal focusing effect due to the limitations of human reaction speed and accuracy.

[0003] Therefore, how to focus quickly and stably and keep the focus clear in a complex environment becomes an urgent problem to be solved in this application.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present application is to provide an automatic focusing method, device, storage medium and computer program product, aiming to solve the technical problem of how to focus quickly and stably and keep the focus clear in a complex environment.

[0006] To achieve the above object, the present application proposes an automatic focusing method, which comprises: Obtaining person position information reported by a pre-built real-time target detection model, and determining the proportion of the face screen based on the person position information; When the face image ratio is within a preset threshold range, determining a focus weight area based on the person position information; Focus priorities are dynamically allocated according to the focus weight areas to perform automatic focusing.

[0007] In one embodiment, before the step of obtaining the person position information reported by the pre-built real-time target detection model and determining the screen proportion of the person position information, the step further includes: Collect diverse face image data as sample datasets; The sample data set is used to train a deep learning target detection framework to obtain a real-time target detection model.

[0008] In one embodiment, the step of obtaining person position information reported by a pre-built real-time target detection model and determining the proportion of the face screen based on the person position information includes: Obtain the person's location information reported by the pre-built real-time object detection model; A face focus algorithm is used to analyze the person position information to obtain a face detection result, and the face area is cropped based on the face detection result to determine the face screen ratio.

[0009] In one embodiment, when the face screen ratio is within a preset threshold range, the step of determining the focus weight area based on the person position information includes: When the face screen ratio is within a preset threshold range, selecting a focus area centered on the face in combination with the face detection result and the person position information; A dynamic weight allocation algorithm based on deep learning is used to select weights for the focus area to obtain a focus weight area.

[0010] In one embodiment, when the face screen proportion is within a preset threshold range, before or after the step of determining the focus weight area based on the person position information, the step further includes: When the face screen ratio is outside the preset threshold range, automatic focusing is performed using the screen center focusing method.

[0011] In one embodiment, the step of dynamically allocating focus priorities according to the focus weight areas to perform automatic focusing includes: The focus priority is calculated by analyzing the image features of the focus weight area; The focusing parameters are automatically adjusted according to the focusing priority to perform automatic focusing.

[0012] In one embodiment, after the step of dynamically allocating focus priorities according to the focus weight areas for automatic focusing, the step further includes: Monitoring and determining the stability of the person during the automatic focusing process; If the stability of the person is determined to be unstable, return to step: obtaining the person's position information reported by a pre-built real-time target detection model, and determining the proportion of the face screen based on the person's position information.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes an automatic focusing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the automatic focusing method described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the automatic focusing method described above are implemented.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the automatic focusing method described above are implemented.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: The pre-built real-time target detection model can quickly and accurately obtain the position information of the person in the picture, providing a basis for fast focusing; the face screen ratio is determined based on the person's position information. When the face screen ratio is within the preset threshold range, it means that the person is moderately far from the camera. Focusing on this basis can obtain a clear picture; when the face screen ratio is within the preset threshold range, the focus weight area is determined based on the person's position information. The focus weight area ensures that in a complex environment, when the person makes slight changes in movement, the focus point is always in the focus weight area to achieve stable focusing; according to the focus weight area, the focus priority is dynamically allocated for automatic focusing, ensuring that in the process of automatic focusing, the changes in the focus weight area are given priority, and the person's movement changes are quickly responded to in a complex environment, and the focal length is adjusted in time to maintain the clarity of the picture. The comprehensive use of real-time target detection, face screen ratio judgment, focus weight area determination, and dynamic focus priority allocation can achieve fast and stable focusing in a complex environment and keep the focus clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 This is a flow chart of the first embodiment of the automatic focusing method of the present application; Figure 2 This is a flow chart of a second embodiment of the automatic focusing method of the present application; Figure 3 This is a schematic diagram of the flow chart of the third embodiment of the automatic focusing method of the present application; Figure 4 This is a flowchart of a fifth embodiment of the automatic focusing method of the present application; Figure 5 A brief flowchart of the automatic focusing method provided in this application; Figure 6 This is a schematic diagram of the module structure of the automatic focusing device according to an embodiment of the present application; Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the automatic focusing method in the embodiment of the present application.

[0020] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0022] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of the embodiment of the present application is: obtaining the person position information reported by a pre-built real-time target detection model, and determining the face screen ratio based on the person position information; when the face screen ratio is within a preset threshold range, determining the focus weight area based on the person position information; dynamically allocating focus priority according to the focus weight area for automatic focusing.

[0024] In this embodiment, for the convenience of description, the following description is made by taking the identification of the automatic focusing system as the execution subject.

[0025] The embodiments of the present application take into account that face focus maintenance is an important function in conference scenes, which can ensure that the conference speaker remains in a locked focus state in the picture. Although the traditional automatic focusing technology can achieve automatic focusing on objects in the picture to a certain extent, when facing complex and changeable conference scenes, especially when the speaker frequently moves or swings in the picture, there will often be problems such as unstable focusing, excessive focusing amplitude or frequent focusing, which will lead to breathing effect. When facing the breathing effect, the traditional automatic focusing technology usually uses manual operation by the focus follower, which not only increases the labor cost, but also in actual operation, it is difficult to achieve the ideal focusing effect due to the limitations of human reaction speed and accuracy.

[0026] Therefore, the present application provides a solution, which uses a pre-built real-time target detection model to quickly and accurately obtain the position information of the person in the picture, providing a basis for fast focusing; based on the person's position information, the face screen ratio is determined. When the face screen ratio is within the preset threshold range, it means that the person is moderately away from the camera. Focusing operations on this basis can obtain a clear picture; when the face screen ratio is within the preset threshold range, the focus weight area is determined based on the person's position information. The focus weight area ensures that in a complex environment, when the person makes slight changes in movement, the focus point is always located in the focus weight area to achieve stable focusing; according to the focus weight area, the focus priority is dynamically allocated for automatic focusing, ensuring that in the process of automatic focusing, the changes in the focus weight area are given priority, and the person's movement changes are quickly responded to in a complex environment. The focal length is adjusted in time to maintain the clarity of the picture. The comprehensive use of real-time target detection, face screen ratio judgment, focus weight area determination, and dynamic allocation of focus priority can achieve fast and stable focusing in a complex environment and keep the focus clear.

[0027] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an automatic focusing system, etc. The following takes the automatic focusing system as an example to illustrate this embodiment and the following embodiments.

[0028] Based on this, the present application embodiment provides an automatic focusing method, referring to Figure 1 , Figure 1 FIG. 1 is a flow chart of the first embodiment of the automatic focusing method of the present application.

[0029] In this embodiment, the automatic focusing method includes steps S10 to S30: Step S10, obtaining person position information reported by a pre-built real-time target detection model, and determining the face screen ratio based on the person position information; The real-time target detection model is a model pre-trained based on a deep learning framework, which can analyze, identify and locate the target objects in each frame captured by the camera in real time (i.e. quickly and continuously).

[0030] The person's location information refers to the specific location information of the person in the picture reported by the real-time target detection model after analyzing the picture captured by the camera, such as the coordinate values ​​of the upper left corner and the lower right corner, or the center point coordinates and parameters such as width and height.

[0031] The face-to-screen ratio refers to the proportion of the face area in the picture, which is usually obtained by calculating the ratio of the face area to the entire picture area. The face-to-screen ratio can be used to evaluate the size or importance of the face in the picture.

[0032] The real-time target detection model is used to detect the input video or image frame and obtain the person's position information. The coordinate range and area of ​​the face area are calculated based on the person's position information. The area of ​​the entire screen is calculated, and the ratio of the face area to the screen area is calculated, that is, the face screen ratio. According to the size of the face screen ratio, the size and position of the person in the screen can be determined, which provides a basis for the subsequent focus weight allocation and automatic focusing process.

[0033] Exemplarily, the person's position information is used to determine the proportion of the face in the picture. If the person's picture accounts for a relatively small proportion of the entire picture, the person's body parts are considered. If the person's body parts are also relatively small, it is believed that the person is far away from the camera. At this time, consider using other focusing methods. However, if the person's face accounts for a high proportion of the entire picture and the quality is reliable, the face area is given priority.

[0034] Step S20, when the face image ratio is within a preset threshold range, determining a focus weight area based on the person position information; The preset threshold range is set according to the actual application scenario and the characteristics of the camera to keep the face clear. It is used to judge the importance or clarity of the face in the picture. When the face screen ratio is within the preset threshold range, the automatic focus system believes that the current face has sufficient clarity and importance in the picture and needs to be focused.

[0035] Furthermore, the autofocus system determines the focus weight area based on the person's position information (including the coordinates and size of the face, etc.). The focus weight area refers to the area that is given priority when performing autofocus, usually centered on the face and adjusted appropriately according to the size and position of the face. By setting the focus weight area, the autofocus system can ensure that the focus can stay stably on the face during the autofocus process, even if the person moves in the picture or makes slight changes in movement.

[0036] It should be noted that determining the focus weight area requires not only reference to the single factor of the person's position information, but also comprehensive consideration of multiple factors, including the size, position, movement speed, environmental factors, etc. of the face. The setting of the preset threshold range needs to be adjusted according to the actual application scenario to ensure the focusing effect and stability of the automatic focusing system.

[0037] Step S30: dynamically assigning focus priorities according to the focus weight regions to perform automatic focusing.

[0038] The autofocus system automatically assigns focus priorities based on the size, shape, and importance of the focus weight areas (e.g., the speaker's face area usually has a higher priority than other face areas). Areas with high priority will receive more attention during the autofocus process, and the system will adjust the focus more frequently to ensure that these areas remain clear.

[0039] After determining the focus priority, the auto focus system will adjust the focus according to the focus priority. If the presenter moves in the picture or changes his posture, the system will automatically update the focus weight area and reallocate the focus priority to ensure that the presenter is always in clear focus.

[0040] Understandably, in the process of dynamically allocating focus priorities, the balance between stability and response speed also needs to be considered. If the speaker's movements are large, the autofocus system needs to quickly adjust the focus to maintain clear focus; but if the speaker remains relatively stable, the autofocus system can reduce the frequency of focus adjustments to avoid unnecessary focusing movements.

[0041] This embodiment provides an automatic focusing method, which uses a pre-built real-time target detection model to quickly and accurately obtain the position information of the person in the picture, providing a basis for fast focusing; based on the person's position information, the face screen ratio is determined. When the face screen ratio is within the preset threshold range, it means that the person is moderately away from the camera. On this basis, focusing operations can obtain a clear picture; when the face screen ratio is within the preset threshold range, the focus weight area is determined based on the person's position information. The focus weight area ensures that in a complex environment, when the person makes slight changes in movement, the focus point is always located in the focus weight area, achieving stable focusing; according to the focus weight area, the focus priority is dynamically allocated for automatic focusing, ensuring that in the process of automatic focusing, the change of the focus weight area is given priority, and the person's movement changes are quickly responded to in a complex environment, and the focal length is adjusted in time to maintain the clarity of the picture. The comprehensive use of real-time target detection, face screen ratio judgment, focus weight area determination, and dynamic allocation of focus priority can achieve fast and stable focusing in a complex environment and maintain clear focus.

[0042] In a feasible implementation, step S10 may include steps S11-S12: Step S11, obtaining person position information reported by a pre-built real-time target detection model; The video frames are input into the real-time object detection model, which analyzes the video frames and outputs the person’s location information, including the person’s bounding box coordinates (i.e., the person’s position and size in the video frame) and possible confidence scores (indicating the credibility of the detected person’s position).

[0043] The auto-focus system and the real-time target detection model communicate in real time through an efficient data transmission protocol to ensure the timeliness and accuracy of the person's location information. At the same time, in order to ensure the accurate transmission and processing of information, the person's location information reported by the real-time target detection model usually adopts a standardized data format. This format defines the structure and content of the information, allowing the auto-focus system to easily parse and process this information.

[0044] Step S12: using a face focus algorithm to analyze the person position information to obtain a face detection result, and based on the face detection result, cropping the face area to determine the face screen ratio.

[0045] The face focus algorithm is an algorithm specifically used to process person position information and detect faces, including face detection, face feature extraction and other steps to accurately identify faces from person position information. The automatic focus system uses the face focus algorithm to analyze the person position information, identify and extract the face area from the provided person position information.

[0046] After processing by the face focus algorithm, the auto focus system will obtain one or more face detection results. The face detection results mainly include the specific face position, size and possible face feature information after detailed analysis of the person position information reported by the real-time target detection model.

[0047] Based on the face detection results, the autofocus system extracts the sub-area containing the face from the original picture taken by the camera. The cropping process ensures that sufficient facial information is retained while minimizing the interference of background and other irrelevant information.

[0048] After cropping the face area, the autofocus system calculates the proportion of this area in the picture. The face screen ratio is an important indicator that reflects the relative size of the face in the picture. This ratio is crucial for the subsequent allocation of focus weights, because a larger face ratio usually means a higher focus priority.

[0049] In this embodiment, the face focus algorithm is used to analyze the person position information reported by the real-time target detection model, and the face area is cropped based on the analysis result to determine the face screen ratio, which provides a key basis for the subsequent allocation of focus weights. It can be understood that the accuracy and robustness of this step are crucial to maintaining a clear focus on the face during the movement of the screen.

[0050] In a feasible implementation, step S20 may include steps S21-S22: Step S21, when the face image ratio is within a preset threshold range, selecting a focus area centered on the face in combination with the face detection result and the person position information; The autofocus system determines whether a specific focus area needs to be selected based on the face ratio in the face detection results. When the face ratio is within the preset threshold range, it means that the face is of moderate size in the picture and of high quality, making it suitable as an area for precise focus.

[0051] When the face screen ratio meets the preset conditions, a focus area centered on the face is selected based on the face detection results (such as the specific position and size of the face) and the person's position information. This focus area is selected to ensure that the camera's focus can always remain on the speaker's face even if the person moves in the picture.

[0052] Step S22: using a deep learning-based dynamic weight allocation algorithm to select a weight for the focus area to obtain a focus weight area.

[0053] The dynamic weight assignment algorithm based on deep learning is an algorithm that uses deep learning technology to dynamically assign weights to focus areas in an autofocus system. The algorithm uses a deep learning model to analyze face detection results and person location information, and then selects appropriate weights for the focus areas based on this information.

[0054] A dynamic weight allocation algorithm based on deep learning is used to select weights for the focus area. The dynamic weight allocation algorithm based on deep learning can dynamically adjust the weight allocation according to the size and position of the focus area and the dynamic changes of the characters in the picture.

[0055] Through the processing of the dynamic weight allocation algorithm, the system will obtain a focus weight area. This area will be given the highest priority during the autofocus process, that is, the camera will give priority to changes in this area to make focus adjustments. The purpose of this is to ensure that when the person moves or makes slight movements, the camera can adjust the focus quickly and stably, so as to keep the face in clear focus.

[0056] In this embodiment, when the face screen ratio is within the preset threshold range, the focus area centered on the face can be accurately selected in combination with the face detection result and the person's position information, ensuring that even if the person moves in the picture, the focus area can always follow the face, thereby ensuring that the speaker in the picture always remains clear. A dynamic weight allocation algorithm based on deep learning is used to select weights for the focus area, obtain the focus weight area, and realize intelligent adjustment of the focus area, so that the focus stability can be maintained when the person moves or the expression changes.

[0057] Based on the first embodiment of the present application, the second embodiment of the present application is proposed. In the second embodiment of the present application, the same or similar contents as those of the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following.

[0058] On this basis, please refer to Figure 2 Before step S10, the automatic focusing method further includes steps S01-S02: Step S01, collecting diverse face image data as a sample data set; Building a high-quality and diverse sample dataset is the key to successfully training a deep learning object detection model. The sample dataset will be used to train the model to recognize and locate the face area in the image.

[0059] Collect face images under different lighting, angles, expressions, occlusions, etc. by shooting, web crawling, etc. Alternatively, use publicly available, annotated face datasets on the Internet, such as LFW (Labeled Faces in the Wild) and WIDER FACE.

[0060] Manually annotate the collected images and mark the location of the face (usually using a rectangular box). Scale, crop, and normalize the images to ensure that the data format of the input model is consistent. Use data enhancement techniques such as rotation, flipping, and adding noise on the normalized images to increase data diversity and improve the generalization ability of the model.

[0061] Furthermore, the sample data set is divided into a training set, a validation set, and a test set. The training set is used to train the model, the validation set is used to adjust the model parameters, and the test set is used to evaluate the model performance.

[0062] Step S02: Use the sample data set to train a deep learning target detection framework to obtain a real-time target detection model.

[0063] Choose a deep learning target detection framework suitable for target detection tasks as a starting point, such as YOLO (YouOnly Look Once). The core idea of ​​YOLO is to transform the target detection problem into a regression problem, and directly predict the position and category of multiple objects in the image through a single neural network. Specifically, the input image is divided into SxS grids (the number of grids in actual applications will be much larger than 3x3, such as 19x19), and each grid is responsible for predicting whether there is a target in the grid and the category and location information of the target. In this way, YOLO can obtain the position and category information of all objects at the same time in one forward propagation, thereby achieving efficient target detection. As an efficient and accurate target detection algorithm, YOLO can serve as the basis for building a real-time target detection model.

[0064] Input the preprocessed training dataset into the deep learning object detection framework, set the training parameters such as learning rate, batch size, number of iterations, etc., and use the back-propagation algorithm and Adam optimizer to update the model weights.

[0065] Furthermore, the performance of the trained model is evaluated using the test set, including accuracy, recall, F1 score and other indicators. Based on the evaluation results, the model is fine-tuned to improve its performance. In order to achieve real-time detection, the model is optimized by pruning, quantization and other optimization operations to reduce the model's computational workload and memory usage. Alternatively, hardware acceleration technology (such as GPU, FPGA, etc.) is used to increase the model's running speed.

[0066] In this embodiment, a deep learning target detection framework is trained using a diverse sample data set to obtain a real-time target detection model. The real-time target detection model learns target features in various complex situations, and is therefore able to quickly process and report character location information, providing a basis for subsequent rapid and stable focusing and maintaining clear focus in complex environments.

[0067] Based on the first embodiment and / or the second embodiment of the present application, the third embodiment of the present application is proposed. In the third embodiment of the present application, the same or similar contents as those of the first embodiment and / or the second embodiment can be referred to the above introduction, and will not be repeated in the following.

[0068] On this basis, please refer to Figure 3 , Figure 3 A schematic diagram of a third embodiment of the automatic focusing method provided in the present application.

[0069] In this embodiment, when the face screen ratio is within a preset threshold range, before or after step S20 of determining the focus weight area based on the person position information, step A20 is further included: Step A20: When the face image ratio is outside a preset threshold range, automatic focusing is performed using a screen center focusing method.

[0070] It can be understood that step S20 and step A20 are in parallel relationship, and are two situations that may occur when the face screen ratio is compared with the preset threshold range.

[0071] When the face ratio is outside the preset threshold range, that is, the face ratio is too small or too large, the center of the screen is used. The center of the screen focus means that the camera's focus is automatically adjusted to the center of the screen. The center of the screen focus is suitable for situations where the face ratio is too small (such as the person is far away from the camera) or too large (such as the person is close to the camera lens). In this case, directly focusing on the center of the screen can ensure the overall clarity of the picture.

[0072] In addition, the present application provides another feasible implementation of step A20: when the face image ratio is outside a preset threshold range, automatic focusing is performed using a multi-point focusing method.

[0073] Set multiple focus points in the picture. When the face screen ratio exceeds the preset threshold, the auto focus system can automatically switch to one of these preset focus points for focusing. These focus points can be intelligently selected or manually set according to the content and structure of the picture. Multi-point focus can cope with changes in the face screen ratio to a certain extent while maintaining the overall clarity of the picture.

[0074] In this embodiment, the situation where the proportion of the face screen is outside the preset threshold range is discussed. Automatic focusing can be performed in a manner not limited to focusing on the center of the screen, and corresponding focusing strategies can be specified to respond flexibly, thereby ensuring fast and stable focusing in complex environments and maintaining clear focus.

[0075] Based on the above embodiments of the present application, a fourth embodiment of the present application is proposed. In the fourth embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above introduction, and will not be described in detail later.

[0076] In this embodiment, the step S30 of dynamically allocating focus priorities according to the focus weight area to perform automatic focusing may include steps S31-S32: Step S31, calculating and obtaining the focus priority by analyzing the image features of the focus weight area; Since the focus weight area is usually based on the position, size and reliability of the face or body in the picture, the autofocus system will conduct an in-depth analysis of the image features in the area, including: texture features, the texture complexity of the image in the area, which is usually related to the richness of the image details; edge features, the edge sharpness of the image in the area, the sharper the edge, the clearer the image; contrast features, the contrast of the image in the area, high contrast helps to improve the visual effect and clarity of the image.

[0077] By analyzing the above image features, the autofocus system can calculate the focus priority of the area. The focus priority is a quantitative value used to indicate the importance of the area in the focusing process relative to other areas in the picture.

[0078] Step S32: automatically adjusting focus parameters according to the focus priority to perform automatic focusing.

[0079] After obtaining the focus priority, the autofocus system will automatically adjust the focus parameters according to the focus priority to perform autofocus. The focus parameters include: focus distance, which is the distance from the camera lens to the object being photographed, which determines the degree to which the lens needs to be adjusted to achieve the best focus effect; focus speed, which is the speed at which the lens adjusts from the current position to the best focus position, which affects the speed and stability of the focusing process. Focus range, the size of the area where the lens can clearly image, which is usually related to the focal length and aperture size of the lens.

[0080] Depending on the focus priority, the autofocus system dynamically adjusts these focus parameters. For example, if the focus weight area has a high focus priority (i.e., the image features in this area are clear and important), the system may speed up the focus speed and reduce the focus range to ensure that the area can be imaged quickly and clearly. On the contrary, if the focus priority is low, the autofocus system may adopt a more conservative focus strategy to avoid unnecessary focus actions and image jitter.

[0081] In this embodiment, by deeply analyzing the image features of the focus weight area and calculating the focus priority, the key area that needs to be clearly focused is accurately identified. The focus parameters are automatically adjusted according to the focus priority, and the changes in the target position and state are quickly responded to to maintain the stability and accuracy of the focus. This process not only improves the efficiency and accuracy of the focus, but also ensures the overall clarity and visual effect of the picture, thereby achieving a fast and stable focusing effect in a complex environment.

[0082] Based on the above embodiments of the present application, a fifth embodiment of the present application is proposed. In the fifth embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above introduction, and will not be described in detail later.

[0083] On this basis, please refer to Figure 4 , Figure 4 A fifth embodiment of the process flow diagram of the automatic focusing method of the present application is provided.

[0084] In this embodiment, the step S30 of dynamically allocating the focus priority according to the focus weight area to perform automatic focusing further includes steps S40 to S50: Step S40, monitoring and determining the stability of the person during the automatic focusing process; During the autofocus process, the stability of the target person is continuously monitored by analyzing the position, size, speed and other parameters of the target person in consecutive frames. The autofocus system determines whether the target person is in a stable state based on the changes in these parameters. For example, if the position of the target person changes very little in consecutive frames, and the size and speed of movement are also relatively stable, then the target person can be considered to be in a stable state.

[0085] If the target person is stable, the autofocus system can maintain the current focus state and avoid unnecessary focus adjustments, thereby maintaining the clarity and stability of the image. On the contrary, if the target person is unstable, the autofocus system needs to adjust the focus parameters in time to ensure that the focus can always be locked on the target person.

[0086] Step S50, if the stability of the person is determined to be unstable, return to step: obtaining the person's position information reported by the pre-built real-time target detection model, and determining the proportion of the face screen based on the person's position information.

[0087] If the autofocus system detects that the target person is unstable, it needs to refocus. Return to the previous step, that is, obtain the person's position information reported by the pre-built real-time target detection model, and determine the proportion of the face in the picture based on this information.

[0088] This step is returned to re-evaluate the position and size of the target person so that the focus parameters can be adjusted based on the latest information. Since the target person may have moved or changed his posture, it is necessary to re-acquire his position information and determine the proportion of the face in the picture. In this way, the autofocus system can reallocate the focus weight area and calculate the focus priority based on the latest information to achieve accurate focus on the target person.

[0089] In this embodiment, by continuously monitoring the stability of the target person, and returning to reacquire the person's position information and determine the proportion of the face screen as needed, continuous and accurate focus on the target person is achieved. This dynamic adjustment capability enables the automatic focus system to focus quickly and stably in complex environments and maintain clear focus.

[0090] For example, to help understand the implementation process of the automatic focusing method obtained by combining this embodiment with the above embodiments, please refer to Figure 5 , Figure 5 A brief flowchart of an automatic focusing method is provided, specifically: In the frame before triggering focus, the automatic focus program checks the person position information reported by the AI ​​model, i.e., the real-time target detection model, and determines whether the selection conditions are met based on the person position information, i.e., whether the face ratio reaches the preset threshold.

[0091] When the person's position information is judged not to meet the selection conditions, the normal focusing process is performed, and the normal focusing process can be performed by selecting the center focus method or the multi-point focus method.

[0092] When the person's position information is judged to meet the selection conditions, the focus weight area is obtained based on the person's position information. The face area with a better proportion is selected in the focus weight area, and the first weight is selected for the area to reduce the problem of excessive focus range caused by losing the target during the focusing process due to the small size of the person. The changes ensure stability.

[0093] In addition, during the automatic focusing process, the position information of the person is regularly observed to see if it changes. If it does, the focus weight of the person is updated in time. If it does not change, continue to follow the previously selected area until the automatic focusing is completed.

[0094] After the automatic focus is completed, the situation information before the focus is completed is retained, and based on the detection situation during the automatic focus process, the stability of the person is judged and it is determined whether to re-trigger the automatic focus. In the scene detection, the person information obtained in the current frame will continue to be judged. If it is considered that the current person is relatively stable, the focus will not be triggered and the focus will be kept at the current position. If it is judged that the person is far away from the center or has changed a lot from the original position, the automatic focus will be re-performed.

[0095] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the automatic focusing method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0096] This application also provides an automatic focusing device, please refer to Figure 6 , the automatic focusing device comprises: The character position information acquisition module 10 is used to acquire the character position information reported by the pre-built real-time target detection model, and determine the proportion of the face screen based on the character position information; The focusing module 20 is used to determine a focusing weight area based on the person's position information when the face screen ratio is within a preset threshold range, and dynamically allocate a focusing priority according to the focusing weight area to perform automatic focusing.

[0097] The automatic focusing device provided by the present application adopts the automatic focusing method in the above embodiment, which can solve the technical problem of automatic focusing. Compared with the prior art, the beneficial effects of the automatic focusing device provided by the present application are the same as the beneficial effects of the automatic focusing method provided by the above embodiment, and other technical features in the automatic focusing device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0098] The present application provides an automatic focusing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the automatic focusing method in the above-mentioned embodiment 1.

[0099] Reference below Figure 7 , which shows a schematic diagram of the structure of an automatic focusing device suitable for implementing an embodiment of the present application. The automatic focusing device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The autofocus device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0100] like Figure 7 As shown, the autofocus device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the autofocus device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the autofocus device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an autofocus device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have instead.

[0101] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0102] The automatic focusing device provided by the present application adopts the automatic focusing method in the above embodiment to solve the technical problem of automatic focusing. Compared with the prior art, the beneficial effects of the automatic focusing device provided by the present application are the same as the beneficial effects of the automatic focusing method provided by the above embodiment, and other technical features in the automatic focusing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0103] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0104] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0105] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the automatic focusing method in the above-mentioned embodiment.

[0106] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0107] The computer-readable storage medium may be included in the automatic focusing device; or may exist independently without being assembled into the automatic focusing device.

[0108] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the automatic focusing device, the automatic focusing device: obtains the person position information reported by a pre-built real-time target detection model, and determines the proportion of the face screen based on the person position information; when the proportion of the face screen is within a preset threshold range, determines the focus weight area based on the person position information; and dynamically allocates the focus priority according to the focus weight area for automatic focusing.

[0109] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0110] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0111] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0112] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned automatic focusing method, and can solve the technical problem of automatic focusing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the automatic focusing method provided in the above-mentioned embodiment, and will not be described in detail here.

[0113] The present application also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the automatic focusing method as described above are implemented.

[0114] The computer program product provided by the present application can solve the technical problem of automatic focusing. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the automatic focusing method provided by the above embodiment, which will not be described in detail here.

[0115] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An automatic focusing method, characterized in that: The method comprises: Obtaining person position information reported by a pre-built real-time target detection model, and determining the proportion of the face screen based on the person position information; When the face image ratio is within a preset threshold range, determining a focus weight area based on the person position information; Focus priorities are dynamically allocated according to the focus weight areas to perform automatic focusing.

2. The method according to claim 1, characterized in that The step of obtaining the person position information reported by the pre-built real-time target detection model and determining the screen proportion of the person position information also includes: Collect diverse face image data as sample datasets; The sample data set is used to train a deep learning target detection framework to obtain a real-time target detection model.

3. The method according to claim 1, characterized in that The step of obtaining person position information reported by a pre-built real-time target detection model and determining the proportion of a face screen based on the person position information includes: Obtain the person's location information reported by the pre-built real-time object detection model; A face focus algorithm is used to analyze the person position information to obtain a face detection result, and the face area is cropped based on the face detection result to determine the face screen ratio.

4. The method according to claim 3, characterized in that When the face screen ratio is within a preset threshold range, the step of determining a focus weight area based on the person position information comprises: When the face screen ratio is within a preset threshold range, selecting a focus area centered on the face in combination with the face detection result and the person position information; A dynamic weight allocation algorithm based on deep learning is used to select weights for the focus area to obtain a focus weight area.

5. The method according to claim 1, characterized in that When the face screen ratio is within a preset threshold range, before or after the step of determining the focus weight area based on the person position information, the step further includes: When the face screen ratio is outside the preset threshold range, automatic focusing is performed using the screen center focusing method.

6. The method according to claim 1, characterized in that The step of dynamically allocating focus priorities according to the focus weight area to perform automatic focusing comprises: The focus priority is calculated by analyzing the image features of the focus weight area; The focusing parameters are automatically adjusted according to the focusing priority to perform automatic focusing.

7. The method according to claim 1, characterized in that After the step of dynamically allocating focus priorities according to the focus weight area for automatic focusing, the following step is further included: Monitoring and determining the stability of the person during the automatic focusing process; If the stability of the person is determined to be unstable, return to step: obtaining the person's position information reported by a pre-built real-time target detection model, and determining the proportion of the face screen based on the person's position information.

8. An automatic focusing device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the automatic focusing method according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the automatic focusing method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the automatic focusing method according to any one of claims 1 to 7 are implemented.

Images