Auxiliary focusing method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses an auxiliary focusing method and device, equipment and a storage medium. The method comprises the following steps: transmitting acquired preview image data to a pre-added peak focusing node, wherein the peak focusing node is a node pre-added in a CHI of a Camx of a hardware abstraction layer; calling an OpenCL library integrated in the peak focusing node to carry out peak focusing processing on the preview image data on a GPU (Graphics Processing Unit) to obtain image data after peak focusing; and sending the image data after peak focusing to an image preview node so as to display an image after auxiliary focusing. According to the embodiment of the invention, the efficiency and accuracy of peak focusing can be improved, and extra overhead and data transmission are reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of auxiliary focus technology, and in particular to an auxiliary focus method, device, equipment and storage medium. Background Art

[0002] The camera's focus mode can be divided into automatic focus and manual focus. Automatic focus can improve the efficiency and success rate of the photographer, but it may also lose some personality and flexibility. Manual focus is a focusing method that makes the target in the picture clear and visible by manually adjusting the focus ring on the lens or the focus scale on the screen. Compared with automatic focus, manual focus can control the focus effect according to the photographer's wishes and creativity, and is more adaptable to some special shooting environments and needs.

[0003] Focus Peaking is a method of assisting focus that uses a specific color to enhance the outline of the in-focus range when shooting with manual focus. It helps photographers display the focus status in real time by highlighting the area with the highest contrast in the picture.

[0004] Implementing focus peaking in an Android application using OpenCV and OpenGL.

[0005] Integrate OpenCV and OpenGL libraries: Add OpenCV and OpenGL libraries to your Android project. Use the official Android SDK and the OpenGL ES library provided by Android, or use third-party library management tools such as OpenCV Manager and GLSurfaceView to simplify the integration process.

[0006] Get camera images: Use the Android camera API or a third-party camera library to get camera image data.

[0007] Image preprocessing: Use the functions provided by the OpenCV library to preprocess camera images, including image enhancement, noise reduction, and color space conversion.

[0008] Edge detection: Calculate the edge probability based on the matching degree of feature points. Use some indicators, such as the number of feature point matches, matching distance or consistency, to evaluate the accuracy of detection.

[0009] Real-time rendering: Use the functions and shader programming provided by the OpenGL library to render the processed image to the screen.

[0010] App calls SDK slowly: Since OpenCV and OpenGL in Android applications need to be called through SDK, this may cause certain performance loss and delay, and the focus efficiency is relatively low. Code execution is not as fast as C++: Compared with code written in native C++, the execution speed of code calling OpenCV and OpenGL through SDK may be slower. This is because there is additional overhead and data transmission during the SDK call process. Summary of the invention

[0011] The embodiments of the present application provide an auxiliary focusing method, device, equipment and storage medium, which can be used to improve the efficiency and accuracy of peak focusing and reduce additional overhead and data transmission. The technical solution is as follows:

[0012] On the one hand, an embodiment of the present application provides an auxiliary focusing method, the method comprising:

[0013] The acquired preview image data is passed to a pre-added peak focus node, where the peak focus node is a node pre-added in the CHI of Camx of the hardware abstraction layer;

[0014] Calling the OpenCL library integrated in the peak focus node to perform peak focus processing on the preview image data on the GPU to obtain image data after peak focus;

[0015] The peak focused image data is sent to an image preview node to display the auxiliary focused image.

[0016] On the other hand, an embodiment of the present application provides an auxiliary focusing device, the device comprising:

[0017] An image data transmission module, used to transmit the acquired preview image data to a pre-added peak focus node, wherein the peak focus node is a node pre-added in the CHI of Camx of the hardware abstraction layer;

[0018] A peak focus data acquisition module, used for calling the OpenCL library integrated in the peak focus node to perform peak focus processing on the preview image data on the GPU to obtain image data after peak focus;

[0019] The focused image preview module is used to send the peak focused image data to the image preview node to display the auxiliary focused image.

[0020] On the other hand, an embodiment of the present application provides a camera, wherein a peak focus node is added to the CHI of Camx of the hardware abstraction layer of the camera, wherein:

[0021] When performing auxiliary focusing, the camera executes any of the above methods.

[0022] On the other hand, an embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method described in the above aspects.

[0023] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the method described in the above aspects.

[0024] On the other hand, an embodiment of the present application provides a computer program product, which, when executed, is used to execute the method described in the above aspects.

[0025] In the technical solution provided in the embodiment of the present application, the acquired preview image data is passed to a pre-added peak focus node, which is a node pre-added in the CHI of Camx of the hardware abstraction layer. The OpenCL library integrated in the peak focus node is called to perform peak focus processing on the preview image data on the GPU to obtain the image data after peak focus. The image data after peak focus is displayed on the image preview node to display the image after auxiliary focus. In the embodiment of the present application, by adding a peak focus node in the CHI of the hardware abstraction layer, the OpenCL library integrated in the peak focus node can perform peak focus on the preview image data on the GPU, which can improve the efficiency and accuracy of peak focus compared to the manual focus method. At the same time, compared to the use of OpenCV and OpenGL in Android applications, which needs to be called through the SDK, which may cause certain performance losses and delays, it can reduce additional overhead and data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A flowchart of the steps of an auxiliary focusing method provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of an overall camera process provided by an embodiment of the present application;

[0028] Figure 3 A schematic diagram of a Node provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of a peak focus process provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of an image data processing flow provided in an embodiment of the present application;

[0031] Figure 6 A block diagram of an auxiliary focusing device provided in an embodiment of the present application;

[0032] Figure 7 A structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0034] Reference Figure 1 , shows a flowchart of the steps of an auxiliary focusing method provided by an embodiment of the present application. Figure 1 As shown, the auxiliary focusing method may include: step 101, step 102 and step 103.

[0035] Step 101: passing the acquired preview image data to a pre-added peak focus node, where the peak focus node is a node pre-added in the CHI of Camx of the hardware abstraction layer.

[0036] In this embodiment, the Hardware Abstraction Layer (HAL) is a software layer between the operating system software and the physical hardware. Its main purpose is to provide a unified and standard hardware platform interface for the upper-level application software and the operating system, thereby reducing the complexity of the software directly communicating with a variety of hardware. HAL isolates the hardware dependency of the operating system to the maximum extent, allowing the operating system core to provide hardware-independent services. In this way, the operating system and application programs can interact with the hardware through a standard interface without having to worry about the specific hardware implementation details.

[0037] The peak focus node is a node pre-added in the CHI of Camx of the hardware abstraction layer.

[0038] Among them, CamX (Camera Extensions) is a core module in the Qualcomm platform camera hardware abstraction layer, which represents a collection of code implementations of common functional interfaces.

[0039] CHI (Camera Hardware Interface) is another important component of the Qualcomm platform camera hardware abstraction layer. It represents a collection of code implementations for customizable requirements.

[0040] In the CamX-CHI architecture, CamX and CHI together form a complete camera hardware abstraction layer solution. CamX is responsible for providing a general functional interface and implementing communication with the kernel, while CHI is responsible for customizable demand implementation and control logic. Through an efficient interaction mechanism, the two together provide powerful camera function support for Android devices.

[0041] For specific architectures, it can be combined Figure 2 The overall process of the camera shown is described as follows. Figure 2 As shown, the system architecture may include:

[0042] 1. The application (in this example, the camera application) includes: a camera manager instance, a camera device, and a camera session instance.

[0043] The camera manager instance is the core component responsible for managing and controlling camera devices. In operating systems such as Android, it usually exists as a system service and is used to enumerate camera devices, obtain camera features, open and close camera sessions, etc. The camera manager instance provides an interface for interacting with camera hardware, allowing applications to easily access and control camera functions.

[0044] A camera device instance represents a specific camera hardware device. When a camera manager instance opens a camera device, it creates a camera device instance that contains all the hardware features and functions of the camera. Through the camera device instance, the application can perform operations such as taking photos and recording videos, and adjust various camera parameters such as exposure and focus.

[0045] A camera session instance is a session established on a camera device instance for performing a series of camera operations. A camera session instance allows an application to set parameters on a camera device, capture images or videos, and close the session when finished. The camera session instance exists to provide a mechanism for isolation and synchronization when sharing camera resources between multiple applications or different parts of an application.

[0046] 2. FW JAVA refers to a software component written in JAVA, which contains the camera's API (Application Program Interface) and defines the camera framework.

[0047] 3. FW Native refers to the native software of the camera. These firmware or software are specially designed for camera hardware to provide the best performance and compatibility. It includes the camera service interface to provide camera services.

[0048] 4. HAL (Hardware Abstraction Layer), which includes: ICameraProvider (camera provider interface), Camxcamera.qcom.so (Qualcomm CamX camera driver library) and CHI.

[0049] ICameraProvider is an interface used to provide camera device services (CameraProvider) in the Android system. It is responsible for generating a list of camera device service names so that these camera devices can be opened and used later. It is an important component of the Camera HAL (Hardware Abstraction Layer), allowing the Camera Service to interact with specific camera hardware.

[0050] camera.qcom.so is a dynamic link library file under the CamX (Camera Extensions) architecture provided by Qualcomm for its camera hardware. CamX is a camera driver architecture launched by Qualcomm, which aims to provide more flexible and scalable camera function support. This library file contains the drivers and algorithms required to implement camera functions, so that the Camera Service can control and manage the camera hardware of the Qualcomm platform through it. Camx provides HAL interface (HAL interface) for data transmission between the camera device service.

[0051] In HAL (Hardware Abstraction Layer), CHI (Camera Hardware Interface) refers to the camera hardware interface. It is an important component of the Qualcomm platform camera system, mainly used to define the interaction interface between the camera hardware and the camera service. CHI aims to provide a flexible and scalable way to connect and manage different components of camera hardware, such as sensors, image signal processors (ISPs), encoders, etc. Among them, CHI pipeline topology refers to the topology used to define the camera operation pipeline in the Qualcomm CamX-CHI architecture. In the camera system, a pipeline refers to a series of image processing steps executed in sequence, which together complete the conversion process from raw image data to the final output image (such as preview, photo or video). CHIpipeline topology describes the connection relationship and parameter configuration between each node in these pipelines (such as Sensor, ISP, Encoder, etc.) in the form of an XML file, allowing developers to flexibly define and adjust the behavior and functions of the camera.

[0052] In this embodiment, a focus peaking node (ie, FocusPeaking Node) is added to the CHI operation pipeline to execute the process of the focus peaking. Meanwhile, the CHI can provide a CHI Interface to communicate with Camx.

[0053] During the image peaking focus process, the acquired preview image data can be passed to the pre-added peaking focus node. In a specific implementation, the camera application can take a preview image and send the preview image to the CPU (Central Processing Unit) for processing. The preview image data processed by the CPU will flow into the Pipeline of the HAL layer. The image data flows through the Pipeline, and each node processes the data in sequence, and finally reaches the peaking focus node, so that the peaking focus node can perform peaking focus based on the preview image data. For the Pipeline in CHI, it can be as follows Figure 3 As shown, it includes:

[0054] 1. Camera IMX299 (image sensor), capable of capturing high-quality videos and images. It may have specific characteristics such as pixel count, dynamic range, low-light performance, etc.

[0055] 2. CSIPHY and CSID are usually related to camera interfaces. CSIPHY (Camera Serial Interface Physical Layer) is responsible for handling the physical layer of high-speed serial communication between the camera module and the main processor. CSID (Camera Serial Interface Device) refers to the device-side implementation of this interface, which is used to transfer image data from the sensor to the processor.

[0056] 3. IFE (Image Front End): Image front-end processing unit, responsible for receiving raw data from the image sensor and performing preliminary processing, such as denoising and color correction, for subsequent processing and analysis.

[0057] 4. Stats Processing: Statistical information processing is a key part of the camera system, which is used to collect and process various statistical information in the image, such as brightness, contrast, automatic exposure, automatic white balance and autofocus parameters.

[0058] 5. Qcom Face Detection refers to the facial detection function provided by Qualcomm. Qualcomm is a leading semiconductor company whose chips are widely used in smartphones and other mobile devices. The facial detection function can automatically identify faces in images and is used in a variety of applications, such as face recognition, autofocus, photo optimization, etc.

[0059] 6. 3A Algorithm refers to the algorithms of Automatic Exposure (AE), Automatic White Balance (AWB) and Auto Focus (AF). These algorithms work together to ensure that clear and color-accurate photos can be taken under various lighting conditions.

[0060] 7. EIS V2: The second generation version of Electronic Image Stabilization. EIS is a technology that uses software algorithms to reduce or eliminate image blur caused by camera shake.

[0061] 8. BPS (Burst Processing System) is used to process the large amount of image data captured by the camera in high frame rate or continuous shooting mode.

[0062] 9. IPE (Image Processing Engine) is responsible for further processing the image data after preliminary processing, including sharpening, compression, color adjustment, etc., to generate the final image or video output.

[0063] 10. Preview (Camera Preview, i.e. the Image Preview node in this example) allows the user to view the real-time image from the camera lens before shooting.

[0064] 11. Video (video recording node) is used to record and save video clips.

[0065] It also includes Focus Peaking (peak focus node), and Pipeline is used to add the FocusPeaking (peak focus node) shown in the figure above.

[0066] like Figure 3As shown, the Focus Peaking node connects the IPE and Preview nodes, and the preview image data can flow into the Focus Peaking node through the IPE node. In this example, the current focus mode, whether MF (Manual Focus) or AF (AutoFocus) can be obtained inside the FocusPeaking node through MetaData data. Among them, MetaData data (i.e. metadata) is descriptive information about data and information resources, which can be used to indicate a specific focus mode. In actual applications, the state migration of the AF function and the MF function can be sent to the focus peaking node by setting MetaData data in the App.

[0067] If it is AF, the FocusPeaking node function is to copy the preview image data unchanged and not process the preview image data.

[0068] If it is MF, the FocusPeaking node draws the preview image data before outputting it, that is, focusing on the preview image data.

[0069] After the acquired preview image data is transferred to the pre-added peak focus node, step 102 is performed.

[0070] Step 102: calling the OpenCL library integrated in the peak focus node to perform peak focus processing on the preview image data on the GPU to obtain peak focus image data.

[0071] After the acquired preview image data is passed to the peak focus node, the OpenCL library integrated in the peak focus node can be called to perform peak focus processing on the preview image data on the GPU to obtain image data after peak focus.

[0072] In a specific implementation of the present application, the OpenCL library can be called to use GPU resources in a parallel processing manner to perform edge detection and RGB value adjustment processing on the preview image data to obtain image data after peak focus.

[0073] In the specific implementation, the kernel object parameters involved may include: 1. Input - the starting address of the original image memory, 2. Output - the starting address of the image memory after drawing, 3. The length of the original image after alignment, 4. The height of the original image after alignment, 5. The critical threshold of the gradient amplitude, and 6. The color of the drawn edge line.

[0074] The input image data can be divided into as many work groups as there are pixels in it. Parallel computing is used to detect and draw each pixel to reduce processing time. When the pixel represented by the work group is the edge pixel of the four sides of the image, the YUV data of this pixel is directly copied from the input to the output memory (i.e., temporary storage of edge pixels), and when the gradient amplitude threshold is 0, direct copying of data is also performed.

[0075] In this embodiment, the OpenCL library can be called to adopt a multi-threaded parallel processing method to perform edge detection on the preview image data and adjust the RGB value of the detected edge pixel points, so as to obtain the image data after peak focus. By adopting a parallel processing method, the processing efficiency of image data can be improved, and the user's shooting experience can be improved.

[0076] In another specific implementation of the present application, the step of performing edge detection and adjusting the RGB values ​​of edge pixels on the preview image data to obtain the peak focused image data may include: sub-step S1, sub-step S2 and sub-step S3.

[0077] Sub-step S1: calling the OpenCL library to use the GPU resources to calculate the gradient amplitude of each pixel in the preview image data.

[0078] In this embodiment, the gradient magnitude can be used for edge detection, image sharpening, texture analysis, etc. The gradient magnitude indicates how fast the pixel value of a point in the image changes within its neighborhood, that is, the rate or intensity of change of the pixel value of the point.

[0079] The OpenCL library can be called to use GPU resources to calculate the gradient magnitude of each pixel in the preview image data.

[0080] In practical applications, the filter templates in the horizontal and vertical directions of the image can be pre-set. Suppose the convolution factors in the x-direction and y-direction are G x and G y The template is as follows, where A is the original image.

[0081]

[0082] After matrix operation, the horizontal gray value G can be obtained. x and the vertical gray value G y, these two grayscale values ​​are the gradient components in the x and y directions of the image. In practical applications, the gradient components can be calculated by applying appropriate filters (such as Sobel operator, Prewitt operator, Roberts operator, etc.) to the image. Taking the Sobel operator as an example, it will apply two 3x3 convolution kernels to calculate G x and G y .

[0083] Then, the gradient magnitude of the pixel is calculated by using the Euclidean distance:

[0084]

[0085] After the gradient magnitude of each pixel in the preview image data is calculated, sub-step S2 is executed.

[0086] Sub-step S2: Obtain edge pixel points in the preview image data according to the gradient amplitude and a preset gradient amplitude threshold.

[0087] The gradient magnitude threshold is a pre-set threshold for detecting edge pixels. This threshold is used to distinguish which pixels are considered edge points and which are not. The choice of the threshold depends on the specific application and the characteristics of the image. If the threshold is set too low, many non-edge noise points may be detected; if it is set too high, some important edge information may be missed.

[0088] After calculating the gradient magnitude of each pixel in the preview image data, the edge pixel in the preview image data can be obtained according to the gradient magnitude and the preset gradient magnitude threshold. For each pixel, if its gradient magnitude is greater than or equal to the threshold, the pixel is marked as an edge point, otherwise it is regarded as a non-edge point.

[0089] After obtaining edge pixel points in the preview image data according to the gradient amplitude and the preset gradient amplitude threshold, sub-step S3 is executed.

[0090] Sub-step S3: adjusting the RGB values ​​of the edge pixels to obtain the peak focused image data.

[0091] After obtaining edge pixels in the preview image data according to the gradient amplitude and the preset gradient amplitude threshold, the RGB values ​​of the edge pixels can be adjusted to obtain image data after peak focus. In a specific implementation, the attribute data of the preset edge pixels sent by the camera application can be obtained, and the attribute data of the edge pixels can include: the target RGB value of the edge pixels. Then, the RGB value of the edge pixels can be adjusted to the target RGB value to obtain the image data after peak focus.

[0092] The processing flow can be as follows Figure 4 As shown, home refers to the main interface of the operation interface, on which the camera APP can be operated to turn on or off the camera.

[0093] Normal picture refers to an unprocessed image. After opening the camera, you can get the normal picture and set the corresponding focus mode. If it is MF mode, you can use the peak focus node to perform peak focus on the normal picture to obtain a peak focus image. Specifically, you can set the properties through the APP to draw the peak focus line of the image to obtain a peak focus image. The updated RGB image data is converted back to YUV data, and then assigned to the output image data memory. The output data is the image data after the edge is drawn.

[0094] After calling the OpenCL library integrated in the peak focus node to perform peak focus processing on the preview image data on the GPU to obtain peak focus image data, step 103 is executed.

[0095] Step 103: Send the peak focused image data to an image preview node to display the auxiliary focused image.

[0096] After calling the OpenCL library integrated in the peak focus node to perform peak focus processing on the preview image data on the GPU to obtain the peak focus image data, the peak focus image data can be sent to the image preview node to display the auxiliary focus image.

[0097] The process of processing image data using OpenCL can be as follows Figure 5 During the processing, the host sends data and tasks to the device, which performs calculations and finally synchronizes them on the host.

[0098] In this example, the host is the CPU and the device is the GPU. Each device corresponds to a context, and the host interacts and manages the device through the context.

[0099] The channel through which the host side controls the computing device, pushing a series of commands for the device side to execute, including data transmission and computing tasks. A command queue can only manage one device, and can be executed sequentially or out of order.

[0100] The kernel part of OpenCL calculation is expressed as a piece of C-style code. When the device needs to perform a computing task, the data will be pushed to the device, and then multiple computing units on the device will execute the kernel program concurrently to complete the predetermined computing process.

[0101] In OpenCL, cl_program is used to represent program objects, which can be created using source code text or binary data.

[0102] The embodiment of the present application uses the OpenCL library inside Camx and uses the GPU for image processing to achieve fast and efficient edge detection. The parallel computing capability of the GPU can accelerate the image processing process and improve the speed and effect of edge detection. Using the GPU for image processing can achieve real-time performance and respond to user operations immediately. This is very important for the peak focus function because the user can see the focus effect immediately, improving the user experience. At the same time, by performing image processing on the GPU, the burden on the CPU can be reduced and system resources can be saved. This can provide more computing resources for other applications and improve the performance and stability of the overall system.

[0103] The auxiliary focus method provided in the embodiment of the present application passes the acquired preview image data to a pre-added peak focus node, which is a node pre-added in the CHI of Camx of the hardware abstraction layer. The OpenCL library integrated in the peak focus node is called to perform peak focus processing on the preview image data on the GPU to obtain the image data after peak focus. The image data after peak focus is displayed in the image preview node to display the image after auxiliary focus. The embodiment of the present application adds a peak focus node in the CHI of the hardware abstraction layer, and the OpenCL library integrated in the peak focus node can perform peak focus on the preview image data on the GPU, which can improve the efficiency and accuracy of peak focus compared to the manual focus method. At the same time, compared to the use of OpenCV and OpenGL in Android applications, which needs to be called through the SDK, which may cause certain performance losses and delays, it can reduce additional overhead and data transmission.

[0104] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0105] Reference Figure 6 , shows a block diagram of an auxiliary focus device provided by an embodiment of the present application. The device has the function of implementing the above method example, and the function can be implemented by hardware or by hardware executing corresponding software. The auxiliary focus device 600 may include: an image data transmission module 610, a peak focus data acquisition module 620 and a focus image preview module 630.

[0106] An image data transmission module 610 is used to transmit the acquired preview image data to a pre-added peak focus node, where the peak focus node is a node pre-added in the CHI of Camx of the hardware abstraction layer;

[0107] The peak focus data acquisition module 620 is used to call the OpenCL library integrated in the peak focus node to perform peak focus processing on the preview image data on the GPU to obtain image data after peak focus;

[0108] The focused image preview module 630 is used to send the peak focused image data to the image preview node to display the auxiliary focused image.

[0109] Optionally, the peak focus data acquisition module includes:

[0110] The focus data acquisition unit is used to call the OpenCL library to use GPU resources in a parallel processing manner to perform edge detection and RGB value adjustment processing on the preview image data, so as to obtain the image data after the peak focus.

[0111] Optionally, the peak focus data acquisition module includes:

[0112] A gradient magnitude calculation unit, configured to call the OpenCL library and use the GPU resources to calculate the gradient magnitude of each pixel in the preview image data;

[0113] An edge pixel point acquisition unit, used for obtaining edge pixel points in the preview image data according to the gradient amplitude and a preset gradient amplitude threshold;

[0114] The peak focus data acquisition unit is used to adjust the RGB value of the edge pixel point to obtain the image data after the peak focus.

[0115] Optionally, the gradient amplitude calculation unit includes:

[0116] A grayscale value calculation subunit, used for calling the OpenCL library to use the GPU resources to calculate the horizontal grayscale value and the vertical grayscale value of each pixel in the preview image data;

[0117] The gradient amplitude calculation subunit is used to calculate the gradient amplitude of each pixel based on the horizontal grayscale value and the vertical grayscale value of each pixel.

[0118] Optionally, the edge pixel acquisition unit includes:

[0119] The edge pixel point acquisition subunit is used to take the pixel points whose gradient amplitude is greater than the gradient amplitude threshold value among all the pixel points of the preview image data as the edge pixel points.

[0120] Optionally, the peak focus data acquisition unit includes:

[0121] The attribute data acquisition subunit is used to acquire the attribute data of the preset edge pixel points sent by the camera application, wherein the attribute data of the edge pixel points includes: the target RGB value of the edge pixel points;

[0122] The RGB value adjustment subunit is used to adjust the RGB value of the edge pixel point to the target RGB value to obtain the image data after the peak focus.

[0123] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0124] In an exemplary embodiment, a camera is further provided, wherein a peak focus node is added to the CHI of Camx of the hardware abstraction layer of the camera, wherein:

[0125] When performing auxiliary focusing, the camera executes any of the auxiliary focusing methods described above.

[0126] The hardware abstraction layer HAL of the camera includes: ICameraProvider (camera provider interface), Camxcamera.qcom.so (Qualcomm CamX camera driver library) and CHI, wherein the Camx camera.qcom.so is connected to the ICameraProvider through the HAL interface, and the CHI is connected to the Camx camera.qcom.so through the CHI interface.

[0127] An image preview node and an IPE node are also provided in the CHI, wherein the peak focus node is connected to the IPE node and the image preview node respectively.

[0128] Please refer to Figure 7 , which shows a block diagram of a computer device provided in one embodiment of the present application. The computer device can be used to implement the auxiliary focusing method provided in the above embodiment. The computer device can be a PC or a server, or other devices with data processing and storage capabilities. Specifically:

[0129] The computer device 700 includes a central processing unit (CPU) 701, a system memory 704 including a random access memory (RAM) 702 and a read-only memory (ROM) 703, and a system bus 705 connecting the system memory 704 and the central processing unit 701. The computer device 700 also includes a basic input / output system (I / O system) 706 that helps to transfer information between various components in the computer, and a large-capacity storage device 707 for storing an operating system 713, application programs 714, and other program modules 715.

[0130] The basic input / output system 706 includes a display 708 for displaying information and an input device 709 such as a mouse and a keyboard for user inputting information. The display 708 and the input device 709 are connected to the central processing unit 701 through an input / output controller 710 connected to the system bus 705. The basic input / output system 706 may also include an input / output controller 710 for receiving and processing inputs from a plurality of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 710 also provides output to a display screen, a printer, or other types of output devices.

[0131] The mass storage device 707 is connected to the central processing unit 701 through a mass storage controller (not shown) connected to the system bus 705. The mass storage device 707 and its associated computer readable media provide non-volatile storage for the computer device 700. That is, the mass storage device 707 may include a computer readable medium (not shown) such as a hard disk or a CD-ROM drive.

[0132] Without loss of generality, the computer readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, CD-ROM, DVD or other optical storage, tape cassettes, magnetic tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media is not limited to the above. The above-mentioned system memory 704 and mass storage device 707 can be collectively referred to as memory.

[0133] According to various embodiments of the present application, the computer device 700 can also be connected to a remote computer on the network through a network such as the Internet. That is, the computer device 700 can be connected to the network 712 through the network interface unit 711 connected to the system bus 705, or the network interface unit 711 can be used to connect to other types of networks or remote computer systems (not shown).

[0134] The memory further includes one or more programs, which are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for executing the auxiliary focusing method.

[0135] In an exemplary embodiment, a computer device is also provided, the computer device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, at least one program, code set or instruction set is configured to be executed by one or more processors to implement the above-mentioned auxiliary focusing method.

[0136] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set implements the above-mentioned auxiliary focusing method when executed by a processor of a computer device.

[0137] Optionally, the computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0138] In an exemplary embodiment, a computer program product is also provided. When the computer program product is executed, it is used to implement the above auxiliary focusing method.

[0139] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0140] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An auxiliary focusing method, characterized in that: The method comprises: The acquired preview image data is passed to a pre-added peak focus node, where the peak focus node is a node pre-added in the CHI of Camx of the hardware abstraction layer; Calling the OpenCL library integrated in the peak focus node to perform peak focus processing on the preview image data on the GPU to obtain image data after peak focus; The peak focused image data is sent to an image preview node to display the auxiliary focused image.

2. The method according to claim 1, characterized in that The calling of the OpenCL library integrated in the peak focus node to perform peak focus processing on the preview image data on the GPU to obtain image data after peak focus, includes: The OpenCL library is called to use GPU resources in a parallel processing manner to perform edge detection and RGB value adjustment processing on the preview image data to obtain the peak focused image data.

3. The method according to claim 2, characterized in that The performing edge detection and RGB value adjustment processing on the preview image data to obtain the peak focused image data includes: Calling the OpenCL library to use the GPU resources to calculate the gradient amplitude of each pixel in the preview image data; Obtaining edge pixel points in the preview image data according to the gradient amplitude and a preset gradient amplitude threshold; The RGB values ​​of the edge pixels are adjusted to obtain the peak focused image data.

4. The method according to claim 3, characterized in that The calling the OpenCL library to use the GPU resources to calculate the gradient amplitude of each pixel in the preview image data includes: Calling the OpenCL library to use the GPU resources to calculate the horizontal grayscale value and the vertical grayscale value of each pixel in the preview image data; Based on the horizontal grayscale value and vertical grayscale value of each pixel, the gradient amplitude of each pixel is calculated.

5. The method according to claim 3, characterized in that: The step of obtaining edge pixel points in the preview image data according to the gradient amplitude and a preset gradient amplitude threshold comprises: Pixels whose gradient magnitudes are greater than the gradient magnitude threshold value among all the pixels of the preview image data are taken as the edge pixels.

6. The method according to claim 3, characterized in that The step of adjusting the RGB value of the edge pixel point to obtain the peak focused image data includes: Acquire the preset attribute data of the edge pixel sent by the camera application, wherein the attribute data of the edge pixel includes: the target RGB value of the edge pixel; The RGB value of the edge pixel is adjusted to the target RGB value to obtain the peak focused image data.

7. An auxiliary focusing device, characterized in that: The device comprises: An image data transmission module, used to transmit the acquired preview image data to a pre-added peak focus node, wherein the peak focus node is a node pre-added in the CHI of Camx of the hardware abstraction layer; A peak focus data acquisition module, used for calling the OpenCL library integrated in the peak focus node to perform peak focus processing on the preview image data on the GPU to obtain image data after peak focus; The focused image preview module is used to send the peak focused image data to the image preview node for peak focused image preview.

8. A camera, characterized in that: A peak focus node is added to the CHI of Camx of the hardware abstraction layer of the camera, wherein: When performing auxiliary focusing, the camera executes the method according to any one of claims 1 to 6.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method as described in any one of claims 1 to 6.

Images