Image processing method and system and electronic equipment

Abstract

The invention provides an image processing method and system and electronic equipment, and relates to the technical field of image processing. The method comprises the specific steps that an application layer initiates a calling request to a user-defined providing program layer through a camera service layer; the user-defined providing program layer responds to a calling request to generate an image request parameter corresponding to the calling request, and sends the image request parameter to the hardware abstraction layer; the hardware abstraction layer obtains image data from data collected by a camera based on image request parameters and sends the image data to the user-defined program providing layer; and the user-defined providing program layer processes the image data according to a preset algorithm, and transmits the processed image data to an application program. According to the method, the user-defined function of the specific equipment is realized through the user-defined providing program layer, and the situation that the realization of the user-defined function is excessively coupled with an application or a chip and cannot be separated due to deep binding is avoided, so that the flexibility of image processing function development is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, system and electronic device. Background Art

[0002] With the rapid advancement of technology and living standards, electronic devices (such as mobile phones, tablet computers, etc.) have become indispensable electronic products in people's lives. In addition, most electronic devices currently have a camera function, and users can use the electronic devices they carry with them to record various contents in real scenes at any time. When taking photos, electronic devices usually expand their functions based on the software architecture of image processing to achieve functions such as image preview, photo taking, and video recording.

[0003] However, the existing framework does not provide a standardized solution, resulting in the implementation of custom functions scattered in the application layer or hardware abstraction layer, deep coupling with the application or chip vendors, unclear functional modules, and poor development and expansion flexibility, resulting in poor versatility of related software architectures for shooting. Summary of the invention

[0004] The present disclosure provides an image processing method, device and system to at least solve the problem that the implementation of custom functions in the related art is too coupled with the application or chip. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0006] The application layer initiates a call request to the custom provider layer through the camera service layer;

[0007] The custom provider layer generates, in response to the call request, an image request parameter corresponding to the call request, and sends the image request parameter to the hardware abstraction layer;

[0008] The hardware abstraction layer obtains image data based on the image request parameters and sends the image data to the custom provider layer;

[0009] The custom provider layer processes the image data according to a preset algorithm and transmits the processed image data to the application layer.

[0010] Optionally, the custom provider layer includes at least one of the following: a hardware interface definition language HIDL service module, a platform control module, a private software algorithm framework module, and a platform adaptation module, and wherein the custom provider layer generates an image request parameter corresponding to the call request in response to the call request, specifically comprising:

[0011] The HIDL service module of the custom provider layer receives the call request and parses the data information in the call request to obtain parsed data;

[0012] The HIDL service module of the custom provider layer maps the image buffer space to the local space, and sends the parsed data and the buffer handle corresponding to the local space to the platform control module.

[0013] Optionally, the step of generating, in response to the calling request, an image request parameter corresponding to the calling request by the custom provider layer further comprises:

[0014] The platform control module of the custom provider layer reconstructs the call request according to the parsed data to determine the original image data and the processing algorithm corresponding to the image processing;

[0015] The platform control module of the custom provider layer constructs a raw image data request according to the raw image data and the processing algorithm, and sends the raw image data request to the platform adaptation module of the custom provider layer.

[0016] Optionally, the step of generating, in response to the calling request, an image request parameter corresponding to the calling request by the custom provider layer further comprises:

[0017] The platform adaptation module of the custom provider layer performs format conversion on the original image data request according to the target platform, and converts the original image data request into a first target format;

[0018] The platform adaptation module of the custom provider layer sends the original image data request to the hardware abstraction layer, wherein the original image data request includes the image request parameters, and the target platform is the hardware abstraction layer driven by the current chip.

[0019] Optionally, the custom provider layer processes the image data according to a preset algorithm, including:

[0020] The platform adaptation module performs format conversion on the image data according to the target platform, and converts the image data into a second target format, wherein the second target format is a data format recognizable by the custom provider layer;

[0021] The platform adaptation module sends the image data in the second target format to the private software algorithm framework module.

[0022] Optionally, the custom provider layer processes the image data according to a preset algorithm, including:

[0023] The private software algorithm framework module of the custom provider layer calls the corresponding target algorithm according to the processing algorithm, and processes the image data according to the target algorithm to obtain the processed image data;

[0024] The private software algorithm framework module of the custom provider layer fills the processed image data into the image buffer space in the local space.

[0025] Optionally, transmitting the processed image data to the application layer includes:

[0026] The application program in the application layer obtains the processed image data from the local space and displays it to the user.

[0027] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing system, comprising: an application layer, a custom provider layer, and a hardware abstraction layer;

[0028] The application layer is used to initiate a call request to the custom provider layer through the camera service layer;

[0029] The custom provider layer is used to generate image request parameters corresponding to the call request in response to the call request, and send the image request parameters to the hardware abstraction layer;

[0030] The hardware abstraction layer is used to obtain image data based on the image request parameters and send the image data to the custom provider layer;

[0031] The custom provider layer is further used to process the image data according to a preset algorithm and transmit the processed image data to the application layer.

[0032] Optionally, the custom provider layer includes: a hardware interface definition language HIDL service module, a platform control module, a private software algorithm framework module and a platform adaptation module.

[0033] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0034] processor;

[0035] a memory for storing instructions executable by the processor;

[0036] The processor is configured to execute the instructions to implement the method as described in any one of the first aspects.

[0037] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute a method as described in any one of the first aspects.

[0038] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0039] Implementing custom functions for specific devices through a custom provider layer avoids over-coupling of custom functions with applications or chips, thus improving the flexibility of image processing function development.

[0040] The custom provider layer enables smart device manufacturers to implement custom functions for specific devices, expand functions on the original architecture, and provide a standardized solution to facilitate the development and evolution of custom functions for smart device manufacturers.

[0041] The custom provider layer can meet the diverse needs of smart device manufacturers, make technical solutions cross-platform, and improve the efficiency of massive product development.

[0042] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0044] Figure 1 The invention is a block diagram of an image processing system according to an exemplary embodiment.

[0045] Figure 2 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0046] Figure 3 The invention is a block diagram of an image processing system according to an exemplary embodiment.

[0047] Figure 4 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0048] Figure 5 It is a block diagram of a device according to an exemplary embodiment.

[0049] Figure 6 It is a block diagram of a device according to an exemplary embodiment. DETAILED DESCRIPTION

[0050] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0051] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0052] It should be noted that the user information involved in this disclosure (including but not limited to user device information, user personal information, etc.) is all information authorized by the user or fully authorized by all parties.

[0053] Figure 1 FIG. 1 is a block diagram of an image processing system according to an exemplary embodiment. Figure 1 As shown in the figure, the camera architecture from top to bottom is the application layer (Camera Application), camera application framework (Camera Framework), camera service layer (Camera Service), hardware abstraction layer (Platform Hardware Abstraction Layer, Platform HAL), camera chip driver layer (Camera Driver), and hardware chip layer (Camera Hardware).

[0054] The application layer can be provided with applications for taking photos (such as system photo application, third-party photo application, etc.); the hardware abstraction layer is an interface layer located between the operating system kernel and the hardware circuit, and its purpose is to abstract the hardware. The hardware abstraction layer is provided with a hardware abstraction module. Taking the Android system as an example, the hardware abstraction module is an Android native module, such as the native camera hardware abstraction module Camera HAL, the media strategy module and the algorithm management module. In addition, the operating system also includes a framework layer (Framework), a driver layer, etc. The framework layer includes application interfaces of various native applications (such as native camera application program interface), application services (such as native camera services), and framework layer interfaces (such as Google HAL3interface). The driver layer includes various drivers (such as screen Display driver, audio Audio driver, etc.), which are used to enable various hardware of mobile terminals, such as image signal processor ISP + front-end image sensor sensors, etc. The overall architecture adopts a modular layered design, which separates the definition and implementation of the interface, allowing developers and suppliers to pay more attention to the implementation of functions, reduce the coupling degree of the system, and make the system run more stable.

[0055] When implementing custom functions for specific devices based on the Android camera architecture (such as beauty processing, filter processing, rotation processing, watermark processing, blur processing, high dynamic range processing, multi-frame processing and other post-processing algorithms), smart device manufacturers need to expand functions. However, the existing framework does not provide a standardized solution for smart device manufacturers, resulting in the implementation of custom functions scattered in the application layer or hardware abstraction layer, deeply coupled with the software implementation of applications or chip suppliers, and unclear functional modules, which is contrary to Google's initial modular and layered design ideas. At the same time, it is not convenient for smart device manufacturers to separate their own functions, and the flexibility of development and expansion is poor.

[0056] In the system, the application layer is at the top of the architecture and interacts directly with the user to complete specific needs such as recording, taking pictures, and previewing. The standard interface Camera Api v2 is used to standardize the data interaction between various camera applications and the camera application framework. The camera application framework and camera services are uniformly implemented by the Android system, providing Android core components and services, such as the graphics library OpenGL ES for 2D and 3D objects, which can be used through Java's OpenGL API. The hardware abstraction layer and camera chip driver layer are implemented by chip suppliers to load hardware components for the camera system.

[0057] Figure 2 FIG. 1 is a flowchart of an image processing method according to an exemplary embodiment. Figure 2 As shown, the method includes:

[0058] Step 101, the application layer (Activity / Service) initiates a call request to the custom provider layer (Custom Provider) through the camera service layer (Camera Service).

[0059] In the embodiment of the present application, when the application in the application layer needs to take an image, it can initiate a call request to the custom provider layer through the application programming interface API interface to the custom provider layer to complete the required image capture. Among them, the custom provider layer can be extracted from the relevant software logic, image processing algorithm, configuration file, etc. that depend on the application framework Framework and the hardware abstraction layer HAL, and wherein the custom program provider layer can be an implementation layer built based on the software development kit (Software Development Kit, SDK). In other words, the custom provider layer can be understood as a plug-in based on SDK, so that the application in the application layer only needs to call the plug-in based on SDK to achieve the image capture it needs. In the related art, the implementation of custom functions is scattered in the application layer or the hardware abstraction layer, deeply coupled with the software implementation of the application or chip supplier, and the functional module is not clear. Through the solution in the embodiment of the present application, the custom function can be separated to avoid the implementation of the custom function from being too coupled with the application or chip, thereby improving the flexibility of image processing function development.

[0060] Step 102, the custom provider layer generates image request parameters corresponding to the call request in response to the call request, and sends the image request parameters to the hardware abstraction layer (HAL);

[0061] After receiving a call request sent by the application layer (e.g., an application APP in the application layer), the custom provider layer can generate image request parameters corresponding to the call request in response to the call request of the APP. The image request parameters are used to indicate that the hardware abstraction layer needs to capture specified data from the image data collected by the camera, as well as control the shooting parameters of the camera device; the specified data is the original data that has not been processed by any algorithm by the hardware abstraction layer, and the specified data is used to subsequently generate the image required for the call request, such as a preview image, a photo image, a video screen, etc. Among them, the call request may carry the switch control of the camera mode and the image processing function, and the camera mode may include photo mode, video mode, night scene mode, portrait mode, slow motion, time-lapse photography, panorama and other modes, which are not limited here; the image processing function may include beauty, filter, blur, request for preview data, request for photo, etc., which are not limited here.

[0062] Step 103, the hardware abstraction layer obtains image data based on the image request parameters, and sends the image data to the custom provider layer;

[0063] In an embodiment of the present application, after the custom provider layer generates the image request parameters corresponding to the call request, the image request parameters can be sent to the hardware abstraction layer. The hardware abstraction layer can configure the camera device (camera) based on the received image request parameters, and / or capture the image data corresponding to the image request parameters from the image data collected by the camera device. Optionally, the hardware abstraction layer can be Camera HAL or PlatformHAL.

[0064] In some embodiments, the image request parameters may include: image request parameters corresponding to the preview stage or the shooting stage. It can be understood that if the call request is a request for the preview stage, the image request parameters should include parameters for obtaining the original image data corresponding to the preview data; if the call request is a request for the shooting stage (such as a trigger request for taking photos or recording videos, etc.), the image request parameters should include parameters for obtaining the original image data for generating the captured image. Thus, the hardware abstraction layer can obtain the unprocessed original image data from the data collected by the camera device based on the image request parameters sent by the custom provider layer at different stages of the shooting process.

[0065] In addition, since the camera device needs to be configured differently under different shooting requirements when shooting images, the image request parameters may also include configuration parameters for configuring the camera device, which are used by the hardware abstraction layer to configure the camera device so that the hardware abstraction layer can capture the required original image data from the image data collected by the camera device.

[0066] Step 104: the custom provider layer processes the image data according to a preset algorithm and transmits the processed image data to the application.

[0067] In the embodiment of the present application, after receiving the image data transmitted by the hardware abstraction layer, the custom provider layer can process the image data by a preset algorithm, thereby obtaining image data for transmission to the application, and transmit the image data obtained after the processing by the preset algorithm to the application layer. After receiving the image data sent by the hardware abstraction layer, the application layer can generate a preview image for display, or generate a captured image for display and storage based on the image data processed by the preset algorithm by the custom provider layer.

[0068] In some embodiments, the preset algorithm processing may include preview processing, post-processing, etc. of the received image data, wherein the preview processing is used to generate image data for screen display by a subsequent application layer; the post-processing is used to perform preview processing or algorithm processing in the process of generating captured images, such as beauty processing, filter processing, rotation processing, watermark processing, blur processing, high dynamic range processing, multi-frame processing, and other image processing.

[0069] The image processing method provided in the embodiment of the present application, since in the image processing process of realizing the shooting process, the relevant processing logic and image processing algorithm in the entire processing process are processed by the custom provider layer, and the hardware abstraction layer only needs to control the basic camera equipment and capture the required original image data, the image processing process does not have to rely on adding processing logic and processing algorithms in the hardware abstraction layer, so that the application layer only needs to call the software development kit to realize image shooting, and when the relevant algorithms for image shooting are transplanted, they only need to be transplanted to the software development kit, so that it has good versatility.

[0070] Optionally, the custom provider layer includes: a hardware interface definition language HIDL service module, a platform control module, a private software algorithm framework module and a platform adaptation module.

[0071] In this embodiment, a custom provider layer (Custom Provider) is added to the original architecture, and the customized content of the smart device manufacturer is integrated into the custom provider layer, and the hardware components and software algorithm control logic of the smart device manufacturer are independently implemented, and the private algorithm is uniformly managed and used for image processing. In the overall architecture, the custom provider layer implements the Android standard Hidl interface to communicate with the camera service layer, and adapts to the hardware abstraction layer interface provided by the chip manufacturer to realize the control of the chip supplier's software process. The custom provider layer can be divided into four parts: HIDL service module (Hidl Interface Impl), platform control module (Platform Control), private software algorithm framework (AlgoFramework) and platform adapter module (Platform Adapter). The HIDL service module is responsible for implementing the process communication service interface, establishing communication with the camera service layer, and maintaining the resources obtained from the camera service layer internally. The chip platform control module is responsible for implementing a customizable data flow construction strategy and reasonably utilizing the hardware module capabilities to obtain the original image information. The private software algorithm framework facilitates the integration of private algorithms of smart device manufacturers, realizes the management of private algorithms of smart device manufacturers, and provides a unique image processing strategy. The chip platform adapter module is responsible for data conversion. Considering the differences in chip platform software implementation, the adapter module needs to have good scalability.

[0072] The above architecture does not affect the original data processing flow of the application layer, camera application framework, and camera service layer. The difference is that after the data requests such as preview, photo taking, and video recording are sent to the custom provider layer, they will be modified differentially so that the image finally presented to the user achieves the desired effect.

[0073] Optionally, the step of generating, by the custom provider layer in response to the call request, an image request parameter corresponding to the call request specifically includes:

[0074] The HIDL service module receives the call request and parses the data information in the call request to obtain parsed data;

[0075] The image buffer space is mapped to the local space, and the parsed data and the buffer handle corresponding to the local space are sent to the platform control module.

[0076] In this embodiment, in response to a call request from the camera service layer, the HIDL service module parses the data information in the call request, and the obtained parsed data is used to indicate that the hardware abstraction layer needs to capture specified data from the image data collected by the camera, as well as to control the shooting parameters of the camera device; the specified data is the original data that has not been processed by any algorithm by the hardware abstraction layer, and the specified data is used to subsequently generate the image required for the call request.

[0077] At the same time, a certain amount of space needs to be reserved in the local storage space as an image buffer space to store the image data. The device driver maps the image buffer space to the local storage space and generates a buffer handle corresponding to the local space, which is used to identify the local storage space allocated for the image buffer space.

[0078] Optionally, the step of generating, in response to the calling request, an image request parameter corresponding to the calling request by the custom provider layer further comprises:

[0079] The platform control module reconstructs the call request according to the parsed data to determine the original image data and processing algorithm corresponding to the image processing;

[0080] An original image data request is constructed according to the original image data and the processing algorithm, and the original image data request is sent to the platform adaptation module.

[0081] Raw image data means original and unprocessed, which represents a pile of pure unprocessed data. The Raw image records the original data of the CMOS or CCD image sensor converting the captured light source (photon) signal into a digital signal.

[0082] In a possible embodiment, the processing algorithm is used to process image data of the camera application in preview, photo taking, and video recording scenarios, and the processing methods include but are not limited to: beauty processing, filter processing, rotation processing, watermark processing, blur processing, high dynamic range processing, multi-frame processing and other image processing.

[0083] Optionally, the step of generating, in response to the calling request, an image request parameter corresponding to the calling request by the custom provider layer further comprises:

[0084] The platform adaptation module performs format conversion on the original image data request according to the target platform, converts the original image data request into a first target format, and sends the original image data request to the hardware abstraction layer, wherein the original image data request includes the image request parameters, and the target platform is the hardware abstraction layer driven by the current chip.

[0085] In this embodiment, the platform adaptation module adapts the original image data request and converts the data format according to the current platform requirements.

[0086] Optionally, the custom provider layer processes the image data according to a preset algorithm, including:

[0087] The platform adaptation module performs format conversion on the image data according to the target platform, and converts the image data into a second target format, wherein the second target format is a data format recognizable by the custom provider layer;

[0088] The platform adaptation module sends the image data in the second target format to the private software algorithm framework module.

[0089] In this embodiment, after the hardware abstraction layer processes and returns the image data, the platform adaptation module also needs to perform data format conversion to convert the image data into data recognizable by the custom provider layer.

[0090] Optionally, the custom provider layer processes the image data according to a preset algorithm, including:

[0091] The private software algorithm framework module calls the corresponding target algorithm according to the processing algorithm, and processes the image data according to the target algorithm to obtain the processed image data;

[0092] The private software algorithm framework module fills the processed image data into the image buffer space in the local space.

[0093] In this embodiment, the private software algorithm framework facilitates the integration of private algorithms of smart device manufacturers, implements the management of private algorithms of smart device manufacturers, and provides unique image processing strategies.

[0094] Optionally, transmitting the processed image data to the application layer includes:

[0095] The application layer obtains the processed image data from the local space and displays it to the user.

[0096] Figure 3 FIG. 1 is a block diagram of an image processing system according to an exemplary embodiment. Figure 3 As shown, the system includes: an application layer 310, a camera application framework 320, a camera service layer 330, a custom provider layer 340, a hardware abstraction layer 350, a camera chip driver layer 360, and a hardware chip layer 370.

[0097] In this embodiment, Figure 1 A custom provider layer 340 is added to the system architecture. The custom provider layer 340 integrates the customized content of the smart device manufacturer, independently implements the hardware components and software algorithm control logic of the smart device manufacturer, and uniformly manages and uses private algorithms for image processing. In the overall architecture, the custom provider layer implements the Android standard Hidl interface to communicate with the camera service layer, and adapts to the hardware abstraction layer interface provided by the chip manufacturer to control the chip supplier's software process. The custom provider layer can be divided into four parts: HIDL service module (Hidl Interface Impl) 341, platform control module (Platform Control) 342, private software algorithm framework (AlgoFramework) 343 and platform adapter module (Platform Adapter) 344. The HIDL service module is responsible for implementing the process communication service interface, establishing communication with the camera service layer, and maintaining the resources obtained from the camera service layer internally. The chip platform control module is responsible for implementing a customizable data flow construction strategy and reasonably utilizing the hardware module capabilities to obtain original image information. The private software algorithm framework facilitates the integration of private algorithms of smart device manufacturers, realizes the management of private algorithms of smart device manufacturers, and provides unique image processing strategies. The chip platform adapter module is responsible for data conversion. Considering the differences in chip platform software implementation, the adapter module needs to have good scalability.

[0098] The above architecture does not affect the original data processing flow of the application layer 310, the camera application framework 320, and the camera service layer 330. The difference is that after the data requests such as preview, photo taking, and video recording are sent to the custom provider layer, they will be modified differentially so that the image finally presented to the user achieves the desired effect.

[0099] Figure 4 FIG. 1 is a flowchart of an image processing method according to an exemplary embodiment. Figure 4 As shown, Figure 3 The data processing flow of the system is as follows.

[0100] Step 401, the underlying data stream request is sent from the camera service layer (Camera Service) to the custom provider layer through the HIDL interface;

[0101] Step 402, the HIDL service module in the custom provider layer parses the data information sent by the interface, maps the image buffer space to the local space, and then sends the parsed data and the local buffer handle to the platform control module;

[0102] Step 403: the platform control module reconstructs the data request according to the data information, specifies the basic original Raw image and processing algorithm required for image processing, and then constructs the Raw data request.

[0103] Step 404: the platform adaptation module adapts the Raw data request and converts the data format according to the current platform requirements.

[0104] Step 405: After the hardware abstraction layer obtains the original image data and returns it, the platform adaptation module also needs to convert the data format into data that can be recognized by the custom provider layer.

[0105] Step 406, after the platform control module obtains the Raw image, the required algorithm information and the Raw image are sent to the private algorithm framework, and the processing result of the software algorithm module is filled in the buffer sent down from the top layer.

[0106] Step 407: The processing result is returned, and the application layer obtains the processing result through the shared memory and presents it to the user.

[0107] Figure 5 8 is a block diagram of an apparatus 800 according to an exemplary embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0108] Reference Figure 5, the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0109] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0110] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0111] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0112] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0113] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0114] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0115] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0116] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0117] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0118] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the device 800 to complete the above method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0119] Figure 6 900 is a block diagram of a device 900 according to an exemplary embodiment. For example, the device 900 may be provided as a server. Figure 6 , the apparatus 900 includes a processing component 922, which further includes one or more processors, and a memory resource represented by a memory 932 for storing instructions, such as an application, that can be executed by the processing component 922. The application stored in the memory 932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 922 is configured to execute the instructions to perform the above method.

[0120] The device 900 may also include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 may operate based on an operating system stored in the memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0121] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0122] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: The application layer initiates a call request to the custom provider layer through the camera service layer; The custom provider layer generates, in response to the call request, an image request parameter corresponding to the call request, and sends the image request parameter to the hardware abstraction layer; The hardware abstraction layer obtains image data based on the image request parameters and sends the image data to the custom provider layer; The custom provider layer processes the image data according to a preset algorithm and transmits the processed image data to the application layer.

2. The method according to claim 1, characterized in that The custom provider layer includes at least one of the following: a hardware interface definition language HIDL service module, a platform control module, a private software algorithm framework module, and a platform adaptation module, and wherein the custom provider layer generates an image request parameter corresponding to the call request in response to the call request, specifically comprising: The HIDL service module of the custom provider layer receives the call request and parses the data information in the call request to obtain parsed data; The HIDL service module of the custom provider layer maps the image buffer space to the local space, and sends the parsed data and the buffer handle corresponding to the local space to the platform control module.

3. The method according to claim 2, characterized in that The step of generating, by the custom provider layer, image request parameters corresponding to the call request in response to the call request further includes: The platform control module of the custom provider layer reconstructs the call request according to the parsed data to determine the original image data and the processing algorithm corresponding to the image processing; The platform control module of the custom provider layer constructs a raw image data request according to the raw image data and the processing algorithm, and sends the raw image data request to the platform adaptation module of the custom provider layer.

4. The method according to claim 3, characterized in that The step of generating, by the custom provider layer, image request parameters corresponding to the call request in response to the call request further includes: The platform adaptation module of the custom provider layer performs format conversion on the original image data request according to the target platform, and converts the original image data request into a first target format; The platform adaptation module of the custom provider layer sends the original image data request to the hardware abstraction layer, wherein the original image data request includes the image request parameters, and the target platform is the hardware abstraction layer driven by the current chip.

5. The method according to claim 4, characterized in that The custom provider layer processes the image data according to a preset algorithm, including: The platform adaptation module performs format conversion on the image data according to the target platform, and converts the image data into a second target format, wherein the second target format is a data format recognizable by the custom provider layer; The platform adaptation module sends the image data in the second target format to the private software algorithm framework module.

6. The method according to claim 5, characterized in that The custom provider layer processes the image data according to a preset algorithm, including: The private software algorithm framework module of the custom provider layer calls the corresponding target algorithm according to the processing algorithm, and processes the image data according to the target algorithm to obtain the processed image data; The private software algorithm framework module of the custom provider layer fills the processed image data into the image buffer space in the local space.

7. The method according to claim 6, characterized in that The step of transmitting the processed image data to the application layer comprises: The application program in the application layer obtains the processed image data from the local space and displays it to the user.

8. An image processing system, characterized in that: include: Application layer, custom provider layer, hardware abstraction layer; The application layer is used to initiate a call request to the custom provider layer through the camera service layer; The custom provider layer is used to generate image request parameters corresponding to the call request in response to the call request, and send the image request parameters to the hardware abstraction layer; The hardware abstraction layer is used to obtain image data based on the image request parameters and send the image data to the custom provider layer; The custom provider layer is further used to process the image data according to a preset algorithm and transmit the processed image data to the application layer.

9. The system according to claim 8, characterized in that: The custom provider layer includes: a hardware interface definition language HIDL service module, a platform control module, a private software algorithm framework module and a platform adaptation module.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 7. 11 . A computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 7.

Images