Image processing method, system architecture, device, medium and program product

Abstract

The present disclosure relates to an image processing method, system architecture, device, medium and program product. The image processing method comprises: receiving abstract layer data storage information sent by a first abstract layer; converting the abstract layer data storage information into general data storage information having a preset data structure; and controlling the first abstract layer to execute at least one hardware processing process based on the general data storage information. By converting the abstract layer data storage information into general data storage information, the general data storage information having a unified preset data structure can be used as a basis for controlling the first abstract layer to execute the hardware processing process, a general data transmission path is provided for different chips, the coupling with the chips is reduced, the data processing process is simplified, the system development efficiency and maintenance efficiency are improved, and the generality of the system architecture is ensured.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, specifically to an image processing method, system architecture, device, medium, and program product. Background Technology

[0002] In recent years, as developers and users have increasingly focused on the diversification of electronic device functions, how to achieve functional expansion through modular layering has gradually become a key issue in system architecture design. After developing and expanding custom functions, it is still necessary to use some data from the chip to perform corresponding hardware processing to ensure the integrity and accuracy of data processing functions.

[0003] However, the use of related technologies for image processing is limited by the fact that image processing data is privately dedicated to specific chips. Different chips need to be configured with different transmission paths to achieve data transmission, which leads to problems such as complex image processing, low development efficiency and low maintenance efficiency, which is not conducive to the functional expansion and versatility of electronic devices. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an image processing method, system architecture, device, medium, and program product.

[0005] According to a first aspect of the present disclosure, an image processing method is provided, the image processing method comprising:

[0006] The abstract layer data storage information sent by the first abstract layer includes the storage address of image processing data generated by the first abstract layer in response to a preset image processing request. The image processing data is used by the first abstract layer to perform at least one hardware processing procedure corresponding to the preset image processing request.

[0007] The abstract layer data storage information is converted into general data storage information with a preset data structure;

[0008] Based on the general data storage information, the first abstraction layer is controlled to execute the at least one hardware processing procedure.

[0009] In some embodiments of this disclosure, the abstract layer data storage information includes storage addresses for multiple image processing data, and the step of converting the abstract layer data storage information into general data storage information includes:

[0010] A storage address data set is generated based on the storage addresses of multiple image processing data sets.

[0011] Based on the preset data structure, the storage address data set is transformed to obtain the general data storage information.

[0012] In some embodiments of this disclosure, controlling the first abstraction layer to execute the at least one hardware processing procedure based on the general data storage information includes:

[0013] For each of the aforementioned hardware processing procedures, in response to the start of the processing thread corresponding to the hardware processing procedure, the general data storage information is converted into the abstract layer data storage information that the first abstract layer can recognize;

[0014] The abstraction layer data storage information is sent to the first abstraction layer so that the first abstraction layer can read the image processing data and execute the hardware processing procedure based on the image processing data.

[0015] In some embodiments of this disclosure, the image processing method further includes:

[0016] The system receives the processing result obtained by the first abstraction layer performing the hardware processing procedure. The processing result includes updated abstraction layer data storage information, which includes the storage address of the updated image processing data.

[0017] The updated abstract layer data storage information is converted into general data storage information with the preset data structure.

[0018] In some embodiments of this disclosure, the image processing method further includes:

[0019] In response to the preset image processing request, a data processing task corresponding to the preset image processing request is generated, the data processing task including at least one hardware processing procedure and at least one software processing procedure;

[0020] The software processing procedures described are executed directly.

[0021] In some embodiments of this disclosure, the preset image processing request is a photo-taking request, and the data processing task includes multiple image processing processes executed sequentially, wherein a portion of the image processing processes are hardware processing processes and another portion of the image processing processes are software processing processes;

[0022] The hardware processing procedure includes: controlling the first abstraction layer to perform hardware processing on the initial image data based on the image processing data to obtain the target image data;

[0023] The software processing procedure includes: processing the initial image data to obtain the target image data;

[0024] The initial image data used in the first image processing step is the original image data, and the initial image data used in the nth image processing step is the target image data obtained in the (n-1)th image processing step.

[0025] In some embodiments of this disclosure, in the frame-fetching mode, the original image data is preview cached image data;

[0026] In the post-frame mode, the original image data is the original format image data sent by the first abstraction layer.

[0027] In some embodiments of this disclosure, the image processing method further includes:

[0028] In response to the total number of data processing tasks to be executed being greater than or equal to a preset number threshold, the general data storage information corresponding to the data processing tasks to be executed is stored in the data storage module.

[0029] In response to the total number of data processing tasks to be executed being less than the preset number threshold, the general data storage information is read from the data storage module to execute the data processing task corresponding to the read general data storage information.

[0030] In some embodiments of this disclosure, the image processing method further includes:

[0031] The reference counts of the general data storage information and the abstract layer data storage information are performed respectively to obtain a first count result and a second count result;

[0032] Based on the first and second counting results, lifecycle management is performed on the image processing data.

[0033] In some embodiments of this disclosure, the reference counting of the general data storage information and the abstract layer data storage information includes:

[0034] In response to using the general data storage information, the first count value is increased by a first preset value;

[0035] In response to the completion of the use of the general data storage information, the first count value is reduced by the first preset value;

[0036] In response to using the data storage information of the abstract layer, the second count value is increased by a second preset value;

[0037] In response to the completion of the use of the data storage information in the abstract layer, the second count value is reduced by the second preset value.

[0038] In some embodiments of this disclosure, the lifecycle management of the image processing data based on the first counting result and the second counting result includes:

[0039] In response to the first counting result and the second counting result being the corresponding initial counting value, the image processing data is deleted.

[0040] In some embodiments of this disclosure, the image processing method further includes:

[0041] In response to the preset image processing request, a first storage space is allocated for the image processing data, and the preset image processing request is sent to the first abstraction layer, so that the first abstraction layer generates the image processing data and stores the image processing data in the first storage space; or,

[0042] In response to the preset image processing request, the preset image processing request is sent to the first abstraction layer, so that the first abstraction layer generates the image processing data and stores the image processing data in the second storage space allocated by the first abstraction layer.

[0043] In some embodiments of this disclosure, the image processing method further includes:

[0044] Generate a first queue corresponding to the first storage space. The first queue includes multiple first reference values, each first reference value corresponding to a different storage region in the first storage space. Each first reference value is used to characterize whether data exists in the corresponding storage region; and / or,

[0045] A second queue corresponding to the second storage space is generated. The second queue includes multiple second reference values. Each second reference value corresponds to a different storage area in the second storage space. Each second reference value is used to characterize whether data exists in the corresponding storage area.

[0046] According to a second aspect of the present disclosure, a system architecture is provided, the system architecture comprising:

[0047] A conversion layer is used to receive abstract layer data storage information sent by a first abstract layer and convert the abstract layer data storage information into general data storage information with a preset data structure. The abstract layer data storage information includes the storage address of image processing data generated by the first abstract layer in response to a preset image processing request. The image processing data is used by the first abstract layer to perform at least one hardware processing procedure corresponding to the preset image processing request.

[0048] The second abstraction layer is used to control the first abstraction layer to execute the at least one hardware processing procedure based on the general data storage information.

[0049] In some embodiments of this disclosure, for each of the hardware processing procedures, the second abstraction layer is used to send the general data storage information to the conversion layer in response to the start of the processing thread corresponding to the hardware processing procedure.

[0050] The conversion layer is used to convert the general data storage information into abstract layer data storage information that the first abstract layer can recognize, and send the abstract layer data storage information to the first abstract layer so that the first abstract layer can read the image processing data and execute the hardware processing process based on the image processing data.

[0051] In some embodiments of this disclosure, the conversion layer is further used for:

[0052] The system receives the processing result obtained by the first abstraction layer performing the hardware processing procedure. The processing result includes updated abstraction layer data storage information, and the updated abstraction layer data storage information includes the storage address of the updated image processing data.

[0053] The updated abstract layer data storage information is converted into general data storage information with the preset data structure and sent to the second abstract layer.

[0054] In some embodiments of this disclosure, the second abstraction layer is further used for:

[0055] In response to the preset image processing request, a data processing task corresponding to the preset image processing request is generated, the data processing task including at least one hardware processing procedure and at least one software processing procedure;

[0056] The software processing procedures described are executed directly.

[0057] In some embodiments of this disclosure, the system architecture further includes an application layer, which is used to send the preset image processing request to the second abstraction layer and receive the target data returned by the second abstraction layer.

[0058] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0059] processor;

[0060] Memory used to store processor-executable instructions;

[0061] The processor is configured to perform the image processing method as described in the first aspect.

[0062] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image processing method as described in the first aspect.

[0063] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the image processing method as described in the first aspect.

[0064] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: by receiving abstract layer data storage information sent by the first abstract layer, converting the abstract layer data storage information into general data storage information with a preset data structure, and then controlling the first abstract layer to execute hardware processing according to the general data storage information, the execution of hardware processing based on image processing data is realized. By converting the abstract layer data storage information into general data storage information, general data storage information with a unified preset data structure can be used as the basis for controlling the first abstract layer to execute hardware processing, providing a common data transmission path for different chips, reducing coupling with chips, simplifying the data processing process, improving system development and maintenance efficiency, and ensuring the universality of the system architecture.

[0065] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0066] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0067] Figure 1 This is a schematic diagram illustrating the interaction between the system architecture and the first abstraction layer according to an exemplary embodiment.

[0068] Figure 2 This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0069] Figure 3 This is a flowchart illustrating, according to an exemplary embodiment, the conversion of abstract layer data storage information into general data storage information.

[0070] Figure 4 This is a flowchart illustrating, according to an exemplary embodiment, a process for controlling a first abstraction layer to perform at least one hardware processing procedure based on general data storage information.

[0071] Figure 5 This is a flowchart illustrating an image processing method according to another exemplary embodiment.

[0072] Figure 6 This is a flowchart illustrating an image processing method according to another exemplary embodiment.

[0073] Figure 7 This is a flowchart illustrating an image processing method according to another exemplary embodiment.

[0074] Figure 8 This is a flowchart illustrating an image processing method according to another exemplary embodiment.

[0075] Figure 9 This is a flowchart illustrating, according to an exemplary embodiment, the reference counting of general data storage information and abstraction layer data storage information.

[0076] Figure 10 This is a flowchart illustrating an image processing method according to another exemplary embodiment.

[0077] Figure 11 This is a schematic diagram of a first queue according to an exemplary embodiment.

[0078] Figure 12 This is a schematic diagram of a second queue according to an exemplary embodiment.

[0079] Figure 13 This is a flowchart illustrating an image processing method according to another exemplary embodiment.

[0080] Figure 14 This is a block diagram of an electronic device according to an exemplary embodiment.

[0081] In the picture:

[0082] 10-Transformation layer; 20-Second abstraction layer; 30-First abstraction layer; 40-Application layer; 101-Processing component; 102-Memory; 103-Power component; 104-Multimedia component; 105-Audio component; 106-Input / output interface; 107-Sensor component; 108-Communication component; 109-Processor. Detailed Implementation

[0083] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0084] In recent years, as developers and users have paid increasing attention to the diversity of electronic device functions, separating the definition and implementation of interfaces through modular layering to achieve functional expansion and stable system operation has gradually become a key issue in electronic device system architecture design.

[0085] In related technologies, by providing a modular, layered structure for electronic devices, custom functions can be developed and extended using a developer-defined Hardware Abstraction Layer (HAL) within the system architecture. When processing data such as images, the HAL requires chip-specific image processing data to implement the hardware services provided by the chip, ensuring the integrity and accuracy of the data processing functions.

[0086] However, when using related technologies for data processing, due to the special nature of the data structure of image processing data, some image processing data is only applicable to the corresponding chip. Different chips need to be configured with different transmission paths to realize the transmission and use of image processing data. This results in a high degree of coupling between the abstraction layer in the system architecture and the chip, leading to problems such as complex image processing, low development and maintenance efficiency, and hindering the functional expansion and versatility of electronic devices.

[0087] Based on this, an exemplary embodiment of this disclosure provides an image processing method. This method receives abstract layer data storage information sent by a first abstract layer, converts this information into general data storage information with a preset data structure, and then controls the first abstract layer to execute hardware processing based on the general data storage information. This achieves the execution of hardware processing based on image processing data. By converting the abstract layer data storage information into general data storage information, the general data storage information with a unified preset data structure can be used as the basis for controlling the first abstract layer to execute hardware processing. This provides a common data transmission path for different chips, reduces coupling with chips, simplifies the data processing process, improves system development and maintenance efficiency, and ensures the universality of the system architecture.

[0088] In one exemplary embodiment, an image processing method is provided, applied to a system architecture and an electronic device configured with that system architecture, such as... Figure 1 As shown, the system architecture can interact with the first abstraction layer 30, and electronic devices such as mobile phones, cameras, and tablets can be included. (Reference) Figure 2 As shown, the image processing methods include:

[0089] S100: Receive abstract layer data storage information sent by the first abstract layer. The abstract layer data storage information includes the storage address of image processing data generated in response to a preset image processing request. The image processing data is used by the first abstract layer to execute at least one hardware processing procedure corresponding to the preset image processing request.

[0090] In step S100, as Figure 1 As shown, the first abstraction layer 30 may include, for example, the abstraction layer corresponding to a chip containing multiple processors and memories such as a central processing unit, a graphics processing unit, a digital signal processor, and dynamic random access memory. The first abstraction layer 30 can generate image processing data in response to a preset image processing request of the system architecture. The image processing data is at least partially exclusive to the corresponding first abstraction layer 30, i.e., data private to the corresponding first abstraction layer 30, and can serve as the basis for the first abstraction layer 30 to execute one or more hardware processing procedures corresponding to the preset image processing request. The first abstraction layers 30 included in chips from different manufacturers and models can generate different types of image processing data. For example, when the preset image processing request is a photo-taking request, the image processing data corresponding to different first abstraction layers 30 may include, for example, meta data and buffer handle data or meta data and 3A parameters. The hardware processing procedure is the image processing procedure implemented at the hardware level, i.e., it needs to utilize the hardware services provided by the chip's first abstraction layer 30.

[0091] like Figure 1 As shown, the system architecture includes a conversion layer 10 (adapter). When the first abstraction layer 30 generates image processing data in response to a preset image processing request, it sends abstraction layer data storage information, including the storage address of the image processing data, to the conversion layer 10. The storage address of the image processing data indicates its storage location in memory. The data structure of the abstraction layer data storage information also matches the corresponding first abstraction layer 30. The first abstraction layer 30 can read and use the corresponding image processing data based on the storage address of the image processing data included in its corresponding abstraction layer data storage information.

[0092] S200: Convert the abstract layer data storage information into general data storage information with a preset data structure.

[0093] In step S200, the conversion layer 10 converts the received abstract layer data storage information into general data storage information with a preset data structure. This unified data structure transforms the abstract layer data storage information, which can only be recognized by the corresponding first abstract layer 30, into general data storage information that can be recognized by the system architecture. For example... Figure 1As shown, the system architecture also includes a second abstraction layer 20. After obtaining the general data storage information, the conversion layer 10 sends the general data storage information to the second abstraction layer 20 of the system architecture.

[0094] S300, based on general data storage information, controls the first abstraction layer to execute at least one hardware processing procedure.

[0095] In step S300, since the general data storage information is converted from the abstract layer data storage information, and the abstract layer data storage information includes the storage address of image processing data used to execute hardware processing procedures, the system architecture can control the first abstract layer 30 to execute the corresponding hardware processing procedures based on the general data storage information. For example, the system architecture can generate data processing tasks through the second abstract layer 20 and send the general data storage information to the conversion layer 10. The conversion layer 10 then converts the general data storage information back into the abstract layer data storage information required by each hardware processing procedure in the data processing task and sends it to the first abstract layer 30, so that the first abstract layer 30 executes the corresponding hardware processing procedures.

[0096] Understandably, compared to existing technologies, the above-mentioned image processing method transforms the data storage information of the abstract layer through the transformation layer 10. Regardless of the system architecture and the type of chip used in conjunction with the first abstract layer 30, it can obtain general data storage information applicable to the system architecture. It is not necessary to configure different data transmission paths for the first abstract layer 30 of different chips when designing the system architecture to control the first abstract layer 30 to execute the corresponding hardware processing process. This makes the system architecture and the above-mentioned image processing method compatible with different first abstract layers 30, improving the versatility of the above-mentioned image processing method, the system architecture, and the system architecture configured with the system architecture.

[0097] In this embodiment, by receiving abstract layer data storage information sent by the first abstract layer 30 and converting it into general data storage information with a preset data structure, and then controlling the first abstract layer 30 to execute hardware processing based on the general data storage information, the execution of hardware processing based on image processing data is realized. By converting the abstract layer data storage information into general data storage information, the general data storage information with a unified preset data structure can be used as the basis for controlling the first abstract layer 30 to execute hardware processing, providing a common data transmission path for different chips, reducing coupling with chips, simplifying the data processing process, improving system development and maintenance efficiency, and ensuring the universality of the system architecture.

[0098] In some embodiments, the abstraction layer data storage information includes the storage addresses of multiple image processing data.

[0099] When the first abstraction layer 30 receives a preset image processing request, it can generate multiple image processing data simultaneously according to the preset image processing request. In order to ensure the integrity of the data storage information, the abstraction layer data storage information sent by the first abstraction layer 30 to the conversion layer 10 of the system architecture includes the storage addresses of multiple image processing data, so as to indicate the storage location of different image processing data through different storage addresses.

[0100] refer to Figure 3 As shown, the abstract layer data storage information is converted into general data storage information, including:

[0101] S210. Generate a set of storage address data based on the storage addresses of multiple image processing data.

[0102] In step S210, the conversion layer 10 can generate a storage address data set based on the storage addresses of multiple image processing data. This encapsulates and packages the multiple image processing data into an image processing data class using storage addresses, facilitating simultaneous conversion of the storage addresses of multiple image processing data. For example, the abstract layer data storage information received by the conversion layer 10 consists of the storage addresses of meta data and 3A parameters. The conversion layer 10 then packages the storage addresses of the meta data and 3A parameters into a single storage address data set, ensuring that the storage address data set includes both the storage addresses of the meta data and the 3A parameters.

[0103] S220. Based on the preset data structure, the storage address data set is transformed to obtain general data storage information.

[0104] In step S220, the data structure represents the composition structure of the information. The preset data structure of the general data storage information can reflect the storage addresses of the image processing data included in the general data storage information and the arrangement of the storage addresses of various image processing data. The conversion layer 10 can perform data structure conversion on the storage address data set including the storage addresses of multiple image processing data according to the preset data structure. By converting different image processing data classes into a unified base class through the form of storage addresses, the original dedicated data structure matching the corresponding first abstraction layer 30 of the storage address data set is converted into the preset data structure to obtain the general data storage information. This ensures that the general data storage information after conversion of different storage address data sets all have a unified preset data structure, facilitating the identification of the general data storage information by the second abstraction layer 20.

[0105] In this embodiment, when the abstract layer data storage information includes the storage addresses of multiple image processing data, a storage address data set is generated based on the storage addresses of the multiple image processing data. Then, according to a preset data structure, the storage address data set is transformed to obtain general data storage information. This realizes the generation of general data storage information and provides a basis for controlling the first abstract layer 30 to execute hardware processing procedures. By sequentially encapsulating and transforming the storage addresses of multiple image processing data, the storage addresses of multiple image processing data can be transformed simultaneously, ensuring the integrity of the image processing data storage addresses. This also ensures that the obtained general data storage information has a unified data structure, facilitating the identification of the general data storage information, reducing coupling with the chip, improving system development and maintenance efficiency, and ensuring the versatility of the system architecture.

[0106] In some embodiments, reference Figure 4 As shown, based on general data storage information, the first abstraction layer 30 is controlled to execute at least one hardware processing procedure, including:

[0107] S310. For each hardware processing procedure, in response to the start of the processing thread corresponding to the hardware processing procedure, the general data storage information is converted into abstract layer data storage information that can be recognized by the first abstract layer.

[0108] In step S310, as Figure 1 As shown, for each hardware processing procedure, when the processing thread corresponding to the hardware processing procedure is started in the second abstraction layer 20, the second abstraction layer 20 sends general data storage information and conversion request to the conversion layer 10, so that the conversion layer 10 converts the general data storage information into abstraction layer data storage information that the first abstraction layer 30 can recognize through its conversion function.

[0109] S320. The abstract layer data storage information is sent to the first abstract layer so that the first abstract layer can read the image processing data and execute the hardware processing procedure based on the image processing data.

[0110] In step S320, the conversion layer 10 sends the obtained abstract layer data storage information to the first abstract layer 30, so that the first abstract layer 30 reads the image processing data according to the storage address of the image processing data included in the abstract layer data storage information, and uses the image processing data to execute the corresponding hardware processing process. This enables the system architecture to control the first abstract layer 30 to execute the hardware processing process according to the general data storage information, thereby realizing the execution of the hardware processing process based on the image processing data.

[0111] In this embodiment, for each hardware processing procedure, when the processing thread corresponding to that hardware processing procedure is started in the second abstraction layer 20, the general data storage information is converted into abstraction layer data storage information that the first abstraction layer 30 can recognize, and the abstraction layer data storage information is sent to the first abstraction layer 30. This allows the first abstraction layer 30 to read the image processing data and execute the hardware processing procedure based on the image processing data, thus realizing the execution of the hardware processing procedure. When the processing thread corresponding to the hardware processing procedure is started, the general data storage information that the system architecture can recognize is converted back into abstraction layer data storage information that the first abstraction layer 30 can recognize. This provides a basis for the first abstraction layer 30 to read image processing data and execute the hardware processing procedure, reduces coupling with the chip, simplifies the data processing process, improves system development and maintenance efficiency, and ensures the versatility of the system architecture.

[0112] In some embodiments, reference Figure 5 As shown, the image processing method also includes:

[0113] S410. Receive the processing result obtained by the hardware processing procedure of the first abstraction layer. The processing result includes the updated abstraction layer data storage information, which includes the storage address of the updated image processing data.

[0114] In step S410, after the first abstraction layer 30 performs the corresponding hardware processing procedure based on the image processing data, the image processing data used and its corresponding storage address will change. The first abstraction layer 30 uses the updated storage address of the image processing data as the updated abstraction layer data storage information, and sends it together with the data after the hardware processing procedure as the processing result of this hardware processing procedure to the conversion layer 10, so that the conversion layer 10 can receive the updated storage address of the image processing data.

[0115] S420: Convert the updated abstract layer data storage information into general data storage information with a preset data structure.

[0116] In this embodiment, the conversion layer 10 converts the updated abstract layer data storage information into general data storage information with a preset data structure to update the general data storage information, and sends the general data storage information to the second abstract layer 20 so that the general data storage information matches the updated image processing data. The general data storage information subsequently identified and used by the second abstract layer 20 is the updated general data storage information, which ensures the timeliness of the general data storage information.

[0117] In this embodiment, by receiving the processing result obtained from the hardware processing process of the first abstract layer 30, and converting the updated abstract layer data storage information into general data storage information with a preset data structure, the abstract layer data storage information and general data storage information are updated after the hardware processing process is executed. This ensures the timeliness of the general data storage information, thereby ensuring the accuracy of the general data storage information as the basis for executing subsequent hardware processing processes, and improving the effect and continuity of data processing.

[0118] In some embodiments, reference Figure 6 As shown, the image processing method also includes:

[0119] S510. In response to a preset image processing request, a data processing task corresponding to the preset image processing request is generated. The data processing task includes at least one hardware processing procedure and at least one software processing procedure.

[0120] In step S510, as Figure 1 As shown, the system architecture also includes an application layer 40. When the second abstraction layer 20 receives a preset image processing request generated by the application layer 40, it generates a data processing task corresponding to the preset image processing request to respond to the preset image processing request. The data processing task can be run by a post-processing module in the second abstraction layer 20, and includes at least one hardware processing procedure and at least one software processing procedure. The hardware processing procedure requires hardware services provided by the first abstraction layer 30 of the chip, and needs to be controlled by the image processing method described above based on general data storage information to execute the hardware processing procedure. It is understood that when the second abstraction layer 20 receives the preset image processing request, it also needs to send the preset image processing request to the first abstraction layer 30 via the conversion layer 10 so that the first abstraction layer 30 generates image processing data according to the preset image processing request.

[0121] S520 directly executes the processing procedures of each software.

[0122] In step S520, for the software processing procedures in the data processing task, since they only need to be implemented through software algorithms, each software processing procedure can be directly executed through the second abstraction layer 20. Each software processing procedure can be executed independently of the first abstraction layer 30. By adding software processing procedures to the data processing task, the corresponding functions can be implemented through the execution of the software processing procedures, thereby realizing the extension of custom functions. When the preset image processing request is a photo-taking request, developers can configure the data processing task to implement the image processing functions built into the first abstraction layer 30 using hardware processing procedures, and to extend the custom image processing functions using software processing procedures.

[0123] In this embodiment, when a preset image processing request is detected, a data processing task corresponding to the preset image processing request is generated, and the software processing processes in the data processing task are directly executed. Based on the execution of the hardware processing process by controlling the first abstraction layer 30, the software processing process is executed through the system architecture itself. This can provide multiple image processing methods based on different layers, making it easier for developers to extend custom functions through the software processing process, improving system development and maintenance efficiency, and ensuring the versatility of the system architecture.

[0124] In some embodiments, the preset image processing request is a photo-taking request, and the data processing task includes multiple image processing procedures executed sequentially, wherein some image processing procedures are hardware processing procedures and others are software processing procedures. The hardware processing procedure includes controlling the first abstraction layer 30 to perform hardware processing on initial image data based on image processing data to obtain target image data. The software processing procedure includes performing software processing on the initial image data to obtain target image data. The initial image data used in the first image processing procedure is the original image data, and the initial image data used in the nth image processing procedure is the target image data obtained in the (n-1)th image processing procedure.

[0125] When the preset image processing request is a photo-taking request, the data processing task generated by the second abstraction layer 20 in response to the photo-taking request includes multiple image processing processes executed sequentially. These multiple image processing processes can be divided into hardware processing processes and software processing processes, so as to perform image processing based on the hardware level, i.e., the first abstraction layer 30, and based on the software level, i.e., the second abstraction layer 20, respectively.

[0126] For each hardware processing step, the initial image data passed to this hardware processing step is processed at the hardware level based on the image processing data to obtain the target image data after this hardware processing step. For each software processing step, the initial image data passed to this software processing step is processed at the software level directly to obtain the target image data after this software processing step. As mentioned above, when performing hardware processing on the initial image data based on the image processing data, the first abstraction layer 30 is controlled to execute the hardware processing step based on the general data storage information.

[0127] In the sequential execution of multiple image processing steps, the initial image data used in the first image processing step is the raw image data, i.e., RAW data, to process the image data acquired by the image sensor, which includes the raw information. The initial image data used in the nth image processing step is the target image data obtained from the (n-1)th, i.e., the previous image processing step, where n is a positive integer greater than 1, to process the image data after each of the previous image processing steps. This ensures that the processing effect of the later image processing steps is based on the earlier image processing steps, thereby guaranteeing the continuity of the entire data processing task and the correlation between each image processing step.

[0128] In this embodiment, when the preset image processing request is a photo-taking request, the data processing task includes multiple image processing processes executed sequentially. By controlling the first abstraction layer 30 to perform hardware processing on the initial image data based on the image processing data, and to perform software processing on the initial image data, the hardware processing and software processing processes in multiple image processing processes are implemented respectively, and the target image data corresponding to each image processing process is obtained. This provides a basis for the generation of the final target image and ensures the continuity of the entire data processing task and the correlation between each image processing process. This enables the system architecture to be used for image processing, improves the system development efficiency and maintenance efficiency, and ensures the versatility of the system architecture.

[0129] In some embodiments, in the forward frame mode, the original image data is preview cached image data. In the backward frame mode, the original image data is the original format image data sent by the first abstraction layer 30.

[0130] The pre-frame capture mode and post-frame capture mode are two shooting modes. In the pre-frame capture mode, also known as zero-delay shooting mode, the electronic device has already generated preview cached image data through its pre-recording and caching functions before the user presses the shutter button and the second abstraction layer 20 receives the shooting request. When performing image processing on the original image data in the pre-frame capture mode, the zero-delay shooting (ZSLQ) stored in the second abstraction layer 20 can be directly retrieved.

[0131] The preview cache image data in the module is used as the raw image data to perform the first image processing procedure.

[0132] In post-frame capture mode, raw image data is generated by the image sensor in the first abstract layer 30 only when the user presses the shutter button, i.e., when the second abstract layer 20 receives the photo request. When performing image processing on the raw image data in post-frame capture mode, it is necessary to receive the raw image data (RAW data) sent by the first abstract layer 30 and use it as the raw image data to perform the first image processing.

[0133] In this embodiment, in both the pre-frame fetching mode and the post-frame fetching mode, the preview cached image data and the original format image data sent by the first abstraction layer 30 are used as the original image data. The original image data is then used for the first image processing step, providing the raw material for the first image processing step. Directly retrieving the preview cached image data in the pre-frame fetching mode ensures the speed of acquiring the original image data, thereby improving the image processing and image generation speed, which is beneficial for achieving zero-latency photography in the pre-frame fetching mode.

[0134] In some embodiments, reference Figure 7 As shown, the image processing method also includes:

[0135] Step S610: In response to the total number of data processing tasks to be executed being greater than or equal to a preset number threshold, store the general data storage information corresponding to the data processing tasks to be executed into the data storage module.

[0136] In step S610, when the second abstraction layer 20 receives multiple preset image processing requests simultaneously, it can generate a corresponding number of data processing tasks and execute each data processing task sequentially according to the received order of the preset image processing requests. When the total number of data processing tasks to be executed is greater than or equal to a preset threshold, it indicates that there are many data processing tasks to be executed and the amount of general data storage information corresponding to all data processing tasks to be executed is large. In this case, it is necessary to store the general data storage information corresponding to the data processing tasks to be executed in the data storage module of the second abstraction layer 20 to retain the general data storage information that will be used later.

[0137] Step S620: In response to the total number of data processing tasks to be executed being less than a preset threshold, read general data storage information from the data storage module to execute the data processing task corresponding to the read general data storage information.

[0138] In step S620, the second abstract layer 20 executes data processing tasks in the execution order, so that the total number of data processing tasks to be executed gradually decreases. When the total number of data processing tasks to be executed is less than a preset threshold, the previously stored general data storage information is read from the data storage module to execute the corresponding data processing task according to the general data storage information.

[0139] In this embodiment, when the total number of data processing tasks to be executed is greater than or equal to a preset threshold, the general data storage information corresponding to the data processing tasks to be executed is stored in the data storage module. When the total number of data processing tasks to be executed is less than the preset threshold, the general data storage information is read from the data storage module to execute the data processing task corresponding to the read general data storage information. This realizes the retention and reading of general data storage information, reduces the data volume pressure of general data storage information, and provides a basis for the subsequent execution of data processing tasks to be executed, ensuring that each data processing task to be executed can be executed.

[0140] In some embodiments, reference Figure 8 As shown, the image processing method also includes:

[0141] S710. Perform reference counting on the general data storage information and the abstract layer data storage information respectively to obtain the first count result and the second count result.

[0142] In step S710, the system architecture can use smart pointers or additional reference counters to perform reference counting on the general data storage information and the data storage information of each abstract layer, so as to determine the first count result corresponding to the general data storage information and the second count result corresponding to the abstract layer data storage information in real time through a two-layer reference counting method. The first count result and the second count result respectively represent whether the second abstract layer 20 uses the general data storage information and whether the first abstract layer 30 uses the abstract layer data storage information.

[0143] S720 performs lifecycle management on image processing data based on the first and second counting results.

[0144] In step S720, the image processing data can be managed for lifecycle based on the first and second counting results determined in real time. That is, the first and second counting results are used to determine whether the image processing data still has value. Based on the determination results, the stored image processing data can be deleted or retained to ensure that the lifecycle and validity of each image processing data can match the actual needs.

[0145] Compared to related technologies that rely on copying image processing data to ensure its validity, the aforementioned image processing method reduces data copying, significantly improving power consumption optimization and dynamic storage management. For example, in photography scenarios, reducing image processing data copying through a dual-layer counting lifecycle management approach optimizes power consumption by 9-13mA in multi-camera preview scenarios, 17-20mA in rear-camera portrait preview scenarios, and 3-9mA in single-camera preview scenarios.

[0146] In this embodiment, by performing reference counting on the general data storage information and the abstract layer data storage information respectively, a first counting result and a second counting result are obtained. Based on the first counting result and the second counting result, lifecycle management of image processing data is performed. This allows for real-time assessment of the usage value of image processing data and selection of whether to retain or delete it, ensuring that the effectiveness of image processing data matches actual needs and improving the effectiveness of power consumption optimization and dynamic storage space management.

[0147] In some embodiments, reference Figure 9 As shown, reference counting is performed on both general data storage information and abstract layer data storage information, including:

[0148] S711. In response to using general data storage information, the first count value is increased by a first preset value.

[0149] S712. In response to the completion of general data storage information use, the first count value is reduced by a first preset value.

[0150] In steps S711 and S712, when the second abstract layer 20 uses general data storage information to control the first abstract layer 30 to perform hardware processing, the first count value is increased by a first preset value. When the use of general data storage information is completed, the first count value is decreased by a first preset value, so that the real-time first count value is used as the first count result.

[0151] For example, the initial first count value is 0, and the first preset value is 1. When general data storage information is used, such as when it is sent to the post-processing module, the first count value is incremented by 1. When the use of general data storage information is completed, such as when the data processing task corresponding to the general data storage information is fully executed, the first count value is decremented by 1. When the first count result changes from 0 to a non-zero value, it indicates that the general data storage information is being used by the second abstraction layer 20, and the image processing data still has value and storage requirements for the second abstraction layer 20. When the first count result changes from a non-zero value to 0, it indicates that the use of general data storage information is complete, and the image processing data no longer has value and storage requirements for the second abstraction layer 20.

[0152] S713. In response to using the abstract layer to store information, the second count value is increased by a second preset value.

[0153] S714. In response to the completion of the use of data storage information in the abstract layer, the second count value is reduced by a second preset value.

[0154] In steps S713 and S714, when the first abstract layer 30 uses the abstract layer data storage information to read image processing data and perform hardware processing, the second count value is increased by a second preset value. When the abstract layer data storage information is used up, the second count value is decreased by a second preset value, so that the real-time second count value is used as the second count result.

[0155] For example, the initial second count value is 0, and the second preset value is 1. When the first abstraction layer 30 uses the abstraction layer data storage information to perform hardware processing, the second count value is incremented by 1. When the use of the abstraction layer data storage information is complete, the second count value is decremented by 1. When the second count result changes from 0 to a non-zero value, it indicates that the abstraction layer data storage information is being used by the first abstraction layer 30, and the image processing data still has usability and storage requirements for the first abstraction layer 30. When the second count result changes from a non-zero value to 0, it indicates that the use of the abstraction layer data storage information is complete, and the image processing data no longer has usability and storage requirements for the first abstraction layer 30.

[0156] In this embodiment, the first count value is increased and decreased by a first preset value when the general data storage information is used and when the general data storage information is used up, respectively. Similarly, the second count value is increased and decreased by a second preset value when the abstract layer data storage information is used and when the abstract layer data storage information is used up, respectively. This achieves a two-level reference count for the general data storage information and the abstract layer data storage information, providing a basis for the lifecycle management of image processing data. The first count result and the second count result can respectively characterize the use value and storage requirements of the image processing data for the second abstract layer 20 and the first abstract layer 30, further ensuring that the effectiveness of the image processing data matches the actual needs and improving the effect of power consumption optimization and dynamic management of storage space.

[0157] In some embodiments, lifecycle management of image processing data is performed based on a first counting result and a second counting result, including deleting the image processing data in response to the first counting result and the second counting result being the corresponding initial counting value.

[0158] As mentioned earlier, when the first count result decreases to the corresponding initial count value (i.e., the initial first count value), it indicates that the general data storage information has been used up, and the image processing data no longer has any use value or storage requirement for the second abstraction layer 20. When the second count result decreases to the corresponding initial count value (i.e., the initial second count value), it indicates that the abstraction layer data storage information has been used up, and the image processing data no longer has any use value or storage requirement for the first abstraction layer 30. When both of the above conditions are met simultaneously, it means that the image processing data has no use value or storage requirement for either the second abstraction layer 20 or the first abstraction layer 30, and the corresponding image processing data is deleted to achieve lifecycle management of the image processing data.

[0159] In this embodiment, when both the first counting result and the second counting result are the corresponding initial counting values, the image processing data is deleted, thereby realizing lifecycle management of the image processing data. When the image processing data has no use value or storage requirement for the second abstraction layer 20 and the first abstraction layer 30, it can be deleted in a timely manner. This is achieved by using a two-level reference counting method to further ensure that the validity of the image processing data matches the actual needs, thereby improving the effectiveness of power consumption optimization and dynamic management of storage space.

[0160] In some embodiments, the image processing method further includes: in response to a preset image processing request, allocating a first storage space for image processing data, and sending the preset image processing request to a first abstraction layer 30, so that the first abstraction layer 30 generates image processing data and stores the image processing data in the first storage space. Alternatively, in response to the preset image processing request, sending the preset image processing request to the first abstraction layer 30, so that the first abstraction layer 30 generates image processing data and stores the image processing data in a second storage space allocated by the first abstraction layer 30.

[0161] When a preset image processing request is received, the conversion layer 10 can allocate a first storage space for the image processing data to be generated by the first abstraction layer 30, and send the preset image processing request to the first abstraction layer 30. This allows the first abstraction layer 30 to respond to the preset image processing request, generate image processing data, and store the image processing data in the first storage space allocated by the conversion layer 10. The first storage space is the storage space in system memory corresponding to the system architecture and is directly managed by the system architecture.

[0162] When a preset image processing request is received, it can be directly sent to the first abstraction layer 30. The first abstraction layer 30 can then allocate a second storage space for the image processing data to be generated, and store the generated image processing data in the second storage space. The second storage space is the storage space in the system memory corresponding to the first abstraction layer 30, and is directly managed by the first abstraction layer 30.

[0163] In this embodiment, when a preset image processing request is received, the system architecture allocates a first storage space for the image processing data or allocates a second storage space for the image processing data through the first abstraction layer 30. This enables the image processing data to be stored in the first or second storage space after it is generated, providing a basis for the storage of image processing data and the generation of corresponding storage addresses. It also provides multiple ways to store image processing data, reducing coupling with the chip and ensuring the versatility of the system architecture.

[0164] In some embodiments, reference Figure 10 As shown, the image processing method also includes:

[0165] S810. Generate a first queue corresponding to the first storage space. The first queue includes multiple first reference values. Each first reference value corresponds to a different storage area in the first storage space. Each first reference value is used to characterize whether data exists in the corresponding storage area.

[0166] In step S810, the image processing data manager can generate a corresponding image for the first storage space, such as... Figure 11 The first queue shown includes multiple first reference values, each corresponding to a different storage region in the first storage space allocated by the system architecture. The first reference value indicates whether data exists in its corresponding storage region. When storing image processing data in the first storage space, the first storage region without data can be determined by the first reference values ​​in the first queue and their order, and the image processing data can be stored in that storage region. For example, a first reference value of 1 indicates the presence of data in its corresponding storage region, and a first reference value of 0 indicates the absence of data in its corresponding storage region. When storing image processing data in the first storage space, the first first reference value with a value of 0 can be found in the first queue, and the image processing data can be stored in the storage region corresponding to that first reference value.

[0167] S820. Generate a second queue corresponding to the second storage space. The second queue includes multiple second reference values. Each second reference value corresponds to a different storage area in the second storage space. Each second reference value is used to characterize whether data exists in the corresponding storage area.

[0168] In step S820, the image processing data manager can generate a corresponding image for the second storage space, such as... Figure 12 The second queue shown includes multiple second reference values. Each second reference value corresponds to a different storage region in the second storage space allocated by the first abstraction layer 30, indicating whether data exists in its corresponding storage region. When storing image processing data in the second storage space, the first storage region without data can be determined by the second reference values ​​in the second queue and their order, and the image processing data can be stored in that storage region. For example, if the value of a second reference value is 1, it means that data exists in its corresponding storage region; if the value of a second reference value is 0, it means that data does not exist in its corresponding storage region. When storing image processing data in the second storage space, the first second reference value with a value of 0 can be found in the second queue, and the image processing data can be stored in the storage region corresponding to that second reference value.

[0169] It should be noted that the second queue corresponding to the second storage space can also be used for lifecycle management of image processing data. For example, when the second reference value in the second queue changes, it indicates a change in the state of whether data exists in the corresponding storage area. If the corresponding storage area first changes from no data to data, and then from data to no data, it means that the image processing data stored therein has been used up. At this time, the first counting result and the second counting result can be combined to select whether to completely destroy the image processing data that once existed in the storage area of ​​the system.

[0170] In this embodiment, by generating a first queue and a second queue corresponding to the first storage space and the second storage space respectively, the existence of data in the corresponding storage area can be characterized by the first reference value in the first queue and the second reference value in the second queue, providing a basis for the management of the first storage space and the second storage space. Furthermore, the lifecycle management of image processing data can be realized by combining the first counting result and the second counting result, thereby improving the convenience and effectiveness of data management.

[0171] In one exemplary embodiment, an image processing method is provided, referring to... Figure 13 As shown, the image processing methods include:

[0172] S1. In response to a preset image processing request, generate a data processing task corresponding to the preset image processing request. The data processing task includes at least one hardware processing procedure and at least one software processing procedure.

[0173] S2. Directly execute each software processing procedure;

[0174] S3. Receive abstract layer data storage information sent by the first abstract layer;

[0175] S4. Generate a set of storage address data based on the storage addresses of multiple image processing data;

[0176] S5. Based on the preset data structure, the storage address data set is transformed to obtain general data storage information;

[0177] S6. In response to the total number of data processing tasks to be executed being greater than or equal to a preset number threshold, store the general data storage information corresponding to the data processing tasks to be executed into the data storage module.

[0178] S7. In response to the total number of data processing tasks to be executed being less than a preset threshold, read general data storage information from the data storage module;

[0179] S8. For each hardware processing procedure, in response to the start of the processing thread corresponding to the hardware processing procedure, convert the general data storage information into abstract layer data storage information that can be recognized by the first abstract layer.

[0180] S9. Send the data storage information of the abstract layer to the first abstract layer so that the first abstract layer can read the image processing data and execute the hardware processing procedure based on the image processing data;

[0181] S10. Receive the processing result obtained by the hardware processing procedure of the first abstract layer. The processing result includes the updated abstract layer data storage information, which includes the storage address of the updated image processing data.

[0182] S11. Convert the updated abstract layer data storage information into general data storage information with a preset data structure;

[0183] S12. Perform reference counting on the general data storage information and the abstract layer data storage information respectively to obtain the first count result and the second count result;

[0184] S13. Based on the first and second counting results, perform lifecycle management on the image processing data;

[0185] S14. Generate a first queue corresponding to the first storage space. The first queue includes multiple first reference values. Each first reference value corresponds to a different storage area in the first storage space. Each first reference value is used to characterize whether data exists in the corresponding storage area.

[0186] S15. Generate a second queue corresponding to the second storage space. The second queue includes multiple second reference values. Each second reference value corresponds to a different storage area in the second storage space. Each second reference value is used to characterize whether data exists in the corresponding storage area.

[0187] In this embodiment, by receiving abstract layer data storage information sent by the first abstract layer 30 and converting it into general data storage information with a preset data structure, and then controlling the first abstract layer 30 to execute hardware processing based on the general data storage information, the execution of hardware processing based on image processing data is realized. By converting the abstract layer data storage information into general data storage information, the general data storage information with a unified preset data structure can be used as the basis for controlling the first abstract layer 30 to execute hardware processing, providing a common data transmission path for different chips, reducing coupling with chips, simplifying the data processing process, improving system development and maintenance efficiency, and ensuring the universality of the system architecture.

[0188] In one exemplary embodiment, a system architecture is provided for performing the image processing method described above, with reference to... Figure 1 As shown, the system architecture includes a conversion layer 10 and a second abstraction layer 20. The conversion layer 10 receives abstraction layer data storage information sent by the first abstraction layer 30 and converts the abstraction layer data storage information into general data storage information with a preset data structure. The abstraction layer data storage information includes the storage address of image processing data generated in response to a preset image processing request. The image processing data is used by the first abstraction layer 30 to execute at least one hardware processing procedure corresponding to the preset image processing request. The second abstraction layer 20 controls the first abstraction layer 30 to execute at least one hardware processing procedure based on the general data storage information.

[0189] In this embodiment, the conversion layer 10 receives the abstract layer data storage information sent by the first abstract layer 30 and converts it into general data storage information with a preset data structure. Then, the second abstract layer 20 controls the first abstract layer 30 to execute hardware processing based on the general data storage information, thus realizing the execution of hardware processing based on image processing data. By converting the abstract layer data storage information into general data storage information, the general data storage information with a unified preset data structure can be used as the basis for controlling the first abstract layer 30 to execute hardware processing. This provides a common data transmission path for different chips, reduces coupling with chips, simplifies the data processing process, improves system development and maintenance efficiency, and ensures the universality of the system architecture.

[0190] In some embodiments, for each hardware processing procedure, the second abstraction layer 20 is used to send general data storage information to the conversion layer 10 in response to the start of the processing thread corresponding to the hardware processing procedure; the conversion layer 10 is used to convert the general data storage information into abstraction layer data storage information that the first abstraction layer 30 can recognize, and send the abstraction layer data storage information to the first abstraction layer 30, so that the first abstraction layer 30 reads the image processing data and executes the hardware processing procedure based on the image processing data.

[0191] In some embodiments, the conversion layer 10 is further configured to: receive the processing result obtained by the first abstraction layer 30 performing the hardware processing procedure, the processing result including the updated abstraction layer data storage information, the updated abstraction layer data storage information including the storage address of the updated image processing data; convert the updated abstraction layer data storage information into general data storage information with a preset data structure and send it to the second abstraction layer 20.

[0192] In some embodiments, the second abstraction layer 20 is further configured to: generate a data processing task corresponding to the preset image processing request in response to the preset image processing request, wherein the data processing task includes at least one hardware processing procedure and at least one software processing procedure; and directly execute each software processing procedure.

[0193] In some embodiments, the system architecture further includes an application layer 40, which is used to send a preset image processing request to the second abstraction layer 20 and receive target data returned by the second abstraction layer 20.

[0194] In one exemplary embodiment, an electronic device is provided, which may include, for example, a mobile phone, a camera, a tablet computer, or other devices that include the system architecture and chips described above.

[0195] refer to Figure 14 As shown, the electronic device may include one or more of the following components: processing component 101, memory 102, power component 103, multimedia component 104, audio component 105, input / output (I / O) interface 106, sensor component 107, and communication component 108.

[0196] Processing component 101 typically controls the overall operation of an electronic device, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 101 may include one or more processors 109 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 101 may include one or more modules to facilitate interaction between processing component 101 and other components. For example, processing component 101 may include a multimedia module to facilitate interaction between multimedia component 104 and processing component 101.

[0197] Memory 102 is configured to store various types of data to support the operation of the electronic device. Examples of such data include instructions for any application or method used to operate on the electronic device, contact data, phonebook data, messages, pictures, videos, etc. Memory 102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0198] Power component 103 provides power to various components of the electronic device. Power component 103 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0199] Multimedia component 104 includes a screen that provides an output interface between the electronic device and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 104 includes a front-facing camera and / or a rear-facing camera. When the electronic device is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0200] Audio component 105 is configured to output and / or input audio signals. For example, audio component 105 includes a microphone (MIC) configured to receive external audio signals when the electronic device is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 102 or transmitted via communication component 108. In some embodiments, audio component 105 also includes a speaker for outputting audio signals.

[0201] I / O interface 106 provides an interface between processing component 101 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0202] Sensor assembly 107 includes one or more sensors for providing state assessments of various aspects of the electronic device. For example, sensor assembly 107 can detect the on / off state of the electronic device, the relative positioning of components such as the display and keypad of the electronic device, changes in the position of the electronic device or a component of the electronic device, the presence or absence of user contact with the electronic device, the orientation or acceleration / deceleration of the electronic device, and temperature changes of the electronic device. Sensor assembly 107 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 107 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 107 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0203] Communication component 108 is configured to facilitate wired or wireless communication between electronic devices and other devices. Devices can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 108 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 108 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0204] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the image processing method applied to the electronic device described above.

[0205] In one exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 102 including instructions, which can be executed by a processor 109 of an electronic device to perform the image processing method applied to the electronic device described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the instructions in the storage medium are executed by the processor 109 of the electronic device, the electronic device is able to perform the image processing method shown in the above embodiments.

[0206] In one exemplary embodiment, a computer program product is also provided, including a computer program that, when executed by processor 109, implements the image processing method shown in the above embodiments.

[0207] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0208] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, The image processing method includes: The abstract layer data storage information sent by the first abstract layer includes the storage address of image processing data generated by the first abstract layer in response to a preset image processing request. The image processing data is used by the first abstract layer to perform at least one hardware processing procedure corresponding to the preset image processing request. The abstract layer data storage information is converted into general data storage information with a preset data structure; Based on the general data storage information, the first abstraction layer is controlled to execute the at least one hardware processing procedure.

2. The image processing method according to claim 1, characterized in that, The abstract layer data storage information includes the storage addresses of multiple image processing data. Converting the abstract layer data storage information into general data storage information with a preset data structure includes: A storage address data set is generated based on the storage addresses of multiple image processing data sets. Based on the preset data structure, the storage address data set is transformed to obtain the general data storage information.

3. The image processing method according to claim 1, characterized in that, The step of controlling the first abstraction layer to execute the at least one hardware processing procedure based on the general data storage information includes: For each of the aforementioned hardware processing procedures, in response to the start of the processing thread corresponding to the hardware processing procedure, the general data storage information is converted into the abstract layer data storage information that the first abstract layer can recognize; The abstraction layer data storage information is sent to the first abstraction layer so that the first abstraction layer can read the image processing data and execute the hardware processing procedure based on the image processing data.

4. The image processing method according to claim 3, characterized in that, The image processing method further includes: The system receives the processing result obtained by the first abstraction layer performing the hardware processing procedure. The processing result includes updated abstraction layer data storage information, which includes the storage address of the updated image processing data. The updated abstract layer data storage information is converted into general data storage information with the preset data structure.

5. The image processing method according to claim 3, characterized in that, The image processing method further includes: In response to the preset image processing request, a data processing task corresponding to the preset image processing request is generated, the data processing task including at least one hardware processing procedure and at least one software processing procedure; The software processing procedures described are executed directly.

6. The image processing method according to claim 5, characterized in that, The preset image processing request is a photo taking request, and the data processing task includes multiple image processing processes executed sequentially, of which a portion of the image processing processes are hardware processing processes and another portion of the image processing processes are software processing processes; The hardware processing procedure includes: controlling the first abstraction layer to perform hardware processing on the initial image data based on the image processing data to obtain the target image data; The software processing procedure includes: processing the initial image data to obtain the target image data; The initial image data used in the first image processing step is the original image data, and the initial image data used in the nth image processing step is the target image data obtained in the (n-1)th image processing step.

7. The image processing method according to claim 6, characterized in that, In the first-frame-fetch mode, the original image data is the preview cached image data; In the post-frame mode, the original image data is the original format image data sent by the first abstraction layer.

8. The image processing method according to claim 5, characterized in that, The image processing method further includes: In response to the total number of data processing tasks to be executed being greater than or equal to a preset number threshold, the general data storage information corresponding to the data processing tasks to be executed is stored in the data storage module. In response to the total number of data processing tasks to be executed being less than the preset number threshold, the general data storage information is read from the data storage module to execute the data processing task corresponding to the read general data storage information.

9. The image processing method according to any one of claims 1 to 8, characterized in that, The image processing method further includes: The reference counts of the general data storage information and the abstract layer data storage information are performed respectively to obtain a first count result and a second count result; Based on the first and second counting results, lifecycle management is performed on the image processing data.

10. The image processing method according to claim 9, characterized in that, The reference counting of the general data storage information and the abstract layer data storage information includes: In response to using the general data storage information, the first count value is increased by a first preset value; In response to the completion of the use of the general data storage information, the first count value is reduced by the first preset value; In response to using the data storage information of the abstract layer, the second count value is increased by a second preset value; In response to the completion of the use of the data storage information in the abstract layer, the second count value is reduced by the second preset value.

11. The image processing method according to claim 9, characterized in that, The lifecycle management of the image processing data based on the first counting result and the second counting result includes: In response to the fact that both the first counting result and the second counting result are the corresponding initial counting values, the image processing data is deleted.

12. The image processing method according to any one of claims 1 to 8, characterized in that, The image processing method further includes: In response to the preset image processing request, a first storage space is allocated for the image processing data, and the preset image processing request is sent to the first abstraction layer, so that the first abstraction layer generates the image processing data and stores the image processing data in the first storage space; or, In response to the preset image processing request, the preset image processing request is sent to the first abstraction layer, so that the first abstraction layer generates the image processing data and stores the image processing data in the second storage space allocated by the first abstraction layer.

13. The image processing method according to claim 12, characterized in that, The image processing method further includes: Generate a first queue corresponding to the first storage space. The first queue includes multiple first reference values, each first reference value corresponding to a different storage region in the first storage space. Each first reference value is used to characterize whether data exists in the corresponding storage region; and / or, A second queue corresponding to the second storage space is generated. The second queue includes multiple second reference values. Each second reference value corresponds to a different storage area in the second storage space. Each second reference value is used to characterize whether data exists in the corresponding storage area.

14. A system architecture, characterized in that, The system architecture includes: A conversion layer is used to receive abstract layer data storage information sent by a first abstract layer and convert the abstract layer data storage information into general data storage information with a preset data structure. The abstract layer data storage information includes the storage address of image processing data generated by the first abstract layer in response to a preset image processing request. The image processing data is used by the first abstract layer to perform at least one hardware processing procedure corresponding to the preset image processing request. The second abstraction layer is used to control the first abstraction layer to execute the at least one hardware processing procedure based on the general data storage information.

15. The system architecture according to claim 14, characterized in that, For each of the aforementioned hardware processing procedures, the second abstraction layer is used to send the general data storage information to the conversion layer in response to the start of the processing thread corresponding to the hardware processing procedure. The conversion layer is used to convert the general data storage information into abstract layer data storage information that the first abstract layer can recognize, and send the abstract layer data storage information to the first abstract layer so that the first abstract layer can read the image processing data and execute the hardware processing process based on the image processing data.

16. The system architecture according to claim 14, characterized in that, The conversion layer is also used for: The system receives the processing result obtained by the first abstraction layer performing the hardware processing procedure. The processing result includes updated abstraction layer data storage information, and the updated abstraction layer data storage information includes the storage address of the updated image processing data. The updated abstract layer data storage information is converted into general data storage information with the preset data structure and sent to the second abstract layer.

17. The system architecture according to claim 14, characterized in that, The second abstraction layer is also used for: In response to the preset image processing request, a data processing task corresponding to the preset image processing request is generated, the data processing task including at least one hardware processing procedure and at least one software processing procedure; The software processing procedures described are executed directly.

18. The system architecture according to claim 14, characterized in that, The system architecture also includes an application layer, which is used to send the preset image processing request to the second abstraction layer and receive the target data returned by the second abstraction layer.

19. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions; The processor is configured to perform the image processing method as described in any one of claims 1 to 13.

20. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the image processing method as described in any one of claims 1 to 13.

21. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image processing method as described in any one of claims 1 to 13.

Images