Image signal processing driving method, control module, processing module, and medium

Abstract

The present application provides an image signal processing driving method, a control module, a processing module, and a medium. The method is used in an image signal processing module including multiple ISP cores or an ISP core supporting time division multiplexing, and a driving architecture is mounted on the image signal processing module. The method includes: an interface layer receives an image signal processing ISP command including an initial control parameter and an image size parameter from an upper application; a connection layer determines a processing mode of an image signal based on the image size parameter, and determines target control parameters for different processing modes based on the initial control parameters; and a driving layer transmits the target control parameters to a target ISP core, thereby driving the target ISP core to perform image signal processing based on the target control parameters, and the processing mode includes a single-core mode or a multi-core mode. By providing a connection layer in the driving architecture, low resolution by a single core and high resolution by a multi-core can be realized under the same driving architecture, and multiple processing modes can be supported.

Description

[Technical Field]

[0001] The present application belongs to the technical field of image processing, and specifically relates to an image signal processing driving method, a control module, a processing module, and a medium. [Background technology]

[0002] ISP (Image Signal Processor) technology improves the imaging quality of image signals. With the rapid development of electronic devices, image resolution has increased from 2K to 4K, 8K, 16K, etc., and people's demands for image resolution are also increasing. In conventional technology, multi-core processors are required to achieve resolutions of 8K or higher. Electronic devices are required to achieve resolutions of 8K or higher, as well as lower resolutions, which is a technical problem that this field aims to solve. Summary of the Invention

[0003] The embodiments of the present application aim to provide an image signal processing driving method, a control module, a processing module and a medium for achieving technical effects that satisfy both high resolution and low resolution implementations.

[0004] In a first aspect of an embodiment of the present application, there is provided an image signal processing driving method, which is used in an image signal processing module including a plurality of ISP cores or a time-division multiplexed ISP core, wherein the image signal processing module is equipped with a driving architecture including an interface layer, a connection layer, and a driving layer, and the driving layer is connected to the ISP cores, and the method includes: a step in which the interface layer receives an image signal processing ISP command from an upper application, the image signal processing command including initial control parameters and image size parameters; a step in which the connection layer determines a processing mode of an image signal based on the image size parameters, and determines target control parameters for different processing modes based on the initial control parameters; and a step in which the driving layer sends the target control parameters to a target ISP core, thereby driving the target ISP core to perform image signal processing based on the target control parameters, and the processing mode includes a single-core mode or a multi-core mode.

[0005] In the above implementation process, the image signal processing module includes multiple ISP cores or an ISP core that supports time-division multiplexing. The image signal processing module also includes a driving architecture. An ISP command from a higher-level application includes initial control parameters and image size parameters. By providing a connection layer in the driving architecture, the connection layer can determine which processing mode to use based on the image size parameters. In addition, for each processing mode, the connection layer can construct different target control parameters based on the initial control parameters. Meanwhile, the driving layer transmits the target control parameters to the target ISP core, which then drives the target ISP core to perform image signal processing. In this way, both low-resolution processing using a single core and high-resolution processing using multiple cores can be achieved under the same driving architecture, and multiple processing modes can be supported.

[0006] Furthermore, the processing mode is a multi-core mode, and the step of determining target control parameters based on the initial control parameters includes the step of constructing a plurality of control parameters based on the initial control parameters to set the target control parameters.

[0007] In the above realization process, the connection layer establishes multiple control parameters under the multi-core mode as target control parameters, thereby realizing the technical effect of supporting multiple processing modes.

[0008] Furthermore, the ISP instruction includes a target function to be called, and the step of constructing a plurality of control parameters based on the initial control parameters to become the target control parameters includes the step of calling a membership function pointed to by a function pointer provided in the multi-core mode so as to construct the initial control parameters as a plurality of control parameters to become target control parameters, and the membership function corresponds to the target function.

[0009] In the above implementation process, the function pointers can point to different membership functions. In multi-core mode, multiple control parameters can be constructed by calling the membership functions pointed to by the corresponding function pointers and corresponding target functions. In this way, desired target control parameters can be constructed according to the processing mode, and the target ISP core can perform image signal processing based on the corresponding target control parameters in different processing modes, thereby achieving the technical effect of supporting multiple processing modes.

[0010] Furthermore, the target function includes one or more of a function module open function, a function module close function, and a driving interface function.

[0011] In the above implementation process, different target functions included in the ISP command can call the membership functions pointed to by the corresponding function pointers under different processing modes to establish corresponding target control parameters, so that the target ISP core can process image signals according to the corresponding target control parameters under different processing modes, thereby achieving the technical effect of supporting multiple processing modes.

[0012] Furthermore, the ISP command indicates a target image processing function to be realized, the target control parameters include size-related control parameters, and the size-related control parameters include one or more of an image width, a parameter related to the image width, an image height, and a parameter related to the image height. The target ISP core performs image signal processing to realize the target image processing function. The step of constructing a plurality of control parameters based on the initial control parameters to set the target control parameters includes, if the target image processing function is a default function, constructing a plurality of different control parameters to set the target control parameters, and if the target image processing function is not the default function, constructing a plurality of the same control parameters to set the target control parameters. The default function is realized by the target ISP core performing image signal processing based on the size-related control parameters.

[0013] In the above realization process, the target control parameters constructed under multi-core mode differ depending on the image processing function. For the default function, the target control parameters used to realize it are size-related control parameters, so the constructed multiple control parameters are different. In this way, image signal processing can be performed using different control parameters. For image processing functions other than the default function, the constructed multiple control parameters are the same. In this way, image signal processing can be performed using the same control parameters. This allows all different image processing functions to be realized under multi-core mode.

[0014] Furthermore, the step of constructing the plurality of different control parameters includes the step of setting the plurality of different control parameters calculated based on a predetermined image position as the target control parameters.

[0015] In the above realization process, if multiple different control parameters need to be obtained, the target ISP performs image signal processing based on the different control parameters by calculating them based on a predetermined image position, thereby realizing the target image processing function.

[0016] Further, under a multi-core mode, the method further includes a step in which the connection layer responds to an interrupt signal of an image signal processing corresponding to a target ISP core, and after receiving all the interrupt signals, synchronously merges all the interrupt signals and sends them to the interface layer.

[0017] In the above implementation process, if an interruption occurs in image signal processing, after all interruption signals are received, the interruption signals are synchronously merged and reported. In any mode, the upper application receives the interruption signal only once, so it does not recognize the existence of two processing modes.

[0018] Furthermore, the method further includes a step of, after receiving an interrupt signal corresponding to the target ISP core, storing the interrupt signal in a corresponding queue; and, after receiving all interrupt signals, a step of synchronously merging all interrupt signals and sending them to the interface layer includes a step of, when interrupt signals are stored in all queues, reading the interrupt signals in each queue, synchronously merging them, and sending them to the interface layer.

[0019] In the above implementation process, when an interrupt signal is received from the target ISP core, the interrupt signal is stored in a corresponding queue and waits until all interrupt signals are received. By providing a queue for storing interrupt signals, the technical effect of synchronously merging and reporting all interrupt signals after receiving them can be achieved.

[0020] Furthermore, if there are multiple target ISP cores, the all interrupt signals include interrupt signals corresponding to all target ISP cores, and the corresponding queues are queues corresponding to the target ISP cores that transmit the interrupt signals; if the target ISP core supports time division multiplexing, the all interrupt signals include interrupt signals transmitted by the target ISP core in all multiple time segments, and the corresponding queues are queues corresponding to the multiple time segments that transmit the interrupt signals.

[0021] In the above implementation process, a corresponding queue is set for each target ISP core or each multiple time segment, and the interrupt signals corresponding to the target ISP core are stored in the corresponding queue, thereby achieving the technical effect of synchronously merging and reporting all the interrupt signals after receiving them.

[0022] Furthermore, the interrupt signal includes a plurality of types, and the step of storing the interrupt signal in the corresponding queue includes a step of storing the interrupt signal at a target position in the queue corresponding to the target type based on the target type to which the interrupt signal belongs, and storing the interrupt signals in all queues includes storing the interrupt signals at target positions in all of the queues.

[0023] In the above implementation process, the interrupt signal includes multiple types, and interrupt signals of the same type are synchronously merged and reported. Therefore, by setting different positions in the queue to correspond to different types of interrupt signals, different types of interrupt signals can be stored in the same queue. When all interrupt signals are stored in the same position in all queues, it is considered that all interrupt signals have been received. This achieves the technical effect of synchronously merging and reporting after all interrupt signals have been received.

[0024] Furthermore, the processing mode is a single-core mode, and determining target control parameters based on the initial control parameters includes determining the initial control parameters as the target control parameters.

[0025] In the above implementation process, the connection layer directly determines the initial control parameters as the target control parameters under the single-core mode, thereby achieving the technical effect of supporting multiple processing modes.

[0026] In a second aspect of an embodiment of the present application, there is provided an image processing control module, the image processing control module being connected to a plurality of ISP cores or an ISP core supporting time division multiplexing, and configured to implement the steps of any one of the methods of the first aspect.

[0027] In a third aspect of an embodiment of the present application, there is provided an image signal processing module, the image signal processing module including an image processing control module of the second aspect and a plurality of ISP cores or an ISP core supporting time division multiplexing, the image processing control module and the plurality of ISP cores or the ISP core supporting time division multiplexing are connected to each other.

[0028] Furthermore, the image signal processing module is an IP core, chip or electronic device for image signal processing.

[0029] In a fourth aspect of an embodiment of the present application, there is provided a computer-readable storage medium storing computer commands, which, when executed by a processor, effect the steps of any one of the methods of the first aspect. [Brief explanation of the drawings]

[0030] In order to more clearly explain the technical solutions of the embodiments of the present application, the drawings necessary for explaining the embodiments of the present application will be briefly described below. The drawings described only illustrate some embodiments of the present application and do not limit the scope. Those skilled in the art can obtain other related drawings based on these drawings without using inventive abilities. [Figure 1] FIG. 1 is a schematic block diagram of a drive architecture based on a single core according to the prior art; [Figure 2]FIG. 1 is a block diagram of a drive architecture configuration according to an embodiment of the present application. [Figure 3] FIG. 10 is a schematic block diagram of another drive architecture according to an embodiment of the present application. [Figure 4] 1 is a flowchart of an image signal processing driving method according to an embodiment of the present application. [Figure 5] 10 is a flowchart of another image signal processing driving method according to an embodiment of the present application. [Figure 6] 10 is a flowchart of another image signal processing driving method according to an embodiment of the present application. [Figure 7] 1 is a flowchart of a process for constructing target control parameters according to an embodiment of the present application. [Figure 8] 10 is a flowchart of another target control parameter construction process according to an embodiment of the present application. [Figure 9] FIG. 1 is a schematic configuration diagram of an image signal processing module according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0031] Hereinafter, the technical solutions according to the embodiments of the present application will be described with reference to the drawings of the embodiments of the present application.

[0032] Since like reference numerals refer to like objects in the drawings, a definition in one drawing does not require further definition or interpretation in other drawings. Also, in the description of this application, terms such as "first," "second," etc., are for descriptive purposes only and should not be understood to express or imply any relative importance.

[0033] ISP (Image Signal Processor) technology improves the imaging quality of image signals. With the rapid development of electronic devices, image resolutions have increased from 2K to 4K, 8K, 16K, and more, and people's demands for higher image resolution are also increasing. In conventional technology, low-resolution images such as 2K and 4K are realized with a single core. That is, low-resolution images can be processed with only one ISP core. In contrast, achieving resolutions of 8K or higher requires multi-core processing. That is, high-resolution images can be processed by using multiple ISP cores, such as two or more ISP cores.

[0034] Electronic devices sometimes process low-resolution images and sometimes high-resolution images. Therefore, electronic devices must be able to process both high-resolution and low-resolution images. However, in the prior art, the ISP driving architecture is written with a driving program based on a single core. The single-core ISP driving architecture shown in Figure 1 includes a test layer (Test), an image engine layer (cam_engine), an interface layer (Cameric), a memory input management control (mim), a buffer pool (buffer_pool), a memory output management control (mom_ctrl), a snapshot output module control (som_ctrl), a hardware abstraction layer (HAL), an ISP driver (ISPdrv), and a memory interface driver (MIdrv). This driving architecture can only drive a single core to perform image signal processing and realize low-resolution image processing.

[0035] To achieve high-resolution image processing, the driver architecture needs to be upgraded so that it can drive multiple ISP cores to perform image signal processing. Also, considering that electronic devices need to simultaneously meet high-resolution and low-resolution image processing, the upgraded driver architecture needs to be able to achieve multi-core driving while also retaining the original single-core driving function.

[0036] Therefore, the present application proposes an image signal processing driving method to be used for an image signal processing module including multiple ISP cores or an ISP core that supports time division multiplexing.

[0037] An ISP core that supports time division multiplexing means that one ISP core can execute different tasks at different times, realizing multitasking execution and fully utilizing the ISP core's computing resources. Multiple time segments can appear alternately according to a predetermined period.

[0038] For example, the image signal processing module may be a hardware module, including, but not limited to, an IP core (Intellectual Property Core), chip, electronic device, and other hardware device or module for image signal processing, including an ISP core. An IP core is an integrated circuit IP core, a multiplexed module with specific circuit functions that have already been designed and verified. Depending on the degree of integration with a specific integrated circuit manufacturing process (foundry), it can be classified as a soft core, firm core, or hard core, with the soft core being independent of the final implementation process. Electronic devices include, but are not limited to, other devices requiring image processing, such as smartphones / mobile phones, personal digital assistants (PDAs), media content players, video game stations / systems, virtual reality systems, augmented reality systems, and wearable devices (e.g., watches, smart bands, gloves, hats, helmets, virtual reality headsets, augmented reality headsets, head-mounted displays (HMDs), headbands, pendants, shoulder patches, leg bands, shoes, or vests).

[0039] The image signal processing module includes a drive architecture 200 shown in Figure 2, which includes an interface layer 210, a connection layer 220, and a drive layer 230. The drive layer 230 is connected to an ISP core. If the image signal processing module includes multiple ISP cores, the drive layer 230 is connected to the multiple ISP cores.

[0040] Illustratively, the driver layer 230 includes the HAL layer, ISPdrv, and MIdrv shown in FIG. 1 . The driver architecture 200 may also include other layers for driving the ISP core, such as a test layer and an image engine layer. For example, FIG. 3 shows a driver architecture obtained by improving a single-core-based ISP driver architecture. As shown in FIG. 3 , this driver architecture includes a connection layer (binder) between the interface layer (Camerice) and the hardware abstraction layer (HAL). The improved driver architecture supports both high-resolution and low-resolution image processing. By providing this connection layer, the driver architecture that drives a single core can also drive multiple cores to perform image signal processing. The improved driver architecture can support both high-resolution and low-resolution image processing, while requiring relatively small changes to the existing driver architecture, thereby reducing the labor costs of development and upgrades.

[0041] Based on the above embodiments, the image signal processing driving method according to the present application is realized by the driving architecture shown in FIG. 2 or FIG. 3, and includes the steps shown in FIG.

[0042] Step S1: The interface layer receives an image signal processing ISP command from an upper application.

[0043] The ISP command includes initial control parameters and image size parameters.

[0044] Step S2: The connection layer determines processing modes of image signals according to the image size parameters, and determines target control parameters for different processing modes according to the initial control parameters.

[0045] The processing mode includes a single-core mode or a multi-core mode.

[0046] Step S3: The driving layer transmits the target control parameters to the target ISP core, thereby driving the target ISP core to perform image signal processing based on the target control parameters.

[0047] Illustratively, the steps of the above method may be implemented by a dedicated device or module based on hardware that realizes a predetermined function or executes a predetermined operation. For example, the steps of the above method may be implemented by an image processing control module. The image processing control module is a hardware module including circuits for implementing specific functions. The image processing control module is connected to multiple ISP cores or an ISP core that supports time division multiplexing, and drives the connected target ISP cores to perform image signal processing by executing the above steps S1 to S3. For example, the image processing control module may be an IP core. Optionally, the image signal processing module may include the above image processing control module, multiple ISP cores, or an ISP core that supports time division multiplexing.

[0048] The upper application can issue an ISP command to drive the ISP core to process the image signal. The ISP command includes an initial control parameter and an image size parameter. The connection layer determines the processing mode of the image signal based on the image size parameter so that the processing mode can be determined at startup.

[0049] The processing modes include single-core mode and multi-core mode. In single-core mode, image signals are processed by one ISP core of the image signal processing module. If the image signal processing module includes multiple ISP cores, image signals are processed by one of the ISP cores. In multi-core mode, image signals are processed by at least two of the ISP cores of the image signal processing module, or the ISP performs different tasks related to image signal processing in different multiple time segments. For example, if the image signal processing module is a dual-core module, the multi-core mode performs image signal processing using two ISP cores. In this case, the multi-core mode is also referred to as dual-core mode. For example, if the image signal processing module has more than two ISP cores, the multi-core mode may perform image signal processing using all ISP cores or at least two ISP cores.

[0050] The connection layer determines a processing mode of the image signal based on an image size parameter. For example, if the image size parameter is equal to or greater than a predetermined size threshold, the connection layer determines the processing mode to be the multi-core mode. If the image size parameter is less than the predetermined size threshold, the connection layer determines the processing mode to be the single-core mode. The image size parameter may be the image resolution. In this case, the connection layer may be configured such that if the image resolution is 8K or greater, the connection layer determines the processing mode to be the multi-core mode, and if the image resolution is less than 8K, the connection layer determines the processing mode to be the single-core mode.

[0051] The connection layer then determines target control parameters for different processing modes based on the initial control parameters. Different processing modes determine different target control parameters. The control parameters are parameters set in registers to enable the chip. The control parameters can be accessed by a user and set according to the user's actual needs. The control parameters include multiple types, such as, but not limited to, size-related control parameters and function module-related control parameters. For example, size-related control parameters include image width, image width parameters, image height, and image height parameters. Function module-related control parameters include chromatic aberration correction (CAC) module parameters, denoising pre-filter (DPF) module parameters, etc. The initial control parameters and the target control parameters are the same type of control parameters.

[0052] After determining the target control parameters, the driving layer sends the target control parameters to the target ISP core, which is an ISP core that performs image signal processing.

[0053] In the case of an image signal processing module including multiple ISP cores, all ISP cores in the image signal processing module may be target ISP cores, i.e., all ISP cores are involved in image signal processing, or some ISP cores may be target ISP cores, i.e., only some ISP cores are involved in image signal processing. On the other hand, in the case of an image signal processing module including an ISP core that supports time division multiplexing, the target ISP is an ISP core that supports time division multiplexing in the image signal processing module and performs image signal processing. A method for determining the target ISP core will be described later.

[0054] The image signal processing driving method of the present application provides a connection layer in the driving architecture, allowing the connection layer to construct target control parameters based on initial control parameters in any processing mode. Compared to the prior art, the connection layer separates the interface layer and the driving layer, allowing corresponding target control parameters to be constructed under different processing modes. The driving layer then transmits the target control parameters to the target ISP core, which then drives the target ISP core to perform image signal processing. In this way, both low-resolution single-core and high-resolution multi-core can be achieved with the same driving architecture, and multiple processing modes can be supported.

[0055] In some embodiments, step S2 may include the steps shown in FIG.

[0056] Step S20: The connection layer determines a processing mode of the image signal according to the image size parameter.

[0057] Step S21: Under the single-core mode, the initial control parameters are determined as the target control parameters.

[0058] Step S22: Under the multi-core mode, a plurality of control parameters are constructed based on the initial control parameters to be the target control parameters.

[0059] In the single-core mode, the initial control parameters are directly determined as target control parameters. There is one target control parameter and one target ISP core. In this way, the driving layer sends this one initial control parameter as the target control parameter to the one target ISP core, so that the one target ISP core performs image signal processing based on the target control parameter.

[0060] In the multi-core mode, the target control parameters include multiple control parameters, and the target ISP core or the multiple time segments of the target ISP core are multiple. The number of control parameters is the same as the number of target ISP cores or multiple time segments. In this way, the driving layer transmits multiple control parameters to the multiple target ISP cores, respectively, so that the multiple target ISP cores perform image signal processing based on their respective control parameters. Alternatively, the target ISP core performs image signal processing based on different control parameters in different multiple time segments.

[0061] In this way, the connection layer constructs different numbers of control parameters under different processing modes, respectively, as target control parameters, thereby achieving the technical effect of supporting multiple processing modes.

[0062] In some embodiments, the ISP command indicates a target image processing function to be implemented. Image processing functions include, but are not limited to, demosaic, gamma correction, and auto white balance (AWB). The upper-level application issues a corresponding ISP command to operate the target ISP core to perform image signal processing based on target control parameters, thereby implementing the image processing function. As described above, the control parameters include size-related control parameters and function module-related control parameters. If the target control parameters used to implement the image processing function are size-related control parameters, including one or more of image width, image width-related parameters, image height, and image height-related parameters, the image processing function is a default function. In other words, when implementing the default function, the target ISP core performs image signal processing based on the size-related control parameters.

[0063] In multi-core mode, multiple target ISP cores can process each image simultaneously or in multiple time segments, respectively, to achieve the corresponding image processing function. For example, using AWB as an example, an image can be divided into multiple sub-images, where the number of sub-images can be the same as the number of target ISP cores or the number of multiple time segments of the target ISP core. In this way, one target ISP core performs AWB processing on one sub-image, or one sub-image within one multiple time segment of the target ISP core. When performing AWB processing, the target ISP core performs image signal processing corresponding to the AWB algorithm based on size-related control parameters. Therefore, AWB is the default function described above. After image signal processing is completed, the image processing results of each target ISP core or each multiple time segment of the target ISP core can be stored in consecutive addresses, resulting in a processed image without overlapping areas due to uneven image division.

[0064] By way of example, default features include, but are not limited to, LSC (Lens Shading Correction), CAC (Chromatic Aberration Correction), AE (Auto Exposure), AWB, DPCC (Defect Pixel Cluster Correction), IS (Image Stabilization), Crop, Scale, VSM (Video Stabilization Measurement), PDAF (Phase Detection Auto Focus), and the like. On the other hand, image functions that are not image signal processing performed based on size-related control parameters, i.e., image processing functions other than default functions, include, but are not limited to, 2DNR (2D Digital Noise Reduction), 3DNR (3D Digital Noise Reduction), EE (Edge Enhance), DPF (De-noising Pre-Filter), CCM (Color Correction Matrix), CSM (Color Space Matrix), WDR (Wide Dynamic Range), BLS (Black Level Subtraction), DGAIN (Digital Gain), and the like.

[0065] The target image processing function indicated by the ISP command may be the above-mentioned default function or may not be the default function. The step of establishing a plurality of control parameters under the multi-core mode in the above-mentioned step S22 may include the steps shown in FIG. 6.

[0066] Step S221: If the target image processing function is a default function, a plurality of different control parameters are constructed as the target control parameters.

[0067] Step S222: If the target image processing function is not the default function, a plurality of identical control parameters are constructed and used as the target control parameters.

[0068] For example, the step of constructing a plurality of different control parameters in the above step S221 may include a step of setting a plurality of different control parameters obtained by calculation based on a predetermined image position as the target control parameters.

[0069] As described above, for the predetermined function, an image is typically divided into multiple sub-images, and image signal processing is performed on each of the different sub-images based on different control parameters. When constructing the different control parameters, calculations are performed based on the image positions of the sub-images in the image, and corresponding control parameters can be obtained. Because the image positions of each sub-image are different, the constructed control parameters are also different. Because the sub-images are divided using a predetermined division method, the image positions are preset. For example, calculations can be performed using an image algorithm based on the image positions to obtain multiple different control parameters. The image algorithm includes a mapping algorithm. After the initial control parameters are processed by the mapping algorithm, multiple different control parameters can be obtained as target control parameters.

[0070] Illustratively, in the above step S222, constructing the same control parameters may include performing an operation using an image algorithm to obtain the same control parameters.

[0071] When constructing a plurality of identical control parameters, it is not necessary to consider the image positions of the sub-images, and the same control parameters can be obtained by directly performing calculations using an image algorithm.

[0072] In this embodiment, in the multi-core mode, the construction process of the multiple control parameters is different for different target image processing functions. For the default function, the target control parameters used to realize it are size-related control parameters, so two different size-related control parameters need to be constructed as target control parameters using a mapping algorithm. Correspondingly, the target ISP core performs image signal processing based on the different size-related control parameters. Image processing functions other than the default function do not need to use size-related control parameters to realize them. Therefore, the multiple control parameters are the same, and processing by the mapping algorithm is not required. The target ISP core performs image signal processing based on the same control parameters.

[0073] In some embodiments, the ISP command includes a target function to be called. The target function includes, but is not limited to, one or more of a function module open function (Open), a function module close function (Close), and a driving interface (ioctl) function. The Open function is used to open a function module, and the Close function is used to close a function module. A function module related to an image processing function is called by the ioctl function. The above target function may be an API (Application Programming Interface) function for calling an API in a driving layer. Illustratively, the driving layer includes a hardware abstraction layer, ISPdrv, and MIdrv, and the target function can be used to call the API in the hardware abstraction layer.

[0074] A function pointer is provided in the multi-core mode. The function pointer is a pointer variable of a function, and its membership function can be pointed to and called. In this case, the step of constructing multiple control parameters under the multi-core mode in step S22 above includes a step of calling the membership function pointed to by the function pointer provided under the multi-core mode, thereby constructing the initial control parameters as multiple control parameters and setting them as target control parameters. This membership function corresponds to the target function.

[0075] Taking a dual-core device as an example, the multi-core mode is the dual-core mode. As shown in Figure 7, when the target function is an Ioctl function, i.e., when transmitting control parameters, the driving layer provides an API for implementing the image processing module, namely, the API BinderHalIoctl. An upper layer, such as the interface layer, can call the API BinderHalIoctl. Then, if the processing mode is determined to be single-core mode, the membership function GeneralIoctl(0) pointed to by the function pointer bindGeneralOps is called. The membership function GeneralIoctl(0) corresponds to the target function Ioctl. The initial control parameters are then passed to the halIoctl(0) function of the driving layer as target control parameters. If the processing mode is determined to be dual-core mode (TileMode), the membership function TileModeIoctl(0) pointed to by the function pointer bindTileModeOps provided in the multi-core mode is called, thereby constructing the initial control parameters into two control parameters as target control parameters. The membership function TileModeIoctl(0) corresponds to the target function Ioctl function, and transmits two control parameters to the driving layer's halIoctl(0) and halIoctl(1) functions, respectively.

[0076] As shown in Figure 8, if the target function is the Close function, that is, if a functional module needs to be closed, the driving layer provides an API, the API BinderHalClose, to close the functional module. An upper layer, such as the interface layer, can call the API BinderHalClose. If the processing mode is determined to be single-core mode, the member function GeneralClose(0) pointed to by the function pointer bindGeneralOps is called. The member function GeneralClose(0) corresponds to the target function Close. The initial control parameters are then passed to the halclose(0) function of the driving layer as target control parameters. If the processing mode is determined to be dual-core mode (TileMode), the member function TileModeClose(0) pointed to by the function pointer bindTileModeOps provided for multi-core mode is called, thereby constructing the initial control parameters into two control parameters, which are then used as target control parameters. The member function TileMode(0) corresponds to the target function Close. The two control parameters are then passed to the halclose(0) and halclose(1) functions of the driving layer, respectively.

[0077] In this embodiment, a function pointer that can point to different membership functions is provided in the multi-core mode. In the multi-core mode, multiple control parameters can be constructed by calling the membership function pointed to by the function pointer and corresponding to the target function. In this way, desired target control parameters can be constructed according to the processing mode, and the target ISP core can perform image signal processing based on the corresponding target control parameters in different processing modes, thereby achieving the technical effect of supporting multiple processing modes.

[0078] In some embodiments, the ISP command includes an ISP core identification number, also referred to as an ISP core ID, for uniquely identifying the ISP core. The image signal processing mode can also be determined by the format of the image processing control module's call to the ISP core identification number. When the image processing control module calls one ISP core identification number only once, it is determined that the processing mode is single-core mode and the target ISP core corresponds to the ISP core identification number, i.e., the target ISP core is the ISP core indicated by the ISP core identification number. When the image processing control module calls multiple ISP core identification numbers, it is determined that the processing mode is multi-core mode and there are multiple target ISP cores, and the multiple target ISP cores correspond one-to-one to the multiple ISP core identification numbers, i.e., the ISP core indicated by each of the multiple ISP core identification numbers is the target ISP core. When the image processing control module calls one ISP core identification number multiple times, it is determined that the processing mode is multi-core mode and image signal processing is performed by time-division multiplexing the target ISP core in this multi-core mode, and the target ISP core is the ISP core indicated by the ISP core identification number.

[0079] In this embodiment, the ISP command includes an ISP core identification number, so that the connection layer can determine whether the processing mode is a single-core mode or a multi-core mode according to the manner in which the ISP core identification number is called, and can determine that the ISP core indicated by the ISP core identification number is the target ISP core for image signal processing, thereby achieving the technical effect of supporting multiple processing modes.

[0080] The target ISP core generates an interrupt signal after completing the image signal processing, and then transmits the interrupt signal to the upper layer. In some embodiments, in the multi-core mode, the connectivity layer responds to the interrupt signals of the image signal processing corresponding to the target ISP core, and after receiving all the interrupt signals, synchronously merges all the interrupt signals and transmits them to the interface layer.

[0081] If there are multiple target ISP cores, the connection layer responds to an image signal processing interruption signal corresponding to at least one target ISP core, and after receiving interruption signals corresponding to all the target ISP cores, synchronizes and merges all the interruption signals and sends them to the interface layer.

[0082] If the target ISP core supports time division multiplexing, the connection layer responds to an interruption signal of image signal processing corresponding to at least one multiple time segment of the target ISP core, and after receiving interruption signals corresponding to all the multiple time segments, synchronously merges all the interruption signals and sends them to the interface layer.

[0083] Illustratively, all interrupt signals are merged into one interrupt signal and reported to the interface layer, informing the interface layer to take appropriate action.

[0084] For example, synchronization of all interrupt signals can be realized by an interrupt handling function provided by the connection layer: The interface layer can be provided with an interrupt handling function by the connection layer.

[0085] As described above, the constructed control parameters may be the same or different depending on the target image function. For example, if the control parameter is the image width, the same control parameters can be used to perform image signal processing on sub-images of the same size (resolution) by multiple target ISP cores or multiple time segments of the target ISP core. In this way, the image signal processing for each sub-image is completed simultaneously or the difference in completion time is within a predetermined time difference range. In this way, the connection layer can receive all interrupt signals simultaneously or within a predetermined time difference range.

[0086] If the control parameters are different, the image signal processing is performed by the multiple target ISP cores on sub-images of different sizes (resolutions). In this way, the image signal processing for the different sub-images is completed at different times. In this way, the connection layer receives interrupt signals corresponding to each target ISP core or interrupt signals corresponding to each multiple time segment back and forth.

[0087] In order to achieve synchronization of all interrupt signals in this way, in some embodiments, a queue may be used to store the interrupt signal that arrived first. When there are multiple target ISP cores, a queue may be set for each target ISP core. In this case, when an interrupt signal corresponding to a certain target ISP core is received, the interrupt signal is stored in the queue corresponding to the target ISP core. When the target ISP core supports time division multiplexing, a queue may be set for each of the multiple time segments. In this case, when an interrupt signal corresponding to a certain multiple time segment is received, the interrupt signal is stored in the queue corresponding to the multiple time segment in which the interrupt signal is transmitted.

[0088] When interrupt signals are stored in all the queues, it is assumed that all the interrupt signals have been received, and the interrupt signals in each queue are read, synchronized, merged, and sent to the interface layer.

[0089] In some embodiments, the image signal processing interrupt includes multiple types, including, but not limited to, a YUV interrupt output from the sp1 (self path 1) path in a memory interface (MI), a YUV interrupt output from the sp2 (self path 2) path, a raw interrupt, and a DMA interrupt of a direct memory access (DMA). Different types of interrupts correspond to different interrupt signals, i.e., multiple types of interrupt signals are included. Thus, each type of interrupt signal has a corresponding storage location in the queue. In other words, different types of interrupt signals are stored in different locations in the queue.

[0090] In this way, after receiving an interrupt signal, the target position of the interrupt signal in the queue can be determined based on the type of the interrupt signal, and the interrupt signal can be stored in the target position in the queue. If interrupt signals are stored in the target positions in all queues, it is considered that the same type of interrupt signals corresponding to all target ISP cores or all multiple time segments of the target ISP core have been received, and the same type of interrupt signals in the target positions in each queue are read, synchronized, merged, and sent to the interface layer.

[0091] Illustratively, a register in the interface layer is set.

[0092] The image signal processing driving method of the present application provides a connection layer in the driving architecture, which can configure different target control parameters based on the initial control parameters in any processing mode. The driving layer sends the target control parameters to the target ISP core, which then drives the target ISP core to perform image signal processing. In this way, both low-resolution processing using a single core and high-resolution processing using multiple cores can be achieved under the same driving architecture, making it possible to support multiple processing modes.

[0093] Based on the image signal processing driving method according to any of the above embodiments, the present application further provides a driving architecture that is mounted in an image signal processing module including multiple ISP cores or an ISP core supporting time division multiplexing, and is used to drive image signal processing. As shown in Figure 2, the driving architecture 200 includes an interface layer 210, a connection layer 220, and a driving layer 230, and the driving layer 230 is connected to the ISP cores.

[0094] The interface layer 210 receives an image signal processing ISP command from an upper application, and the ISP command includes an initial control parameter and an image size parameter.

[0095] The connection layer 220 determines a processing mode of the image signal based on the image size parameter, and determines target control parameters for different processing modes based on the initial control parameters, where the processing modes include a single-core mode or a multi-core mode.

[0096] The drive layer 230 transmits the target control parameters to the target ISP core, thereby driving the target ISP core to perform image signal processing based on the target control parameters.

[0097] In some embodiments, specifically, the connection layer 220 determines the initial control parameters as the target control parameters under the single-core mode, and constructs multiple control parameters based on the initial control parameters as the target control parameters under the multi-core mode.

[0098] In some embodiments, the ISP command indicates a target image processing function to be realized, and the target control parameters include size-related control parameters, which include one or more of image width, parameters related to image width, image height, and parameters related to image height, and the target ISP core performs image signal processing to realize the target image processing function. Specifically, if the target image processing function is a default function, the connection layer 220 constructs a plurality of different control parameters as the target control parameters, and if the target image processing function is not the default function, the connection layer 220 constructs a plurality of the same control parameters as the target control parameters. The default function is realized by the target ISP core performing image signal processing based on the size-related control parameters.

[0099] Specifically, in some embodiments, the connectivity layer 220 computes an image algorithm based on a given image location to obtain a number of different control parameters.

[0100] In some embodiments, the ISP command includes a target function to be called, and specifically, the connectivity layer 220 calls a membership function pointed to by a function pointer provided in the multi-core mode, thereby constructing the initial control parameters into a plurality of control parameters as target control parameters, and the membership function corresponds to the target function.

[0101] In some embodiments, the objective function includes one or more of a function module open function, a function module close function, and a driving interface function.

[0102] In some embodiments, under multi-core mode, the connectivity layer 220 further responds to interrupt signals of image signal processing corresponding to the target ISP core, and after receiving all the interrupt signals, synchronously merges all the interrupt signals and sends them to the interface layer.

[0103] In some embodiments, the connection layer 220 further receives a pause signal corresponding to the target ISP core, stores the pause signal in a corresponding queue, and when pause signals are stored in all queues, reads the pause signals in each queue, merges them synchronously, and sends them to the interface layer.

[0104] In some embodiments, when there are multiple target ISP cores, the all interrupt signals include interrupt signals corresponding to all target ISP cores, and the corresponding queues are queues corresponding to the target ISP cores that send the interrupt signals.

[0105] If the target ISP core supports time division multiplexing, the all interrupt signals include interrupt signals sent by the target ISP core in all multiple time segments, and the corresponding queues are queues corresponding to the multiple time segments in which the interrupt signals are sent.

[0106] In some embodiments, the interrupt signal includes multiple types, and specifically, the connection layer 220 stores the interrupt signal in a target position corresponding to the target type in the queue based on the target type to which the interrupt signal belongs.

[0107] In some embodiments, the image size parameters include a resolution of the image.

[0108] The process for realizing the functions and effects of each module in the above device can be referred to the process for realizing the corresponding step in the above method, so the description will be omitted here.

[0109] Based on the image signal processing driving method according to any of the above embodiments, the present application further provides an image processing control module. The image processing control module is a hardware module including circuits for realizing specific functions. The image processing control module is connected to multiple ISP cores or an ISP core that supports time division multiplexing, and drives the connected target ISP cores to perform image signal processing by executing the image signal processing driving method according to any of the above embodiments.

[0110] In some embodiments, the image processing control module may be an IP core.

[0111] Based on the image signal processing driving method according to any of the above embodiments, the present application further provides a schematic configuration diagram of an image signal processing module shown in FIG. 9. Optionally, as shown in the left diagram of FIG. 9, the image signal processing module includes an image processing control module and multiple ISP cores. The image processing control module is connected to the multiple ISP cores. The image processing control module may call the multiple ISP cores to realize image signal processing in multi-core mode, or may call a certain ISP core multiple times and realize image signal processing in multi-core mode by time-division multiplexing the called ISP cores, or may call a certain ISP core only once to realize image signal processing in single-core mode.

[0112] Optionally, as shown in the right diagram of FIG. 9 , the image signal processing module includes an image processing control module and one ISP core. The image processing control module is connected to an ISP core that supports time division multiplexing. The image processing control module may call the ISP core multiple times to realize image signal processing in a multi-core mode through time division multiplexing of the ISP core, or may call the ISP core only once to realize image signal processing in a single-core mode. The image processing control module drives the connected target ISP core to perform image signal processing by executing the image signal processing driving method according to any of the above embodiments.

[0113] In some embodiments, the image processing control module and the ISP core may be configured together as an IP core, i.e., the image signal processing module may be an IP core. Optionally, the IP core may be implemented in another hardware device, such as a chip.

[0114] In some embodiments, the image signal processing module may be another hardware device or module that includes an ISP core, such as a chip or electronic device for image signal processing.

[0115] The present application further provides a computer storage medium storing a computer program, which, when executed by a processor, performs the image signal processing driving method according to any of the above embodiments.

[0116] In some embodiments provided by the present application, the described apparatus and method may be implemented in other ways. The apparatus examples described above are merely illustrative. For example, the flowcharts and block diagrams in the drawings illustrate possible architectures, functions, and operations based on the apparatus, method, and computer program products according to various embodiments of the present application. Here, each block in the flowchart or block diagram may represent a module, program segment, or part of code, including one or more executable instructions that can implement a given logical function. In alternative implementations, the functions described in the blocks may be implemented in a different order than shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or may even be executed in the reverse order, depending on the required functionality. In addition, each block in the block diagram and / or flowchart, or a combination of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs a given function or operation, or by a combination of dedicated hardware and computer instructions.

[0117] In addition, each functional module according to each embodiment of the present application may be integrated to form a single independent part, each module may exist independently, or two or more modules may be integrated to form a single independent part.

[0118] The functions can be implemented in the form of software functional modules and stored in a computer-readable storage medium when sold or used as an independent product. From this understanding, the technical solution of the present application itself, or a portion of the technical solution that contributes to the prior art, can be implemented in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of commands for causing a computer device (such as a personal computer, a server, or a network device) to execute all or some of the steps of the above-described methods in each embodiment of the present application. The storage medium includes various media capable of storing program code, such as a USB disk, a portable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0119] The above description is merely an example of the present application and does not limit the scope of protection of the present application. Those skilled in the art may have various modifications and variations to the present application. As long as they do not deviate from the spirit and principles of the present application, any modifications, equivalent substitutions, improvements, etc., fall within the scope of protection of the present application. Note that similar symbols indicate similar objects in the drawings, and therefore, once a definition is made in one drawing, it is unnecessary to further define and interpret it in other drawings.

[0120] The above description is merely a specific embodiment of the present application and does not limit the scope of protection of the present application. Those skilled in the art may modify or replace the technical solutions within the technical scope disclosed in the present application, and these modifications or replacements are also included in the scope of protection of the present application. Therefore, the scope of protection of the present application is subject to the content set forth in the claims.

[0121] It should be noted that, in this specification, relational terms such as "first" and "second" are merely used to distinguish one entity or action from another and do not necessarily require or imply any actual relationship or order between those entities or actions. Furthermore, terms such as "comprises," "includes," and any variations thereof are intended to cover non-exclusive inclusions. Thus, a process, method, article, or apparatus that includes a set of elements is not necessarily limited to those elements and may include other elements not explicitly stated or inherent in those processes, methods, articles, or apparatus. Unless otherwise specified, an element limited by the expression "comprises" does not exclude the situation in which the process, method, article, or apparatus that includes the element also includes other similar elements.

Claims

1. An image signal processing driving method, comprising: The image signal processing driving method is used in an image signal processing module including a plurality of ISP cores or an ISP core supporting time division multiplexing, wherein the image signal processing module is equipped with a driving architecture including an interface layer, a connection layer, and a driving layer, and the driving layer is connected to the ISP cores, and the image signal processing driving method comprises: the interface layer receiving an image signal processing ISP command including an initial control parameter and an image size parameter from an upper application; the connection layer determines a processing mode of an image signal based on the image size parameter, and determines target control parameters for different processing modes based on the initial control parameters; the drive layer transmitting the target control parameters to a target ISP core, thereby driving the target ISP core to perform image signal processing based on the target control parameters; The processing mode includes a single-core mode or a multi-core mode.

1. An image signal processing driving method comprising:

2. the processing mode is a multi-core mode, The step of determining a target control parameter based on the initial control parameter includes: a step of constructing a plurality of control parameters based on the initial control parameters to be the target control parameters.

2. The image signal processing driving method according to claim 1.

3. The ISP instruction includes a target function to be called, and the step of constructing a plurality of control parameters based on the initial control parameters as the target control parameters comprises: calling a membership function pointed to by a function pointer provided in the multi-core mode so as to construct the initial control parameters into a plurality of control parameters to become target control parameters; The membership function corresponds to the objective function.

3. The image signal processing driving method according to claim 2.

4. The target function includes one or more of a function module open function, a function module close function, and a driving interface function.

4. The image signal processing driving method according to claim 3.

5. the ISP command indicates a target image processing function to be realized, the target control parameters include size-related control parameters, the size-related control parameters include one or more of an image width, a parameter related to the image width, an image height, and a parameter related to the image height, the target ISP core performs image signal processing to realize the target image processing function, and a step of constructing a plurality of control parameters based on the initial control parameters to become the target control parameters includes: If the target image processing function is a default function, constructing a plurality of different control parameters as the target control parameters; If the target image processing function is not the default function, constructing a plurality of identical control parameters as the target control parameters; The predetermined function is realized by the target ISP core performing image signal processing based on the size-related control parameters.

3. The image signal processing driving method according to claim 2.

6. the ISP command indicates a target image processing function to be realized, the target control parameters include size-related control parameters, the size-related control parameters include one or more of an image width, a parameter related to the image width, an image height, and a parameter related to the image height, the target ISP core performs image signal processing to realize the target image processing function, and a step of constructing a plurality of control parameters based on the initial control parameters to become the target control parameters includes: If the target image processing function is a default function, constructing a plurality of different control parameters as the target control parameters; If the target image processing function is not the default function, constructing a plurality of identical control parameters as the target control parameters; The predetermined function is realized by the target ISP core performing image signal processing based on the size-related control parameters.

4. The image signal processing driving method according to claim 3.

7. The step of constructing a plurality of different control parameters as the target control parameter includes: and a step of setting a plurality of different control parameters calculated based on a predetermined image position as the target control parameters.

6. The image signal processing driving method according to claim 5.

8. The step of constructing a plurality of different control parameters as the target control parameter includes: and a step of setting a plurality of different control parameters calculated based on a predetermined image position as the target control parameters.

7. The image signal processing driving method according to claim 6.

9. In a multi-core mode, the image signal processing driving method includes: The connection layer further includes a step of responding to an interrupt signal of the image signal processing corresponding to the target ISP core, and after receiving all the interrupt signals, synchronously merging all the interrupt signals and sending them to the interface layer.

2. The image signal processing driving method according to claim 1.

10. The image signal processing driving method includes: after receiving a corresponding interrupt signal from the target ISP core, storing the interrupt signal in a corresponding queue; After receiving all the interrupt signals, synchronously merging all the interrupt signals and sending them to the interface layer, When the interrupt signals are stored in all the queues, the interrupt signals in each queue are read out, synchronized, merged, and transmitted to the interface layer.

10. The image signal processing driving method according to claim 9.

11. When the target ISP core is a plurality of cores, the all-of-the-interrupt signals include interrupt signals corresponding to all of the target ISP cores, and the corresponding queues are queues corresponding to the target ISP cores that transmit the interrupt signals; If the target ISP core supports time division multiplexing, the all interrupt signals include interrupt signals sent by the target ISP core in all multiple time segments, and the corresponding queues are queues corresponding to the multiple time segments in which the interrupt signals are sent.

11. The image signal processing driving method according to claim 10.

12. The interrupt signal includes a plurality of types, and the step of storing the interrupt signal in a corresponding queue includes: storing the interrupt signal in the queue at a target position corresponding to the target type based on the target type to which the interrupt signal belongs; The fact that all cues have a stop signal stored in them storing an interrupt signal at a target position in all of said queues; 11. The image signal processing driving method according to claim 10.

13. the processing mode is a single-core mode, The step of determining a target control parameter based on the initial control parameter includes: determining the initial control parameters as the target control parameters.

2. The image signal processing driving method according to claim 1.

14. The image signal processing method according to any one of claims 1 to 13 is configured to be connected to a plurality of ISP cores or an ISP core that supports time division multiplexing, and to implement each step of the image signal processing method according to any one of claims 1 to 13.

1. An image processing control module comprising:

15. 15. An image processing control module according to claim 14, comprising a plurality of ISP cores or an ISP core supporting time division multiplexing, wherein the image processing control module and the plurality of ISP cores or the ISP core supporting time division multiplexing are connected to each other. An image signal processing module comprising:

16. The image signal processing module is an IP core, chip or electronic device for image signal processing.

16. The image signal processing module according to claim 15.

17. A computer-readable storage medium storing computer commands, which, when executed by a processor, implement the steps of the image signal processing driving method according to any one of claims 1 to 13. A computer-readable storage medium comprising:

Images