Image signal processor and image processing system performing interrupt control

Abstract

An image signal processor includes a command queue circuit, an image processing engine, and an interrupt control circuit. The command queue circuit receives and stores a plurality of commands from a control processor, each containing an interrupt control value corresponding to a plurality of image units, and provides the plurality of commands one by one in sequence. The image processing engine receives image data including the plurality of image units and performs image processing on the plurality of image units sequentially based on the plurality of commands provided sequentially from the command queue circuit. The interrupt control circuit receives the interrupt control value from the command queue circuit, determines one or more output interrupt event signals among a plurality of interrupt event signals based on the interrupt control value, and generates an interrupt signal based on the output interrupt event signals.

Description

Technology Field

[0001] The present invention relates to a semiconductor integrated circuit, and more specifically, to an image signal processor and an image processing system that perform interrupt control. Background Technology

[0002] An image signal processor equipped in an image capturing device, such as a camera or smartphone, can perform image processing such as changing the data format of image data provided from an image sensor to a data format like RGB or YUV, removing noise from the image data, or adjusting brightness. The image signal processor can process image data output from the image sensor on a frame-by-frame basis. Recently, as image capturing devices support operation modes that provide images at high frame rates, such as slow motion mode and super slow motion mode, image sensors can generate and output image data at high frame rates. Therefore, an image signal processor capable of efficiently processing image data at high frame rates is required. The problem to be solved

[0003] One objective of the present invention to solve the above-mentioned problems is to provide an image signal processor capable of efficiently performing interrupt control.

[0004] In addition, one objective of the present invention is to provide an image processing system comprising an image signal processor capable of efficiently performing interrupt control. means of solving the problem

[0005] To achieve the above objective, an image signal processor according to embodiments of the present invention comprises a command queue circuit, an image processing engine, and an interrupt control circuit.

[0006] The command queue circuit receives and stores a plurality of commands from a control processor, each containing an interrupt control value corresponding to each of the plurality of image units, and provides the plurality of commands one by one in sequence.

[0007] The image processing engine receives image data including the plurality of image units and sequentially performs image processing on the plurality of image units based on the plurality of commands sequentially provided from the command queue circuit.

[0008] The interrupt control circuit receives the interrupt control value from the command queue circuit, determines one or more output interrupt event signals among a plurality of interrupt event signals based on the interrupt control value, and generates an interrupt signal based on the output interrupt event signals.

[0009] To achieve the above objective, an image processing system according to embodiments of the present invention comprises a control processor that generates a plurality of commands including interrupt control values ​​corresponding to each of a plurality of image units and executes an interrupt service routine based on an interrupt signal, and an image signal processor that performs image processing for the plurality of image units and generates the interrupt signal.

[0010] The image signal processor comprises: a command queue circuit that receives and stores the plurality of commands from a control processor and provides the plurality of commands one by one sequentially; an image processing engine that receives image data including a plurality of image units corresponding to each of the plurality of commands and performs image processing on the plurality of image units sequentially based on the plurality of commands provided sequentially from the command queue circuit; and an interrupt control circuit that receives the interrupt control value from the command queue circuit, determines one or more output interrupt event signals among a plurality of interrupt event signals based on the interrupt control value, and generates the interrupt signal based on the output interrupt event signals.

[0011] To achieve the above objective, an image signal processor according to embodiments of the present invention comprises a command queue circuit, an image processing engine, and an interrupt control circuit.

[0012] The command queue circuit receives and stores a plurality of commands from a control processor, each including an interrupt control value and an image unit identifier corresponding to each of the plurality of image units, and provides the plurality of commands one by one in sequence.

[0013] The image processing engine receives image data including the plurality of image units and sequentially performs image processing on the plurality of image units based on the plurality of commands sequentially provided from the command queue circuit.

[0014] The interrupt control circuit receives the interrupt control value and the image unit identifier from the command queue circuit, determines one or more output interrupt event signals and one or more log interrupt event signals among a plurality of interrupt event signals based on the interrupt control value, generates an interrupt signal based on the output interrupt event signals, and stores interrupt log information including whether the log interrupt event signals corresponding to each of the plurality of image units are activated and the image unit identifier corresponding to each of the image units based on the log interrupt event signals. Effects of the invention

[0015] An image signal processor and an image processing system according to embodiments of the present invention can efficiently perform communication between an image signal processor and a control processor and improve the operating speed and operating efficiency of the image signal processor and the image processing system including the same by pre-storing a plurality of commands corresponding to a plurality of image units in a command queue circuit and performing image processing based on the stored commands.

[0016] In addition, the image signal processor and image processing system according to the embodiments of the present invention can improve the operating speed and operating efficiency of the image signal processor and the image processing system including the same by determining the conditions for generating an interrupt signal for each image unit where image processing is performed based on an interrupt control value included in a command.

[0017] In addition, the image signal processor and image processing system according to embodiments of the present invention can efficiently perform a recovery procedure in the event of an error by determining log interrupt event signals to be monitored based on an interrupt control value included in a command and storing interrupt log information to include an image unit identifier included in the command. Brief explanation of the drawing

[0018] FIG. 1 is a block diagram showing a system according to embodiments of the present invention. FIG. 2 is a block diagram showing an image signal processor according to embodiments of the present invention. FIG. 3 is a flowchart illustrating an interrupt control method of an image signal processor according to embodiments of the present invention. FIG. 4 is a diagram showing one embodiment of a command received by an image signal processor according to embodiments of the present invention. FIG. 5 is a drawing for explaining an image unit of an image signal processor according to embodiments of the present invention. FIG. 6 is a block diagram showing an example of an interrupt control circuit included in an image signal processor according to embodiments of the present invention. FIG. 7 is a diagram showing an example of an output interrupt control circuit included in the interrupt control circuit of FIG. 6. FIG. 8 is a diagram showing an example of a log interrupt control circuit included in the interrupt control circuit of FIG. 6. FIG. 9 is a timing diagram showing an example of the operation of an image processing system according to embodiments of the present invention. Figure 10 is a diagram showing interrupt log information according to the operation of Figure 9. FIG. 11 is a timing diagram showing an example of the operation of an image processing system according to embodiments of the present invention. Figure 12 is a diagram showing interrupt log information according to the operation of Figure 11. FIG. 13 is a diagram showing an example of a command queue circuit included in the interrupt control circuit of FIG. 6. FIG. 14 is a timing diagram showing an example of the operation of an image processing system according to embodiments of the present invention. FIG. 15 is a diagram showing interrupt log information according to the operation of FIG. 14. FIG. 16 is a block diagram showing an image signal processor according to embodiments of the present invention. FIGS. 17 and 18 are block diagrams illustrating an image processing system according to embodiments of the present invention. FIG. 19 is a block diagram showing a system according to embodiments of the present invention. Specific details for implementing the invention

[0019] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. Identical components in the drawings are given the same reference numerals, and redundant descriptions of identical components are omitted.

[0020] FIG. 1 is a block diagram showing a system according to embodiments of the present invention.

[0021] The system (1000) of FIG. 1 may be implemented as an electronic device that captures an image, displays the captured image, or performs operations based on the captured image. The system (1000) may be implemented, for example, as a PC (personal computer), an IoT (Internet of Things) device, or a portable electronic device. Portable electronic devices may include a laptop computer, a mobile phone, a smartphone, a tablet PC, a PDA (personal digital assistant), an EDA (enterprise digital assistant), a digital still camera, a digital video camera, an audio device, a PMP (portable multimedia player), a PND (personal navigation device), an MP3 player, a handheld game console, an e-book, a wearable device, etc. Additionally, the system (1000) may be mounted on electronic devices such as drones or Advanced Driver Assistance Systems (ADAS), or on electronic devices equipped as components in vehicles, furniture, manufacturing equipment, doors, various measuring instruments, etc.

[0022] Referring to FIG. 1, the system (1000) may include an image sensor (1100) and an image processing system (1200). The system (1000) may further include other components such as a display and a user interface. The image processing system (1200) may include an image signal processor (100), a control processor (200), and a memory device (300). The image signal processor (100), the control processor (200), and the memory device (300) may be implemented as a single or multiple semiconductor chips. For example, the image signal processor (100) and the control processor (200) may be integrated into a single semiconductor chip.

[0023] The image sensor (1100) can convert an optical signal of an object incident through an optical lens (LS) into an electrical signal and generate and output image data (IDT) based on the electrical signals. The image sensor (1100) may include, for example, a pixel array comprising a plurality of pixels arranged in two dimensions and a readout circuit, and the pixel array can convert the received optical signals into electrical signals. The pixel array may be implemented as a photoelectric conversion device such as a CCD (Charge Coupled Devices) or CMOS (Complementary Metal Oxide Semiconductor), for example, and may also be implemented as various other types of photoelectric conversion devices. The readout circuit generates raw data based on the electrical signal provided from the pixel array and can output the raw data, or the raw data after preprocessing such as removing bad pixels, as image data (IDT). The image sensor (1100) may be implemented as a semiconductor chip or package including a pixel array and a readout circuit.

[0024] The image signal processor (100) can perform image processing on image data (IDT) provided from the image sensor (1100). For example, the image signal processor (100) may include image processing for image data (IDT) that changes the data format (e.g., changing Bayer pattern image data to YUV or RGB format), image processing for image quality improvement such as noise removal, brightness adjustment, and sharpness adjustment. The image signal processor (100) can configure the hardware of the image processing system (1200).

[0025] The image signal processor (100) may include an image signal processing core (110) (ISP core), a command queue circuit (CQC) (400), and an interrupt control circuit (ICC) (500).

[0026] The ISP core (110) may be referred to as an image processing engine. The image processing engine (110) can perform image processing on image data (IDT) output from an image sensor (1100) in units of image units. As described below with reference to FIG. 5, an image unit may be an image frame, an image strip, or an image tile. Hereinafter, embodiments of the present invention will be described with an image frame among the image units, but it will be understood that embodiments of the present invention may also be applied to any image unit other than an image frame. An image frame may be briefly referred to as a frame.

[0027] Processing data (PDT) generated by image processing, such as an image-processed frame (hereinafter, converted image data) and / or result data generated according to image processing (statistical data, histogram, etc.) can be stored in a memory device (300).

[0028] The command queue circuit (400) receives and stores a plurality of commands (CMDs) including interrupt control values ​​corresponding to each of the plurality of image units from the control processor (200), and can provide the plurality of commands one by one in sequence. In one embodiment, as described below with reference to FIG. 13, the command queue circuit (400) can determine the order of providing commands according to the priority of the commands. The image processing engine (110) receives image data (IDT) including the plurality of image units and can perform image processing on the plurality of image units sequentially based on the plurality of commands provided sequentially from the command queue circuit (400).

[0029] In this way, an image signal processor (100) and an image processing system (1200) including the image signal processor (100) according to embodiments of the present invention can efficiently perform communication between the image signal processor (100) and the control processor (200) and improve the operating speed and operating efficiency of the image processing system (1200) including the image signal processor (100) and the image signal processor (100) by storing a plurality of commands corresponding to a plurality of image units in a command queue circuit (400) and performing image processing based on the stored commands.

[0030] The interrupt control circuit (500) receives the interrupt control value from the command queue circuit (400) and can determine one or more output interrupt event signals among a plurality of interrupt event signals based on the interrupt control value. The interrupt control circuit (500) can generate an interrupt signal (INT) based on the output interrupt event signals. Embodiments of the configuration and operation of the interrupt control circuit (500) will be described in more detail later with reference to FIGS. 6 to 14.

[0031] The control processor (200) can control the image signal processor (100) to perform image processing. The control processor (200) can configure the software of the image processing system (1200). The control processor (200) may be a CPU (Central Processing Unit), a microprocessor, an ARM processor, an X86 processor, a MIPS (Microprocessor without Interlocked Pipeline Stages) processor, a graphics processing unit (GPU), a general-purpose GPU, or any other processor configured to execute program instructions stored in memory. The control processor (200) can generate a control signal (CONS) that controls the control processor (200) by processing or executing instruction code (or programs) and data that include the execution algorithm of the image signal processor (100).

[0032] The control signal (CONS) may be considered to include the command (CMD) or may be considered as a signal distinct from the command (CMD). In addition to the command (CMD), the control signal (CONS) may include various information or signals for controlling image quality for image processing, the address of the memory device (300) where processing data (PDT) is stored, setting values ​​for the operation of the image signal processor (100), operation timing, etc. Some information, for example, information regarding image quality and / or storage address, may be included in the command (CMD).

[0033] The control processor (200) transmits a corresponding command (CMD) to the image signal processor (100) before the image unit included in the image data (IDT) is input to the image processing engine (110), and the command queue circuit (400) can store the received command (CMD). The control processor (200) can execute an interrupt service routine (ISR) based on an interrupt signal (INT) received from the image signal processor (100).

[0034] The memory device (300) stores processing data (PDT) received from the image signal processor (100) and can provide the processing data (PDT) to the image signal processor (100), the control processor (200), or other components of the system (1000).

[0035] The memory device (300) may be implemented as volatile memory or non-volatile memory. Volatile memory may include DRAM (Dynamic Random Access Memory), SRAM (Static RAM), etc. Volatile memory may include ROM (Read Only Memory), PROM (Programmable ROM), EPROM (Electrically Programmable ROM), EEPROM (Electrically Erasable and Programmable ROM), flash memory, PRAM (Phase-change RAM), MRAM (Magnetic RAM), RRAM (Resistive RAM), FRAM (Ferroelectric RAM), etc.

[0036] FIG. 2 is a block diagram showing an image signal processor according to embodiments of the present invention. FIG. 2 shows the image signal processor (100) of FIG. 1 in detail, and FIG. 2 also shows a control processor (200) and a memory device (300).

[0037] Referring to FIG. 2, the image signal processor (100) may include an image processing engine (110), a command queue circuit (400), an interrupt control circuit (500), and a DMA controller (130). The image processing engine (110) may include a controller (111) and a plurality of IP (intellectual property) blocks (112). In FIG. 2, the plurality of IP blocks (112) are shown to include first to third IP blocks (11, 12, 13), but are not limited thereto, and the plurality of IP blocks (112) may include two or more IP blocks.

[0038] The controller (111) receives a control signal (CONS) from the control processor (200) and can control the overall operation of the image signal processor (100) based on the control signal (CONS). As described above, the control signal (CONS) may include a command (CMD) and various setting information, and the controller (111) may provide the setting information to a plurality of IP blocks (112) or a DMA controller (130) when image processing is performed for each image unit (e.g., image frame).

[0039] Each of the plurality of IP blocks (112), namely the first to third IP blocks (11, 12, 13), can perform image processing set as an image processing block, and the first to third IP blocks (11, 12, 13) can perform different image processing. As a non-limiting example, the first IP block (11) can change the data format of image data (IDT), the second IP block (12) can adjust brightness, and the third IP block (13) can adjust contrast. The first to third IP blocks (11, 12, 13) can perform image processing sequentially on a frame. The result data according to the image processing of each of the first to third IP blocks (11, 12, 13) and / or the frame for which image processing is completed, e.g., the converted image data, can be stored in a memory device (300). A plurality of IP blocks (112) receive a setting value corresponding to a frame being processed from a controller (111) and / or a command queue circuit (400), and can perform image processing on the frame based on the setting value.

[0040] The DMA controller (130) can store processing data (PDT) received from at least one IP block among the first to third IP blocks (11, 12, 13) in the memory device (300). The processing data (PDT) may include result data and / or converted image data resulting from image processing. At this time, the DMA controller (130) can receive an address (or address register value) from the controller (111) and / or the command queue circuit (400) and store the processing data (PDT) in a storage area corresponding to the address on the memory device (300).

[0041] The command queue circuit (400) receives and stores a plurality of commands (CMD) including interrupt control values ​​corresponding to each of the plurality of image units from the control processor (200), and can provide the plurality of commands one by one in sequence. The image processing engine (110) receives image data (IDT) including the plurality of image units and can perform image processing on the plurality of image units sequentially based on the plurality of commands provided sequentially from the command queue circuit (400).

[0042] The interrupt control circuit (500) can receive an interrupt control value and an image unit identifier corresponding to each image unit from the command queue circuit (400).

[0043] For example, when image processing is performed for a first frame, the command queue circuit (400) can extract an interrupt control value and an image unit identifier included in a first command corresponding to the first frame and provide them to the interrupt control circuit (500). Subsequently, when image processing is performed for a second frame, the command queue circuit (400) can extract an interrupt control value and an image unit identifier included in a second command corresponding to the second frame and provide them to the interrupt control circuit (500).

[0044] The interrupt control circuit (500) can determine one or more output interrupt event signals among a plurality of interrupt event signals based on the interrupt control value. The interrupt control circuit (500) can generate an interrupt signal (INT) based on the output interrupt event signals. In addition, the interrupt control circuit (500) can generate and store interrupt log information based on the interrupt control value and the image unit identifier. Embodiments of the configuration and operation of the interrupt control circuit (500) will be described in more detail later with reference to FIGS. 6 to 14.

[0045] FIG. 3 is a flowchart illustrating an interrupt control method of an image signal processor according to embodiments of the present invention.

[0046] Referring to FIGS. 1 to 3, a command queue circuit (400) included in an image signal processor (100) can receive and store a plurality of commands (CMD) including interrupt control values ​​corresponding to each of a plurality of image units from a control processor (200) (S100).

[0047] The image processing engine (110) included in the image signal processor (100) can sequentially perform image processing on the plurality of image units based on a plurality of commands (CMD) provided sequentially from the command queue circuit (400) (S200).

[0048] The interrupt control circuit (500) can determine one or more output interrupt event signals and one or more log interrupt event signals among a plurality of interrupt event signals based on the interrupt control value (S300).

[0049] The interrupt control circuit (500) can generate an interrupt signal (INT) based on the output interrupt event signals (S400).

[0050] Additionally, the interrupt control circuit (500) can store interrupt log information regarding whether the log interrupt event signals are enabled (S500).

[0051] Examples of the method for determining the output interrupt event signals and the log interrupt event signals, the method for generating the interrupt signal (INT), and the method for storing the interrupt log information are described below with reference to FIGS. 6 to 14.

[0052] FIG. 4 is a diagram showing one embodiment of a command received by an image signal processor according to embodiments of the present invention.

[0053] Referring to FIG. 4, a command (CMD) transmitted from a control processor (200) to an image signal processor (100) may include at least an interrupt control value (ICV) and an image unit identifier (UID). The interrupt control value (ICV) and the image unit identifier (UID) may each include a plurality of bits.

[0054] FIG. 4 illustrates only information for explaining embodiments of the present invention. A command (CMD) may include various information regarding a corresponding image unit. A control processor (200) may transmit a command (CMD) to an image signal processor (100) in the form of a packet containing a plurality of fields. The image signal processor (100) may parse the received packet-form command (CMD) and store the processed form of the command in a command queue circuit (400).

[0055] FIG. 5 is a drawing for explaining an image unit of an image signal processor according to embodiments of the present invention.

[0056] Referring to FIG. 5, an image frame (IFM) may include multiple pixel data arranged in the form of a matrix of multiple rows and multiple columns. The image sensor (1100) of FIG. 1 may provide image data (IDT) to an image signal processor (100) in units of image frames (IFM).

[0057] The image signal processor (100) can perform image processing on an image frame (IFM) in units of an image frame (IFM), an image strip (ISTR), or an image tile (ITL). That is, the "image unit" referred to in the present disclosure may correspond to an image frame (IFR), an image strip (ISTR), or an image tile (ITL) as illustrated in FIG. 5.

[0058] Although not illustrated in FIG. 1, the image signal processor (100) may include a frame buffer for buffering an image frame (IFM) and may sequentially process a plurality of image units provided sequentially from the frame buffer.

[0059] FIG. 6 is a block diagram showing an example of an interrupt control circuit included in an image signal processor according to embodiments of the present invention.

[0060] Referring to FIG. 6, the interrupt control circuit (500) may include a decision bit generator (600), an output interrupt control circuit (OICC) (700), and a log interrupt control circuit (LICC) (800).

[0061] The decision bit generator (600) may include an output decision bit generator (ODBG) (610) and a log decision bit generator (LDBG) (620).

[0062] The output determination bit generator (610) can generate a plurality of output determination bits (ODB) for determining output interrupt event signals among a plurality of interrupt event signals (IEV) based on an interrupt control value (ICV).

[0063] The log decision bit generator (620) can generate a plurality of log decision bits (LDB) for determining log interrupt event signals among a plurality of interrupt event signals (IEV) based on an interrupt control value (ICV).

[0064] For example, the interrupt control value (ICV) included in each command (CMD) may include p bits, and there may be n interrupt event signals (IEV). The output decision bit generator (610) may generate n-bit output decision bits (ODB) based on q bits among the p bits of the interrupt control value (ICV). The log decision bit generator (620) may generate n-bit log decision bits (LDB) based on r bits among the p bits of the interrupt control value (ICV). The n-bit output decision bits (ODB) and the n-bit log decision bits (LDB) may correspond one-to-one with the n interrupt event signals (IEV). Each bit value (0 or 1) of the output decision bits (ODB) may indicate whether the corresponding interrupt event signal is an output interrupt event signal or not, and each bit value (0 or 1) of the log decision bits (LDB) may indicate whether the corresponding interrupt event signal is a log interrupt event signal or not.

[0065] In one embodiment, the q bits and the r bits may be bits obtained by dividing the p bits of the interrupt control value (ICV) into two groups, i.e., p may be equal to q+r. In another embodiment, some of the q bits and the r bits may overlap, i.e., q+r may be greater than p.

[0066] In one embodiment, the log decision bit generator (620) can generate a plurality of log decision bits (LDB) based on an external interrupt control value (EICV) comprising r bits and s bits of an interrupt control value (ICV). The external interrupt control value (EICV) may be a value provided in real time from the control processor (200) and not included in the command (CMD).

[0067] As described above, each command (CMD) includes an interrupt control value (ICV) and an image unit identifier (UID) corresponding to the image unit. Accordingly, the decision bit generator (600) can generate a plurality of output decision bits (ODB) and a plurality of log decision bits (LDB) for each image unit where image processing is performed, based on the interrupt control value (ICV) included in each command. In other words, the decision bit generator (600) can determine output interrupt event signals and log interrupt event signals for each image unit where image processing is performed, based on the interrupt control value (ICV) included in each command.

[0068] In one embodiment, the log decision bit generator (620) may receive at least some of a plurality of interrupt event signals (IEV). The log decision bit generator (620) may change the values ​​of the log decision bits (LDB) while image processing for the corresponding image is performed based on the activation of the received interrupt event signals.

[0069] As described below with reference to FIG. 7, an output interrupt control circuit (700) can generate an interrupt signal (INT) based on a plurality of interrupt event signals (IEV) and a plurality of output decision bits (ODB). As described below with reference to FIG. 8, a log interrupt control circuit (800) can store interrupt log information (ILINF) indicating whether log interrupt event signals are enabled based on a plurality of interrupt event signals (IEV) and a plurality of log decision bits (LDB). In one embodiment, a log decision bit generator (620) can control the storage of interrupt log information (ILINF) based on the activation of received interrupt event signals.

[0070] FIG. 7 is a diagram showing an example of an output interrupt control circuit included in the interrupt control circuit of FIG. 6.

[0071] Referring to FIG. 7, the output interrupt control circuit (700) may include an output interrupt accumulation circuit (720) and an interrupt generator (740).

[0072] The output interrupt accumulation circuit (720) can generate a plurality of output status bits, namely the first to nth output status bits (B0 to Bn-1), indicating whether the output interrupt event signals are enabled, based on a plurality of interrupt event signals (IEV), namely the first to nth interrupt event signals (IEV0 to IEVn-1) and a plurality of output determination bits (ODB), namely the first to nth output determination bits (A0 to An-1). The interrupt generator (740) can generate an interrupt signal (INT) based on the first to nth output status bits (B0 to Bn-1).

[0073] The output interrupt accumulation circuit (720) may include an output determination register circuit (721), an output comparison circuit (722), and an output status register circuit (723).

[0074] The output determination register circuit (721) can store the first to nth output determination bits (A0 to An-1) provided by the determination bit generator (600) of FIG. 6. The output comparison circuit (722) can generate the first to nth output status bits (B0 to Bn-1) by comparing each of the first to nth output determination bits (A0 to An-1) stored in the output determination register circuit (721) with each of the first to nth interrupt event signals (IEV0 to IEVn-1). The output status register circuit (723) can store the first to nth output status bits (B0 to Bn-1). The output status register circuit (723) can be initialized based on an external reset signal (ECLR) provided by the control processor (200).

[0075] In one embodiment, the interrupt generator (740) may receive an interrupt enable value from the control processor (200). For example, the interrupt enable value may correspond to a plurality of interrupt enable bits (IEB), namely the first to nth interrupt enable bits (C0 to Cn-1).

[0076] The interrupt generator (740) can determine one or more enable interrupt event signals among the output interrupt event signals based on the first to nth interrupt enable bits (C0 to Cn-1) and generate an interrupt signal (INT) based on the enable interrupt event signals.

[0077] In one embodiment, the interrupt generator (740) may include an enable register circuit (741) and an enable comparison circuit (742).

[0078] The enable register circuit (741) can store the first to nth interrupt enable bits (C0 to Cn-1) provided by the control processor (200). The enable comparison circuit (742) can generate an interrupt signal (INT) by comparing each of the first to nth output status bits (B0 to Bn-1) and each of the first to nth interrupt enable bits (C0 to Cn-1).

[0079] As illustrated in FIG. 7, in one embodiment, the output comparison circuit (722) may be implemented with n AND gates. In this case, among the first to nth output decision bits (A0 to An-1), the output decision bits corresponding to the output interrupt event signals may be set to a value of 1, and the remaining bits may be set to a value of 0. An interrupt event signal corresponding to an output decision bit of a value of 1 may correspond to an output interrupt event signal. An AND gate corresponding to an output decision bit of a value of 0 outputs a value of 0 regardless of the logic level (i.e., whether it is enabled) of the corresponding interrupt event signal. That is, an AND gate corresponding to a value of 0 can perform the role of masking the corresponding interrupt event signal. In this case, the first to nth output status bits (B0 to Bn-1) may all be initialized to a value of 0 when an external reset signal (ECLR) is enabled.

[0080] Meanwhile, as illustrated in FIG. 7, in one embodiment, the enable comparison circuit (742) can be implemented with n AND gates and one OR gate. In this case, among the first to nth interrupt enable bits (C0 to Cn-1), the output determination bits corresponding to the enable interrupt event signals can be set to a value of 1, and the remaining bits can be set to a value of 0. In this case, the AND gate corresponding to the interrupt enable bit with a value of 0 outputs a value of 0 regardless of the value of the corresponding output status bit. That is, the AND gate corresponding to the value of 0 can perform the role of masking the corresponding interrupt event signal.

[0081] Consequently, the output interrupt control circuit (700) of FIG. 7 can enable an interrupt signal (INT) when one or more of the enable interrupt event signals are enabled, regardless of whether the remaining interrupt event signals, excluding the enable interrupt event signals among the first to nth interrupt event signals (IEV0~IEVn-1), are enabled.

[0082] In one embodiment, the control processor (200) may not provide a plurality of interrupt enable bits (C0 to Cn-1), and the enable register circuit (741) may be omitted. In this case, AND gates included in the enable comparison circuit (742) may be omitted, and a OR gate may generate an interrupt signal (INT) by performing a OR operation on a plurality of output status bits (B0 to Bn-1).

[0083] FIG. 8 is a diagram showing an example of a log interrupt control circuit included in the interrupt control circuit of FIG. 6.

[0084] Referring to FIG. 8, the log interrupt control circuit (800) may include a log interrupt accumulation circuit (820) and a log storage unit (840).

[0085] The log interrupt accumulation circuit (820) can generate a plurality of log status bits, namely the first to nth log status bits (F0 to Fn-1), indicating whether the log interrupt event signals are enabled, based on a plurality of interrupt event signals, namely the first to nth interrupt event signals (IEV0 to IEV3) and a plurality of log determination bits, namely the first to nth log determination bits (E0 to En-1). The log storage unit (840) can store interrupt log information (ILINF) including the first log status bits (F0 to Fn-1). As described below, the log storage unit (840) receives an image unit identifier (UID) included in each command and can store interrupt log information (ILINF) to include an image unit identifier (UID) corresponding to the image unit.

[0086] The log storage unit (840) can store a set of first to nth log state bits (F0 to Fn-1) provided from the log interrupt accumulation circuit (820) and an image unit identifier (UID) provided from the command queue circuit (400) as one unit interrupt information when the storage enable signal (STEN) is activated. In one embodiment, the storage enable signal (STEN) can be generated by the decision bit generator (600) of FIG. 6.

[0087] The log interrupt accumulation circuit (820) may include a log determination register circuit (821), a log comparison circuit (822), and a log status register circuit (823).

[0088] The log determination register circuit (821) can store the first to nth log determination bits (E0 to En-1) provided by the determination bit generator (600) of FIG. 6. The log comparison circuit (822) can generate the first to nth log status bits (F0 to Fn-1) by comparing each of the first to nth log determination bits (E0 to En-1) stored in the log determination register circuit (821) with each of the first to nth interrupt event signals (IEV0 to IEVn-1). The log status register circuit (823) can store the first to nth log status bits (F0 to Fn-1). The log status register circuit (823) can be initialized based on a reset signal (CLR). The reset signal (CLR) may be a signal generated internally within the image signal processor (100) and activated based on the timing at which image processing for each image unit is terminated.

[0089] As illustrated in FIG. 8, in one embodiment, the log comparison circuit (822) may be implemented with n AND gates. In this case, among the first to n log decision bits (E0 to En-1), the log decision bits corresponding to the log interrupt event signals may be set to a value of 1, and the remaining bits may be set to a value of 0. An interrupt event signal corresponding to an output decision bit of a value of 1 may correspond to an output interrupt event signal. An AND gate corresponding to a log decision bit of a value of 0 outputs a value of 0 regardless of the logic level (i.e., whether it is enabled) of the corresponding interrupt event signal. That is, an AND gate corresponding to a value of 0 may perform the role of masking the corresponding interrupt event signal. In this case, the first to n log state bits (F0 to Fn-1) may all be initialized to a value of 0 when the reset signal (CLR) is enabled.

[0090] FIG. 9 is a timing diagram showing an example of the operation of an image processing system according to embodiments of the present invention, and FIG. 10 is a diagram showing interrupt log information according to the operation of FIG. 9.

[0091] Hereinafter, for the convenience of illustration and explanation, embodiments of the present invention are described based on the first to fourth interrupt event signals (IEV0 to IEV3) among the first to nth interrupt event signals (IEV0 to IEVn-1).

[0092] For example, as illustrated in FIG. 10, the first interrupt event signal (IEV0) may be activated when a frame start event occurs, the second interrupt event signal (IEV1) may be activated when a frame end event occurs, the third interrupt event signal (IEV2) may be activated when a first error (ERR1) occurs, and the fourth interrupt event signal (IEV3) may be activated when a second error (ERR1) occurs.

[0093] Referring to FIG. 9, an image signal processor (ISP) can sequentially perform image processing for the first to fourth frames (IFM0 to IFM3) in each of the first to fourth processing intervals (TP0 to TP3) between time points (t1 to t8).

[0094] As described above, the first to fourth commands (CMD0~CMD3) corresponding to the first to fourth frames (IFM0~IFM3), respectively, can be received from the control processor (200) as a control signal (CONS) and stored in the command queue circuit (400) before image processing for the corresponding frame is performed.

[0095] FIG. 9 illustrates examples of values ​​of output decision bits (ODB), log decision bits (LDB), and image unit identifiers (UID) corresponding to the first to fourth frames (IFM0 to IFM3), respectively.

[0096] As described, frame start events (INT_S0~INT_S3) and frame end events (INT_E0~INT_E3) may occur at each of the time points (t1~t8). Output interrupt event signals among the multiple interrupt event signals are determined according to the values ​​of the multiple output decision bits (ODB), and log output interrupt event signals may be determined according to the values ​​of the multiple log decision bits (LDB).

[0097] For example, in the first processing section (TP0), the values ​​of the first to fourth output determination bits corresponding to the first frame (IFM0), 'A3A2A1A0', can be set to '0101'. The first output determination bit (A0) corresponding to the frame start event and the third output determination bit (A2) corresponding to the first error (ERR1) occurrence event have a value of 1, and the second output determination bit (A1) corresponding to the frame end event and the fourth output determination bit (A3) corresponding to the second error (ERR2) occurrence event have a value of 0. Accordingly, in the first processing section (TP0), the first interrupt event signal (IEV0) and the third interrupt event signal (IEV2) correspond to output interrupt event signals. In Fig. 9, interrupt events that contribute to the activation of the interrupt signal (INT) are indicated by black arrows, and interrupt events that do not affect the activation of the interrupt signal (INT), i.e., are masked, are indicated by white arrows.

[0098] For example, while image processing for the third frame (IFM2) is being performed at time point (tr), a first error (ERR1) may occur and a first error event (INT_ERR1) may occur. Even if the interrupt signal (INT) is activated by interrupt events (INT_S0, INT_ERR1) at times points (t1, tr), a certain delay may occur depending on the operating state of the control processor (200), and the interrupt service routine may be executed at the delayed times points (t1', tr').

[0099] As shown in FIG. 9, at the points where the first to fourth processing intervals (TP0~TP3) end, a storage enable signal (STEN) is activated so that unit interrupt information corresponding to each image unit as shown in FIG. 10 can be stored in the log storage unit (840). After the unit interrupt information is stored, a reset signal (CLR) is activated so that the log status bits can be initialized to a value of 0.

[0100] FIG. 10 illustrates output status bits (B0~B3), log status bits (F0~F3), and first to third unit interrupt information (UINF0~UINF2) included in interrupt log information (ILINF) corresponding to the time point (tr') of FIG. 9. As described above, output status bits (B0~B3) can be stored in an output status register circuit (723), and log status bits (F0~F3) can be stored in a log status register circuit (823). The log storage unit (840) may include a plurality of storage units (51~53) that each store a plurality of unit interrupt information (UINF0~UINF2).

[0101] As illustrated in FIG. 10, each of the first to third unit interrupt information (UINF0~UINF2) includes the value of an image unit identifier (UID) and the values ​​of log status bits (F0~F3) corresponding to the first to third frames (IFM0~IFM2), respectively.

[0102] Since it is not possible to determine when the first error (ERR1) occurred based solely on the output status bits (B0~B3), the control processor (200) drops all of the first to third frames (IFM0~IFM2). On the other hand, the control processor (200) can determine that the first error (ERR1) occurred during the processing of the third frame (IFM2) by referring to the stored interrupt log information (ILINF), and therefore the control processor (200) can drop only the third frame (IFM2).

[0103] In this way, by determining a plurality of output determination bits, for example, the first and second output determination bits (A0 to A3), in the first to fourth processing intervals (TP0 to TP3) for each frame, various operation modes including a fast readout (FRO) mode that requires high-speed image processing such as 120fps, 240fps, and 480fps can be efficiently implemented.

[0104] In conventional cases, FRO mode was implemented so that the control processor software controlled interrupt generation in units of N frames, distinguishing it from general frame processing. In FRO mode, the image signal processor was equipped with a counter that increments for each frame processed, and using a fixed-pattern logic, only the frame start interrupt was allowed for the first frame of FRO mode, and only the frame end interrupt for the last frame.

[0105] In this case, due to the overhead of having to provide N sets of configuration registers, the said N sets of configuration registers were applied only to parts that absolutely require control for every frame (e.g., DMA buffer address register), and image quality parameters were processed by applying them identically to all frames in FRO mode. Furthermore, since switching between normal operation mode and FRO mode requires suspending hardware processing of the image signal processor and software control by the control processor, it is difficult to smoothly utilize the interrupt history or interrupt log functions.

[0106] Meanwhile, even when processing high frame rates such as general 60fps or 120fps, cases are emerging where significant variations in interrupt delay occur within the system, such as in the application processor (AP). With current implementation technology, at the moment when the control processor performing software control receives an interrupt and checks the hardware interrupt status of the image signal processor, it is only possible to check whether errors from accumulated past processing have occurred.

[0107] Without an interrupt history function, it is impossible to distinguish which frames have errors during accumulated frame processing while others are processed normally, resulting in the dropping of all unverifiable frames. Additionally, processors handling interrupts have a control bound on the maximum number of interrupts that can be processed per unit of time. In the field of image signal processors, the number of image sensors and frame rates requiring simultaneous processing are continuously increasing.

[0108] According to embodiments of the present invention, software control of the control processor (200) can be reserved in the command queue circuit (400) to efficiently control interrupt generation and recording. An FRO mode is integrated into general high frame rate processing, so that the activation frequency of the interrupt signal (INT) can be reduced even in general high frame rate processing without overhead due to mode switching. That is, the pattern of output decision bits (ODB) for interrupt reduction can be varied in all operating modes, including the FRO mode. In addition, interrupt recording is possible for specific processing intervals (image frames, image strips, or image tiles) in all operating modes without being limited to a specific operating mode, so that the frame drop rate can be reduced even in interrupt delay situations.

[0109] FIG. 11 is a timing diagram illustrating an embodiment of the operation of an image processing system according to embodiments of the present invention, and FIG. 12 is a diagram illustrating interrupt log information according to the operation of FIG. 11. Below, descriptions that overlap with FIG. 9 and 10 will be omitted, and only the differences will be explained.

[0110] In the embodiments of FIGS. 9 and 10, the storage enable signal (STEN) is activated at each end point (t2, t4, t6, t8) of the first to fourth processing intervals (TP0 to TP3). Accordingly, the log storage unit (840) of the interrupt control circuit (500) sequentially stores unit interrupt information for all of the plurality of image units, namely the first to fourth frames (IFM0 to IFM3).

[0111] On the other hand, in the embodiments of FIGS. 11 and 12, the storage enable signal (STEN) is activated only at the end time (t6) of the third processing section (TP2) where the first error (ERR1) occurs. Accordingly, the log storage unit (840) of the interrupt control circuit (500) stores only the unit interrupt information (UINF0) corresponding to the third frame (IFM2) as shown in FIG. 12. Thus, according to the embodiments, the interrupt control circuit (500) can determine whether to store the unit interrupt information corresponding to each image unit based on whether some of the log interrupt event signals among the log interrupt event signals are activated.

[0112] FIG. 13 is a diagram showing an embodiment of a command queue circuit included in the interrupt control circuit of FIG. 6, FIG. 14 is a timing diagram showing an embodiment of the operation of an image processing system according to embodiments of the present invention, and FIG. 15 is a diagram showing interrupt log information according to the operation of FIG. 14. Hereinafter, descriptions that overlap with FIG. 9 and 10 will be omitted, and only the differences will be explained.

[0113] Referring to FIG. 13, the command queue circuit (400) may include a first command queue (CQA) for storing commands of relatively lower priority, a second command queue (CQB) for storing commands of relatively higher priority, and an arbitration unit (410).

[0114] Referring to FIG. 14, an image signal processor (ISP) can perform image processing on the first to fourth frames (IFM0_A to IFM3_A) of a first stream having a relatively low priority and the frame (IFM0_B) of a second stream having a relatively high priority in each of the first to fifth processing intervals (TP0 to TP4) between time points (t1 to t10).

[0115] As described above, first to fourth commands (CMD0_A to CMD3_A) corresponding to each of the first to fourth frames (IFM0_A to IFM3_A) of the first stream can be received from the control processor (200) as a control signal (CONS) before image processing for the corresponding frame is performed, and can be stored in the first command queue (CQA) of the command queue circuit (400). The left part of FIG. 13 shows the state in which the first to fourth commands (CMD0_A to CMD3_A) are stored in the first command queue (CQA) of the command queue circuit (400) at time point (t1).

[0116] For example, at time point (ta), a fifth command (CMD0_B) corresponding to a frame (IFM0_B) of the second stream may be received from the control processor (200) as a control signal (CONS) and stored in the second command queue (CQB) of the command queue circuit (400). The right part of FIG. 13 shows a state in which, at time point (t5), the third and fourth commands (CMD2_A, CMD3_A) are stored in the first command queue (CQA) of the command queue circuit (400) and the fifth command (CMD0_B) is stored in the second command queue (CQB).

[0117] The arbitration unit (410) of the command queue circuit (400) can provide the fifth command (CMD0_B), which has a higher priority, before the third command (CMD2_A), which has a lower priority, based on priority. As a result, as shown in FIG. 14, the frame (IFM0_B) of the second stream is processed first in the third processing section (TP2), and then the third and fourth frames (IFM2_A, IFM3_A) of the first stream can be processed sequentially in the fourth and fifth processing sections (TP3, TP4).

[0118] As described, frame start events (INT_S0~INT_S4) and frame end events (INT_E0~INT_E4) may occur at each of the time points (t1~T10). Output interrupt event signals among the multiple interrupt event signals are determined according to the values ​​of the multiple output decision bits (ODB), and log output interrupt event signals may be determined according to the values ​​of the multiple log decision bits (LDB).

[0119] For example, in the third processing section (TP2), the values ​​of the first to fourth output determination bits 'A3A2A1A0' corresponding to the frame (IFM0_B) of the second stream can be set to '0111'. The first output determination bit (A0) corresponding to the frame start event, the second output determination bit (A1) corresponding to the frame end event, and the third output determination bit (A2) corresponding to the first error (ERR1) occurrence event have a value of 1, and the fourth output determination bit (A3) corresponding to the second error (ERR2) occurrence event has a value of 0. Accordingly, in the third processing section (TP2), the first to third interrupt event signals (IEV0~IEV2) correspond to output interrupt event signals. As explained with reference to FIG. 9, in FIG. 14, events that contribute to the activation of the interrupt signal (INT) among the interrupt events are indicated by black arrows, and interrupt events that do not affect the activation of the interrupt signal (INT), i.e., are masked, are indicated by white arrows.

[0120] For example, while image processing for the third frame (IFM2_A) of the first stream is being performed at time point (tr), a first error (ERR1) may occur and a first error event (INT_ERR1) may occur. Even if an interrupt signal (INT) is activated by interrupt events (INT_S2, INT_E2, INT_ERR1) at times points (t5, t6, tr), a certain delay may occur depending on the operating state of the control processor (200), and an interrupt service routine may be executed at the delayed time point (t5').

[0121] As shown in FIG. 14, at the points where the first to fifth processing intervals (TP0 to TP4) end, a storage enable signal (STEN) is activated so that unit interrupt information corresponding to each image unit as shown in FIG. 15 can be stored in the log storage unit (840). After the unit interrupt information is stored, a reset signal (CLR) is activated so that the log status bits can be initialized to a value of 0.

[0122] FIG. 15 illustrates first to fourth unit interrupt information (UINF0~UINF3) that correspond to the time point (t5') of FIG. 14 and are included in the interrupt log information (ILINF). The log storage unit (840) may include a plurality of storage units (51~54) that each store a plurality of unit interrupt information (UINF0~UINF3).

[0123] As illustrated in FIG. 15, each of the first to fourth unit interrupt information (UINF0~UINF3) includes the values ​​of an image unit identifier (UID) and log status bits (F0~F3) corresponding to the first to third frames (IFM0_A~IFM2_A) of the first stream and the frame (IFM0_B) of the second stream, respectively.

[0124] Since it is not possible to determine when the first error (ERR1) occurred solely from the output status bits (B0~B3) stored in the output status register circuit (723), the control processor (200) drops all of the first to third frames (IFM0_A~IFM2_A) of the first stream and the frame (IFM0_B) of the second stream. On the other hand, the control processor (200) can determine that the first error (ERR1) occurred during the processing of the third frame (IFM2_A) of the first stream by referring to the stored interrupt log information (ILINF), and therefore the control processor (200) can drop only the third frame (IFM2_A) of the first stream.

[0125] In this way, by determining a plurality of output determination bits, for example, the first and second output determination bits (A0 to A3), in the first to fourth processing intervals (TP0 to TP3) for each frame, various operation modes including a fast readout (FRO) mode that requires high-speed image processing such as 120fps, 240fps, and 480fps can be efficiently implemented.

[0126] In this way, according to embodiments of the present invention, the types of interrupts for activating an interrupt signal (INT) provided to a control processor (200) can be controlled by variably modifying the pattern of a plurality of output decision bits (ODB) without software control of the processor for urgent processing changes.

[0127] In addition, even when multi-stream processing is mixed, the interrupt history can be easily understood and limited storage space can be utilized efficiently.

[0128] FIG. 16 is a block diagram showing an image signal processor according to embodiments of the present invention.

[0129] The image signal processor (100c) of FIG. 16 may include an image processing engine (110c), a command queue circuit (400), an interrupt control circuit (500), a first DMA controller (140), and a second DMA controller (150). The image processing engine (110c) may include a controller (111c), a plurality of IP blocks (112c), and a post-processing block (113c).

[0130] The configuration and operation of the image processor (100c) of FIG. 16 are similar to the configuration and operation of the image processor (100) of FIG. 2. Therefore, the explanation will focus on the differences.

[0131] Referring to FIG. 16, a post-processing block (113c) provided in an image processing engine (110c) can perform post-processing on converted image data generated from a plurality of IP blocks (112c). For example, the post-processing block (113c) may include a scaler, a JPEG circuit, etc.

[0132] Multiple IP blocks (112c) can directly transmit the converted image data to the post-processing block (113c) or store it in the memory device (300) through the first DMA controller (140).

[0133] The post-processing block (113c) can receive converted image data from a plurality of IP blocks (112c) or receive converted image data stored in the memory device (300) through the second DMA controller (150). The post-processing block (113c) can store the post-processed image data (IDT') in the memory device (300) through the second DMA controller (150) or output it to other components provided in the system (1000 of FIG. 1), such as a display.

[0134] FIGS. 17 and 18 are block diagrams illustrating an image processing system according to embodiments of the present invention.

[0135] Referring to FIG. 17, the image processing system (20) may include a main processor (210), a ROM (Read Only Memory) (220), a RAM (230), an image signal processor (240), a non-volatile memory interface (250), a camera interface (260), a memory interface (270), and a display interface (280). The components of the image processing system (20), namely the main processor (210), ROM (220), RAM (230), image signal processor (240), non-volatile memory interface (250), camera interface (260), memory interface (270), and display interface (280), may transmit and receive data through a system bus (290). In an embodiment, the image processing system (20) may be implemented as a system-on-chip (SoC). In an embodiment, the image processing system (20) may be an application processor.

[0136] The main processor (210) can control the overall operation of the image processing system (20). The main processor (210) can be implemented, for example, as a CPU, microprocessor, ARM processor, X86 processor, MIPS processor, etc., and, depending on the embodiment, can be implemented as a computing component having two or more independent processors (or cores), i.e., a multi-core processor. The main processor (210) can process or execute instruction code and / or data stored in the ROM (220) or RAM (230).

[0137] The ROM (220) can store programs and / or data that are used continuously. The ROM can be implemented as an EPROM (erasable programmable ROM) or an EEPROM (electrically erasable programmable ROM), etc.

[0138] RAM (230) can temporarily store programs, data, and / or instructions. Depending on the embodiment, RAM (230) may be implemented as DRAM or SRAM. RAM (230) may temporarily store image data processed by an image signal processor (240) or input / output through interfaces (250, 260, 270, and 280).

[0139] The non-volatile memory interface (250) can interface data input from the non-volatile memory device (255) or data output to the non-volatile memory. The non-volatile memory device (255) can be implemented, for example, as a memory card (MMC (Multi Media Card), eMMC, SD (Secure Digital) card, micro SD card, etc.).

[0140] The camera interface (260) can interface with image data (e.g., raw image data) input from a camera (265) located outside the image processing system (20). The camera (265) can generate data for an image captured using a plurality of light-sensing elements. The image data received through the camera interface (260) can be provided to an image signal processor (240) or stored in a memory (275) through a memory interface (270).

[0141] The memory interface (270) can interface data input from the memory (275) or data output to the memory (275). According to an embodiment, the memory (275) may be implemented as a volatile memory such as DRAM or SRAM, or as a non-volatile memory such as ReRAM, PRAM, or NAND flash.

[0142] The display interface (280) can interface with data (e.g., image data) output to the display device (285). The display device (285) can output a video signal according to the image data through a display such as an LCD (Liquid-crystal display) or an AMOLED (active matrix organic light emitting diodes).

[0143] The image signal processor (240) can generate converted image data by performing image processing on image data provided from the camera (265), store the converted image data in memory (275), or scale the converted image data and provide the scaled image to the display device (285).

[0144] With reference to FIGS. 1 to 15, the aforementioned control processor and image signal processor may be applied as a main processor (210) and an image signal processor (240), respectively. In a high-speed operation mode, the main processor (210) transmits a plurality of commands to the image signal processor (240), and the image signal processor (240) can generate an interrupt signal and store interrupt log information based on an interrupt control value and an image unit identifier included in the commands. Accordingly, even in a high-speed operation mode, the image signal processor (100) can normally perform image processing and efficiently control interrupts.

[0145] Referring to FIG. 18, the image processing system (30) may include a CPU (310), a ROM (320), a post-processing block (330), a sensor interface (340), an image processing engine (350), a circuit (360) composed of a command queue circuit (CQC) and an interrupt control circuit (ICC) as described above, and a DMA controller (370), and the CPU (310), ROM (320), post-processing block (330), sensor interface (340), image processing engine (350) and DMA controller (370) may transmit and receive data through a system bus (380).

[0146] The CPU (310) can control the overall operation of the image processing system (30) and can control the image processing operation by processing or executing programs and / or data stored in the ROM (320).

[0147] The ROM (320) can store instruction code (i.e., programs) and / or data including image processing algorithms.

[0148] The post-processing block (330) can perform post-processing (e.g., adjusting the size of the data, compressing the data, etc.) on the converted image data generated by the image processing engine (350). The post-processed image data can be stored in memory (375) through the DMA controller (370).

[0149] In an embodiment, the image processing system (30) may further include a display interface, and post-processed image data may be provided to a display device through the display interface. Alternatively, image data stored in memory (375) may be read out through a DMA controller (370) and provided to a display device through the display interface.

[0150] The sensor interface (340) communicates with the image sensor (345) and can receive image data, such as raw image data, from the image sensor (345).

[0151] FIG. 19 is a block diagram showing a system according to embodiments of the present invention.

[0152] Referring to FIG. 19, a system (2000) according to an exemplary embodiment of the present disclosure may include an application processor (2100), an image sensor (2200), a display device (2400), a working memory (2500), a storage (2600), a user interface (2700), and a wireless transceiver (2800), and the application processor (2100) may include an image signal processor (2300). The image signal processor (100) of FIG. 1 may be applied as the image signal processor (2300). In the embodiment, the image signal processor (100) may be implemented as an integrated circuit separate from the application processor (2100).

[0153] The application processor (2100) controls the overall operation of the system (2000) and can be provided as a system-on-chip (SoC) that runs applications, operating systems, etc.

[0154] The application processor (2100) can control the operation of the image signal processor (2300) and can provide the converted image data generated by the image signal processor (2300) to the display device (2400) or store it in the storage (2600).

[0155] The image sensor can generate image data, such as raw image data, based on the received optical signal and provide the image data to the image signal processor (2300).

[0156] The image signal processor described with reference to FIGS. 1 to 16 can be applied as an image signal processor (2300). The image signal processor (2300) receives a plurality of commands from a processor provided in the AP (2100) and can perform image processing and interrupt control for a plurality of image units based thereon.

[0157] The working memory (2500) can be implemented as a volatile memory such as DRAM, SRMA, or a non-volatile resistive memory such as FeRAM, RRAM, or PRAM. The working memory (2500) can store programs and / or data that the application processor (2100) processes or executes.

[0158] The storage (2600) may be implemented as a non-volatile memory device such as NADN flash or resistive memory, and, for example, the storage (2600) may be provided as a memory card (MMC, eMMC, SD, micro SD), etc. The storage (2600) may store data and / or programs for an execution algorithm that controls the image processing operation of the image signal processor (2300), and the data and / or programs may be loaded into the working memory (2500) when the image processing operation is performed. In an embodiment, the storage (2600) may store image data generated by the image signal processor (2300), such as converted image data or post-processed image data.

[0159] The user interface (2700) can be implemented with various devices capable of receiving user input, such as a keyboard, a curtain key panel, a touch panel, a fingerprint sensor, and a microphone. The user interface (2700) can receive user input and provide a signal corresponding to the received user input to the application processor (2100).

[0160] The wireless transceiver (2800) may include a transceiver (2810), a modem (2820), and an antenna (2830).

[0161] As described above, an image signal processor and an image processing system according to embodiments of the present invention can efficiently perform communication between an image signal processor and a control processor and improve the operating speed and operating efficiency of the image signal processor and the image processing system including the same by pre-storing a plurality of commands corresponding to a plurality of image units in a command queue circuit and performing image processing based on the stored commands.

[0162] In addition, the image signal processor and image processing system according to the embodiments of the present invention can improve the operating speed and operating efficiency of the image signal processor and the image processing system including the same by determining the conditions for generating an interrupt signal for each image unit where image processing is performed based on an interrupt control value included in a command.

[0163] In addition, the image signal processor and image processing system according to embodiments of the present invention can efficiently perform a recovery procedure in the event of an error by determining log interrupt event signals to be monitored based on an interrupt control value included in a command and storing interrupt log information to include an image unit identifier included in the command. Industrial applicability

[0164] Embodiments of the present invention can be usefully utilized in devices requiring image processing and systems including the same.

[0165] In particular, embodiments of the present invention can be more usefully applied to electronic devices such as computers, laptops, cellular phones, smartphones, MP3 players, Personal Digital Assistants (PDA), Portable Multimedia Players (PMP), digital TVs, digital cameras, portable game consoles, navigation devices, wearable devices, Internet of Things (IoT) devices, Internet of Everything (IoE) devices, e-books, virtual reality (VR) devices, augmented reality (AR) devices, etc.

[0166] Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the following claims.

Claims

Claim 1 A command queue circuit that receives and stores a plurality of commands, each including an interrupt control value corresponding to each of a plurality of image units, from a control processor, and provides the plurality of commands one by one in sequence; and an image processing engine that receives image data including the plurality of image units and performs image processing on the plurality of image units sequentially based on the plurality of commands provided sequentially from the command queue circuit. An image signal processor comprising an interrupt control circuit that receives an interrupt control value from the command queue circuit, determines one or more output interrupt event signals among a plurality of interrupt event signals based on the interrupt control value, and generates an interrupt signal based on the output interrupt event signals, wherein the interrupt control circuit determines one or more log interrupt event signals among the plurality of interrupt event signals based on the interrupt control value, stores interrupt log information regarding whether the log interrupt event signals are activated, and stores unit interrupt information corresponding to each image unit by accumulating whether the log interrupt event signals are activated while image processing is performed for each image unit of the plurality of image units. Claim 2 An image signal processor according to claim 1, wherein the interrupt control circuit determines the output interrupt event signals for each image unit where the image processing of the plurality of image units is performed, based on the interrupt control value included in each of the plurality of commands. Claim 3 delete Claim 4 An image signal processor according to claim 1, wherein each of the plurality of commands received from the control processor further includes an image unit identifier corresponding to each of the image units, and the interrupt control circuit receives the image unit identifier from the command queue circuit and stores the unit interrupt information to include the image unit identifier corresponding to each of the image units. Claim 5 A command queue circuit that receives and stores a plurality of commands, each including an interrupt control value corresponding to each of a plurality of image units, from a control processor and provides the plurality of commands one by one sequentially; an image processing engine that receives image data including the plurality of image units and performs image processing on the plurality of image units sequentially based on the plurality of commands provided sequentially from the command queue circuit; and an interrupt control circuit that receives the interrupt control value from the command queue circuit, determines one or more output interrupt event signals among a plurality of interrupt event signals based on the interrupt control value, and generates an interrupt signal based on the output interrupt event signals, wherein the interrupt control circuit includes: a decision bit generator that generates a plurality of output decision bits representing the output interrupt event signals based on the interrupt control value; and an output interrupt accumulation circuit that generates a plurality of output status bits representing whether the output interrupt event signals are enabled based on the plurality of interrupt event signals and the plurality of output decision bits. An image signal processor characterized by including an interrupt generator that generates the interrupt signal based on the plurality of output status bits. Claim 6 An image signal processor according to claim 5, wherein the output interrupt accumulation circuit comprises: an output decision register circuit that stores the plurality of output decision bits provided from the decision bit generator; an output comparison circuit that generates the plurality of output status bits by comparing each of the plurality of output decision bits stored in the output decision register circuit with each of the plurality of interrupt event signals; and an output status register circuit that stores the plurality of output status bits. Claim 7 An image signal processor according to claim 5, wherein the decision bit generator further generates a plurality of log decision bits representing one or more log interrupt event signals among the plurality of interrupt event signals based on the interrupt control value, and the interrupt control circuit further comprises: a log interrupt accumulation circuit that generates a plurality of log status bits representing whether the log interrupt event signals are enabled based on the plurality of interrupt event signals and the plurality of log decision bits; and a log storage unit that stores interrupt log information including the plurality of log status bits. Claim 8 An image signal processor according to claim 7, wherein each of the plurality of commands received from the control processor further includes an image unit identifier representing each of the image units, and the log storage unit receives the image unit identifier from the command queue circuit and stores unit interrupt information corresponding to each of the image units to include the image unit identifier corresponding to the image unit. Claim 9 An image signal processor according to claim 7, wherein the log interrupt accumulation circuit comprises: a log determination register circuit that stores a plurality of log determination bits provided from the determination bit generator; a log comparison circuit that generates values ​​of a plurality of log status bits by comparing each of the plurality of log determination bits stored in the log determination register circuit with each of the plurality of interrupt event signals; and a log status register circuit that stores the plurality of log status bits. Claim 10 A control processor that generates a plurality of commands including interrupt control values ​​corresponding to each of a plurality of image units and executes an interrupt service routine based on an interrupt signal; and an image signal processor that performs image processing for the plurality of image units and generates the interrupt signal, wherein the image signal processor includes: a command queue circuit that receives and stores the plurality of commands from the control processor and provides the plurality of commands one by one sequentially; and an image processing engine that receives image data including a plurality of image units corresponding to each of the plurality of commands and performs image processing for the plurality of image units sequentially based on the plurality of commands provided sequentially from the command queue circuit. An image processing system comprising an interrupt control circuit that receives an interrupt control value from the command queue circuit, determines one or more output interrupt event signals among a plurality of interrupt event signals based on the interrupt control value, and generates the interrupt signal based on the output interrupt event signals, wherein the interrupt control circuit determines one or more log interrupt event signals among the plurality of interrupt event signals based on the interrupt control value, stores interrupt log information regarding whether the log interrupt event signals are activated, and stores unit interrupt information corresponding to each image unit by accumulating whether the log interrupt event signals are activated while image processing is performed for each image unit of the plurality of image units.

Images