Image sensing device, image processing device, and imaging system

Abstract

The present application relates to image sensing devices, image processing devices, and imaging systems. An image sensing device is disclosed. The image sensing device includes a pixel array including odd rows and even rows, a first row control circuit configured to control the pixel array to perform pixel operations on the odd rows in a top-to-bottom order, and a second row control circuit configured to control the pixel array to perform pixel operations on the even rows in a bottom-to-top order.

Description

Technical Field

[0001] The technologies and implementations disclosed in this article generally relate to an image sensing device, an image processing device, and an imaging system. Background Technology

[0002] Imaging systems capture optical images by converting light into electrical signals using photosensitive semiconductor materials that react to light. With the development of the automotive, medical, computer, and communications industries, the demand for high-performance imaging systems is increasing in various fields such as smartphones, digital cameras, gaming consoles, the Internet of Things (IoT), robotics, security cameras, and medical miniature cameras.

[0003] The raw images captured by the imaging system may contain image distortions related to moving objects, so it may be necessary to perform image correction on the moving objects to improve the performance of the imaging system. Summary of the Invention

[0004] Various embodiments of this disclosure relate to an image sensing apparatus that performs pixel operations on odd-numbered and even-numbered rows of a pixel array in different directions.

[0005] Various embodiments of this disclosure relate to an image sensing apparatus that performs pixel operations simultaneously on multiple rows of a pixel array by including multiple row control circuits.

[0006] Various embodiments of this disclosure relate to an image processing apparatus for extracting moving objects from an image.

[0007] Various embodiments of this disclosure relate to an image processing apparatus for correcting image distortion of moving objects in an image.

[0008] According to embodiments of the present disclosure, an image sensing device may include: a pixel array including odd rows and even rows; a first row control circuit configured to control the pixel array to perform pixel operations on the odd rows in a top-down order; and a second row control circuit configured to control the pixel array to perform pixel operations on the even rows in a bottom-up order.

[0009] In some implementations, pixel operations include reset, exposure, and readout.

[0010] In some implementations, the first row control circuit is configured to control the pixel array: after the reset of the first odd row in the odd rows stops, the reset of the second odd row in the odd rows begins, the second odd row being the next odd row after the first odd row.

[0011] In some implementations, the first row control circuit is configured to control the pixel array: exposure of the first odd row begins after the reset of the first odd row in the odd rows has stopped.

[0012] In some implementations, the first row control circuit is configured to control the pixel array: after the exposure of the first odd row in the odd rows stops, the readout of the first odd row begins.

[0013] In some implementations, the second row control circuit is configured to control the pixel array: after the reset of the first even row in the even rows stops, the reset of the second even row in the even rows, which is the even row preceding the first even row, begins.

[0014] In some implementations, the second row control circuit is configured to control the pixel array: exposure of the first even row begins after the reset of the first even row in the even row has stopped.

[0015] In some implementations, the second row control circuit is configured to control the pixel array: after the exposure of the first even row in the even row stops, the readout of the first even row begins.

[0016] In some implementations, the first row control circuit is configured to control the pixel array to sequentially perform the reset, exposure, and readout of the first row of the pixel array; and the second row control circuit is configured to control the pixel array to simultaneously perform the reset, exposure, and readout of the last row of the pixel array with the reset, exposure, and readout of the first row, wherein the last row is one of the even-numbered rows of the pixel array.

[0017] In some implementations, the first row control circuit is configured to control the pixel array to sequentially perform the reset, exposure, and readout of the third row of the pixel array; and the second row control circuit is configured to control the pixel array to simultaneously perform the reset, exposure, and readout of the even-numbered rows before the last row in the even-numbered rows, along with the reset, exposure, and readout of the third row.

[0018] According to another embodiment of this disclosure, an image processing apparatus may include: an image generator coupled to a pixel array and configured to generate a first image corresponding to odd-numbered rows of the pixel array and a second image corresponding to even-numbered rows of the pixel array; an image extractor configured to extract a third image including a moving object from the first image and the second image; and an image corrector configured to correct distortion of the moving object in the third image caused by the movement of the moving object.

[0019] In some implementations, the image generator can be configured to generate a first image based on first image data generated by reading odd-numbered rows in a top-down order; and to generate a second image based on second image data generated by reading even-numbered rows in a bottom-up order.

[0020] In some implementations, the image extractor can be configured to: vertically invert the second image to generate a vertically inverted second image; and extract a third image using the first image and the vertically inverted second image.

[0021] In some implementations, the image extractor can be configured to extract a third image by performing an XOR operation on a first image and a vertically inverted second image.

[0022] In some implementations, the image corrector can be configured to correct distortion of moving objects in a third image caused by the movement of moving objects, based on the difference in readout time between rows of the pixel array.

[0023] In some implementations, the difference in readout times can be a multiple of the time required to reset a row of the pixel array.

[0024] According to another embodiment of this disclosure, an imaging system may include: an image sensing device including a pixel array comprising odd rows and even rows and configured to generate first image data by performing pixel operations on the odd rows in a top-down order and to generate second image data by performing pixel operations on the even rows in a bottom-up order; and an image processing device configured to generate a first image based on the first image data, generate a second image based on the second image data, and correct distortion of moving objects included in the first and second images.

[0025] In some implementations, pixel operations may include reset, exposure, and readout.

[0026] In some implementations, the image processing apparatus may be configured to: vertically invert the second image to generate a vertically inverted second image; and use the first image and the vertically inverted second image to extract a third image including a moving object.

[0027] In some implementations, the image processing apparatus may be configured to correct distortion of moving objects in a third image caused by the movement of moving objects, based on the difference in readout time between rows of the pixel array.

[0028] It will be understood that both the foregoing general description and the following detailed description of this disclosure are illustrative and explanatory, and are intended to provide further explanation of the claimed disclosure. Attached Figure Description

[0029] The above and other features and advantages of this disclosure will become readily apparent when considered in conjunction with the accompanying drawings, with reference to the following detailed description.

[0030] Figure 1 This is a block diagram illustrating an example of an imaging system based on some embodiments of the present disclosure.

[0031] Figure 2 This is a block diagram illustrating an image sensing apparatus according to an embodiment of the present disclosure.

[0032] Figure 3 Pixel operations according to embodiments of the present disclosure are shown.

[0033] Figure 4 This is a block diagram illustrating an image processing apparatus according to an embodiment of the present disclosure.

[0034] Figure 5 Pixel operations according to embodiments of the present disclosure are shown.

[0035] Figure 6 Pixel operations according to embodiments of the present disclosure are shown.

[0036] Figure 7 Pixel operations according to embodiments of the present disclosure are shown.

[0037] Figure 8 Pixel operations according to embodiments of the present disclosure are shown.

[0038] Figure 9 An image processing method according to an embodiment of the present disclosure is shown.

[0039] Figure 10 It is shown that... Figure 1 A block diagram of an example computing device corresponding to an image processing device. Detailed Implementation

[0040] This disclosure provides implementations and examples of an image sensing apparatus, an image processing apparatus, and an imaging system, which can be used in a configuration that substantially solves one or more technical or engineering problems and mitigates limitations or disadvantages encountered in some other image sensing apparatuses. Some implementations of this disclosure relate to an image sensing apparatus that performs pixel operations on odd-numbered and even-numbered rows of a pixel array in different directions. Some implementations of this disclosure relate to an image sensing apparatus that simultaneously performs pixel operations on multiple rows of a pixel array by including multiple row control circuits. Some implementations of this disclosure relate to an image processing apparatus for extracting moving objects from an image. Some implementations of this disclosure relate to an image processing apparatus for correcting image distortion of moving objects in an image. Recognizing the above problems, this disclosure provides an image sensing apparatus capable of performing pixel operations on odd-numbered and even-numbered rows of a pixel array in different directions. This disclosure provides an image sensing apparatus capable of simultaneously performing pixel operations on multiple rows of a pixel array by including multiple row control circuits. This disclosure provides an image processing apparatus capable of extracting moving objects from an image. This disclosure provides an image processing apparatus capable of correcting image distortion of moving objects in an image.

[0041] Reference will now be made in detail to embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to denote the same or similar parts. While the present disclosure is readily adaptable to various modifications and alternatives, specific embodiments are shown in the drawings by way of example. However, the present disclosure should not be construed as limiting itself to the embodiments set forth herein.

[0042] Various embodiments will be described below with reference to the accompanying drawings. However, it should be understood that this disclosure is not limited to the specific embodiments, but includes various modifications, equivalents, and / or substitutions of the embodiments. Embodiments of this disclosure can provide various effects that can be directly or indirectly recognized through this disclosure.

[0043] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0044] In the following description of embodiments of this disclosure, detailed descriptions of known functions and configurations contained herein will be omitted where such omissions may obscure the subject matter. In the accompanying drawings, portions irrelevant to the description of this disclosure are omitted for clarity, and similar reference numerals will be used throughout this specification to denote similar portions.

[0045] In this disclosure, when a component is referred to as "connected," "linked," or "joined" to another component, it may include not only a direct connection but also an indirect connection between the two components. Furthermore, when a component "comprises," "includes," or "has" another component, this means that, unless specifically stated above, the component does not exclude other components but may also include other components.

[0046] In this disclosure, terms such as "first" and "second" are used only to distinguish one element from others and are not used to limit the elements, and unless otherwise specified, they do not limit the order or importance of the elements. Therefore, within the scope of this disclosure, a first element in one embodiment may be referred to as a second element in another embodiment, and similarly, a second element in one embodiment may be referred to as a first element in another embodiment.

[0047] In the following description, components are distinguished from each other to clearly describe their characteristics, but this does not necessarily mean that they are physically separated. That is, multiple components may be integrated into a single hardware or software module, and a single component may be divided into multiple hardware or software modules. Therefore, even without specific description, integration or division implementations are within the scope of this disclosure.

[0048] In the following description, not all components described with reference to various embodiments are necessary, and some components may be used selectively. Therefore, embodiments consisting of some of the components described in one embodiment are also within the scope of this disclosure. Furthermore, embodiments implemented by adding components to various embodiments are also within the scope of this disclosure.

[0049] For ease of explanation, this disclosure uses the terms "top", "upper", "bottom", "lower", "left", "right", etc. When the accompanying drawings shown in this specification are viewed in reverse, the positional relationships described in the specification can be interpreted in the opposite way.

[0050] In this disclosure, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” can include any one or all possible combinations of the items listed together in a corresponding phrase. In the description of this disclosure, the term “and / or” can include a combination of multiple items or any one of multiple listed items. For example, “A or B” can include “A only,” “B only,” or “both A and B.”

[0051] In the following text, reference will be made to Figures 1 to 10 The embodiments of this disclosure are described in detail.

[0052] Figure 1 This is a block diagram illustrating an example of an imaging system 10 based on some embodiments of the present disclosure.

[0053] Reference Figure 1 Imaging system 10 may refer to, for example, a digital still camera that captures still images or a digital video camera that captures moving images. For example, imaging system 10 may be implemented as a digital SLR (DSLR) camera, a mirrorless camera, or a smartphone, but is not limited thereto. Imaging system 10 may include means having an image pickup element, such that the means can capture (or photograph) a target object, thereby creating an image of the target object.

[0054] The imaging system 10 may include an image sensing device 100 and an image processing device 200. The image processing device 200 may correspond to an image signal processor (ISP).

[0055] Image sensing device 100 may be a complementary metal-oxide-semiconductor image sensor (CIS) configured to convert incident light into an electrical signal. Image sensing device 100 may include pixel array 110, row control circuitry 120, ramp generator 130, analog-to-digital converter (ADC) 140, output buffer 150, column driver 160, and timing controller (TC) 170. The components of image sensing device 100 are merely illustrative, and at least some components may be added or omitted as needed.

[0056] Pixel array 110 may include a plurality of pixels arranged in rows and columns. In one embodiment, the plurality of pixels may be arranged as a two-dimensional (2D) pixel array including rows and columns. In another embodiment, the plurality of image pixels may be arranged as a three-dimensional (3D) pixel array. The plurality of pixels may convert optical signals into electrical signals based on pixels or groups of pixels, wherein pixels in a group of pixels share at least certain internal circuitry. Pixel array 110 may receive a row control signal (RCS) from row control circuitry 120, including a row selection signal, a pixel reset signal, and a transfer signal. Upon receiving an RCS, a corresponding pixel in pixel array 110 may be enabled to perform an operation corresponding to the row selection signal, the pixel reset signal, and the transfer signal.

[0057] Each pixel in the pixel array 110 may have at least two different sensitivities. Here, sensitivity may refer to an increase in the amount of image data IDATA (or an increase in response) relative to an increase in the intensity of incident light. That is, as sensitivity increases, the increase in image data IDATA increases in response to an increase in the intensity of incident light. As sensitivity decreases, the increase in image data IDATA decreases in response to an increase in the intensity of incident light. In this disclosure, sensitivity may be determined by the conversion gain.

[0058] The row control circuit 120 can enable the pixel array 110 to perform specific operations on the pixels included in a corresponding row based on command and control signals (TS1) provided by the timing controller (TC) 170. The row control circuit 120 may include a row driver. In some embodiments, the row control circuit 120 may select at least one pixel in at least one row arranged in the pixel array 110. The row control circuit 120 may generate a row selection signal to select at least one row among multiple rows. The row control circuit 120 may sequentially enable a pixel reset signal and a transfer signal for the pixel corresponding to at least one selected row. Thus, a reference signal and an image signal, as analog signals generated by the individual pixels of the selected row, may be sequentially transmitted to the ADC 140. The reference signal may be an electrical signal provided to the ADC 140 when the sensing node of the pixel (e.g., a floating diffusion region) is reset. The image signal may be an electrical signal provided to the ADC 140 when the photocharge generated by the pixel accumulates in the sensing node. The reference signal, indicating the unique reset noise of each pixel, and the image signal, indicating the intensity of the incident light, may be collectively referred to as pixel signals.

[0059] Complementary metal-oxide-semiconductor (CMOS) image sensors can use correlated double sampling (CDS) to remove unwanted offset values ​​(referred to as fixed-pattern noise) of pixels by sampling the pixel signal twice to remove the difference between the two samples. In one example, CDS can remove unwanted offset values ​​of pixels by comparing the pixel output voltage obtained before and after the photocharge generated by the incident light accumulates in the sensing node, making it possible to measure only the pixel output voltage based on the incident light. In some implementations, the ADC 140 may sequentially sample and hold a reference signal and an image signal provided from each of multiple column lines in the pixel array 110.

[0060] The ramp generator 130 can generate the ramp signal (RS) required for the analog-to-digital conversion operation of the ADC 140 when it receives a timing signal (TS2) from the timing controller (TC) 170, and can supply the ramp signal (RS) to the ADC 140.

[0061] ADC 140 can sample and hold pixel signals from each column line of pixel array 110 based on timing signal (TS3) of timing controller (TC) 170, convert the resulting signal into a digital signal, and output a digital signal. In some embodiments, ADC 140 can be implemented as a ramp comparator type ADC. A ramp comparator type ADC may include comparator circuitry for comparing analog pixel signals with a ramp signal that ramps up or down over time, and a counter for performing counting until the ramp signal matches the analog pixel signal.

[0062] Output buffer 150 can temporarily hold or store column-based image data (i.e., data obtained through digital conversion of pixel signals (IDATA)) provided by ADC 140, to output the stored image data. The image data IDATA output from ADC 140 can be temporarily stored in output buffer 150 based on the timing signal (TS4) of timing controller (TC) 170. Output buffer 150 can provide an interface to compensate for differences in data rate or transmission rate between image sensing device 100 and other devices.

[0063] The column driver 160 can select the column of the output buffer 150 based on the control signal (TS5) from the timing controller (TC) 170, and can control the output buffer 150 to sequentially output the image data IDATA temporarily stored in the selected column of the output buffer 150. In some embodiments, upon receiving an address signal from the timing controller (TC) 170, the column driver 160 can generate a column selection signal (CSS) based on the address signal, and can select the column of the output buffer 150, thereby outputting the image data IDATA as an output signal from the selected column of the output buffer 150.

[0064] The timing controller (TC) 170 can control the operation of at least one of the row control circuit 120, ramp generator 130, ADC 140, output buffer 150, or column driver 160.

[0065] The timing controller (TC) 170 may provide at least one of the row control circuit 120, the ramp generator 130, the ADC 140, the output buffer 150, or the column driver 160 with clock signals required for the operation of the various components of the image sensing device 100, control signals for timing control, and address signals for selecting rows or columns. In some embodiments, the timing controller (TC) 170 may include logic control circuitry, phase-locked loop (PLL) circuitry, timing control circuitry, and communication interface circuitry.

[0066] The image processing apparatus 200 can perform image signal processing on image data IDATA received from the image sensing apparatus 100. The image processing apparatus 200 can reduce noise in the image data IDATA and can perform various types of image signal processing for improving the image quality of the image data IDATA (e.g., interpolation, compositing, gamma correction, color filter array interpolation, color matrix, color correction, color enhancement, and lens distortion correction). Additionally, the image processing apparatus 200 can compress the image data created by performing image signal processing for image quality improvement, so that the image processing apparatus 200 can use the compressed image data to create an image file. Alternatively, the image processing apparatus 200 can recover image data from an image file. In this case, the format used to compress such image data can be a reversible format or an irreversible format. As examples of compression formats, for still images, the Joint Image Experts Group (JPEG) format, JPEG 2000 format, etc., can be used. Furthermore, for moving images, multiple frames can be compressed according to the Moving Picture Experts Group (MPEG) standard to create a moving image file. For example, an image file can be created according to the Exif (Exchangeable Image File Format) standard.

[0067] Image processing apparatus 200 can generate a high dynamic range (HDR) image by synthesizing at least two images with different sensitivities. For example, image sensing apparatus 100 can output a low-sensitivity image generated from low-sensitivity pixels (e.g., low conversion gain (LCG) pixels) with relatively lower sensitivity and a high-sensitivity image generated from high-sensitivity pixels (e.g., high conversion gain (HCG) pixels) with relatively higher sensitivity. Image processing apparatus 200 can combine the low-sensitivity image and the high-sensitivity image to generate an HDR image. Here, low sensitivity and high sensitivity can correspond to relative concepts, and image sensing apparatus 100 can generate image data IDATA with at least N different sensitivities (where N is an integer greater than or equal to 2). Image processing apparatus 200 can generate an HDR image from image data IDATA.

[0068] The image processing apparatus 200 can send image data that has undergone image processing to a host device (not shown). The host device (not shown) can be a processor (e.g., an application processor) for processing the image data received from the image processing apparatus 200, a memory (e.g., a non-volatile memory) for storing the image data, or a display device (e.g., a liquid crystal display (LCD)) for visually displaying the image data.

[0069] The image processing device 200 can send control signals to the image sensing device 100 for controlling the operation of the image sensing device 100 (e.g., whether to operate, operation timing, operation mode, etc.).

[0070] Imaging system 10 may include image sensing device 100, which generates first image data by performing pixel operations on odd-numbered rows of pixel array 110 in a top-down order, and generates second image data by performing pixel operations on even-numbered rows of pixel array 110 in a bottom-up order. Additionally, imaging system 10 may include image processing device 200, which generates a first image based on the first image data, generates a second image based on the second image data, and corrects distortion of moving objects included in the first and second images. A more detailed description of the image sensing device 100 and image processing device 200 included in imaging system 10 will be provided below.

[0071] Figure 2 This is a block diagram illustrating an image sensing device 20 according to an embodiment of the present disclosure.

[0072] Figure 3 Pixel operations according to embodiments of the present disclosure are shown.

[0073] Now refer to Figure 3 describe Figure 2 .

[0074] Reference Figure 2 The image sensing device 20 may include a first row control circuit 210, a second row control circuit 220, and a pixel array 230. The image sensing device 20 may correspond to... Figure 1 Image processing device 200.

[0075] The first row control circuit 210 can control the pixel array 230 such that pixel operations are performed on specific rows (e.g., odd-numbered rows) of the pixel array 230 in a top-down order. Alternatively, the first row control circuit 210 can control the pixel array 230 such that pixel operations are performed on the odd-numbered rows of the pixel array 230 in a bottom-up order. Pixel operations may include resetting, exposure, and readout.

[0076] For example, refer to Figure 3The first row control circuit 210 controls the pixel array 230 to perform pixel operations on the odd-numbered rows of the pixel array 230 in a top-down order. Specifically, the first row control circuit 210 controls the pixel array 230 to start resetting the second odd-numbered row, which is the next odd-numbered row after the first odd-numbered row, immediately after the reset of the first odd-numbered row stops. For example, the first row control circuit 210 controls the pixel array 230 to start resetting the third row immediately after the reset of the first row stops. Additionally, the first row control circuit 210 controls the pixel array 230 to start exposure of the first odd-numbered row immediately after the reset of the first odd-numbered row stops. Furthermore, the first row control circuit 210 controls the pixel array 230 to start readout of the first odd-numbered row immediately after the exposure of the first odd-numbered row stops. Additionally, the first row control circuit 210 controls the pixel array 230 to start exposure of the second odd-numbered row immediately after the reset of the second odd-numbered row stops. In addition, the first row control circuit 210 can control the pixel array 230 so that the readout of the second odd row begins immediately after the exposure of the second odd row stops.

[0077] The second row control circuit 220 can control the pixel array 230 such that pixel operations are performed on the even-numbered rows of the pixel array 230 in a bottom-up order. Alternatively, the second row control circuit 220 can control the pixel array 230 such that pixel operations are performed on the even-numbered rows of the pixel array 230 in a top-down order.

[0078] For example, refer to Figure 3 The second row control circuit 220 controls the pixel array 230 to perform pixel operations on the even-numbered rows of the pixel array 230 in a bottom-up order. Specifically, the second row control circuit 220 controls the pixel array 230 to start resetting the second even-numbered row, which is the previous even-numbered row, immediately after the reset of the first even-numbered row stops. For example, the second row control circuit 220 controls the pixel array 230 to start resetting the tenth row immediately after the reset of the twelfth row stops. Additionally, the second row control circuit 220 controls the pixel array 230 to start exposure of the first even-numbered row immediately after the reset of the first even-numbered row stops. Furthermore, the second row control circuit 220 controls the pixel array 230 to start readout of the second even-numbered row immediately after the exposure of the first even-numbered row stops. Also, the second row control circuit 220 controls the pixel array 230 to start exposure of the second even-numbered row immediately after the reset of the second even-numbered row stops. In addition, the second row control circuit 220 can control the pixel array 230 so that the readout of the second even row begins immediately after the exposure of the second even row stops.

[0079] The first row control circuit 210 and the second row control circuit 220 can control the pixel array 230, enabling pixel operations to be performed simultaneously on different rows of the pixel array 230. Specifically, the second row control circuit 220 can control the pixel array 230 such that the last row of the pixel array (e.g., Figure 3 The reset, exposure, and readout of the twelfth row of the pixel array are performed simultaneously with the reset, exposure, and readout of the first row of the pixel array. In this case, the last row can be one of the even-numbered rows of the pixel array. Additionally, the second row control circuit 220 can control the pixel array 230 such that the even-numbered rows preceding the last row (e.g., ...) are executed concurrently. Figure 3 The reset, exposure, and readout operations (in line 10) are performed simultaneously with the reset, exposure, and readout operations (in line 3). In other words, assuming a pixel array of 230... Figure 3 The diagram shows twelve rows. The reset, exposure, and readout operations in the first row can be performed simultaneously with those in the twelfth row. Additionally, the reset, exposure, and readout operations in the third row can be performed simultaneously with those in the tenth row. Furthermore, the reset, exposure, and readout operations in the fifth row can be performed simultaneously with those in the eighth row. Moreover, the reset, exposure, and readout operations in the sixth row can be performed simultaneously with those in the seventh row.

[0080] Pixel array 230 can correspond to Figure 1 The pixel array 110. The pixel array 230 can receive a first row control signal RCS1 from the first row control circuit 210 and a second row control signal RCS2 from the second row control circuit 220. Pixel operations can be performed according to the first row control signal RCS1 and the second row control signal RCS2.

[0081] Figure 4 This is a block diagram illustrating an image processing apparatus according to an embodiment of the present disclosure.

[0082] Figure 5 Pixel operations according to embodiments of the present disclosure are shown.

[0083] Figure 6 Pixel operations according to embodiments of the present disclosure are shown.

[0084] Figure 7 Pixel operations according to embodiments of the present disclosure are shown.

[0085] Figure 8 Pixel operations according to embodiments of the present disclosure are shown.

[0086] Now refer to Figure 3 , Figure 5 , Figure 6 , Figure 7 and Figure 8 To describe Figure 4 .

[0087] Reference Figure 4 The image processing apparatus 400 may include an image generator 410, an image extractor 420, and an image corrector 430.

[0088] Image generator 410 can generate a first image corresponding to the odd-numbered rows of the pixel array and a second image corresponding to the even-numbered rows of the pixel array. Specifically, image generator 410 can generate the first image based on first image data generated by reading out the odd-numbered rows in a top-down order. For example, Figure 5 Showing the Figure 3 Pixel operations are performed on the odd-numbered rows. (See reference...) Figure 5 The rows of the pixel array can be arranged in a top-down order (e.g., Figure 3 Pixel operations are performed on the odd-numbered rows in the image. Therefore, the image generator 410 can be based on reading... Figure 5 The first image is generated from the first image data generated from the rows in the image. For example, using... Figure 5 The readout method can generate Figure 6 The first image 600 may include static objects 611, 612 and 613 (still objects) and dynamic objects 620 (moving objects).

[0089] Additionally, the image generator 410 can generate a second image based on second image data generated by reading out even-numbered rows in a bottom-up order. For example, Figure 7 Showing the Figure 3 Pixel operations are performed on even-numbered rows in the table. (See reference...) Figure 7 The rows of the pixel array can be arranged in a bottom-up order (e.g., Figure 3 Pixel operations are performed on the even-numbered rows in the image. Therefore, the image generator 410 can be based on reading... Figure 7 The second image is generated from the rows in the image. For example, using... Figure 7 The readout method can generate a second image. (Refer to...) Figure 8 The second image 800 may be an image obtained by vertically inverting a second image generated based on the second image data. The vertically inverted image of the second image 800 may include static objects 811, 812 and 813 (still objects) and dynamic objects 820 (moving objects).

[0090] The rows of a pixel array can be read at different times, and the direction of pixel operations performed on odd-numbered rows differs from the direction of pixel operations performed on even-numbered rows. Therefore, as the same object, Figure 6 Dynamic objects 620 and Figure 8 The dynamic object 820 in the image may be imaged in different shapes. For example, dynamic objects 620 and 820 may correspond to a vehicle moving to the right.

[0091] Image extractor 420 can extract a third image of dynamic objects 620 and 820 from the first image 600 and the second image 800. Specifically, image extractor 420 can generate a vertically inverted second image 800 by vertically inverting the second image, and then extract the third image from the first image 600 and the vertically inverted second image 800. Image extraction will be described in more detail later.

[0092] Image corrector 430 can correct distortion in the third image caused by the movement of dynamic objects 620 and 820. For example, image corrector 430 can correct distortion in the third image caused by the movement of dynamic objects 620 and 820 based on the difference in readout time between rows of the pixel array. Image correction will be described in more detail later.

[0093] Figure 9 An image processing method according to an embodiment of the present disclosure is shown.

[0094] Reference Figure 9 According to embodiments of the present disclosure, the image processing apparatus can generate a first image 910 and a vertically inverted second image 920. The first image 910 and the vertically inverted second image 920 can respectively correspond to... Figure 6 The first image 600 and Figure 8 The second image 800 is a vertically inverted image. Specifically, the first image 910 can be generated based on first image data generated by reading out the odd-numbered rows of the pixel array of the image sensing device in a top-down order. The second image 920 is a vertically inverted image generated based on second image data generated by reading out the even-numbered rows of the pixel array of the image sensing device in a bottom-up order. The first image 910 may include static objects 911, 912, and 913 (still objects) and dynamic object 914 (moving object). The second image 920 is a vertically inverted image that may include static objects 921, 922, and 923 (still objects) and dynamic object 924 (moving object).

[0095] The image extractor of the image processing apparatus can extract a third image 930 of dynamic objects 914 and 924 from a first image 910 and a vertically inverted second image 920. For example, the image extractor can perform an XOR operation on the first image 910 and the vertically inverted second image 920 to extract the third image 930. Specifically, static objects 911, 912, and 913 in the first image 910 and static objects 921, 922, and 923 in the vertically inverted second image 920 are the same static objects, so they can be imaged identically even if the readout times of each row are different. Conversely, dynamic object 914 in the first image 910 and dynamic object 924 in the vertically inverted second image 920 are the same object existing in different positions at different times. Therefore, for dynamic objects, the readout times of each row are different from each other, and the readout direction of odd-numbered rows is different from that of even-numbered rows. Therefore, dynamic objects can be imaged differently. Therefore, performing an XOR operation on the first image 910 and the vertically inverted second image 920 yields a third image 930 (e.g., a combined image including dynamic objects 914 and 924) that contains only the dynamic objects 914 and 924 with different imaging. In the third image 930, object 931 may correspond to dynamic object 924, and object 932 may correspond to dynamic object 914.

[0096] The image corrector of the image processing device can correct distortion in the third image 930 caused by object movement based on the difference in readout time between rows of the pixel array of the image sensing device. The difference in readout time between rows can be a multiple of a specific time. For example, referring to... Figure 3 The difference in readout time between rows can be a multiple of the time required to perform a reset operation on a single row of the pixel array. Specifically, Figure 3 The time difference between reading the first and second rows can be equal to five times the duration of the reset operation. Additionally, the time difference between reading the first and third rows can be equal to one time the duration of the reset operation. Considering the different readout timings of each row, the image corrector can correct the third image to generate a fourth image 940 including the dynamic object 941, which is free from distortion caused by movement. Furthermore, the image processing device can generate a distortion-free fifth image 950 based on the first image 910, the second image 920, or the fourth image 940, which includes the distortion-corrected dynamic object 941 and static objects 911, 912, 913, 921, 922, and 923 (e.g., static objects corresponding to each of 911 and / or 921, 912 and / or 922, and 913 and / or 923).

[0097] Figure 10 It is shown that... Figure 1 A block diagram of an example of a computing device 1000 corresponding to an image processing device.

[0098] Reference Figure 10 The computing device 1000 can represent a device for performing... Figure 1 An implementation of the hardware configuration for operating the image processing device 200.

[0099] The computing device 1000 can be mounted on a chip independent of the chip on which the image sensing device is mounted. According to one embodiment, the chip on which the image sensing device is mounted and the chip on which the computing device 1000 is mounted can be implemented in a single package (e.g., a multi-chip package (MCP)), but the scope of this disclosure is not limited thereto.

[0100] Furthermore, the internal configuration or arrangement of the computing device 1000 and the image sensing device may vary depending on the implementation. For example, at least a portion of the image sensing device may be included in the computing device 1000. Alternatively, at least a portion of the computing device 1000 may be included in the image sensing device. In this case, at least a portion of the computing device 1000 may be mounted together on a chip on which the image sensing device is mounted.

[0101] The computing device 1000 may include a processor 1010, a memory 1020, an input / output (I / O) interface 1030, and a communication interface 1040.

[0102] Processor 1010 can handle execution Figure 1 The data and / or instructions required for the operation of the components of the image processing apparatus 200 described herein.

[0103] The memory 1020 may store data and / or instructions required to perform the operation of the components of the image processing apparatus 200 and may be accessed by the processor 1010. For example, the memory 1020 may be volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM), etc.) or non-volatile memory (e.g., programmable read-only memory (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, etc.).

[0104] That is, a computer program for performing the operation of the image processing apparatus 200 disclosed herein can be recorded in the memory 1020 and executed and processed by the processor 1010, thereby realizing the operation of the image processing apparatus 200.

[0105] The input / output (I / O) interface 1030 may be an interface for connecting external input devices (e.g., keyboard, mouse, touch panel, etc.) and / or external output devices (e.g., display) to the processor 1010 to allow sending and receiving data.

[0106] The communication interface 1040 is a component capable of sending and receiving various data to and from an external device (e.g., an application processor, external memory, etc.), and can be a device capable of supporting wired or wireless communication.

[0107] It is evident from the above description that the image sensing apparatus according to some embodiments of the present disclosure can perform pixel operations on odd-numbered rows and even-numbered rows of a pixel array in different directions.

[0108] The image sensing apparatus according to some embodiments of the present disclosure can perform pixel operations on multiple rows of a pixel array simultaneously by including multiple row control circuits.

[0109] The image processing apparatus according to some embodiments of the present disclosure can extract moving objects from an image.

[0110] The image processing apparatus according to some embodiments of the present disclosure can correct image distortion of moving objects in an image.

[0111] The embodiments disclosed herein can provide various effects that can be directly or indirectly understood through the aforementioned patent documents.

[0112] Those skilled in the art will understand that this disclosure may be implemented in other specific ways than those set forth herein. Furthermore, claims not expressly set forth in the appended claims may be offered as a combination of embodiments or may be included as new claims through subsequent amendments after filing.

[0113] Although several exemplary embodiments have been described, it should be understood that modifications and enhancements to the disclosed embodiments and other embodiments may be conceived based on the description and / or illustrations in this patent document.

[0114] Cross-reference to related applications

[0115] This patent document claims priority and benefit to Korean Patent Application No. 10-2025-0039727, filed on March 27, 2025, the disclosure of which, in its entirety, is incorporated herein by reference as part of the disclosure of this patent document.

Claims

1. An image sensing device, the image sensing device comprising: A pixel array comprising odd-numbered rows and even-numbered rows; A first row control circuit controls the pixel array to perform pixel operations on the odd-numbered rows in a top-to-bottom order; as well as The second row control circuit controls the pixel array to perform the pixel operation on the even-numbered rows in a bottom-up order.

2. The image sensing device according to claim 1, wherein, The pixel operations include reset, exposure, and readout.

3. The image sensing device according to claim 1, wherein, The first row of control circuits controls the pixel array: After the reset of the first odd row in the odd rows stops, the reset of the second odd row in the odd rows begins, the second odd row being the next odd row after the first odd row.

4. The image sensing device according to claim 1, wherein, The first row of control circuits controls the pixel array: After the reset of the first odd row in the odd rows stops, the exposure of the first odd row begins.

5. The image sensing device according to claim 1, wherein, The first row of control circuits controls the pixel array: After the exposure of the first odd row in the odd rows stops, the readout of the first odd row begins.

6. The image sensing device according to claim 1, wherein, The second row of control circuitry controls the pixel array: After the reset of the first even row in the even rows stops, the reset of the second even row in the even rows begins, the second even row being the even row preceding the first even row.

7. The image sensing device according to claim 1, wherein, The second row of control circuitry controls the pixel array: After the reset of the first even-numbered row in the even-numbered rows stops, the exposure of the first even-numbered row begins.

8. The image sensing device according to claim 1, wherein, The second row of control circuitry controls the pixel array: After the exposure of the first even-numbered row in the even-numbered rows stops, the readout of the first even-numbered row begins.

9. The image sensing device according to claim 1, wherein, The first row control circuit controls the pixel array to sequentially perform the reset, exposure, and readout of the first row of the pixel array; and The second row control circuit controls the pixel array to simultaneously execute the reset, exposure, and readout of the last row of the pixel array, along with the reset, exposure, and readout of the first row. The last row is one of the even-numbered rows of the pixel array.

10. The image sensing device according to claim 9, wherein, The first row control circuit controls the pixel array to sequentially perform the reset, exposure, and readout of the third row of the pixel array; and The second row control circuit controls the pixel array to simultaneously perform the reset, exposure, and readout of the even-numbered rows before the last row in the even-numbered rows, along with the reset, exposure, and readout of the third row.

11. An image processing apparatus, the image processing apparatus comprising: An image generator, which is coupled to a pixel array, and Generate a first image corresponding to the odd-numbered rows of the pixel array, and Generate a second image corresponding to the even-numbered rows of the pixel array; An image extractor that extracts a third image including a moving object from the first image and the second image; as well as An image corrector that corrects distortion of the moving object in the third image caused by the movement of the moving object.

12. The image processing apparatus according to claim 11, wherein, The image generator: The first image is generated based on first image data generated by reading the odd-numbered rows in a top-down order; and The second image is generated based on second image data generated by reading out the even-numbered rows in a bottom-up order.

13. The image processing apparatus according to claim 11, wherein, The image extractor: The second image is vertically flipped to generate a vertically flipped second image; and The third image is extracted using the first image and the vertically inverted second image.

14. The image processing apparatus according to claim 13, wherein, The image extractor: The third image is extracted by performing an XOR operation on the first image and the vertically inverted second image.

15. The image processing apparatus according to claim 11, wherein, The image corrector: The distortion of the moving object in the third image caused by the movement of the moving object is corrected based on the difference in readout time between rows of the pixel array.

16. The image processing apparatus according to claim 15, wherein, The difference in readout time is a multiple of the time required to reset one row of the row in the pixel array.

17. An imaging system comprising: An image sensing device includes a pixel array comprising odd-numbered rows and even-numbered rows, and: First image data is generated by performing pixel operations on the odd-numbered rows in a top-down order; and The second image data is generated by performing the pixel operations on the even-numbered rows in a bottom-up order. as well as Image processing apparatus, the image processing apparatus: Generate a first image based on the first image data; A second image is generated based on the second image data; and Correct the distortion of moving objects included in the first and second images.

18. The imaging system according to claim 17, wherein, The pixel operations include reset, exposure, and readout.

19. The imaging system according to claim 17, wherein, The image processing device: The second image is vertically flipped to generate a vertically flipped second image; and The first image and the vertically inverted second image are used to extract a third image that includes the moving object.

20. The imaging system according to claim 19, wherein, The image processing device: The distortion of the moving object in the third image caused by the movement of the moving object is corrected based on the difference in readout time between rows of the pixel array.

Images