Integrated circuit for optimizing automatic zoom within a moving area of ​​an image - Patent Application 20070122997

Abstract

The present invention relates to an integrated circuit of an image capture device for automatically optimizing the ratio (A1 / A0) of the output image area (A1) to the main window area (A0), and for optimizing (preferably maximizing) the product D1×A1 of the output pixel density (D1) multiplied by the output image area (A1) of the motion detection area that defines an output image (P1) consisting of a predetermined maximum number (N1m) of pixels. The integrated circuit includes an array light sensor for capturing an input image (P0) and a processing unit configured to perform the following actions: - in an image detection mode, a series of consecutive reference images (Prefj) are captured at different times by a subarray of Nr reference light sensors, and a number (Nm) of motion pixels (pm) whose value difference between the two reference images exceeds a predetermined threshold deviation for detecting motion within an area of ​​the reference image is determined, and then switching; - in an image capture mode, the parameters of the captured output image are defined as follows: ● an output window (W1) with an output window area (A1) surrounding at least a predetermined number (Nmt) of the motion pixels (pm), and ● an output pixel density D1 = N1 / A1 ≦ DO, where N1 ≦ N1m; ● capturing an output image (P1) defined by the output window (W1) having the output window area (A1) and the output pixel density (D1) and transferring the output image (P1) including N1 pixels from the integrated circuit.

Description

[Technical Field]

[0001] The present invention relates to an integrated circuit (=IC) for an image capture / processing device configured to capture images when motion is detected. When motion is detected within a frame of an image, the inventive integrated circuit automatically optimizes the ratio between the resolution of the captured image and its size, both in terms of frame dimensions and number of pixels, prior to transferring the captured image to a processor for further image processing. Such optimization, performed within the integrated circuit prior to transferring the output image from the IC, significantly reduces the processing resources and power consumption of the image capture / processing device. Surveillance cameras equipped with the inventive IC require less powerful processing equipment and have long autonomy. [Background technology]

[0002] Image capture devices for monitoring motion within an area visible within a primary window having a primary image area (A0) are known in the art. However, processing a large number of full-scale images requires significant processing power and energy usage, thus increasing the cost of the device and its use. Energy consumption management can be important, for example, when the image capture device monitors an area isolated from any external power source and is powered, for example, by a battery (possibly linked to a photovoltaic panel). Among the most power- and resource-intensive operations is the transfer of images from an IC containing an array of photosensors that captures the images to a processor (for further processing of the captured images). Power consumption and processing resources are proportional to both the number of images transferred and their size (i.e., their number of pixels). Therefore, reducing power consumption can be achieved by transferring fewer images of smaller size.

[0003] U.S. Patent Application Publication No. 20220141409 describes a method and device for detecting regions of an imaging plane where an image of a subject has changed between two images. The number of acquired pixel signals for detecting motion is less than the number of pixels contained in this partial region. In response to the detection of motion within this region, the device proceeds to a normal image capture mode to capture one or more images. The full-size images thus captured are then transferred to a processor for compression and possible resizing. This method reduces the number of transferred images, since images are transferred only upon detection of motion between successive images containing fewer pixels than the full-size image. However, this method does not reduce the size of the transferred images.

[0004] U.S. Patent No. 7,279,671 describes an image capture device for use with a microscope, including an image pickup element for capturing an image, and a change detector for detecting an amount of change between an image captured by the image pickup element and a subsequent image captured by the image pickup element. The device includes binning means for performing a binning process by calculating and outputting cumulative pixel information of a plurality of similar pixels of the input image as output pixels to form a reduced data output image based on the amount of change detected by the change detector, or for outputting pixel information of the input image as output data without reduced data processing.

[0005] US Patent No. 1,102,5822 describes a method for detecting a predetermined event by detecting the presence or absence of object movement from a previous frame to a current frame as the presence or absence of the predetermined event. This method performs pixel addition for each divided block to detect changes in the object image within the divided block unit.

[0006] U.S. Patent Application Publication No. 2014 / 0118592 describes an imaging sensor and method for detecting motion, the method including operating the sensor in a motion detection mode at a first power level to detect motion from a received image detected by pixels in a pixel array, switching the imaging sensor from the motion detection mode to a feature extraction mode in response to detecting motion, and operating the imaging sensor in an extraction mode at a second power level higher than the first power level. The motion detection mode operates based on pixel data received from multiple rows or columns of pixel data from the pixel array.

[0007] Chinese Patent Application Publication No. 106657780A describes an image preview method and device applicable to a user device. The image preview method includes the steps of generating an image preview image in an initial pixel mode in response to a preview command sent by a user, the initial pixel mode being a pixel blending mode; receiving a zoom command sent by the user; and outputting a zoom preview image in a zoom pixel mode based on the received zoom command, the pixel merging ratio in the zoom pixel mode being smaller than the pixel merging ratio in the initial pixel mode. Therefore, compared with existing zoom preview images, the image preview method improves the resolution of the zoom preview image. The output pixel density (D1) is always increased from the initial preview. The initial preview image is acquired in an initial pixel mode, which is a pixel binning mode. Then, in response to receiving a zoom command sent by a user, the zoom image is acquired in the zoom pixel mode, where the binning ratio of the pixels in the zoom pixel pattern is smaller than the binning ratio of the pixels in the initial pixel pattern.

[0008] US Pat. No. 1,121,8641 describes an image capture device including a processor configured to transition an image sensor between image capture modes that define different binning levels in case the digital zoom level satisfies a predetermined threshold.

[0009] There remains a need to reduce the power consumption of image capture and processing devices configured to capture images when motion is detected, while maintaining high resolution of the captured images prior to transferring them for further processing after their capture. Summary of the Invention

[0010] The invention is defined in the accompanying independent claims. Preferred embodiments are defined in the dependent claims. In particular, the invention relates to an integrated circuit for automatically optimizing the ratio (A1 / A0) of the output image area (A1) to the main window area (A0) and for optimizing (preferably maximizing) the product D1×A1 of the output pixel density (D1) multiplied by the output image area (A1) of a motion detection area that defines an output image (P1) consisting of a predetermined maximum number (N1m) of pixels. The integrated circuit includes an image sensor and a processing unit.

[0011] The image sensor includes an array formed by a nominal number (N0) of photosensors configured to capture values ​​for a nominal number N0 of pixels defining corresponding input images (P0), each contained within a primary window (W0) having a primary window area (A0), the input images having a nominal pixel density D0=N0 / A0.

[0012] The processing unit is configured to perform the following actions in the motion detection mode: controlling a predetermined sub-array of Nr (Nr≦N0) reference photosensors to capture a series of successive reference images (Prefj) at different times, each reference image defined by Nr corresponding reference pixels (pr); • Once a given reference image is captured, determining the value difference between corresponding pixels from the given reference image and the reference image preceding the given reference image in the sequence, and selecting a number (Nm) of motion pixels (pm) whose value difference exceeds a predetermined threshold deviation.

[0013] The gist of the present invention is that, on the one hand, the processing unit saves pixels with a predetermined maximum output number N1m < N0 so as not to exceed (i.e., N1 ≦ N1m < N0) the number of output pixels (N1) defining the output image (P1) transferred from the integrated circuit, and on the other hand, the processing unit is configured to pass pixels with a predetermined maximum output number N1m < N0 in the image capture mode when the number Nm of moving pixels (pm) satisfies the trigger condition. When the trigger condition is satisfied, the processing unit ● is defined as follows: ○ An output window (W1) of an output window area (A1) surrounding at least a predetermined lower trigger number (Ntu) of moving pixels (pm) (where A1 = n × A0 and 0 < n < 1), and ○ An output pixel density D1 = N1 / A1 ≦ D0 (where the number of output pixels N1 = m × N0 ≦ N1m and 0 < m ≦ n < 1), ● controls the array of N0 optical sensors to capture an output image (P1) defined by the output window (W1) having the output window area (A1) and the output pixel density (D1), ● is configured to transfer an output image (P1) including N1 pixels from the integrated circuit.

[0014] In one embodiment, the main window (W0) has a length (L0) and a height (H0) and A0 = L0 × H0, and the output window (W1) has a length (L1) and a height (H1) such that L1 = nL × L0 and H1 = nH × H0 and A1 = L1 × H1, where nL and nH are each less than or equal to 1 (i.e., nL ≦ 1, nH ≦ 1), and preferably each is independently equal to 1, 1 / 2, 1 / 3, or 1 / 4, and the aspect ratio (L1 / H1) of the output window (W1) is preferably equal to the nominal aspect ratio (L0 / H0) (i.e., L1 / H1 = L0 / H0 and n = nL 2 = nH 2 ).

[0015] Alternatively or additionally, the predetermined lower trigger number (Ntu) of moving pixels (pm) is included between 50% and 100%, preferably between 75% and 95%, or preferably equal to 100% of the number Nr of reference pixels (pr).

[0016] Alternatively or additionally, the output window (W1) is rectangular, and a predetermined number of moving pixels (pm) are inscribed within the output window (W1).

[0017] Alternatively or additionally, the length (L1) and height (H1) of the output window (W1) are selected from a predetermined set of output image window lengths and output image window heights.

[0018] Alternatively or additionally, the output window area (A1) is greater than or equal to a predetermined minimum output window area (A1min) such that 0 < A1min ≤ A1 ≤ A0.

[0019] Alternatively or additionally, the output pixel density (D1) is greater than or equal to a predetermined minimum output pixel density (D1min) such that 0 < D1min ≤ D1 ≤ D0.

[0020] The processing unit that defines the output window (W1) and the output pixel density (D1) of the output image (P1) defined by the output number (N1 ≤ N1m) of pixels may be configured to maximize the output window area (A1 = n1A0) such that, for a given output pixel density (D1 = mD0), the output number (N1 = (m × n1)N0) of pixels tends to approach the maximum output number (N1m) of pixels (preferably equal to the maximum output number (N1m) of pixels, i.e., N1 = (m × n1)N0 × N1m). The processing unit may also be configured to maximize the output pixel density (D1 = mtD0) such that, for the output window area (A1 = nA0), the output number (N1 = (mt × n)N0) of pixels tends to approach the maximum output number (N1m) of pixels (preferably equal to the maximum output number (N1m) of pixels, i.e., N1 = (mt × n)N0 × N1m).

[0021] The processing unit is preferably configured to allow modification of the number (Nr) and position of the reference photosensors. When a number Nm of motion pixels (pm) satisfy a trigger condition, the processing unit is preferably configured to generate an interrupt signal indicating the incoming output image (P1) and passage from the motion detection mode to the image capture mode. [Brief explanation of the drawings]

[0022] For a fuller understanding of the nature of the present invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings.

[0023] [Figure 1a] FIG. 1(a) shows array pixels of a first reference image (Pref1) and a second reference image (Pref2) containing two birds in motion, with the number Nr of identical reference pixels indicated by white and shaded squares, respectively. [Figure 1b] FIG. 1(b) shows the array pixels of a first reference image (Pref1) and a second reference image (Pref2) containing two birds in motion, with the number Nr of the same reference pixels indicated by white and shaded squares, respectively. [Figure 2a] FIG. 2(a) shows a first example of a motion pixel (pm) with a black square, whose value difference between the first and second reference pixels in FIGS. 1(a) and 1(b) exceeds a predetermined threshold deviation. [Figure 2b] FIG. 2(b) shows one way to define the output window (W1) of the output image area (A1=H1×L1) into which the motion pixels of FIG. 2(a) are engraved, where H1=nH×H0 and L1=nL×L0, where nL is independent of hL. [Figure 2c] FIG. 2(c) shows an alternative way of defining the output window (W1) of the output image area (A1=H1×L1) containing the motion pixels of FIG. 2(a), where H1=nH×H0 and L1=nL×L0, and nL=hL. [Figure 3a] FIG. 3(a) shows a second example of a motion pixel (pm) within a black square whose value difference between the first and second reference pixels of FIGS. 1(a) and 1(b) exceeds a predetermined threshold deviation. [Figure 3b] FIG. 3(b) shows an output window (W1) with an output image area (A1) that is the same as the main window area (A0) (i.e., A1=A0), but with four times fewer pixels (i.e., N1=1 / 4N0) due to a 2×2 binning operation. [Figure 4a] FIG. 4(a) shows the array pixels of a first reference image (Pref1) and a second reference image (Pref2) containing two birds, with the same number Nr of reference pixels as shown in FIGS. 1(a) and 1(b), the array divided into pixel blocks (Bi) and including a mask area (MA) of no interest. [Figure 4b] FIG. 4(b) shows the array pixels of a first reference image (Pref1) and a second reference image (Pref2) containing two birds, with the same number Nr of reference pixels as shown in FIGS. 1(a) and 1(b), the array divided into pixel blocks (Bi) and including a mask area (MA) of no interest. [Figure 5a] FIG. 5(a) shows an example of a motion pixel (pm) within a black square whose value difference between the first and second reference pixels of FIGS. 4(a) and 4(b) exceeds a predetermined threshold deviation. [Figure 5b] FIG. 5(b) shows one way of defining an output window (W1) of the output image area (A1=H1×L1) that includes all blocks (Bi) that contain at least one motion pixel (pm). [Figure 5c] FIG. 5(c) shows an alternative way of defining an output window (W1) of an output image area (A1=H1×L1) that contains only blocks where all reference pixels (pr) are motion pixels (pm). [Figure 6] FIG. 6 shows a flow chart of how the processing unit of the integrated circuit of the present invention is configured to work. [Figure 7] FIG. 7 shows a flow chart of how the processing unit of the integrated circuit of the present invention is configured to optimize the product A0×D0. DETAILED DESCRIPTION OF THE INVENTION

[0024] The present invention relates to an integrated circuit for automatically optimizing the ratio (A1 / A0) of the output image area (A1) to the primary window area (A0) and for optimizing (preferably maximizing) the product D1×A1 of the output pixel density (D1) multiplied by the output image area (A1) of the motion detection area, which defines an output image (P1) consisting of a predetermined maximum number (N1m) of pixels. The ratio A1 / A0 defines the size of the power window compared to the size of the primary window (W0). Since the output window cannot be as large as the primary window, A1 / A0≦1. Maximizing the product D1×A1 results in the maximum output area (A1) of the output image (P1) satisfying the condition N1≦N1m, which results in the maximum resolution allowed. When motion is detected within the motion detection area, the integrated circuit of the present invention automatically defines the parameters for obtaining the output image (P1) by optimizing these two ratios, respecting the condition that the output image must not exceed the predetermined maximum number (N1m) of pixels. In this way, an optimized output image (P1) is obtained and can be transferred for further processing. In this way, only images not exceeding N1m pixels are transferred, thus making it possible to save energy and resources compared to transferring a full-size image (e.g., as described in U.S. Patent Application Publication No. 20220141409).

[0025] The integrated circuit of the present invention includes an image sensor and a processing unit. The image sensor includes an array formed by a nominal number (N0) of photosensors configured to capture values ​​of N0 pixels that define a corresponding input image (P0). Each input image (P0) is contained within a primary window (W0) having a primary window area (A0) and a nominal pixel density D0 = N0 / A0, defined as the number of pixels per unit area, and characterizing the resolution of the input image (P0).

[0026] The processing unit is configured to identify the area of the main window (W0) in which motion is detected between two time points. To detect these areas and motion, the processing unit performs the following actions. In a first step, the processing unit controls a predetermined sub-array of Nr (Nr ≦ N0 (more preferably Nr < N0)) reference optical sensors to capture a series of consecutive reference images (Prefj) at various times. Each reference image (Prefj) is defined by Nr corresponding reference pixels (pr).

[0027] Referring to FIG. 1(b), when a given reference image (Pref2) showing two flying birds is captured, the value difference between the corresponding reference pixels (pr) from the given reference image (Pref2) and the reference image (Pref1) (see FIG. 1(a)) preceding the given reference image (Pref2) in the series is determined. As shown in FIG. 2(a), a number (Nm) of motion pixels (pm) (see black squares) whose value difference exceeds a predetermined threshold deviation (indicating that the bird has moved within the area containing the motion pixels) are selected.

[0028] The reference image (Pref1) to which a given reference image (Pref2) is compared can be the first image in a series of all images preceding the given image when the trigger condition was not satisfied. For example, two consecutive reference images (Pref1, Pref2) are captured and the values of their reference pixels (pr) are compared to each other. If no motion pixels (pm) satisfying the trigger condition are identified, or if motion pixels (pm) that are not sufficient to satisfy the trigger condition are identified, the values of the third reference image (Pref3) and its reference pixels (pr) are compared to the first reference image (Pref1) until motion is detected when the trigger condition is satisfied, and so on. When the trigger condition is met, the processing unit continuously switches from the motion detection mode to the image capture mode described.

[0029] In the image capture mode, the processing unit is configured to first optimize the characteristics of the output image (P1) to be captured before capturing the output image (P1). The processing unit is configured to perform the following actions.

[0030] The processing unit stores pixels up to a predetermined maximum output number N1m so as not to exceed the number of output pixels (N1) that define the output image (P1) transferred from the integrated circuit. The predetermined maximum number of pixels (N1m) is smaller than the nominal number (N0) (i.e., N1 ≦ N1m < N0). The processing unit is preferably configured to enable the user to modify the pixels of the maximum output number (N1m) (since this parameter will affect the power consumption and resources required to operate the image capture device).

[0031] To optimize the parameters of the output image (P1) before acquisition, the processor defines an output window (W1) of an output window area (A1) below the main window area (A0) that surrounds at least a predetermined number (Nmt) of moving pixels (pm) (i.e., A1 = n × A0 and 0 < n < 1). The output window (W1) defines the size and area of the output image (P1) to be captured. Based on the output window area (A1) thus defined, the processor also defines the number of pixels (N1) that form the output image (P1) (N1 = m × N0 ≦ N1m, where 0 < m ≦ n < 1). The ratio (N1 / A1) of the number of outputs (N1) of the pixels thus defined for the output window area (A1) defines the output pixel density D1 = N1 / A1 ≦ D0 of the output image (P1) to be captured.

[0032] The processor is configured to control the array of N0 photosensors to capture an output image (P1) defined by an output window (W1) having an output window area (A1) and an output pixel density (D1). The output image (P1) thus captured has an output image less than or equal to a predetermined maximum output number N1m, which is important for limiting the weight of the output image (P1) to be transferred. At the same time, by optimizing the size and area within the main window (W0) of the output image (P1) on the one hand, and maximizing the number of pixels (N1) that satisfy the condition that the predetermined maximum output number of pixels (N1m) is not exceeded (i.e., N1≦N1m), the captured output image has a maximum resolution with the highest possible output pixel density (D1) that satisfies this condition. The processing unit is configured to transfer the output image (P1) containing N1 pixels from the integrated circuit for further processing.

[0033] Main window (W0), main window area (A0), nominal number (N0) of pixels The integrated circuit includes an array of a nominal number (N0) of photosensors. Each photosensor is configured to capture the value of a corresponding pixel. Several pixels combine to form an image. If the values ​​of all photosensors in the array were used to form an image, the image would be formed by N0 pixels and would be confined within a main window (W0) of the output image area (A0). The resolution of the image is characterized by pixel density. The maximum density for a given array is the nominal pixel density (D0 = N0 / A0). The pixel density of an image captured by a given array of photosensors can be less than D0 (e.g., by binning), even if fewer than N0 pixels form the image (because there cannot be more than N0 pixels captured by N0 photosensors), and the size of each pixel is constant.

[0034] For example, a video graphics array (VGA) has a resolution of 640 pixels long by 480 pixels high, resulting in N = 307,200 pixels. If each pixel has a pixel area (a), then the primary window area would be A = 307,200 x a. An output image (P) having less than N can be captured by the VGA. For example, 2 x 2 binning results in a 1 / 4 VGA (= QVGA) with an output number of pixels (N = 320 x 240 pixels = 76,800 pixels). Similarly, 4 x 4 binning results in a 1 / 4 x 1 / 4 VGA (= QQVGA) with an output number of pixels (N = 160 x 120 pixels = 19,200 pixels).

[0035] On the other hand, if the area surrounding fewer than N0 pixels is reduced, the output density D1 = N1 / A1 remains constant (without binning) because the output number (N1) of pixels enclosed within the output area (A1) is reduced in proportion to the reduction in output area (A1). Reducing the output number (N1) of pixels in a given output image area (A1) can be done by aggregation (or binning) or by thinning. The output pixel density (D1) of the output image (P1) is reduced as a result of such an operation.

[0036] Continuously capturing images with as many as a nominal number (N0) of pixels when no events occur in front of the camera is a waste of energy and processing resources. For this reason, the integrated circuit includes a processing unit configured to initially operate in a motion detection mode configured to detect whether motion has occurred within a field defined by a primary window (W0). The motion detection mode is also configured to detect regions within the primary window (W0) where motion has occurred. The motion detection mode is discussed in more detail subsequently.

[0037] When motion is detected within the field defined by the primary window (W0), the processing unit of the integrated circuit can switch to image capture mode. While it is easy to capture a full-size output image of the primary window area (A0) and nominal density (D0), transferring such a heavy output image (P1) from the integrated circuit to the image processing unit consumes significant energy and resources. For this reason, the processing unit of the integrated circuit is configured to capture an image containing up to a predetermined maximum output number of pixels (N1m) but maximize the resolution of the output image by optimizing (preferably maximizing) the product D1 x A1, i.e., by maximizing the output pixel density D1 = N1 / A1 for a given output image area A1 and respecting the condition D1 x A1 = N1 ≦ N1m, and / or by maximizing the output image area A1 for a given output pixel density D1.

[0038] In this way, the output image is sufficiently lightweight for transfer from the integrated circuit for further processing (since it is formed by only N1 pixels and has the highest possible resolution allowed by a predetermined maximum output number (N1m) of pixels, with D1 tending toward or even equal to the density N1m / A1). In some embodiments, it is preferable that N1=N1m. This is the case, for example, because many artificial intelligence algorithms for processing the output image (P1) are configured to operate on pictures of a predetermined and fixed size. By capturing and transmitting a fixed-size (N1=N1m) output image from the integrated circuit, the artificial intelligence algorithm can process the output image (P1) directly without having to reformat it before processing it.

[0039] Motion Detection Mode In the motion detection mode, the processing unit controls a sub-array of only a number Nr of reference optical sensors to capture a series of consecutive reference images (Prefj) at various times. Each reference image (Prefj) is defined by Nr corresponding reference pixels (pr). The number Nr of optical sensors and reference pixels (pr) can be the same as the nominal number N0 of optical sensors and pixels, but for energy saving purposes, it is preferably Nr < N0 as shown in FIGS. 1(a) and 1(b), where the reference pixels (pr) are represented by white squares and shaded squares.

[0040] As shown in FIGS. 6 loops (g)-(g1)-(f), the processing unit can compare the values of the reference pixels (pr) between a given reference image (Pref1) and each consecutive reference image (Pref2) to detect any value difference exceeding a predetermined threshold deviation. Alternatively, as shown in FIGS. 6 loops (g)-(g2)-(f), the processing unit can compare a consecutive reference image (Pref2) within a series of reference images (Pref1-Pref3) with a preceding reference image (Pref1) preceding the consecutive reference image (Pref2) until motion is detected. Values having reference pixels (pr) that differ by more than the predetermined threshold deviation are identified and stored as motion pixels (pm). For example, by comparing the birds captured in the reference images (Pref1, Pref2) in FIGS. 1(a) and 1(b), it can be understood that "the values of six reference pixels have changed due to the displacement of two birds within the time separating the capture series of the reference images (Pref1, Pref2), and thus are identified by the processing unit as motion pixels (pm)". These motion pixels (pm) are represented as black squares in FIG. 2(a).

[0041] The reference pixels (pr) may be uniformly distributed throughout the primary window area (A0), or alternatively, may be located in critical areas of the window (W0) where motion is expected. The processing unit is preferably configured to allow modification of the number (Nr) and location of the reference photosensors depending on the specific application to which the image capture device is assigned. The value of the predetermined threshold deviation depends on the type of event expected to occur and may also preferably be modified by the user. For example, if the image capture device is programmed to detect the motion of a flying bird, the motion of leaves in surrounding trees should not be identified as motion, and value fluctuations associated with moving leaves should be below the predetermined threshold deviation. Those skilled in the art know how to program the predetermined threshold deviation to identify only motion significant for the specific purpose of the image capture device.

[0042] In areas where there are no objects (such as walls in the foreground) that are relevant for the purposes of the image capture device, the main window (W0) may include one or more masked portions (MA) as shown in Figures 4(a) and 4(b). The masked portions (MA) are: does not contain any reference pixels (pr, pri), or - It contains reference pixels (pr, pri) but is either not considered by the processing unit with respect to the trigger condition or with respect to a predetermined number of motion pixels (pm).

[0043] When the number Nm of moving pixels (pm) satisfies the trigger condition, the processing unit is configured to move from the motion detection mode to the image capture mode. For example, the trigger condition may be considered satisfied if the number Nm of moving pixels (pm) is greater than or equal to a predetermined lower trigger number (Ntd) (i.e., 0 < Ntd ≤ Nm). Additionally or alternatively, the trigger condition may be considered satisfied if the number Nm of moving pixels (pm) is less than or equal to a predetermined upper trigger number (Ntu) (i.e., if 0 < Ntd ≤ Nm). By combining the above two conditions, the trigger condition will be satisfied if 0 < Ntd ≤ Nm ≤ Ntu. For example, the trigger condition may be satisfied if the number (Nm) of moving pixels (pm) is between 50% and 100%, preferably between 75% and 95%, or preferably equal to 100% of the number Nr of reference pixels.

[0044] The predetermined lower trigger number (Ntd) may be required, for example, to prevent the processing unit from interpreting changes in the limit number of reference pixels as motion. For example, it may not be desirable to consider that the branches at the corners of the main window (W0) moving with the wind satisfy the trigger condition. The predetermined lower trigger number (Ntd) can be expressed as the absolute value (Ntd) of the lower limit of the number Nm of moving pixels (less than this, the trigger condition is not satisfied (i.e., TIFF2026501729000002.tif8170)). Alternatively, the predetermined lower trigger number (Ntd) of moving pixels can be defined in terms of the ratio (nmd%) of the number (Nr) of reference pixels (pr) (i.e., Ntd = nmd% × Nr). In the latter case, the predetermined lower trigger number (Ntd) can be included within 50 - 100%, preferably 70 - 95%, more preferably 75 - 80% of the number (Nr) of reference pixels (pr). Ntd = 100% means that the trigger condition is satisfied only when all reference pixels are moving pixels (i.e., Nm = Nr).

[0045] The predetermined upper trigger number (Ntu) may be needed, for example, to prevent the processing unit from interpreting as motion a sudden change in light (e.g., a light being turned on or off, or a dark cloud blocking the sunlight) that would affect the signals of all reference pixels. As with the previously discussed predetermined lower trigger number (Ntd), the predetermined upper trigger number (Ntu) is the upper limit of the number of motion pixels Nm, above which the trigger condition is not met (i.e., The predetermined upper trigger number (Ntu) of motion pixels may be expressed as an absolute value (Ntu) of the number of reference pixels (pr) (Nr). Alternatively, the predetermined upper trigger number (Ntu) of motion pixels may be defined in terms of a percentage (nmu%) of the number of reference pixels (pr) (Nr) (i.e., Ntu = nmu% x Nr). In the latter case, the predetermined upper trigger number (Ntu) may be comprised within 75-100%, preferably 85-98%, and more preferably 90-95% of the number of reference pixels (pr) (Nr).

[0046] In a preferred embodiment shown in Figures 4(a) and 4(b), the image sensor is divided into Nb photosensor blocks. Each photosensor block is formed by Ni adjacent photosensors. The blocks cover the entire primary window area without overlapping, so that each photosensor of the nominal number (N0) of photosensors forming the array belongs to only one photosensor block. Each of the Nb photosensor blocks independently includes Nri block reference photosensors, defined as reference photosensors belonging to the block. The input image (P0) is similarly divided into Nb pixel blocks (Bi) corresponding to the Nb photosensor blocks, each of which includes Nri reference pixels (pr). If the value difference of a given reference pixel in the pixel block (Bi) exceeds a predetermined threshold deviation, the given reference pixel (pr) is identified as a motion pixel (pm) in the same manner as previously discussed. Therefore, in motion detection mode, each pixel block (Bi) may include a block number Nmi of motion pixels (pm). All pixel blocks (Bi) do not necessarily have to have the same size, and the geometry of the pixel blocks may tailor the actual topography of the scene enclosed within the frame defined by the primary window (W0).

[0047] The advantage of dividing the main window (W0) into pixel blocks (Bi) is that it allows for defining more target trigger conditions. For example, the trigger condition can be satisfied if the number of blocks Nmi of moving pixels (pm) in at least one pixel block (Bi) is greater than or equal to a predetermined lower block trigger number (Ntdi) (i.e., if 0 < Ntdi ≤ Nmi). Additionally or alternatively, the trigger condition can be satisfied if the number of blocks Nmi of moving pixels (pmi) is less than or equal to a predetermined upper block trigger number (Ntui) (i.e., if 0 < Ntdi ≤ Nmi). By combining the above two conditions, the trigger condition will be satisfied if 0 < Ntdi ≤ Nmi ≤ Ntui. As previously explained with respect to the predetermined lower and upper trigger numbers (Ntd, Ntu), the predetermined lower and upper trigger block numbers (Ntdi, Ntui) can be expressed as the absolute values (Ntd, Ntu) of the upper and lower limits of the number of moving pixels Nm within the block (i), and the trigger condition is not satisfied respectively if it is less than or more than that (i.e., Ntdi and TIFF2026501729000004.tif7170). Alternatively, the predetermined lower and upper trigger block numbers (Ntdi, Ntui) of moving pixels can be defined in terms of the ratios (nmdi%, nmui%)) of the number (Nri) of block reference pixels (pri) (i.e., Ntd = nmdi% × Nr and Ntu = nmui% × Nr). The predetermined lower and upper trigger block numbers (Ntdi, Ntui) can have the same value in all blocks (Bi), or can vary from one block (Bi) to another.

[0048] In one embodiment, each pixel block contains a single reference pixel (pr). If the value difference of the reference pixel (pr) of a given pixel block (Bi) exceeds a predetermined threshold deviation, the trigger condition is satisfied for the given block. However, a pixel block (Bi) may contain one or more reference pixels (pr) (i.e., Nri≧1), and the number of reference pixels may be the same or different in two different pixel blocks. If a pixel block (Bi) contains two or more reference pixels (pr), the conditions for satisfying the trigger condition may vary. For example, the trigger condition is satisfied for a block as soon as the block contains one motion pixel (pm) (see FIG. 5(b)). Alternatively, the pixel block (Bi) must contain at least a predetermined lower block trigger number (Ntdi) of motion pixels to satisfy the trigger condition. For example, the trigger condition may be satisfied for a given pixel block (Bi) only if all reference pixels (pr) of the given block (Bi) are motion pixels (pm) (see FIG. 5(c)). It can be seen that dividing the main window (W0) into pixel blocks (Bi) allows for a constant variation of trigger conditions, which can be adapted to the specific topography and purpose of the image capture device. For example, a finer and denser network of blocks (Bi) and / or reference pixels (pr) can be concentrated in areas of the main window (W0) where an event is expected with a greater probability. In the case where the main window includes mask areas (MA), these can form one or more pixel blocks (Bi).

[0049] When a number Nm of moving pixels (pm) satisfy the trigger condition, the processing unit is preferably configured to generate an interrupt signal indicating a switch to image capture mode and an incoming output image (P1).

[0050] Output window (W1) and output window area (A1) When a trigger condition indicating that motion has been detected within the region of the main window (W0) is satisfied, the processing unit switches from the motion detection mode to the image capture mode. Preferably, the processing unit is configured to generate an interrupt signal indicating the motion detected within the region of the main window and the switch to the image capture mode. Before capturing the output image (P1) to be transferred for further processing, the processing unit of the integrated circuit is configured to optimize the output window area (A1) and the output number of pixels (N1) of the output window (W1) (for example, while satisfying the condition N1 ≦ N1m and maximizing the output pixel density (D1 = N1 / A1) of the output image (P1)). This optimization process is performed within the integrated circuit prior to capturing the output image (P1) and, of course, prior to transferring it outside the integrated circuit for further processing. This solution results in a significant energy savings and requires fewer processing resources as the weight of the output image (i.e., the output number of pixels (N1) is limited to be less than or equal to a predetermined maximum number of pixels (N1m), i.e., N1 ≦ N1m).

[0051] The processing unit defines an output window (W1) of the output window area (A1) that satisfies the condition that the "output window (W1) encloses at least a predetermined lower trigger number (Ntd) of motion pixels (pm)" (where A1 = n × A0 and 0 < n < 1). Once the output window (W1) is defined, the processing unit defines an output pixel density (D1 = N1 / A1 ≦ D0) that satisfies the condition that the "output number of pixels (N1) does not exceed a predetermined maximum output number (N1m): i.e., N1 = m × N0 ≦ N1m, where 0 < m ≦ n < 1". The output area (A1) and the output number of pixels (N1) of the output image (W1) can be optimized in various ways. For example, they can be optimized by satisfying one or more of the following criteria.

[0052] The primary window (W0) is generally rectangular and has a length (L0) and height (H0), A0 = L0 x H0, as shown in Figures 1(a) and 1(b), 2(a), 3(a), 4(a) and 4(b), and 5(a). The output window (W1) is also generally rectangular and has a length (L1) and height (H1), A1 = L1 x H1, such that L1 = nL x L0 and H1 = nH x H0, where nL and nH are each less than or equal to 1 (i.e., nL < 1, nH < 1). The output window (W0) may be selected according to various criteria.

[0053] In one embodiment shown in Figure 2(b), nL is independent of nH and can have different values ​​(i.e., nH ≠ nL is possible). In an alternative embodiment shown in Figure 2(c), the aspect ratio (L1 / H1) of the output window (W1) is such that L1 / H1 = L0 / H0 ⇔ nL = nH and n = nL. 2 =nH 2 In a preferred embodiment, both or each of nH and nL independently have values ​​selected from among preselected values ​​of 1, 1 / 2, 1 / 3, or 1 / 4. Alternatively, the length (L1) and height (H1) of the output window (W1) may be selected from among a predetermined set of output image window lengths (L1) and heights (H1).

[0054] The output window (W1) must enclose at least a predetermined percentage of the moving pixels (pm) of the number (Nm). This enables focusing on the output window (W1) within the region of the main window (W0) where the event occurred along with the detected movement therein. For example, the processing unit may be configured to ensure that the output window (W1) encloses a number of moving pixels that is between 50% and 100% of the number (Nm) of moving pixels, preferably between 75% and 95%, or preferably equal to 100%. In one embodiment, the processing unit may be configured to enclose only the moving pixels that include at least one or two, or more, directly adjacent reference pixels (pr) which are also moving pixels within the output window (W1). This would avoid including some portions of the main window (W0) where only minimal movement has occurred, as indicated by the fact that "a single (isolated) reference pixel (pr) has changed to become a moving pixel (pm) within that portion of W0".

[0055] The output window (W1) may be restricted to an output window area (A1) that is greater than or equal to a predetermined minimum output window area (A1min) such that 0 < A1min ≤ A1 ≤ A0. This enables avoiding selecting an output window (W1) of dimensions that are too small (see, for example, FIG. 2(b) which shows only a certain percentage of both large and small birds) in order to capture a sufficiently wide frame having a sufficiently large overall image of the scene.

[0056] If the image sensor is divided into Nb photosensor blocks, the output window (W1) can encompass all blocks (Bi) that satisfy various criteria. For example, as shown in FIG. 5(b), it is sufficient that one reference pixel (pr) of a pixel block (Bi) be identified as a motion pixel (pm) for the entire pixel block (Bi) to be encompassed within the output image (W0), regardless of whether the pixel block (Bi) contains other reference pixels (pr) that are not motion pixels (pm). Alternatively, to be encompassed within the output window (W0), a pixel block may require that all reference pixels contained therein be identified as motion pixels (pm), as shown in FIG. 5(c). This, of course, can substantially reduce the output window area (A1) of the output window (W1). Between these two limits, a pixel block (Bi) can be encompassed within the output window (W0) as soon as at least a predetermined percentage of its reference pixels (pr) are identified as motion pixels (at least 50% of the reference pixels (pr), preferably at least 67% or 75% of the reference pixels (pr)).

[0057] Having determined the size (A1, L1, H1) and location of the output window (W1) within the primary window (W0), the processing unit is configured to determine the output number (N1) of pixels to optimize the resolution of the output image (P1) while satisfying the condition that the resolution does not exceed a predetermined maximum output number (N1m) of pixels. This process is described subsequently.

[0058] Number of output pixels (N1) and output pixel density (D1) Capturing an output image (P1) of a given output image area (A1) with a reduced output number (N1≦N1m) of pixels has the advantage of facilitating transfer of the output image thus captured for further processing, but also has the disadvantage of possibly reducing its resolution due to a possibly lower output pixel density (D1=N1 / A1). The processing unit of the integrated circuit is configured to maximize the output pixel density (D1) while keeping the output number of pixels (N1) below a predetermined maximum output number of pixels (N1m).

[0059] For example, the processing unit may be configured to ensure that the output pixel density (D1) is not less than a predetermined minimum output pixel density (D1min), i.e., 0 < D1min ≤ D1 ≤ D0. This avoids capturing an output image (P1) with too poor resolution for further processing. If it is not possible to define A1 and N1 such that D1 = A1 / N1 ≥ D1min and at the same time N1 ≤ N1m, the processing unit may be configured to define an alternative output image area (A1) (smaller than the previous one) by using various previously defined criteria. Next, the processing unit checks whether it is possible to define a value of N1 that satisfies the criteria for both D1 ≥ D1min and N1 ≤ N1m.

[0060] Ideally, it would be preferable in all cases to maximize the output pixel density (D1) to its maximum possible value of D1 = D0 (i.e., all the photosensors of the subarray defining the output window (W1) generate the values of the corresponding pixels). Since A1 = n × A0 and N1 = m × N0 ≤ N1m, if m = n, then naturally D1 = D0 (i.e., D1 = mN0 / nA0 = m / nD0). Whenever possible, i.e., if N1 ≤ N1m, the number of output pixels (N1) of the output image (P1) is selected as N1 = nN0, where n is the proportionality constant of A1 to A0 that results in the highest possible value of the output pixel density D1 = D0 (i.e., A1 = n × A0 and N = n × N0 such that D1 = D0) provided by the image sensor. However, if the number of output pixels (N1 = n × N0) of the output image (P1) is greater than the predetermined maximum number of output pixels (N1m) of the pixels (i.e., if N1 = nN0 > N1m), then the proportionality coefficient m must be less than the area proportionality constant n (i.e., m < n) until N1 ≤ N1m. Naturally, the resolution of the output image (P1) is also reduced (the output pixel density (D1) is less than the nominal pixel density (D0)) (i.e., D1 = m / nD0, where m / n < 1).

[0061] However, reducing the output number of pixels (N1) cannot be done arbitrarily. Two common ways to reduce the output number of pixels (N1) in an output image (W1) for a given output image area (A1) include thinning and aggregation (or binning). In thinning, only the pixels in every block, such as 2x2, 4x4, etc., are read out. Thus, the information contained in all unread pixels is lost. To avoid losing information, pixels can be "connected" together to average their signals in a process called aggregation or binning. If the pixels are fully connected in parallel (2x2), the aggregated result will look like a pixel four times larger than the individual pixels (i.e., an aggregation of kxk pixels in pixel area a0 will result in an aggregation area k). 2 (This results in k × a0 aggregated pixels.) This is illustrated in Figures 3(a) and 3(b), the latter of which shows a 2 × 2 aggregation of the image in Figure 3(a), reducing the number of pixels in the output window area (A1) by a factor of 4. According to Pelgrom's law, the signal-to-noise ratio is TIFF2026501729000005.tif9170 should increase, and in the case of 2x2 aggregation Resulting in TIFF2026501729000006.tif8170.

[0062] The processing unit of the present invention can be programmed to reduce the number of output pixels (N1) either by aggregation or decimation. However, aggregation is preferred because in many applications it is not desirable to completely lose the information of unread pixels. The processing unit can be configured to maximize the A1 / A0 ratio and to maximize the product D1×A1 as follows. The output window (W1) has an output area A1=nA0, where n<1 (determined as previously explained depending on the number and position of moving pixels). The maximum output pixel density D1=D0 results in a number of pixels N1 (N1=D1×A1=D0×nA0=nN0) of the output. To meet the requirement that the number of output pixels does not exceed the maximum number (N1m) of output pixels having the maximum output pixel density (i.e., N1≦N1m), the number of output pixels N1 must satisfy N1=nN0≦N1m. If this condition is satisfied, the output image (P1) can have an output area A1=nA0 with a maximum output pixel density of D1=D0. On the other hand, if nN0>N1m, the output pixel density (D1) must be reduced such that D1=mD0 (where m<1). At this time, the number of output pixels N1 is equal to N1=(n×m)×N0≦N1m. For a given value of n, the maximum value (m_max) of m that allows the condition N1≦N1m is defined as m_max=N1m / (nN0).

[0063] However, for technical reasons, the output pixel density (D1) cannot always be reduced by just any value of the coefficient m without distorting the output image. For example, aggregation (or binning) can reduce the number of pixels per unit area (= density) by a factor of N×N (where TIFF2026501729000007.tif7170). In this case, it is quite possible that the technically maximum available value (mt) of m is smaller than the previously defined value of m_max (i.e., m=mt<m_max). If the output pixel density (D1) is smaller than m_max D0, the processing unit can be configured to perform the following two alternative options. ● Initially, the processing unit may determine to capture an output image (P1) having an output window (W0) with an output area A1 = nA0 and a resolution defined by the number of output pixels (N1) having an output pixel density D1 = mtD0 (where N1 = mtN0 < N1m). ● Alternatively, the processing unit may be configured to increase the output image area (A1) until the number of output pixels (N1) is close to or equal to (but not less than) N1m with an output pixel density of D1 = mtD0. This option has the advantage of maximizing the output image area (A1) at the maximum available technical output pixel density (D1 = mtD0).

[0064] For example, if the following is the case: ● The main window (W0) has a video graphics array (VGA) resolution of 640 pixels long × 480 pixels high, resulting in N0 = 307,200 pixels. ● The predetermined maximum number of output pixels (N1m) of the output image (P1) is set to N1m = 76,800 pixels (which would correspond to a 1 / 4 VGA (QVGA) resolution of 320 pixels long × 240 pixels within the output window of the main image area (A0)), and ● The output window (W1) has an output image area A1 = nA0 (where n < 1); by maintaining the maximum output pixel density D1 = D0, the number of output pixels (N1) is N1 = n × 307,200 pixels.

[0065] If N1 = n307,200 < 76,800 pixels = N1m, the output image (P1) may maintain the maximum output pixel density (D1 = D0). On the other hand, if N1 = n307,200 > 76,800 pixels = N1m, the output pixel density (D1) must be reduced to reduce the number of output pixels (N1) such that N1(m × n) = N0 and D1 = mD0. The maximum value of the coefficient m (m_max) is defined as m_max = N1m / (nN0).

[0066] For example, if n = 50%, the maximum value of the coefficient m (m_max) is m_max = N1 76,800 / (0.5×307,200) = 0.5, resulting in pixels of the maximum output number (N1 = N1m), where N1 = (m_max×n)N0 = 1 / 4N0 = 76,800 pixels = N1m, and the output pixel density D1 = m_max D0 = 1 / 2D0.

[0067] And 2×2 binning reduces the number of pixels of the output image (P1) in a given output image area (A1) by a factor of 4 with respect to the number of pixels of the input number (N0) (i.e., 2×2 binning reduces the output pixel density of the output image (P1) by a factor of 4 (D1 = 1 / 4D0)). This means that with respect to the output image area A1 = 1 / 2A0, the technically maximum available value of the coefficient m (mt = 1 / 4) is substantially smaller than the maximum value (m_max = 0.5) (i.e., mt = 0.25 < m_max = 0.5), and the number of output pixels (N1) of the pixels of the output image (P1) will be N1 = (mt×n)N0 = N0 / 8 = 38,400 pixels. Therefore, since N1m = N0 / 4, it is possible to increase the output image area (A1) by increasing the proportionality constant n by n1 = n×mt / m_max = 2n (where n1 > n) until the number of output pixels (N1) of the pixels becomes equal to N1m. In this example, the initially defined output area (A1) has the value A1 = nA0 (where n = 0.5). Therefore, with the values considered in this example, the processing unit can be configured to maximize the product A1×D1, and define an output image (P1) having an output pixel density (D1 = mtD0 = D0 / 4) that defines the QVGA resolution, and having an output area A1 = n1A0 = A0 (i.e., n1 = 2n = 1), and having a number of output pixels (N1) equal to N1m (i.e., N1 = N1m = 76,800).

[0068] In the alternative example, N1m = 19,200 pixels. This would correspond to a 1 / 4×1 / 4 VGA (QQVGA) resolution of 160 pixels in length × 120 pixels in height within the output window of the main image area (A0). If n = 50%, the maximum value of the coefficient m (m_max) is m_max = 19,200 / (0.5×307,200) = 0.125 (= 1 / 8), and the maximum output number (N1) with an output pixel density D1 = m_max D0 = D0 / 8 (where N1 = (m_max×n)N0 = N0 / 8 = 19,200 pixels = N1m) results in pixels. As seen above, 2×2 binning results in a value of m = 1 / 4 > 1 / 8 = m_max. This means that 4×4 binning, which results in a value of mt = 1 / 16 < 1 / 8 = m_max, may be assumed. Again, if desired, the output image area (A1) can be increased by a factor of mt / m_max = 2 while maintaining the output number of pixels (N1) below N1m (i.e., N1≦N1m).

[0069] Regardless of the actual value of the output pixel density (D1), if the resulting output number of pixels (N1) is less than N1m (i.e., N1 < N1m), the processing unit can be configured to maintain A1 = nA0, or alternatively, to increase the output image area (A1 = n1A0, where n1 > n) until N1 approaches N1m or preferably becomes equal (but never higher) to it.

[0070] The processing unit is described as first determining the output window area (A1) and then maximizing the output pixel density (D1 = N1 / A1), where a given output window area (A1) is maximized according to at least the number (Nm) and / or position of moving pixels (pm). As previously described, in many cases, due to technical reasons, it becomes impossible to exactly match the number of pixel outputs (N1) to a predetermined maximum output number of pixels (i.e., generally N1 = mtN0 < N1m, where mt is the maximum technically available value of m). Therefore, the output window area (A1) can be increased by expanding the boundaries of the output window (W1) to include the number of pixels required to result in N1 = N1m or at least to result in N1 (i.e., N1→N1m) that tends towards N1m. In other words, if m_max = N1m / N0, then N1 = m_maxN0 = N1m, or if m_max→N1m / N0, then at least N1 = m_maxN0→N1m. This additional operation has the advantage of increasing the field of the output image without reducing its resolution and still satisfying the condition N1≦N1m. Considering the previous example with N1m = 100,000 pixels and N1 = 76,200 pixels (= QVGA), the output image area (A1) can be increased by 24% by adding 23,800 pixels up to N1 = N1m = 100,000 pixels. The additional pixels can be added by adding pixels in consecutive rows of length (L1) or in consecutive columns of height (H1) or both, thus similarly increasing the output window area (A1) of the output window (W1).

[0071] A processing unit that optimizes A1 and D1 such that N1≦N1m Figure 7 shows an embodiment of how the processing unit can be configured to optimize A1 and D1 such that N1 ≤ N1m, as shown in a previous section. The maximum output number of pixels (N1m) is defined. The array of optical sensors imposes values for the main image area (A0) and the nominal number of pixels (N0), resulting in a nominal pixel density (D0 = N0 / A0) (see Figure 7(a)). As shown in Figure 7(b), the first value of the output window area (A1) is defined by defining a value of n such that A1 = nA0. This is performed as previously explained, depending on the number and position of the moving pixels (pm). Since it is always preferable for the maximum output pixel density to result in an output image (P1) of the highest possible resolution, the coefficient m that defines the output pixel density D1 = mD0 is set to 1 (i.e., m = 1 and D1 = D0) (as shown in Figure 7(c)). The output number of pixels N1 = (m × n)N0 of the output image (P1) with output image area A1 = nA0 and output pixel density D1 = mD0 = D0 is equal to N1 = mN0.

[0072] As shown in Figure 7(d), in a first situation where N1 = mN0 ≤ N0, the processing unit can evaluate whether N1 is equal to N1m. If N1 = N1m, the processing unit sets the parameters of the output image as indicated in Figures 7(e) and 7(n1): i.e., A1 = nA0, D1 = mD0 = D0 (i.e., m = 1), and N1 = nN0 = N1m.

[0073] On the other hand, if N1 < N1m, the processing unit can be satisfied and set the parameters of the output image as indicated in Figures 7(g) and 7(n1): i.e., A1 = nA0, D1 = mD0 = D0 (i.e., m = 1), and N1 = nN0 < N1m.

[0074] On the other hand, the processing unit is configured to maximize the product A1×D1 (see FIG. 7(h)). In this case, as shown in FIG. 7(h), the processing unit is configured to determine the value of the coefficient (n) such that the number of output pixels (N1) of the pixel tends to go towards N1m (preferably equal to N1m) (i.e., N1=n1N0→N1m, preferably N1=n1N0=N1m). Next, the processing unit may set the parameters of the output image as shown in FIG. 7(n2): namely, A1=n1A0, D1=mD0=D0 (i.e., m=1), and N1=n1N0→N1m, preferably N1=n1N0=N1m.

[0075] In the second situation identified in FIG. 7(i), N1=nN0>N1m, resulting in an output image (P0) that is too heavy to export. The number of output pixels (N1) must be reduced. Since the output image area cannot be reduced and is determined to include a number of moving pixels according to various rules discussed previously, the only option to reduce the number of output pixels (N1) is to reduce the output pixel density: D1=mD0<D0 (i.e., m<1). As discussed previously, for technical reasons, the output pixel density cannot be freely changed. This is the case where binning is applied that aggregates the same number of pixels (along the rows) as along the columns of the array. Thus, the processing unit may be configured to determine the maximum technically available value of m (mt) that results in an output number of pixels (N1) below N1m (see FIG. 7(i)). If N1=(mt×n)N0=N1m, the processing unit sets the parameters of the output image as indicated in FIGS. 7(j) and 7(n3): namely A1=nA0, D1=mtD0, and N1=(mt×n)N0=N1m.

[0076] Now, as shown in the previous example, the output number N1=(mt×n)N0<N1m may be lower than N1m. In this case, the processing unit may be satisfied and may set the parameters of the output image as indicated in FIGS. 7(l) and 7(n3): namely A1=nA0, D1=mtD0, and N1=(mt×n)N0<N1m.

[0077] On the other hand, the processing unit is configured to maximize the product A1×D1 (see FIG. 7(m)). In this case, as shown in FIG. 7(m), the processing unit is configured to determine the value of the coefficient (n) (n1>n) such that the output number of pixels (N1) tends toward (preferably equals) N1m (i.e., N1=n1N0→N1m, preferably N1=n1N0=N1m). The processing unit may then set the parameters of the output image as shown in FIG. 7(n4): A1=n1A0, D1=mtD0, and N1=(mt×n1)N0→N1m, preferably N1=(mt×n1)N0=N1m.

[0078] Once the parameters of the output image (P1), including the resolution of the output window (W1), the output window area (A1), and the output pixel density (D1), which satisfy the condition that "the output number of pixels (N1) must not exceed a predetermined maximum output number of pixels (N1m) (i.e., N1≦N1m)," are defined as previously described, the processing unit of the integrated circuit is configured to control the array of photosensors to capture the output image (P1). Once captured, the output image can be exported for further processing. Because the weight of the output image (P1) is limited to no more than N1m pixels, the power consumed for transmission is minimized by avoiding the transmission of large and dense frames, and the processing resources required for the transmission of the output image are also reduced.

[0079] The gist of the present invention is to move the processing for selecting the area of ​​interest to the integrated circuit prior to exporting the output image (P1) (rather than capturing the full size output image (P1) and re-framing it in a separate image processing unit after transfer of the output image). This means that the camera can maintain a large field of view without losing resolution and will only send the relevant subset of the picture to the processing unit.

[0080] Image capture / processing device The present invention also relates to an image capture / processing device comprising an integrated circuit as previously defined, a lens in optical communication with an array of N0 photosensors of the image sensor of the integrated circuit, and an image processing unit in communication with the integrated circuit, the image processing unit configured to receive and suitably store an output image (P1) from the integrated circuit.

[0081] The image processing unit is preferably configured to receive an interrupt signal (indicating an incoming output image (P1)) generated by the processing unit of the integrated circuit, such that upon receiving the interrupt signal, the image processing unit may switch from an image processing unit sleep mode to an image processing unit wake-up mode.

[0082] FIG. 6 shows a flowchart for explaining various operations configured to be executed by the processing unit of the integrated circuit of the present invention. By an integrated circuit including an array optical sensor for capturing an input image (P0) framed within a main window (W0) of a main window area (A0) having a nominal pixel density (D0), the processing unit stores pixels for which the output number (N1) of pixels defining the output image (P1) transferred from the integrated circuit does not exceed a predetermined maximum output number N1m < N0 (i.e., N1 ≦ N1m < N0) (see boxes (a) and (b) of FIG. 6). The sub-array of reference pixels (pr) must be defined as shown in box (c) of FIG. 6. Of course, this operation is only executed when the camera is installed. However, this operation can preferably be modified by the user at any time. To detect motion within the area included within the frame of the main window (W0), the processing unit is configured to capture a series of consecutive reference images (Pref1, Pref2) at various times (see boxes (d) and (e) of FIG. 6). Reference pixels (pr) that satisfy the criterion of being considered moving pixels (pm) are identified, and the processing unit checks whether a trigger condition is satisfied by the moving pixels (pm) in order to conclude that there was motion between the first reference image and the second reference image (Pref1, Pref2) (see boxes (f) and (g) of FIG. 6). If the trigger condition is not satisfied, the processing unit captures the next reference image (Pref3). In the first embodiment shown in loop (g1), the processing unit repeats steps (f) and (g), comparing the next reference image (Pref3) with the first reference image (Pref1) until the trigger condition is satisfied, and motion is detected. In an alternative embodiment shown by loop (g2), the next reference image (Pref3) is compared with the previous reference image (Pref2), and the processing unit repeats steps (f) and (g) by, for example, replacing the first reference image (Pref1) and the second reference image (Pref2) for comparison until the trigger condition is satisfied, and motion is detected. Next, the processing unit moves to the image capture mode, where the parameters of the captured output image (P1) are first optimized before capturing the output image (see box (h) of FIG. 6).In particular, an output window (W1) and an output image area (A1) are defined to focus on the region of the primary window (W0) where motion occurs. The output pixel density (D1 = N1 / A1) of the output image (P1) is then maximized by ensuring that the condition N1 ≤ N1m is satisfied. Once the parameters of the output image are set, the processing unit is configured to capture the output image (P1) with those parameters and transfer the output image (P1) from the integrated circuit for further processing or analysis (see boxes (i) and (j) in Figure 6). The processing unit is then ready to transition back into motion detection mode, capture a new series of consecutive reference images (Pref1; Pref2) at various times, and repeat the aforementioned steps (see the loop between boxes (j) and (d) in Figure 6).

[0083] TIFF2026501729000008.tif230170

Claims

1. 1. An integrated circuit for automatically optimizing a ratio (A1 / A0) of an output image area (A1) to a main window area (A0) and for optimizing (preferably maximizing) a product D1×A1 of an output pixel density (D1) multiplied by said output image area (A1) of a motion detection area defining an output image (P1) consisting of a predetermined maximum number (N1m) of pixels, said integrated circuit comprising: an image sensor including an array formed by a nominal number (N0) of photosensors configured to capture values ​​of N0 pixels defining corresponding input images (P0), each input image being contained within a primary window (W0) having said primary window area (A0), said input images having a nominal pixel density D0=N0 / A0; ●The following actions: controlling a predetermined sub-array of Nr (Nr≦N0) reference photosensors to capture a series of successive reference images (Prefj) at different times, each reference image defined by Nr corresponding reference pixels (pr); a processing unit configured to, when a given reference image is captured, determine a value difference between corresponding pixels from said given reference image and a reference image preceding said given reference image in said sequence, and to select a number (Nm) of motion pixels (pm) whose value difference exceeds a predetermined threshold deviation; In an integrated circuit comprising: The processing unit is configured to store a predetermined maximum output number N1m<N0 (i.e., N1≦N1m<N0) of pixels that is not exceeded by the output number (N1) of pixels defining the output image (P1) transferred from the integrated circuit; When the number Nm of motion pixels (pm) satisfies a trigger condition, the processing unit: ●The following: an output window (W1) of an output window area (A1) enclosing at least a predetermined lower trigger number (Ntu) of motion pixels (pm), where A1 = n x A0 and 0 < n ≤ 1; ○ At output pixel density D1 = N1 / A1≦D0, the number of output pixels N1 = m × N0≦N1m, where m<1 and 0<m≦n<1 To define controlling the array of N0 photosensors to capture the output image (P1) defined by the output window (W1) having the output window area (A1) and output pixel density (D1); - An integrated circuit, characterized in that it is adapted to transfer from said integrated circuit an output image (P1) comprising N1 pixels.

2. 2. The integrated circuit of claim 1, wherein W1 and N1 are determined based on the following criteria: the primary window (W0) has a length (L0) and a height (H0) and A0=L0×H0, and the output window (W1) has a length (L1) and a height (H1) and A1=L1×H1 such that L1=nL×L0 and H1=nH×H0, where nL and nH are each less than or equal to 1 (i.e., nL≦1, nH≦1), and preferably each independently equal 1, 1 / 2, 1 / 3, or 1 / 4, and the aspect ratio (L1 / H1) of the output window (W1) is preferably equal to the nominal aspect ratio (L0 / H0) (i.e., L1 / H1=L0 / H0 and n=nL ２ = nH ２ ); said predetermined lower trigger number (Ntu) of motion pixels (pm) is comprised between 50% and 100% of said number Nr of reference pixels (pr), preferably between 75% and 95%, or preferably equal to 100%; The output window (W1) is rectangular, and the predetermined number of motion pixels (pm) are inscribed within the output window (W1); the length (L1) and height (H1) of the output window (W1) are selected from a predetermined set of output image window lengths and heights; The output window area (A1) is greater than or equal to a predetermined minimum output window area (A1min), such that 0<A1min≦A1≦A; The output pixel density (D1) is equal to or greater than a predetermined minimum output pixel density (D1min) so that 0<D1min≦D1≦D0. An integrated circuit characterized by satisfying one or more of the following.

3. 3. The integrated circuit of claim 1, wherein a processing unit for defining the output window (W1) and the output pixel density (D1) of the output image (P1) defined by the output number (N1≦N1m) of pixels is as follows: For a given output pixel density (D1=mD0), maximize the output window area (A1=n1A0) such that the output number of pixels (N1=(m×n1)N0) tends towards (and preferably equals) the maximum output number of pixels (N1m), i.e., N1=(m×n1)N0×N1m; For an output window area (A1=nA0), maximize the output pixel density (D1=mtD0) such that the output number of pixels (N1=(mt×n1)N0) tends towards (and preferably equals) the maximum output number of pixels (N1m) (i.e., N1=(mt×n1)N0×N1m).

1. An integrated circuit comprising:

4. 4. An integrated circuit according to any one of claims 1 to 3, characterized in that the processing unit is configured to enable modification of the number (Nr) and positions of the reference photosensors.

5. 5. The integrated circuit according to claim 1, wherein the reference light sensor comprises: uniformly distributed throughout the primary window area (A0), or An integrated circuit, characterized in that it is placed in a critical area of ​​said window (W0) where movement is expected.

6. 6. An integrated circuit according to any one of the preceding claims, characterized in that the processing unit is configured to enable modification of the maximum output number (N1m) of pixels.

7. 7. An integrated circuit according to any one of claims 1 to 6, characterized in that the processing unit is arranged to generate an interrupt signal indicating an incoming output image (P1) when the number Nm of moving pixels (pm) satisfies the trigger condition.

8. 8. An integrated circuit according to any one of claims 1 to 7, wherein the input image has a Video Graphics Array (VGA) resolution of 640 pixels long by 480 pixels high resulting in N0 = 307,200 pixels, the predetermined maximum output number (N1m) is a maximum of 76,800 pixels, and the output image (P1) is: Quarter VGA (QVGA) resolution of 320 pixels long by 240 pixels high yielding N1 = 76,800 pixels, or An integrated circuit characterized by having a 1 / 4 x 1 / 4 VGA (QQVGA) resolution of 160 pixels wide by 120 pixels high, yielding N1 = 19,200 pixels.

9. 9. An integrated circuit according to any one of claims 1 to 8, characterized in that the trigger condition is satisfied if the number Nm of moving pixels (pm) is greater than or equal to a predetermined lower trigger number (Ntd) (i.e. 0<Ntd≦Nm), and preferably the number Nm of moving pixels (pm) is also less than or equal to the predetermined lower trigger number (Ntd) (i.e. 0<Ntd≦Nm).

10. 9. An integrated circuit according to any one of claims 1 to 8, the image sensor is divided into Nb photosensor blocks, each formed by Ni adjacent photosensors, such that each photosensor of the nominal number (N0) of photosensors forming the array belongs to a single photosensor block, each of the Nb photosensor blocks independently including a number Nri of block reference photosensors defined as the reference photosensors belonging to the block; ●An integrated circuit characterized in that each of the input images (P0) includes Nb pixel blocks (Bi) corresponding to the Nb photodetector blocks, and each of the pixel blocks (Bi) includes Nri reference pixels (pri) and a block number Nmi of motion pixels (pmi).

11. 11. An integrated circuit according to claim 10, characterized in that the trigger condition is satisfied when the block number Nmi of moving pixels (pm) of at least one pixel block (Bi) is greater than or equal to a predetermined lower block trigger number (Ntdi) (i.e., if 0<Ntdi≦Nmi), and the block number Nmi of moving pixels (pmi) is also preferably less than or equal to a predetermined upper block trigger number (Ntui) (i.e., if 0<Ntdi≦Nmi (≦Ntui)).

12. 12. An integrated circuit according to claim 11, characterized in that said output image (P1) comprises at least all pixel blocks (Bi) which satisfy said trigger condition.

13. 13. An integrated circuit according to any one of claims 1 to 12, wherein the main window (W0) comprises one or more mask portions (MA), each mask portion (MA) comprising: does not contain any reference pixels (pr, pri), or An integrated circuit comprising reference pixels (pr, pri) which are either not taken into account with regard to said trigger condition or with regard to said predetermined number of motion pixels (pm).

14. an integrated circuit according to any one of claims 1 to 13, a lens in optical communication with an array of N0 photosensors of the image sensor of the integrated circuit; ●An image capture / processing device characterized in that it includes an image processing unit in communication with the integrated circuit, the image processing unit being configured to receive and suitably store the output image (P1) from the integrated circuit.

15. 15. An image capture / processing device as described in claim 14, characterized in that the image processing unit is configured to receive an interrupt signal (indicating an incoming output image (P1)) generated by the processing unit of the integrated circuit, and to switch from the image processing unit sleep mode to the image processing unit wake-up mode upon receiving the interrupt signal.

Images