Data processing system for processing pixel data to indicate contrast

Abstract

The present disclosure relates to a data processing system configured to receive data of a pixel from each image of a sequence of images captured by an imaging device as a system input, the data of the pixel comprising center pixel data and adjacent pixel data, and configured to generate a system output, comprising: a network comprising a plurality of input nodes, each input node is configured to receive pixel data from a corresponding pixel as an input, each input node includes an input node leakage integrator (LI) having a first time constant and an input weight, and each input node is configured to receive the pixel data from the corresponding pixel by applying the input node leakage integrator and the corresponding weight to the input. To generate an input node output; and an output node comprising an output node weight and a multiple input leakage integrator having a second time constant, where the output node is configured to receive each input node output, where the output node is configured to receive pixel data from the center pixel, wherein the output node is configured to combine pixel data from the center pixel with output node weights to produce weighted center pixel data, and wherein the output node is configured to apply a multiple input leakage integrator to the weighted center pixel data and each node output, and wherein the output node is configured to generate a system output with a combination of 15-weighted center pixel data and each node output, and wherein the second time constant is less than the first time constant and the system output is used to indicate the contrast.

Description

Technical Field

[0001] The present disclosure relates to a data processing system, system, apparatus, methods, computer program products and non-transitory computer-readable storage media for processing pixel data. More specifically, the present disclosure relates to a data processing system, system, apparatus, method, computer program product and non-transitory computer-readable storage media for processing pixel data as defined in the introductory parts of the independent claims. Background Art

[0002] Event-based cameras or event cameras are known. Each pixel within an event camera operates independently and asynchronously, reporting when there is a change in brightness or contrast and remaining silent otherwise. There are several types of event cameras. One type of event camera is an event-based camera with a fixed contrast threshold. This camera is described by Cedric Scheerlinck et al. in “Asynchronous Spatial Image Convolution for Event Cameras” (In: IEEE Robotics & Automation Letters 4(2), pp. 816-822, 2019). Another type of event camera is an event-based camera with a dynamic contrast threshold. This camera is described by Feng, Yang et al. in “Event Density-Based Denoising Method for Dynamic Vision Sensors” (In: Applied Sciences, 10.6, 2020).

[0003] However, these types of event-based cameras can be noisy (especially when high magnification is required) and / or have a high level of complexity.

[0004] Therefore, there may be a need for a (camera) system that reduces noise and / or reduces complexity. Preferably, such a system provides or achieves one or more of the following: improved performance, greater reliability, increased efficiency, uses less computer power, takes up less storage space, is less complex, and / or consumes less energy.

[0005] US 2021 / 0142086 A1 discloses a device comprising: an image sensor comprising a plurality of pixel cells; a frame buffer; and a sensor computing circuit configured to: receive a first image frame comprising a first valid pixel and a first invalid pixel from the frame buffer, the first valid pixel being generated by a first subset of pixel cells selected based on first programming data; perform an image processing operation on the first subset of pixels of the first image frame, whereby a second subset of pixels of the first image frame is excluded from the image processing operation to generate a processing output; generate second programming data based on the processing output; and transmit the second programming data to the image sensor to select a second subset of pixel cells to generate a second valid pixel for a second image frame.

[0006] However, noise reduction may still be desired. In addition, better noise stability may be desired. Moreover, the processing in US 2021 / 0142086 A1 includes many operations for indicating contrast (e.g., because the method involves convolution operations and / or because the method involves operations on blocks of pixels). Therefore, faster processing speed, higher efficiency and / or lower complexity may be desired. Summary of the invention

[0007] An object of the present disclosure is to mitigate, alleviate or eliminate one or more of the above-mentioned defects and deficiencies in the prior art and to at least solve the above-mentioned problems.

[0008] According to a first aspect, a data processing system for processing pixel data is provided. The data processing system is configured to receive pixel data from each image of an image sequence captured by an imaging device as a system input, the pixel data including data of a central pixel and data of adjacent pixels, and the data processing system is configured to have a system output, including: a plurality of input nodes, each input node is configured to receive pixel data from a corresponding pixel as input, each input node includes an input node leakage integrator having a first time constant and a weight for the input, and each input node is configured to generate an input node output by applying the input node leakage integrator and the corresponding weight to the input; and an output node, which includes an output node weight and a multi-input leakage integrator having a second time constant. a multi-input leaky integrator configured to apply a multi-input leaky integrator to the weighted center pixel data and each input node output, and wherein the output node is configured to generate a system output that includes only difference data generated by subtracting each input node output (134a, ..., 134i) from the weighted center pixel data; and wherein the second time constant is less than the first time constant, and wherein the system output indicates a contrast.

[0009] According to some embodiments, the method further comprises obtaining a temporal difference signal indicative of contrast between pixels at the same position in different images by applying a leaky integrator and feeding pixel data from a plurality of images to a data processing system.

[0010] According to some embodiments, the method further comprises obtaining a spatial difference signal indicating a contrast between the central pixel and adjacent pixels in the same image by inputting data from the central pixel and data from adjacent pixels into a data processing system.

[0011] According to some embodiments, a data processing system comprises a threshold unit configured to receive a system output, and the threshold unit is configured to compare the system output to a threshold, and the threshold unit is configured to output a signal indicating that an event is detected.

[0012] According to some embodiments, the closer the central pixel is to the pixel, the greater the weight of the input node.

[0013] According to some embodiments, the sum of all weights of the input nodes is equal to the output node weight.

[0014] According to a second aspect, there is provided an apparatus comprising the data processing system of the first aspect or any of the above embodiments and an imaging device, the imaging device being configured to capture pixel data and transmit the captured pixel data to the data processing system.

[0015] According to some embodiments, the imaging device is a multiphoton microscope, a confocal laser microscope, or a digital camera.

[0016] According to a third aspect, a device is provided, comprising: a data processing system of the first aspect or any of the above embodiments, the system being used to process each pixel in the pixel data; a database comprising images of known objects; and a post-processing unit, which is configured to receive a system output from each data processing system, is configured to compare the system output with the image of the known object, and is configured to identify, based on the comparison, an (unknown / unidentified) object present in the pixel data as a known object having the smallest difference between the image and the system output.

[0017] According to a fourth aspect, a computer-implemented or hardware-implemented method for processing pixel data is provided. The method includes: receiving data from a pixel of each image in an image sequence as a system input, the pixel data including data from a center pixel and data from neighboring pixels; providing each pixel as an input to a corresponding input node; applying a corresponding input node leak integrator with a first time constant and a corresponding weight to each input, thereby generating an input node output; receiving each input node output and pixel data from the center pixel by an output node; combining (by multiplication) the pixel data from the center pixel with the output node weight, thereby generating weighted center pixel data; applying a multi-input leak integrator with a second time constant to the weighted center pixel data and each input node output, thereby combining the weighted center pixel data and each input node output to generate a system output, the system output including differential data generated by subtracting each input node output from the weighted center pixel data, the second time constant being less than the first time constant; and indicating contrast using the system output.

[0018] According to a fifth aspect, a computer-implemented or hardware-implemented method for automatically focusing an imaging device is provided. The method includes: receiving a plurality of images in an image sequence, each image including pixel data; determining the size of a preliminary region of interest (ROI) of the image according to a user input; selecting the preliminary region of interest based on the determined size; providing each pixel of the preliminary region of interest as an input to a corresponding input node; applying a corresponding input node leak integrator with a first time constant and a corresponding weight to each input to generate an input node output; receiving each input node output and pixel data from a center pixel by an output node; combining the pixel data from the center pixel with the output node weight to generate weighted center pixel data; applying a multi-input leak integrator with a second time constant to the weighted center pixel data and each input node output, thereby combining the weighted center pixel data and each input node output to generate a system output, the system output including the weighted center pixel data obtained by subtracting each input node output (13) from the weighted center pixel data. 4a, ..., 134i), the second time constant is less than the first time constant, and the system output indicates contrast; if the system output is higher than the value currently stored in the first position of the memory, the system output is stored in the first position of the memory, and the position of the preliminary region of interest is stored in the second position of the memory; the preliminary region of interest is moved by one pixel step; the steps of receiving (410) the image, determining, selecting, providing, applying, receiving (435) each input node output and pixel data from the center pixel, combining, applying, storing and moving are repeated until the stopping criteria are met; the position stored in the second position of the memory is set as the position of the region of interest; scanning the focus range to find the highest contrast value of the region of interest; and focusing the imaging device using the focus at the highest contrast.

[0019] According to some embodiments, the stopping criterion is that all possible regions of interest within the pixel's data have been examined or the preliminary region of interest has reached an end position.

[0020] According to some embodiments, the imaging device is a digital camera (240).

[0021] According to some embodiments, the preliminary region of interest is a region of interest located at the upper left corner of the image or a region of interest located at the lower left corner of the image.

[0022] According to a sixth aspect, a computer program product is provided, comprising instructions which, when executed on at least one processor of a processing device, cause the processing device to perform a method according to the fourth aspect, the fifth aspect or any embodiment mentioned herein.

[0023] According to the seventh aspect, a non-temporary computer-readable storage medium is provided, which stores one or more programs configured to be executed by one or more processors of a processing device, and the one or more programs include instructions that, when executed by the processing device, cause the processing device to perform a method according to the fourth aspect, the fifth aspect, or any embodiment mentioned herein.

[0024] The effects and features of the second, third, fourth, fifth and sixth aspects are largely similar to those described in conjunction with the first aspect, and vice versa.

[0025] The embodiments mentioned in relation to the first aspect are largely or completely compatible with the second, third, fourth, fifth and sixth aspects, and vice versa.

[0026] An advantage of some embodiments is reduced noise.

[0027] Another advantage of some embodiments is improved noise sensitivity (eg, compared to dedicated event-based camera hardware).

[0028] Another advantage of some embodiments is that only differential data (in space and time, ie, across the spatial and temporal domains) is output, thereby providing the functionality of an event-based camera (eg, with lower complexity) and / or reducing the amount of data.

[0029] Yet another advantage of some embodiments is that, for example, by combining a data processing system with a digital video camera, functionality equivalent to that of an event-based camera can be achieved with much lower hardware complexity than prior art event-based cameras.

[0030] Yet another advantage of some embodiments is that a simpler camera than used as a prior art event-based camera may be used and achieve the same accuracy as a prior art event-based camera.

[0031] Another advantage of some embodiments is the reduction of noise such as salt and pepper noise in environments such as dark environments where high magnification is required.

[0032] Yet another advantage of some embodiments is that noise in night vision or thermal imaging is reduced.

[0033] Yet another advantage of some embodiments is that moving objects (e.g., moving through a two-dimensional plane parallel to the two-dimensional plane of the sensor) can be identified regardless of speed. Other advantages of some embodiments include better noise stability, faster processing speed, higher efficiency, lower complexity, improved performance, higher / higher reliability, higher accuracy, higher efficiency, less computer power required, less storage space required, lower complexity, and / or lower energy consumption.

[0034] The present disclosure will become clear from the detailed description given below. The detailed description and specific examples disclose preferred embodiments of the present disclosure only by way of illustration. Those skilled in the art can understand from the guidance in the detailed description that changes and modifications can be made within the scope of the present disclosure.

[0035] Therefore, it should be understood that the disclosure disclosed herein is not limited to the specific components of the described equipment or the steps of the described method, because such equipment and methods may be different. It should also be understood that the terms used here are only used for the purpose of describing specific embodiments and are not intended to be limiting. It should be noted that the articles "one", "a kind of", "the", and "said" used in the specification and the appended claims are intended to indicate that there are one or more elements, unless the context clearly indicates otherwise. Therefore, for example, references to "a unit" or "the unit" may include multiple devices, etc. In addition, "including", "comprising", "having" and similar expressions do not exclude other elements or steps. In addition, the term "configuration" or "adaptation" is intended to indicate that a unit or the like is shaped, sized, connected, connectable or otherwise adjusted for a certain purpose. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above objects and other objects, features and advantages of the present disclosure will be more fully understood by referring to the following illustrative and non-limiting detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings.

[0037] Figure 1 is a schematic block diagram illustrating a data processing system according to some embodiments; Figure 2 is a schematic block diagram illustrating an input node included in a data processing system according to some embodiments; Figure 3 is a schematic block diagram illustrating an output node included in a data processing system according to some embodiments; Figure 4 is a schematic diagram illustrating pixel data according to some embodiments; Figure 5 is a schematic diagram illustrating pixel data according to some embodiments; Fig. 6A is a schematic block diagram illustrating an apparatus according to some embodiments; Figure 6B is a schematic block diagram illustrating an apparatus according to some embodiments; Figure 7 is a flow chart illustrating method steps according to some embodiments; Figure 8 is a flow chart illustrating method steps according to some embodiments; Fig. 9is a schematic diagram illustrating an example computer-readable medium in accordance with some embodiments. DETAILED DESCRIPTION

[0038] The present disclosure will now be described with reference to the accompanying drawings, which show preferred example embodiments of the present disclosure. However, the present disclosure may be embodied in other forms and should not be construed as being limited to the embodiments disclosed herein. The disclosed embodiments are provided to fully convey the scope of the present disclosure to the skilled person.

[0039] the term The term "node" mentioned below can refer to a neuron, such as a neuron in an artificial neural network, another processing element (such as a processor) in a processing element network, or a combination thereof. Therefore, the term "network" (NW) can refer to an artificial neural network, a processing element network, or a combination thereof.

[0040] Reference is made hereinafter to a "time-lapse sequence". A "time-lapse sequence" is a sequence (of images) whose images are captured at a lower frequency than the frequency used to view the sequence, for example 10 times or 100 times lower. Thus, when played back at normal speed, time appears to pass faster and thus time passes. However, a "time-lapse sequence" may also be a sequence (of images) whose images are captured at the same frequency as the frequency used to view the sequence. Furthermore, a "time-lapse sequence" may also be a sequence (of images) whose images are captured at a higher frequency than the frequency used to view the sequence, for example 10 times or 100 times higher, so that when played back at normal speed, time appears to pass slower, for example presenting a slow motion effect.

[0041] Reference is made to "contrast" below. Contrast is determined by differences in color, brightness and / or (luminous) intensity, e.g. between one object and other objects or between an object and the background in the same field of view, which is detected (in time and / or space) in a photodetector such as a CCD (Charge-coupled Device) sensor or a CMOS (Complementary MOS) sensor.

[0042] Reference is made to spatial and temporal "differential data" below. Spatially differential data is the difference between data at a first location and data at a second location adjacent to the first location (and subsequent to the first location, i.e., below or to the right of the first location). Temporally differential data is the difference between data at a first location of a first image and data at a first location of a second image that immediately follows (in time or sequence) the first image.

[0043] Reference is made below to "data of a pixel". The pixel data (data) from a pixel may be a scalar value, such as a grayscale value carrying only a single light intensity value. However, the pixel data from a pixel may also be a set of (light) intensity values, such as an intensity value for each of 3 different colors (such as RGB).

[0044] For ease of explanation, only 9 or 25 pixels are depicted in the figures and / or below for images and regions of interest. However, an image may include multiple pixels, such as 1080×1080, 1920×1080, or 3840×2160. In addition, a region of interest may include multiple pixels, such as 100, 1000, or 2000, and may be a square area, a rectangular area, or a circular area.

[0045] Example The following will describe an embodiment, in which Figure 1 1 is a schematic block diagram showing a data processing system 101 according to some embodiments. The data processing system 101 is configured to process pixel data. In addition, the data processing system 101 is configured to receive pixel data (such as Figure 4 , P4, P6, ..., P9 as system input. The data processing system 101 receives or is configured to receive data from pixels P1, P2, ..., P9 of each image in an image sequence, such as a sequence of images captured by an imaging device. The sequence may be a time-lapse sequence. In some embodiments, the data processing system 101 receives (and processes) one image at a time. In addition, the pixel data of each image includes data from a central pixel P5 and data from neighboring pixels P1, ..., P4, P6, ..., P9. Therefore, the central pixel P5 is a pixel surrounded by neighboring pixels P1, ..., P4, P6, ..., P9. Moreover, the data processing system 101 is configured to have or have a system output 120. In some embodiments, the data processing system 101 is or includes a network (NW) 130. In some embodiments, the network 130 is or includes a deep neural network, a deep belief network, a deep reinforcement learning system, a recursive neural network, or a convolutional neural network.

[0046] The data processing system 101 (or network 130) includes a plurality of input nodes 130a, ..., 130e, ..., 130i. Each input node is configured to receive pixel data from a corresponding pixel P1, ..., P9 as input 132a, ..., 132e, ..., 132i. In addition, each input node 130a, ..., 130i includes an input node leakage integrator LIa. This Figure 2130a. Each input node leakage integrator LIa has a first time constant TC1. The equation of each leakage integrator is dX / dt=-Ax+C, where C is the input and A is the leakage rate. The time constant of the leakage integrator (i.e., the first time constant TC1) is the inverse of the leakage rate, i.e., 1 / A (1 divided by A). Moreover, Figure 2 As shown (for input node 130a), each input node 130a, ..., 130i includes corresponding weights Wa, ..., Wi for input 132a, ..., 132i (to input nodes 130a, ..., 130i). Each input node 130a, ..., 130i is configured to generate (or produce) input node outputs 134a, ..., 134e, ..., 134i. Each input node output 134a, ..., 134i is generated by applying an input node leakage integrator LIa and corresponding weights Wa, ..., Wi to the input 132a, ..., 132i. In some embodiments, the inputs 132a, ..., 132i are first weighted by corresponding weights Wa, ..., Wi, and then the corresponding input node leakage integrator LIa is applied to the weighted inputs. Alternatively, the input node leakage integrator LIa is first applied to the inputs 132a, ..., 132i, and then the corresponding weights Wa, ..., Wi are applied. Although Figure 2 Only input node 130a is shown, but all other input nodes 130b, ..., 130i operate in the same way, ie including weights and a leaky integrator (with time constant TC1).

[0047] The data processing system 101 (or the network 130) includes an output node 140a. The output node 140a includes an output node weight Wnw1. In addition, the output node 140a includes a multi-input leaky integrator LIz. Udaya Rongala et al. give an example of a multi-input leaky integrator suitable for the data processing system 101 in "Non-Spiking Neuron Model with Dynamic Leakage to Avoid Recurrent Network Instability" (https: / / www.frontiersin.org / articles / 10.3389 / fncom.2021.656401 / full). The multi-input leaky integrator LIz has a second time constant TC2. In some embodiments, the multi-input leaky integrator LIz is an adder with a storage function and a time constant, and the equation (LIz) for each input is in the form of dX / dt=-Bx+C, where C is the input and B is the leakage rate. The time constant of the multi-input leaky integrator (i.e., the second time constant TC2) is the reciprocal of B, i.e., TC2=1 / B (1 divided by B). Furthermore, the output node 140a is configured to receive (or receive) each input node output 134a, ..., 134i. The output node 140a is configured to receive (or receive) pixel data from the center pixel P5. Figure 3 As shown, output node 140a is configured to combine (or perform combination) the pixel data from center pixel P5 with output node weight Wnw1 to produce weighted center pixel data. In some embodiments, the combination is performed by multiplying the pixel data with output node weight Wnw1. Furthermore, as shown in FIG. Figure 3As shown, the output node 140a is configured to apply (or have applied) the multi-input leak integrator LIz to the weighted central pixel data and each input node output 134a, ..., 134i. In some embodiments, the multi-input leak integrator Liz receives (or is configured to receive) the weighted central pixel data and each input node output 134a, ..., 134i, and applies (or is configured to apply) the above LIz equation to the weighted central pixel data and all input node outputs 134a, ..., 134i. The output node 140a is configured to generate (or generate) a system output 120 as a combination of the weighted central pixel data and each input node output 134a, ..., 134i. In some embodiments, the combination is generated by adding or subtracting each input node output 134a, ..., 134i from the weighted central pixel data. Alternatively, the combination is generated by adding or subtracting the weighted central pixel data from the sum of each input node output 134a, ..., 134i. In some embodiments, subtracting each input node output 134a, ..., 134i from the weighted center pixel data can be achieved by changing the sign of each input node output 134a, ..., 134i and then adding each input node output 134a, ..., 134i (after the sign change) to the weighted center pixel data. In some embodiments, the second time constant TC2 is different from (smaller than, larger than) the first time constant TC1. In some embodiments, the second time constant TC2 is smaller than the first time constant TC1, for example, TC2 is 10 times smaller than TC1 or TC2 is 20 times smaller than TC1. In some embodiments, TC1 or TC2 approaches zero. The system output indicates contrast. By making the second time constant TC2 smaller than the first time constant TC1, noise can be reduced (and / or better noise stability can be achieved). In addition, better noise stability can be achieved than (prior art) dedicated event-based camera hardware. Moreover, by applying a leaky integrator and providing pixel data from multiple (continuous) images to the data processing system 101, a time difference signal indicating contrast (between pixels at the same position in different images) can be obtained. By inputting data from the central pixel P5 and data from neighboring pixels P1, ..., P4, P6, ..., P9 into the data processing system 101, a spatial difference signal indicating the contrast (between the central pixel and neighboring pixels in the same image) can be obtained.

[0048] In some embodiments, the data processing system 101 includes a threshold unit 142. The threshold unit 142 is configured to receive (or receive) the system output 120. In addition, the threshold unit 142 is configured to compare (or compare) the system output 120 with a threshold (value). The threshold (value) can be fixed or adaptive, for example, it can be defined by a user. Moreover, the threshold unit 142 is configured to output a signal 144 indicating that an event has been detected. The detected event may be the presence of a (one pixel) object in the center pixel P5. Alternatively, the detected event may be an object that has been identified.

[0049] In some embodiments, the closer the central pixel P5 is to the pixels P1, ..., P9 associated with the input nodes 130a, ..., 130i, the greater the weights Wa, ..., Wi of the input nodes 130a, ..., 130i. Figure 5 As shown, an image may consist of 25 pixels (or more). Figure 5 , the center pixel is P9. The center pixel P9 has 4 4-adjacent pixels P6, P8, P34, P43. In addition, the center pixel P9 has 8 8-adjacent pixels P5, P6, P8, P24, P34, P42, P43, P44. Moreover, the center pixel P9 has 24 24-adjacent pixels P1, ..., P8, P14, P15, P24, P25, P34, P35, P41, ..., P45, P51, ..., P55. Therefore, in some embodiments, the weights of pixels belonging to 4-adjacent pixels are greater than the associated weights of pixels belonging to 8-adjacent pixels but not to 4-adjacent pixels. In addition, in some embodiments, the weights of pixels belonging to 8-adjacent pixels are greater than the associated weights of pixels belonging to 24-adjacent pixels but not to 8-adjacent pixels. Moreover, in some embodiments, the weights of pixels belonging to 24-adjacent pixels are greater than the associated weights of pixels belonging to 48-adjacent pixels (not shown) but not to 24-adjacent pixels. The same is true for more distant neighboring pixels.

[0050] In some embodiments, the data processing system 101 outputs only differential data (differential spatial data and / or differential temporal data), that is, the system output 120 includes only differential data. In some embodiments, the differential data is generated by subtracting each input node output 134a, ..., 134i from the weighted central pixel data. In addition, in some embodiments, the sum of all weights Wa, ..., Wi of the input nodes 130a, ..., 130i is equal to the output node weight Wnw1. By making the sum of all weights Wa, ..., Wi of the input nodes 130a, ..., 130i equal to the output node weight Wnw1, the data processing system 101 outputs only differential data (in space and time, that is, in the spatial and temporal domains). For example, if the value of the central pixel is the same as the value of the adjacent pixels, the output of the data processing system is zero, and if the value of one (or more) adjacent pixels is different from the value of the central pixel, the output is non-zero. Therefore, the functionality of an event-based camera is provided in this way.

[0051] Reference Fig. 6A , a device 230 is provided. The device 230 includes one or more data processing systems 101 (same as the data processing system 101 described above). In addition, the device 230 includes an imaging device. The imaging device can be a multiphoton microscope, a laser confocal microscope, a telescope, a radio telescope, a thermal imaging / infrared camera, or a digital camera 240. The imaging device is configured to capture (or capture to) data of pixels P1, P2, ..., P9. In addition, the imaging device is configured to transmit (or send) the data of the captured pixels to one or more data processing systems 101. The data processing system 101 is configured to receive the data of the captured pixels. By combining the data processing system 101 with the digital camera 240 (or other imaging device), an event-based camera (imaging device) can be realized, which is less complex than the event-based camera (imaging device) of the prior art, and / or the requirements for the actual camera (imaging device) are reduced. In addition, in environments where high amplification is required (e.g., high amplification for a charge-coupled device sensor, i.e., a CCD sensor, or a photosensor of a CCD, a photomultiplier tube, or a similar photoelectric conversion electronic device; because high amplification may produce high noise levels), such as dark environments, and / or for night vision or thermal imaging, noise such as salt and pepper noise is reduced.

[0052] Figure 6B An apparatus 200 is shown. The apparatus 200 includes a data processing system 101, 102, ..., 109 (such as in combination with) for each pixel P1, P2, ..., P9 in the data of the pixels P1, P2, ..., P9 Figure 1-5) (for illustrative purposes, the number of pixels and the number of processing systems are limited to 9, but may be much higher, for example 8294400). In addition, in some embodiments, the device 200 includes a database 210. The database 210 includes representations / images of known objects. Therefore, each representation / image in the database 210 is associated with a known object. Moreover, in some embodiments, the database 210 includes a post-processing unit 220. The post-processing unit 220 is configured to receive (or received) the system output 120a, 120b, ..., 120i from each data processing system 101, 102, ..., 109. In addition, the post-processing unit 220 is configured to compare (or compared) the system output 120a, 120b, ..., 120i with the representation / image of the known object (for example, it is configured to compare each system output 120a, 120b, ..., 120i with the corresponding pixel of the representation / image of the known object). Moreover, the post-processing unit 220 is configured to identify (or have identified) an (unknown / unrecognized) object (present in the pixel data P1, P2, ..., P9) based on the comparison. In some embodiments, the object is identified when the difference between the system output 120a, 120b, ..., 120i and the representation / image of the known object is less than a first threshold (value). In some embodiments, the object is not identified if the difference between the system output 120a, 120b, ..., 120i and each representation / image of the known object is not less than the first threshold (value). In some embodiments, the (unknown / unrecognized) object is identified as a known object (of all objects in the database) whose representation / image differs least from the system output 120a, 120b, ..., 120i, for example, if the difference between the system output 120a, 120b, ..., 120i and the representation / image of the known object is less than the first threshold (value). In some embodiments, when / if the object in the database 210 has been identified, the post-processing unit 220 outputs a signal 222 indicating the identified object. In some embodiments, if the object in the database 210 is not recognized (e.g., because the difference between each representation / image in the database 210 and the system output 120a, 120b, ..., 120i is too large, such as greater than a threshold), the post-processing unit 220 outputs a negative signal or a zero signal. In addition, in some embodiments, the device 230 includes the apparatus 200, and the imaging device captures the data of the pixels P1, P2, ..., P9 and transmits it to each data processing system 101, 102, ..., 109.

[0053] Figure 7Method steps of a computer-implemented or hardware-implemented method 300 for processing pixel data according to some embodiments are shown. The method 300 includes receiving 310 data from pixels P1, P2, ..., P9 of each image in a sequence of images as a system input. The sequence can be a time-delayed sequence. Furthermore, the sequence is captured by an imaging device. The imaging device can be a multiphoton microscope, a laser confocal microscope, a telescope, or a digital camera 240. In some embodiments, the sequence is a time-continuous sequence. Furthermore, in some embodiments, the images are received one at a time (starting with the earliest captured image and then receiving the images in the order in which they were captured, e.g., in real time). The pixel data of each image includes data from a center pixel P5 and data from neighboring pixels P1, ..., P4, P6, ..., P9. Furthermore, the method 300 includes providing 320 each pixel P1, ..., P9 as an input 132a, ..., 132i to a corresponding input node 130a, ..., 130i, for example, in combination Figure 1-5 The data processing system 101 described herein. The method 300 includes applying 330 a corresponding input node leakage integrator LIa having a first time constant TC1. In addition, the method 300 includes applying a corresponding weight Wa, ..., Wi to each input 132a, ..., 132i (or the output of the corresponding input node leakage integrator LIa) to produce an input node output 134a, ..., 134i. The weights Wa, ..., Wi may be applied before or after the corresponding input node leakage integrator LIa is applied. Therefore, in some embodiments, the inputs 132a, ..., 132i are first weighted by the corresponding weights Wa, ..., Wi, and then the corresponding input node leakage integrator LIa is applied to the weighted inputs. Alternatively, the input node leakage integrator LIa is first applied to the inputs 132a, ..., 132i, and then the corresponding weights Wa, ..., Wi are applied. Furthermore, the method 300 includes (for example, in combination with Figure 1-5) output node 140a of the described data processing system 101 receives 340 each input node output 134a, ..., 134i. The method 300 includes receiving 345 pixel data from the center pixel P5 by the output node 140a. In addition, the method 300 includes combining 350 the pixel data from the center pixel P5 with the output node weight Wnw1, thereby generating weighted center pixel data. Moreover, the method 300 includes applying 360 a multi-input leak integrator LIz to the weighted center pixel data and each input node output 134a, ..., 134i. The weighted center pixel data and each input node output 134a, ..., 134i are thereby combined to generate the system output 120. The multi-input leak integrator LIz has a second time constant TC2. The second time constant TC2 is less than the first time constant TC1. The method 300 includes utilizing 370 the system output 120 to indicate contrast. In some embodiments, the method 300 includes receiving data of pixels P1, ..., P9 from one image at a time. In some embodiments, method 300 includes repeatedly performing steps 320 - 370 until all images in the sequence of images have been processed.

[0054] Figure 8 Method steps of a computer-implemented or hardware-implemented method 400 for autofocusing an imaging device are shown. The method 400 includes receiving 410 images in a sequence of images, for example continuously (dynamically, in real time). Each image includes data of pixels P1, P2, ..., P25 (an image may include many pixels, such as 1080×1080 or 1920×1080, but for ease of illustration, the image size is limited to 5×5 here). The sequence can be a time-lapse sequence. In addition, the sequence is captured by an imaging device. The imaging device can be a multiphoton microscope, a laser confocal microscope, a telescope, or a digital camera 240. In some embodiments, the sequence is a time-continuous sequence. In addition, in some embodiments, the images are received one by one continuously (starting with the earliest captured image, and then receiving the images in the order in which the images were captured, such as in real time). Moreover, in some embodiments, the images are images in a continuous video image stream from a surrounding digital camera. Furthermore, the method 400 includes determining 415 a size (e.g., 1×1, 3×3, 5×5, or larger) of a preliminary region of interest (ROI) for one or more images. In some embodiments, the determination 415 is based on user input, i.e., user-definable. Alternatively, the determination 415 is automatic, based on an expected size of objects that may be present in the image. Furthermore, the method 400 includes selecting 420 a preliminary region of interest P1, ..., P9 based on the determined size. In some embodiments, the preliminary region of interest is selected as a region of interest located in the upper left corner of the image ( Figure 5Alternatively, the preliminary region of interest is selected as the region of interest located at the lower left corner of the image. In addition, the method 400 includes providing 425 each pixel P1, ..., P9 of the preliminary region of interest as input 132a, ..., 132i to (for example, in combination with Figure 1-5 The method 400 includes applying 430 a corresponding input node leaky integrator LIa having a first time constant TC1 and a corresponding weight Wa, ..., Wi to each input 132a, ..., 132i, thereby generating an input node output 134a, ..., 134i. The step of applying 430 may be performed in combination with Figure 1-5 In addition, method 400 includes (for example, combining Figure 1-5 The output node 140a of the data processing system 101 described above receives 435 each of the input node outputs 134a, ..., 134i and the pixel data from the center pixel P5. Furthermore, the method 400 includes combining 440 the pixel data from the center pixel P5 with the output node weight Wnw1 to generate weighted center pixel data. The combining 440 step can be performed in accordance with the combination of Figure 1-5 The method 400 includes applying 445 a multi-input leaky integrator LIz having a second time constant TC2 to the weighted center pixel data and each input node output 134a, ..., 134i, thereby combining the weighted center pixel data and each input node output 134a, ..., 134i to produce a system output 120. The system output is used to indicate contrast. In some embodiments, the second time constant TC2 is less than the first time constant TC1, for example, 10 times or 100 times less than TC1. In addition, the method 400 includes storing 450 the system output 120 at the first location (or address) of the memory if the system output is higher than the value currently stored at the first location (or address) of the memory, and storing the location of the preliminary region of interest at the second location (or address) of the memory. The first location (or address) of the memory can be initially set to 0 or a large negative number.

[0055] Furthermore, in some embodiments, method 400 includes moving 455 the preliminary region of interest by one pixel step (downward or to the right). In some embodiments, one pixel step is the minimum change that can be made to the preliminary region of interest. Figure 5, if the preliminary region of interest includes pixels P1, ..., P9 (before moving 455), the minimum change that can be made to the preliminary region of interest may be to move the pixels so that the preliminary region of interest after moving 455 includes pixels P4, ..., P9, P14, P24, P34 (i.e., moving downward). Alternatively, if the preliminary region of interest includes pixels P1, ..., P9 (before moving 455), the minimum change will be to move the pixels so that the preliminary region of interest after moving 455 includes pixels P2, P3, P5, P6, P8, P9, P41, P42, P43 (i.e., moving to the right). Similarly, if the preliminary region of interest includes pixels P4, ..., P9, P14, P24, P34 (before moving 455), the minimum change may be to move the pixels so that the preliminary region of interest after moving 455 includes pixels P7, P8, P9, P14, P24, P34, P15, P25, P35. The same sequence may apply to all other possible preliminary regions of interest, i.e., the preliminary regions of interest (P1, ..., P9) are moved downward or rightward in the image (P1, ..., P55), for example, either when the rightmost position is reached, the leftmost position of the next row is selected, or when the bottommost position is reached, the top position of the right column is selected. Alternatively, the method 400 includes moving 455 the preliminary region of interest by two or more pixel steps (downward or rightward). By moving the preliminary region of interest by two or more pixel steps (downward or rightward) instead of one pixel step, the method is faster, i.e., more time-saving. The method 400 includes repeatedly performing 460 steps 410, 415, 420, 425, 430, 435, 440, 445, 450, and 455 until a stopping criterion is met. In some embodiments, the stopping criterion is that all possible regions of interest within the pixel data have been examined. For an image with 5×5 pixels and a preliminary region of interest size of 1×1, all possible regions of interest are all 25 pixel positions of the image. Furthermore, for an image with 5×5 pixels and a preliminary region of interest size of 3×3, all possible regions of interest are 9 possible 3×3 pixel blocks included in the 5×5 pixel block of the image. Therefore, by knowing the image size and the preliminary region of interest size, the number and positions of all possible regions of interest can be determined (and processed). In some embodiments, the stopping criterion is that the preliminary region of interest has reached the end position, such as the position of the lower right corner of the image, for example, the preliminary region of interest includes the positions of pixels P9, P43, P53, P34, P44, P54, P35, P45, P55.

[0056] In addition, method 400 includes setting 465 the location stored in the second location / address of the memory to the location of the region of interest (after / when the stopping criterion is met). This location is the location of the region of interest that has the highest / largest area / largest numerical contrast (among all possible regions of interest) (for the focus used in steps 410-460). Moreover, method 400 includes scanning 470 the focal depth range to find the highest / largest area / largest numerical contrast (value) of the region of interest (within the focal depth range). Scanning 470 may include scanning the full focal depth range in incremental steps of, for example, 0.1%, 0.2%, 0.5%, 1%, 2%, 3%, 4% or 5% of the full focal depth range, and calculating the contrast (value) for each step. In some embodiments, the contrast (value) is determined by the method described above (for example in combination with Figure 7 and method 300 as described herein. In addition, in some embodiments, scanning 470 includes storing the system output 120 (or calculated contrast value) at a first memory location / address and storing the corresponding focus at a second memory location / address after each incremental step if the system output is higher than the value currently stored at the first memory location. That is, the system output (or calculated contrast value) is stored at the first memory location / address and the corresponding focus is stored at the second memory location / address (only if) the system output (or calculated contrast value) is higher than the value currently stored at the first memory location. The first memory location (or address) may initially be set to 0 or a large negative number. Method 400 includes focusing the imaging device using 475 the focus with the highest / largest area / largest value contrast (value), i.e., the focus stored at the second memory location / address after scanning 470 is completed (which is the focus corresponding to the highest / largest area / largest value contrast value within the depth of focus scanned). That is, focusing the imaging device by using 475 the focus with the highest / largest area / largest value contrast. In some embodiments, method 400 is repeated each time it is determined that auto focus is required.

[0057] According to some embodiments, a computer program product is provided, which includes a non-transitory computer readable medium 900, such as a compact disc (CD) ROM, a read only memory (ROM), a digital versatile disc (DVD), an embedded drive, a plug-in card, a random-access memory (RAM) or a universal serial bus (USB) memory. Fig. 9An example computer readable medium in the form of a compact disc (CD) ROM 900 is shown. A computer program including program instructions is stored on the computer readable medium. The computer program may be loaded into a data processor 920, which may be included in a computer or computing device 910, for example. When loaded into a data processing unit, the computer program may be stored in a memory 930 associated with or included in the data processing unit. According to some embodiments, when the computer program is loaded into and executed by the data processing unit, it may cause the execution of, for example, the operations described herein. Figure 7-8 In addition, in some embodiments, a computer program product is provided, which includes instructions that, when executed on at least one processor of a processing device, cause the processing device to perform Figure 7-8 Furthermore, in some embodiments, a non-transitory computer-readable storage medium is provided, which stores one or more programs configured to be executed by one or more processors of a processing device, wherein the one or more programs include instructions that, when executed by the processing device, cause the processing device to execute Figure 7-8 Any of the methods shown in .

[0058] Example table: Example 1 A data processing system (101) for processing pixel data, configured to receive data (including central pixel data (P5) and adjacent pixel data (P1, ..., P4, P6, ..., P9) of each image in a sequence of images captured by an imaging device as system input, and configured to have a system output (120), the data processing system comprising: a plurality of input nodes (130a, ..., 130i), each input node configured to receive pixel data from a corresponding pixel (P1, ..., P9) as an input (132a, ..., 132i), each input node (130a, ..., 130i) comprising an input node leakage integrator (LIa) having a first time constant (TC1) and weights (Wa, ..., Wi) for the inputs (132a, ..., 132i), and each input node configured to generate input node outputs (134a, ..., 134i) by applying the input node leakage integrator and the corresponding weights (Wa, ..., Wi) to the inputs (132a, ..., 132i); and an output node (140a) comprising an output node weight (Wnw1) and a multi-input leak integrator (LIz) having a second time constant (TC2), wherein the output node (140a) is configured to receive each input node output (134a, ..., 134i), wherein the output node (140a) is configured to receive pixel data from a center pixel (P5), wherein the output node (140a) is configured to combine the pixel data from the center pixel (P5) with the output node weight (Wnw1) to produce weighted center pixel data, and wherein the output node (140a) is configured to apply the multi-input leak integrator (LIz) to the weighted center pixel data and each node output (134a, ..., 134i), and wherein the output node (140a) is configured to produce a system output (120) as a combination of the weighted center pixel data and each node output (134a, ..., 134i); and Wherein the second time constant (TC2) is less than the first time constant (TC1), and wherein the system output is used to indicate contrast.

[0059] Example 2 The data processing system of Example 1 comprises a threshold unit (142) configured to receive a system output (120), wherein the threshold unit (142) is configured to compare the system output (120) to a threshold, and wherein the threshold unit (142) is configured to output a signal (144) indicating that an event has been detected.

[0060] Example 3 The data processing system of any of Examples 1-2, wherein the weight (Wa, ..., Wi) of the input node (130a, ..., 130i) is larger when the central pixel (P5) is closer to the pixel (P1, ..., P9).

[0061] Example 4 The data processing system of any of Examples 1-3, wherein the sum of all weights (Wa, ..., Wi) of the input nodes (130a, ..., 130i) is equal to the output node weight (Wnw1).

[0062] Example 5 A device (230) includes a data processing system (101, 102, ..., 109) of any one of Examples 1-4 and an imaging device, such as a multiphoton microscope, a laser confocal microscope, or a digital camera (240), wherein the imaging device is configured to capture data of pixels (P1, P2, ..., P9) and transmit the captured pixel data to the data processing system (101, 102, ..., 109).

[0063] Example 6 A device (200), comprising: The data processing system (101, 102, ..., 109) according to any one of Examples 1-4, for processing each pixel (P1, P2, ..., P9) in data of pixels (P1, P2, ..., P9); a database (210) comprising representations of images of objects; and A post-processing unit (220) is configured to receive the system output (120a, ..., 120i) from each data processing system (101, 102, ..., 109), is configured to compare the system output (120a, ..., 120i) with the object image, and is configured to identify the object based on the comparison.

[0064] Example 7 A computer-implemented or hardware-implemented method (300) for processing pixel data, comprising: Receiving (310) as system input data of pixels (P1, P2, ..., P9) from each image in the sequence of images, including data from a center pixel (P5) and data from neighboring pixels (P1, ..., P4, P6, ..., P9); providing (320) each pixel (P1, ..., P9) as an input (132a, ..., 132i) to a corresponding input node; applying (330) a corresponding input node leaky integrator (LIa) having a first time constant (TC1) and corresponding weights (Wa, ..., Wi) to each input (132a, ..., 132i) to generate input node outputs (134a, ..., 134i); receiving (340) each input node output (134a, ..., 134i) and pixel data from a center pixel (P5) by an output node (140a); combining (350) the pixel data from the center pixel (P5) with the output node weight (Wnw1) to produce weighted center pixel data; applying (360) a multi-input leaky integrator (LIz) having a second time constant (TC2) less than the first time constant (TC1) to the weighted center pixel data and each node output (134a, ..., 134i), thereby combining the weighted center pixel data and each node output (134a, ..., 134i) to produce a system output (120); and The system output (120) is utilized (370) to indicate contrast.

[0065] Example 8 A computer-implemented or hardware-implemented method (400) for autofocusing an imaging device such as a digital camera, comprising: a) receiving (410) an image in a sequence of images, each image comprising data of pixels (P1, P2, ..., P25); b) determining (415) the size of a preliminary region of interest (ROI) (P1, ..., P9) of the image; c) selecting ( 420 ) a preliminary region of interest (P1 , . . . , P9 ) based on the determined size; d) providing (425) each pixel (P1, ..., P9) of the preliminary ROI as input (132a, ..., 132i) to the corresponding input node (130a, ..., 130i); e) applying (430) to each input (132a, ..., 132i) a corresponding input node leaky integrator having a first time constant (TC1) and a corresponding weight (Wa, ..., Wi) to generate an input node output (134a, ..., 134i); f) receiving (435) by the output node (140a) each input node output (134a, ..., 134i) and pixel data from the center pixel (P5); g) combining (440) the pixel data from the center pixel (P5) with the output node weight (Wnw1) to produce weighted center pixel data; h) applying (445) a multi-input leaky integrator (LIz) having a second time constant (TC2) less than the first time constant (TC1) to the weighted center pixel data and each node output (134a, ..., 134i), thereby combining the weighted center pixel data and each node output (134a, ..., 134i) to produce a system output (120), the system output being indicative of contrast; i) if the system output is higher than the value currently stored in the first location of the memory, storing (450) the system output (120) in the first location of the memory and storing the location of the preliminary ROI in the second location of the memory; j) moving the preliminary region of interest by (455) a pixel step; k) repeating steps d)-j) of (460) until a stopping criterion is met; l) setting (465) the location stored in the second location of the memory as the location of the region of interest; and m) scanning (470) the focal depth range to find maximum contrast in the region of interest; and n) focusing the imaging device using (475) the focal point having the greatest contrast.

[0066] Example 9 The computer-implemented or hardware-implemented method of Example 8, wherein the stopping criterion is that all possible regions of interest within the pixel data have been examined.

[0067] Example 10 A computer program product comprises a non-transitory computer-readable medium (900) on which a computer program comprising program instructions is stored, the computer program being loadable into a data processing unit (920) and configured to enable the method of any one of Examples 7-9 to be performed when the computer program is run by the data processing unit (920).

[0068] Those skilled in the art will recognize that the present disclosure is not limited to the preferred embodiments described above. Those skilled in the art will also recognize that modifications and variations may be made within the scope of the appended claims. In addition, those skilled in the art will be able to understand and implement variations to the disclosed embodiments when studying the drawings, the disclosure, and the appended claims.

Claims

1. A data processing system (101) for processing pixel data, configured to receive as system input data of pixels (P1, ..., P9) from each image of a sequence of images captured by an imaging device, the data of the pixels comprising data of a center pixel (P5) and data of neighboring pixels (P1, ..., P4, P6, ..., P9), and the system is configured to have a system output (120), comprising: a plurality of input nodes (130a, ..., 130i), each of the input nodes being configured to receive pixel data from a corresponding pixel (P1, ..., P9) as an input (132a, ..., 132i), each of the input nodes (130a, ..., 130i) comprising an input node leakage integrator (LIa) having a first time constant (TC1) and weights (Wa, ..., Wi) for the input (132a, ..., 132i), and each of the input nodes being configured to generate an input node output (134a, ..., 134i) by applying the input node leakage integrator and the corresponding weights (Wa, ..., Wi) to the input (132a, ..., 132i); and an output node (140a) comprising an output node weight (Wnw1) and a multi-input leaky integrator (LIz) having a second time constant (TC2), wherein the output node (140a) is configured to receive each of the input node outputs (134a, ..., 134i), wherein the output node (140a) is configured to receive pixel data from the center pixel (P5), wherein the output node (140a) is configured to integrate the pixel data from the center pixel (P5) with the output node weight (Wnw1) w1) to produce weighted center pixel data, and wherein the output node (140a) is configured to apply the multi-input leak integrator (LIz) to the weighted center pixel data and each of the input node outputs (134a, ..., 134i), and wherein the output node (140a) is configured to produce the system output (120) comprising only difference data generated by subtracting each of the input node outputs (134a, ..., 134i) from the weighted center pixel data; and wherein the second time constant (TC2) is less than the first time constant (TC1), and wherein the system output is indicative of contrast.

2. The method according to claim 1, wherein the method further comprises obtaining a time difference signal indicating the contrast between pixels at the same position in different images by applying the leak integrator and feeding pixel data from multiple images to the data processing system (101).

3. A method according to claim 1 or 2, wherein the method further comprises obtaining a spatial differential signal indicating the contrast between the central pixel and adjacent pixels in the same image by inputting data from the central pixel (P5) and data from the adjacent pixels (P1, ..., P4, P6, ..., P9) into the data processing system (101).

4. The data processing system of claim 1 , comprising a threshold unit ( 142 ) configured to receive the system output ( 120 ), wherein the threshold unit ( 142 ) is configured to compare the system output ( 120 ) to a threshold value, and wherein the threshold unit ( 142 ) is configured to output a signal ( 144 ) indicating a detected event.

5. A data processing system according to any one of claims 1-4, wherein the closer the central pixel (P5) is to the pixel (P1, ..., P9), the greater the weight (Wa, ..., Wi) of the input node (130a, ..., 130i).

6. The data processing system according to any one of claims 1-5, wherein the sum of all weights (Wa, ..., Wi) of the input nodes (130a, ..., 130i) is equal to the output node weight (Wnw1).

7. A device (230) comprising a data processing system (101, 102, ..., 109) as described in any one of claims 1 to 6 and an imaging device, wherein the imaging device is configured to capture pixel (P1, P2, ..., P9) data and transmit the captured pixel data to the data processing system (101, 102, ..., 109).

8. The apparatus according to claim 7, wherein the imaging device is a multiphoton microscope, a laser confocal microscope or a digital camera (240).

9. An apparatus (200), comprising: A data processing system (101, 102, ..., 109) according to any one of claims 1 to 6, for processing data of pixels (P1, P2, ..., P9) for each of said pixels (P1, P2, ..., P9); a database (210) comprising images of known objects; and A post-processing unit (220) is configured to receive system outputs (120a, ..., 120i) from each of the data processing systems (101, 102, ..., 109), is configured to compare the system outputs (120a, ..., 120i) with the image of the known object, and is configured to identify the object present in the data of the pixels (P1, P2, ..., P9) as the known object whose image has the smallest difference with the system outputs (120a, 120b, ..., 120i) based on the comparison.

10. A computer-implemented or hardware-implemented method (300) for processing data of a pixel, comprising: Receiving (310) data from pixels (P1, P2, ..., P9) of each image in the image sequence as system input, the data for the pixels including data from a center pixel (P5) and data from neighboring pixels (P1, ..., P4, P6, ..., P9); providing (320) each of said pixels (P1, ..., P9) as input (132a, ..., 132i) to a corresponding input node; applying (330) a respective input node leaky integrator (LIa) having a first time constant (TC1) and respective weights (Wa, ..., Wi) to each of said inputs (132a, ..., 132i) to generate input node outputs (134a, ..., 134i); receiving (340) by an output node (140a) each of said input node outputs (134a, ..., 134i) and pixel data from said center pixel (P5); combining (350) the pixel data from the center pixel (P5) with the output node weight (Wnw1) by multiplication to produce weighted center pixel data; applying (360) a multi-input leaky integrator (LIz) having a second time constant (TC2) to the weighted center pixel data and each of the input node outputs (134a, ..., 134i), wherein the second time constant (TC2) is less than the first time constant (TC1), thereby combining the weighted center pixel data with each of the input node outputs (134a, ..., 134i) to produce a system output (120), the system output comprising difference data generated by subtracting each of the input node outputs (134a, ..., 134i) from the weighted center pixel data; and The system output (120) is utilized (370) to indicate contrast.

11. A computer-implemented or hardware-implemented method (400) for automatically focusing an imaging device, comprising: Receiving (410) a plurality of images in a sequence of images, each of said images comprising data of pixels (P1, P2, ..., P25); Determining (415) the size (P1, ..., P9) of a preliminary region of interest ROI of the image according to user input; Selecting (420) a preliminary region of interest ROI based on the determined sizes (P1, ..., P9); providing (425) each pixel (P1, ..., P9) in the preliminary region of interest as an input (132a, ..., 132i) to a corresponding input node (130a, ..., 130i); applying (430) a corresponding input node leaky integrator having a first time constant (TC1) and a corresponding weight (Wa, ..., Wi) to each of said inputs (132a, ..., 132i) to generate an input node output (134a, ..., 134i); receiving (435) by an output node (140a) each of said input node outputs (134a, ..., 134i) and pixel data from a center pixel (P5); combining (440) pixel data from the center pixel (P5) with an output node weight (Wnw1) to produce weighted center pixel data; applying (445) a multi-input leaky integrator (LIz) having a second time constant (TC2) to the weighted central pixel data and each of the input node outputs (134a, ..., 134i), wherein the second time constant (TC2) is less than the first time constant (TC1), thereby combining the weighted central pixel data with each of the input node outputs (134a, ..., 134i) to produce a system output (120), the system output comprising difference data generated by subtracting each of the input node outputs (134a, ..., 134i) from the weighted central pixel data, the system output being indicative of contrast; If the system output is above a value currently stored in a first memory location, storing (450) the system output (120) in a first memory location and storing the location of the preliminary region of interest in a second memory location; Moving (455) the preliminary region of interest by a pixel step; Repeating (460) the steps of receiving (410) an image, determining (415), selecting (420), providing (425), applying (430), receiving (435) each of the input node outputs (134a, ..., 134i) and pixel data from the center pixel (P5), combining (440), applying (445), storing (450) and moving (455) until a stopping criterion is met; setting (465) the position stored at the second location of the memory as the position of the region of interest; Scanning (470) the focus range to find the highest contrast value of the region of interest; as well as The imaging device is focused using (475) the focal point corresponding to the highest contrast value.

12. A computer-implemented or hardware-implemented method according to claim 11, wherein the stopping criterion is that all possible regions of interest within the data of the pixel have been examined, or wherein the stopping criterion is that the preliminary region of interest has reached an end position.

13. The computer-implemented or hardware-implemented method of claim 11 or 12, wherein the imaging device is a digital camera (240).

14. The computer-implemented or hardware-implemented method according to any one of claims 11 to 13, wherein: The preliminary region of interest (P1, ..., P9) is a region of interest located at the upper left corner of the image or a region of interest located at the lower left corner of the image.

15. A computer program product comprising instructions which, when executed on at least one processor of a processing device, cause the processing device to perform the method according to any one of claims 10 to 14.

16. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a processing device, the one or more programs comprising instructions that, when executed by the processing device, cause the processing device to perform the method according to any one of claims 10-14.

Images