Image processing method, device and system

Abstract

The invention relates to an image processing method, device and system. The image processing method comprises the following steps: for each cycle of a current cycle and at least one reference cycle adjacent to the current cycle, acquiring a plurality of substrate images of a plurality of fluorescent channels of a sequencing substrate in each cycle, the current cycle and the at least one reference cycle are a plurality of continuous biochemical reaction and imaging cycles which sequentially occur, each fluorescence channel corresponds to one substrate image, and the pixel value of the substrate image of each fluorescence channel represents the fluorescence intensity value under each fluorescence channel; acquiring a fluorescence intensity value in a pixel neighborhood of the same position on each substrate image corresponding to each cycle; and identifying the base at the base position corresponding to the current cycle according to the fluorescence intensity value in the pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle. According to the invention, the accuracy of image recognition can be improved.

Description

Image processing method, device and system Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method, device and system, and a computer-storable medium. Background Art

[0002] In biology, sequencing refers to a method for determining the primary structure or sequence of biopolymers, such as nucleic acids like DNA (deoxyribonucleic acid) and RNA (ribonucleic acid).

[0003] In related technologies, the base at the base position corresponding to the current biochemical reaction and imaging cycle is identified based on the fluorescence intensity values ​​within the pixel neighborhood of the same position on multiple substrate images of the sequencing substrate in multiple fluorescence channels during the current biochemical reaction and imaging cycle. Each biochemical reaction and imaging cycle involves performing a base-pairing biochemical reaction at the same position on a nanosphere based on at least one nucleic acid template on the sequencing substrate. The substrate after the biochemical reaction is then imaged under different excitation light sources to obtain substrate images under multiple fluorescence channels. Different excitation light sources correspond to different fluorescence channels.

[0004] Summary of the Invention

[0005] In related technologies, bases may be lost or added during the cyclic replication of nucleic acid templates to produce nanospheres. This can result in different base positions for each nucleic acid template within the same nanosphere during a single biochemical reaction. This situation can interfere with the fluorescence intensity of the pixel surrounding the nanosphere on the substrate image during a single biochemical reaction and imaging cycle, resulting in poor base recognition accuracy.

[0006] In response to the above technical problems, the present disclosure proposes a solution that can improve the accuracy of image recognition.

[0007] According to a first aspect of the present disclosure, there is provided an image processing method, comprising: for each cycle in a current cycle and at least one reference cycle adjacent to the current cycle, obtaining multiple substrate images of a sequencing substrate in multiple fluorescent channels in the current cycle, wherein the current cycle and the at least one reference cycle are multiple consecutive biochemical reactions and imaging cycles occurring sequentially, each fluorescent channel corresponds to a substrate image, and the pixel values ​​of the substrate image of each fluorescent channel represent the fluorescence intensity values ​​under the each fluorescent channel; obtaining the fluorescence intensity values ​​within a pixel neighborhood at the same position on each substrate image corresponding to each cycle; and identifying the base at the base position corresponding to the current cycle based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle.

[0008] In some embodiments, obtaining the fluorescence intensity value within the pixel neighborhood at the same position on each substrate image corresponding to each cycle includes: using a shift register-based cache to obtain the fluorescence intensity value within the pixel neighborhood at the same position on each substrate image corresponding to each cycle.

[0009] In some embodiments, the shift register-based cache includes a first cache area corresponding to each fluorescence channel, the first cache area includes a plurality of first sub-cache areas connected sequentially, the size of each first sub-cache area is the same as the pixel size of the substrate image, the plurality of first sub-cache areas are the same as the number of cycles of the current cycle and the at least one reference cycle, and the connection order of the plurality of first sub-cache areas is consistent with the occurrence order of the current cycle and the at least one reference cycle, each first sub-cache area is configured to shift the fluorescence intensity value of each pixel on the substrate image stored therein to the first sub-cache area sequentially connected before it based on the shift register, and output the fluorescence intensity value within the pixel neighborhood at the same position during each shift process.

[0010] In some embodiments, each first sub-buffer area includes a first area and a second area sequentially connected to the first area, wherein the first area is configured to shift the fluorescence intensity value of each pixel based on a shift register and output the fluorescence intensity value within the pixel neighborhood at the same position during each shift process, and the second area is configured to shift the fluorescence intensity value of each pixel.

[0011] In some embodiments, the first area includes a plurality of shift registers connected in sequence, the number of shift registers in the first area is the product of the number of pixels of the width of the substrate image and the number of pixels of the width or height of the pixel neighborhood at the same position, each shift register in the first area is configured to shift the fluorescence intensity value of each pixel, and the shift registers located at a preset sequential position in the first area are also configured to output the fluorescence intensity value within the pixel neighborhood at the same position during each shift process, and the preset sequential position is the arrangement position of the shift registers corresponding to each pixel included in the pixel neighborhood at the same position in the plurality of shift registers.

[0012] In some embodiments, the second area includes a plurality of shift registers connected in sequence, the number of shift registers in the second area is the product of the number of pixels of the width of the substrate image and the pixel difference, the pixel difference is the difference between the number of pixels of the height of the substrate image and the number of pixels of the width or height of the pixel neighborhood at the same position, and each shift register in the second area is configured to shift the fluorescence intensity value of each pixel.

[0013] In some embodiments, the second region includes a read address decoder, a write address decoder, and a linear storage region. The linear storage region within the second region includes a plurality of storage units, each storage unit being configured to store the fluorescence intensity value of each pixel. The address of the storage unit pointed to by the write address decoder is the previous address of the address of the storage unit pointed to by the read address decoder. The read address decoder and the write address decoder are configured to shift the fluorescence intensity values ​​of each pixel within the plurality of storage units.

[0014] In some embodiments, the shift register-based cache also includes a second cache area corresponding to each fluorescent channel, the second cache area is sequentially connected after the first cache area corresponding thereto, and the second cache area is configured to store substrate images of multiple fluorescent channels of multiple buffer cycles that occur sequentially after the current cycle and the at least one reference cycle, and to shift the substrate images of multiple fluorescent channels of the multiple buffer cycles.

[0015] In some embodiments, the second buffer area includes a plurality of sequentially connected second sub-buffer areas, each second sub-buffer area includes a plurality of sequentially connected shift registers, and the number of shift registers in each second sub-buffer area is the number of pixels of the substrate image.

[0016] In some embodiments, the second cache area includes a first linear storage area and a ping-pong storage area, wherein the first linear storage area is connected after the ping-pong storage area, and the ping-pong storage area is configured to shift the substrate images of the multiple fluorescent channels of the multiple buffer cycles based on the ping-pong cache mechanism.

[0017] In some embodiments, the ping-pong storage area includes a second linear storage area and its corresponding address decoder and a third linear storage area and its corresponding address decoder for implementing a ping-pong cache mechanism.

[0018] In some embodiments, the ping-pong storage area includes a first shift register area and a second shift register area, and the first shift register area and the second shift register area each include a plurality of shift registers for implementing a ping-pong buffer mechanism.

[0019] In some embodiments, identifying the base at the base position corresponding to the current cycle based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle includes: using a linear cache to output the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle from the shift register-based cache at a preset rate; and using an optical computing chip to identify the base at the base position corresponding to the current cycle based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle, the preset rate being consistent with the rate at which the optical computing chip identifies the base at the base position corresponding to the current cycle.

[0020] In some embodiments, under the control of a central processing unit, one or more coprocessors are used to obtain substrate images of multiple fluorescence channels of the sequencing substrate in each cycle; obtain fluorescence intensity values ​​within a pixel neighborhood at the same position on each substrate image corresponding to each cycle; and identify the base at the base position corresponding to the current cycle based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle.

[0021] In some embodiments, acquiring multiple substrate images of the sequencing substrate in multiple fluorescent channels in each cycle includes: for each cycle, using an imaging device, acquiring multiple field-of-view images of the sequencing substrate in each fluorescent channel; and acquiring a substrate image of each fluorescent channel of the sequencing substrate in each cycle based on the multiple field-of-view images of the sequencing substrate in each fluorescent channel.

[0022] In some embodiments, based on the multiple field-of-view images of the sequencing substrate in each fluorescent channel, obtaining the substrate image of the sequencing substrate in each fluorescent channel in the cycle includes: for each cycle, performing at least one of an alignment and stitching operation, a super-resolution reconstruction operation, and an image boundary extension operation on the multiple field-of-view images of the sequencing substrate in each fluorescent channel to obtain the substrate image of the sequencing substrate in each fluorescent channel in the cycle.

[0023] According to a second aspect of the present disclosure, there is provided an image processing method, comprising: for a current time unit and each of at least one reference time unit adjacent to the current time unit, obtaining an image of an object to be identified in each time unit, wherein the current time unit and the at least one reference time unit are continuous in time sequence; obtaining pixel features within a pixel neighborhood at the same position on each image to be identified corresponding to each time unit by utilizing a shift register-based cache; and identifying a target object within the pixel neighborhood at the same position corresponding to the current time unit based on the pixel features within the pixel neighborhood at the same position corresponding to the current time unit and the at least one reference time unit, wherein the target object belongs to at least a part of the object to be identified.

[0024] In some embodiments, the image to be identified includes images to be identified corresponding to multiple image channels, each image channel corresponds to an image to be identified, the shift register-based cache includes a first cache area corresponding to each image channel, the first cache area includes multiple first sub-cache areas connected sequentially, the size of each first sub-cache area is the same as the pixel size of the image to be identified, the multiple first sub-cache areas have the same number of time units as the current time unit and the at least one reference time unit, and the connection order of the multiple first sub-cache areas is consistent with the occurrence order of the current time unit and the at least one reference time unit, each first sub-cache area is configured to shift the pixel features of each pixel on the image to be identified stored therein to the first sub-cache area sequentially connected before it based on the shift register, and output the pixel features in the pixel neighborhood at the same position during each shift process.

[0025] In some embodiments, each first sub-buffer area includes a first area and a second area sequentially connected to the first area, wherein the first area is configured to shift the pixel features of each pixel based on a shift register and output the pixel features within the pixel neighborhood at the same position during each shift process, and the second area is configured to shift the pixel features of each pixel.

[0026] In some embodiments, the first area includes a plurality of shift registers connected in sequence, the number of shift registers in the first area is the product of the number of pixels of the width of the image to be identified and the number of pixels of the width or height of the pixel neighborhood at the same position, each shift register in the first area is configured to shift the pixel features of each pixel, and the shift registers located at a preset sequential position in the first area are also configured to output the pixel features in the pixel neighborhood at the same position during each shift process, and the preset sequential position is the arrangement position of the shift registers corresponding to each pixel included in the pixel neighborhood at the same position in the plurality of shift registers.

[0027] In some embodiments, the second area includes a plurality of shift registers connected in sequence, the number of shift registers in the second area is the product of the number of pixels of the width of the image to be identified and the pixel difference, the pixel difference is the difference between the number of pixels of the height of the image to be identified and the number of pixels of the width or height of the pixel neighborhood at the same position, and each shift register in the second area is configured to shift the pixel features of each pixel.

[0028] In some embodiments, the second area includes a read address decoder, a write address decoder, and a linear storage area. The linear storage area within the second area includes multiple storage units, each storage unit is configured to store pixel characteristics of each pixel, the address of the storage unit pointed to by the write address decoder is the previous address of the address of the storage unit pointed to by the read address decoder, and the read address decoder and the write address decoder are configured to shift the pixel characteristics of each pixel in the multiple storage units.

[0029] In some embodiments, the shift register-based cache also includes a second cache area corresponding to each image channel, and the second cache area is sequentially connected after the first cache area corresponding to it. The second cache area is configured to store the images to be identified of multiple image channels of the object to be identified for multiple buffer time units that are arranged continuously in time sequence after the current time unit and the at least one reference time unit, and shift the images to be identified of multiple image channels for the multiple buffer time units.

[0030] In some embodiments, the second cache area includes a plurality of second sub-cache areas connected in sequence, each second sub-cache area includes a plurality of shift registers connected in sequence, and the number of shift registers in each second sub-cache area is the number of pixels of the image to be identified.

[0031] In some embodiments, the second cache area includes a first linear storage area and a ping-pong storage area, the first linear storage area is connected after the ping-pong storage area, and the ping-pong storage area is configured to shift the images to be identified of the multiple image channels of the multiple buffer time units based on the ping-pong cache mechanism.

[0032] In some embodiments, the ping-pong storage area includes a second linear storage area and its corresponding address decoder and a third linear storage area and its corresponding address decoder for implementing a ping-pong cache mechanism.

[0033] In some embodiments, the ping-pong storage area includes a first shift register area and a second shift register area, and the first shift register area and the second shift register area each include a plurality of shift registers for implementing a ping-pong buffer mechanism.

[0034] In some embodiments, identifying a target object within the pixel neighborhood at the same position corresponding to the current time unit and the at least one reference time unit based on pixel features within the pixel neighborhood at the same position corresponding to the current time unit and the at least one reference time unit includes: utilizing a linear cache to output pixel features within the pixel neighborhood at the same position corresponding to the current time unit and the at least one reference time unit from the shift register-based cache at a preset rate; and utilizing an optical computing chip to identify a target object within the pixel neighborhood at the same position corresponding to the current time unit based on pixel features within the pixel neighborhood at the same position corresponding to the current time unit and the at least one reference time unit, the preset rate being consistent with a rate at which the optical computing chip identifies a target object within the pixel neighborhood at the same position corresponding to the current time unit.

[0035] In some embodiments, under the control of a central processing unit, one or more coprocessors are used to obtain an image of the object to be identified in each time unit; obtain pixel features within a pixel neighborhood at the same position on each image to be identified corresponding to each time unit; and identify a target object within the pixel neighborhood at the same position corresponding to the current time unit based on the pixel features within the pixel neighborhood at the same position corresponding to the current time unit and the at least one reference time unit.

[0036] In some embodiments, obtaining an image of the object to be identified in each time unit includes: for each time unit, using an imaging device, obtaining multiple field-of-view images of the object to be identified; and obtaining an image of the object to be identified in each time unit based on the multiple field-of-view images of the object to be identified.

[0037] In some embodiments, based on the multiple field-of-view images of the image to be identified, obtaining the image to be identified of the image to be identified in each time unit includes: for each time unit, performing at least one of registration and stitching operations, super-resolution reconstruction operations, and image boundary extension operations on the multiple field-of-view images of the image to be identified to obtain the image to be identified of the object to be identified in each time unit.

[0038] According to a third aspect of the present disclosure, an image processing device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the image processing method described in any of the above embodiments based on instructions stored in the memory.

[0039] According to a fourth aspect of the present disclosure, an image processing system is provided, comprising: the image processing device described in any one of the above embodiments.

[0040] In some embodiments, the image processing device includes: a shift register-based cache, configured to obtain the fluorescence intensity value within the pixel neighborhood at the same position on each substrate image corresponding to each cycle or to obtain the pixel features within the pixel neighborhood at the same position on each image to be identified corresponding to each time unit.

[0041] In some embodiments, the image processing device further includes: a linear cache configured to output fluorescence intensity values ​​within a pixel neighborhood at the same position corresponding to a current cycle and at least one reference cycle from the shift register-based cache at a preset rate, or configured to output pixel features within a pixel neighborhood at the same position corresponding to a current time unit and at least one reference time unit from the shift register-based cache at a preset rate; and a processing unit configured to, using an optical computing chip, identify a base at a base position corresponding to the current cycle based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle, wherein the preset rate is consistent with the rate at which the optical computing chip identifies the base at the base position corresponding to the current cycle, or configured to, using an optical computing chip, identify a target object within the pixel neighborhood at the same position corresponding to the current time unit based on pixel features within the pixel neighborhood at the same position corresponding to the current time unit and the at least one reference time unit, wherein the preset rate is consistent with the rate at which the optical computing chip identifies the target object within the pixel neighborhood at the same position corresponding to the current time unit.

[0042] In some embodiments, the base recognition system includes a central processing unit and one or more image processing devices, wherein the one or more image processing devices serve as one or more coprocessors, and the central processing unit is configured to control the one or more coprocessors.

[0043] According to a fifth aspect of the present disclosure, a computer storable medium is provided, on which computer program instructions are stored, and when the instructions are executed by a processor, the image processing method described in any of the above embodiments is implemented.

[0044] In the above embodiment, the accuracy of image recognition can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0046] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0047] FIG1 is a flowchart illustrating an image processing method according to some embodiments of the present disclosure;

[0048] FIG2 is a schematic diagram showing the structure of a shift register-based cache according to some embodiments of the present disclosure;

[0049] FIG3 is a schematic diagram illustrating storage of a first buffer area corresponding to a fluorescence channel according to some embodiments of the present disclosure;

[0050] FIG4A is a schematic diagram illustrating the structure of a first sub-cache area according to some embodiments of the present disclosure;

[0051] FIG4B is a schematic diagram showing the structure of a first sub-cache area according to some other embodiments of the present disclosure;

[0052] FIG5A is a schematic diagram illustrating the structure of a second cache area according to some embodiments of the present disclosure;

[0053] FIG5B is a schematic diagram showing the structure of a second cache area according to some other embodiments of the present disclosure;

[0054] FIG6A is a schematic diagram illustrating the structure of a ping-pong storage area according to some embodiments of the present disclosure;

[0055] FIG6B is a schematic diagram showing the structure of a ping-pong storage area according to some other embodiments of the present disclosure;

[0056] 7A is a schematic diagram illustrating storage of a first buffer area and a second buffer area corresponding to a fluorescent channel according to some embodiments of the present disclosure;

[0057] 7B is a schematic diagram illustrating storage of a first buffer area and a second buffer area corresponding to a fluorescent channel according to other embodiments of the present disclosure;

[0058] FIG8 is a flowchart illustrating an image processing method according to other embodiments of the present disclosure;

[0059] FIG9 is a block diagram illustrating an image processing apparatus according to some embodiments of the present disclosure;

[0060] FIG10 is a block diagram illustrating an image processing system according to some embodiments of the present disclosure;

[0061] FIG11 is a block diagram illustrating an image processing system according to other embodiments of the present disclosure;

[0062] FIG12 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0063] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0064] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0065] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0066] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0067] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0068] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0069] FIG1 is a flowchart illustrating an image processing method according to some embodiments of the present disclosure.

[0070] As shown in FIG1 , the image processing method includes: step S10, for each cycle in a current cycle and at least one reference cycle adjacent to the current cycle, acquiring multiple substrate images of the sequencing substrate in multiple fluorescence channels in each cycle; step S20, acquiring fluorescence intensity values ​​within a pixel neighborhood at the same position on each substrate image corresponding to each cycle; and step S30, identifying the base at the base position corresponding to the current cycle based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and at least one reference cycle. For example, the image processing method is performed by an image processing device.

[0071] In the above embodiment, the base at the base position corresponding to the current cycle is identified using the fluorescence intensity values ​​within the pixel neighborhood at the same position in at least one reference cycle that has a temporal sequence relationship with the current cycle. In this manner, the fluorescence intensity values ​​of the reference cycle can be used to assist the base identification process in the current cycle, thereby reducing interference and improving base identification accuracy.

[0072] In step S10, multiple substrate images of the sequencing substrate are acquired for each of the current cycle and at least one reference cycle adjacent to the current cycle, across multiple fluorescence channels. The current cycle and the at least one reference cycle represent multiple consecutive biochemical reactions and imaging cycles occurring sequentially. Each fluorescence channel corresponds to a substrate image. The pixel values ​​of the substrate image for each fluorescence channel represent the fluorescence intensity value in each fluorescence channel.

[0073] In some embodiments, the above step S10 can be implemented as follows.

[0074] First, for each cycle, an imaging device is used to acquire multiple field of view images of the sequencing substrate in each fluorescent channel. The multiple field of view images are multiple small fields of view (FOV) obtained by scanning with the imaging device.

[0075] Then, based on the multiple field images of the sequencing substrate in each fluorescent channel, a substrate image of each fluorescent channel of the sequencing substrate in each cycle is acquired.

[0076] In some embodiments, for each cycle, at least one of a registration and stitching operation, a super-resolution reconstruction operation, and an image boundary extension operation can be performed on multiple field-of-view images of the sequencing substrate in each fluorescent channel to obtain a substrate image of the sequencing substrate in each fluorescent channel in each cycle.

[0077] For example, when a pixel corresponds to multiple nucleic acid templates, super-resolution reconstruction of the image can be performed. Through super-resolution reconstruction, the resolution of the substrate image can be improved so that the resolution of the substrate image can distinguish nucleic acid templates in adjacent positions.

[0078] In some embodiments, the alignment and stitching operations can be performed by a central processing unit or coprocessor. In some embodiments, a pattern for identifying positions can be pre-set on the sequencing substrate, so that the multiple field images obtained by imaging have a pre-set pattern. The alignment and stitching operations described above can be achieved by matching these patterns. For example, the pre-set pattern can be a line segment with a pre-set mark.

[0079] In some embodiments, for an image size N a ×N b For an image with one or more pixel neighborhoods corresponding to a nucleic acid template, in order to facilitate parallel processing, the size of the substrate image can be expanded to (N a +N neighbor )×(N b +N neighbor ), where N neighbor is the number of pixels on the side of the pixel neighborhood. Image expansion allows pixels at the edge of the substrate image to be processed in the same way as pixels at the center of the image, without requiring additional changes, thus improving parallel processing capabilities.

[0080] In step S20, the fluorescence intensity value within the pixel neighborhood at the same position on each substrate image corresponding to each cycle is obtained. A pixel area on the substrate image covers at least the pixel area occupied by a nanosphere. In some embodiments, the size of the pixel neighborhood is 3×3, and the unit is pixel. The size of the pixel neighborhood can also be set to other values ​​according to actual conditions. For example, the number of nucleic acid templates corresponding to pixels can be artificially controlled by setting the optical parameters of the imaging device, such as the objective lens focal length, the eyepiece focal length, and the image distance.

[0081] In some embodiments, a shift register-based cache can be used to obtain the fluorescence intensity value of a pixel neighborhood at the same position on each substrate image corresponding to each cycle. For example, the shift register-based cache is a multi-cycle image data cache.

[0082] The shift register-based cache in some embodiments of the present disclosure will be described in detail below with reference to FIG. 2 .

[0083] FIG2 is a schematic diagram illustrating the structure of a shift register-based cache according to some embodiments of the present disclosure.

[0084] In some embodiments, the shift register-based buffer includes a first buffer region corresponding to each fluorescent channel.

[0085] As shown in FIG2 , the shift register-based buffer 2 includes multiple first buffer regions 10. Different first buffer regions 10 correspond to different fluorescence channels, with each fluorescence channel corresponding to a first buffer region 10. The ellipsis in FIG2 indicates that a larger number of first buffer regions can be provided depending on the number of fluorescence channels.

[0086] In some embodiments, taking DNA sequencing as an example, four fluorescence channels may be set, each corresponding to one of four types of bases. For example, fluorescence channels 1, 2, 3, and 4 correspond to bases A, T, C, and G, respectively.

[0087] In other embodiments, taking DNA sequencing as an example, different types of excitation light can be used to perform two imaging operations to achieve combined encoding of the fluorescence signal intensities and imaging sequences of the two fluorescence channels. For example, a high value in fluorescence channel 1 and a high value in fluorescence channel 2 correspond to base A, a high value in fluorescence channel 1 and a low value in fluorescence channel 2 correspond to base G, a low value in fluorescence channel 1 and a high value in fluorescence channel 2 correspond to base T, and a low value in fluorescence channel 1 and a low value in fluorescence channel 2 correspond to base C.

[0088] Taking the first cache area 10 of Figure 2 as an example, the first cache area 10 includes a plurality of first sub-cache areas 11 connected in sequence. The size of each first sub-cache area 11 is the same as the pixel size of the substrate image. The number of cycles of the plurality of first sub-cache areas 11 is the same as the number of cycles of the current cycle and at least one reference cycle, and the connection order of the plurality of first sub-cache areas 11 is consistent with the occurrence order of the current cycle and at least one reference cycle. Each first sub-cache area 11 is configured to shift the fluorescence intensity value of each pixel on the substrate image stored therein to the first sub-cache area 11 connected sequentially before it based on a shift register, and output the fluorescence intensity value within the pixel neighborhood at the same position during each shift process. The size of the cache area refers to the size of the storage space. In some embodiments, the size of each first sub-cache area 11 can be the same as the pixel size of the substrate image after image expansion.

[0089] The arrows in Figure 2 indicate the direction of stored data transmission. For example, the direction of the shift arrow and the direction of the output data arrow can be different. The arrows between different cache areas or sub-cache areas also indicate the order of connection between the sub-cache areas or between different cache areas. The sub-cache area or cache area at the tail of the arrow is sequentially connected to the sub-cache area or cache area at the head of the arrow. The cycle corresponding to the substrate image stored in the sub-cache area at the tail of the arrow occurs after the cycle corresponding to the substrate image stored in the sub-cache area at the head of the arrow. Each sub-cache area is configured to store the fluorescence intensity value of each pixel in the substrate image.

[0090] The following will describe in detail with reference to FIG. 3 how the first cache area of ​​some embodiments of the present disclosure stores substrate images of multiple cycles.

[0091] FIG3 is a schematic diagram illustrating storage of a first cache area corresponding to a fluorescent channel according to some embodiments of the present disclosure.

[0092] As shown in Figure 3, taking the current cycle as cycle 3 and the reference cycles as cycles 1, 2, 4, and 5 as an example, the first cache area 10 corresponding to a particular fluorescence channel includes five first sub-cache areas 11a, 11b, 11c, 11d, and 11e. First sub-cache area 11a stores the substrate image of the fluorescence channel in cycle 1, first sub-cache area 11b stores the substrate image of the fluorescence channel in cycle 2, first sub-cache area 11c stores the substrate image of the fluorescence channel in cycle 3, first sub-cache area 11d stores the substrate image of the fluorescence channel in cycle 4, and first sub-cache area 11e stores the substrate image of the fluorescence channel in cycle 5. The total number of cycles for the current cycle and the reference cycles can also be a number other than 5.

[0093] In some embodiments, taking the total number of cycles of the current cycle and the reference cycle as 5 as an example, if the current cycle is cycle 1 or cycle 2, two known bases can be spliced ​​at the beginning of the nucleic acid template to ensure that two reference cycles occur before and after each current cycle. In this case, base recognition can be controlled to start from cycle 1. In some embodiments, when cycle 1 is the current cycle, only cycles 2 and 3 can be obtained as reference cycles. When cycle 2 is the current cycle, only cycles 1, 3, and 4 can be obtained as reference cycles. In some embodiments, the biochemical reaction may be ahead or behind, and with cycle 3 as the current cycle, cycles 1, 2, 4, and 5 can be obtained as reference cycles.

[0094] As shown in Figure 3, the first sub-cache area 11b is configured to shift the fluorescence intensity value of each pixel on the substrate image in cycle 2 to the first sub-cache area 11a, which is sequentially connected to the first sub-cache area 11b, based on a shift register, in a row-first or column-first manner. During each shift, the fluorescence intensity value of the pixel neighborhood at the same position is output. For example, the location of the pixel neighborhood is identified by the center pixel position of the pixel neighborhood. For the first sub-cache area 11a, the fluorescence intensity value of each pixel is simply shifted out. The shifted fluorescence intensity value can be discarded or stored in another storage area.

[0095] In some embodiments, each first sub-buffer area includes a first area and a second area sequentially connected to the first area. The first area is configured to shift the fluorescence intensity value of each pixel based on a shift register and output the fluorescence intensity value of a neighborhood of pixels at the same location during each shift. The second area is configured to shift the fluorescence intensity value of each pixel. The shifting here is forward shifting, which refers to shifting in the direction of data transmission.

[0096] The structure and function of the first sub-cache area will be described in detail below by taking the first sub-cache areas 11 b and 11 c of FIG. 3 as an example.

[0097] FIG4A is a schematic diagram illustrating the structure of a first sub-cache area according to some embodiments of the present disclosure.

[0098] As shown in FIG4A , the first sub-cache area 11b includes a first area 111 and a second area 112 sequentially connected to the first area 111. The arrows in FIG4 indicate the transfer direction or shift direction of the stored data. The first sub-cache area 11c has the same structure as the first sub-cache area 11b.

[0099] In some embodiments, the first region 111 includes a plurality of sequentially connected shift registers. The number of shift registers in the first region 111 is the product of the number of pixels in the width of the substrate image and the number of pixels in the width or height of the pixel neighborhood at the same location. As shown in FIG4 , taking a substrate image with a pixel size of 5×5 and a pixel neighborhood with a pixel size of 3×3 as an example, the first region 111 includes shift registers 1 through 15. Each shift register stores the fluorescence intensity value of a pixel in the substrate image. Shift register 1 stores the fluorescence intensity value at pixel location 1 of the substrate image, and the relationship between the other shift registers and the pixel locations of the substrate image is similar.

[0100] Each shift register within the first region is configured to shift the fluorescence intensity value of each pixel along a shift direction. The shift registers within the first region at predetermined sequential positions are further configured to output the fluorescence intensity value of a neighborhood of pixels at the same position during each shift. The predetermined sequential position is the arrangement position of the shift registers corresponding to each pixel within the neighborhood of pixels at the same position within the plurality of shift registers.

[0101] Taking FIG4A as an example, using row-first storage as an example, the pixel neighborhood to which pixel position 7 belongs includes pixel positions 1, 2, 3, 6, 7, 8, 11, 12, and 13, corresponding to shift registers 1, 2, 3, 6, 7, 8, 11, 12, and 13. Each of shift registers 1, 2, 3, 6, 7, 8, 11, 12, and 13 is configured to shift the fluorescence intensity value stored therein to another shift register sequentially connected to the preceding shift register, and during the shifting process, output the fluorescence intensity value within the pixel neighborhood to which pixel position 7 belongs. As shown in FIG4A , the shift registers 1, 2, 3, 6, 7, 8, 11, 12, and 13 in the shaded area of ​​the first sub-buffer area 11b are configured to shift and output the fluorescence intensity value within the pixel neighborhood.

[0102] 4A , during the shifting process, shift register 1 in first sub-buffer region 11c shifts the fluorescence intensity value stored therein to shift register 25 in first sub-buffer region 11b . The structure and function of first sub-buffer region 11c are the same as those of first sub-buffer region 11b .

[0103] In some embodiments, the second region includes a plurality of sequentially connected shift registers. The number of shift registers in the second region is the product of the number of pixels of the width of the substrate image and the pixel difference. The pixel difference is the difference between the number of pixels of the height of the substrate image and the number of pixels of the width or height of a pixel neighborhood at the same location. Each shift register in the second region is configured to shift the fluorescence intensity value of each pixel.

[0104] Taking the 5×5 pixel size of the substrate image in FIG4A as an example, second region 112 includes sequentially connected shift registers 16 through 25. Each shift register in shift registers 16 through 25 is configured to shift the fluorescence intensity value of each pixel stored therein. For example, shift register 16 shifts the fluorescence intensity value stored therein to shift register 15, and shift register 17 shifts the fluorescence intensity value stored therein to shift register 16. The shift registers shift one fluorescence intensity value per clock cycle.

[0105] In some embodiments, the second region includes a read address decoder, a write address decoder, and a linear storage region. The linear storage region within the second region includes a plurality of storage cells, each storage cell being configured to store the fluorescence intensity value of each pixel. The address of the storage cell pointed to by the write address decoder is the address immediately preceding the address of the storage cell pointed to by the read address decoder. The read address decoder and the write address decoder are configured to shift the fluorescence intensity values ​​of each pixel within the plurality of storage cells.

[0106] In some embodiments, each time data is shifted and transferred, the read pointer decoder and the write pointer decoder add a location identifier of a storage unit to themselves, and when the corresponding address of the read address decoder reaches the end of the linear storage area, it automatically returns to the head of the linear storage area, which is equivalent to circular storage in effect, and is also equivalent to the second area composed of multiple shift registers shown in Figure 4A.

[0107] In some embodiments, the first sub-cache area may further include a third area. The third area includes a plurality of register areas whose number is equal to the neighborhood height and whose width is equal to the width of the expanded image. These register areas may also be implemented by an address decoder.

[0108] FIG4B is a schematic diagram showing the structure of a first sub-cache area according to other embodiments of the present disclosure.

[0109] The difference between FIG4B and FIG4A is that FIG4B shows the second region 112 of some other embodiments. Only the difference between FIG4B and FIG4A will be described below, and the similarities will not be repeated.

[0110] As shown in Figure 4B, taking the first sub-cache area 11b as an example, the second area 112 includes a read address decoder 1121, a write address decoder 1122, and a linear storage area 1123. The linear storage area 1123 includes multiple storage cells. Taking a substrate image with a pixel size of 5×5 as an example, the multiple storage cells in the linear storage area 1123 can be identified as storage cells 16 to 25, which are used to store the fluorescence intensity values ​​at pixel positions 16 to 25 of the substrate image. The read pointer decoder and the write pointer decoder can also be referred to as the read address decoder and the write address decoder. The read pointer decoder 1121, the write pointer decoder 1122, and the linear storage area 1123 in the second area 112 collectively constitute the cache manager. The structure and function of the first sub-cache area 11c are identical to those of the first sub-cache area 11b.

[0111] In the above embodiment, by combining a read pointer decoder and a write pointer decoder to shift the fluorescence intensity values ​​within the linear storage area, the number of chip logic gates can be further reduced, resource consumption can be lowered, resource utilization can be improved, and implementation on an FPGA (Field-Programmable Gate Array) can be more convenient.

[0112] 4A and 4B utilize the data transfer of the shift register to implement the sliding operation of the pixel neighborhood window, so that the fluorescence intensity value within a pixel area can be read at one time within one clock cycle. The shift rate of each part of the above buffer is the same.

[0113] In some embodiments, a shift register-based buffer can shift multiple pixels at a time and one row at a time, so that the shift register outputting fluorescence intensity values ​​for a pixel neighborhood does not overlap and moves quickly. For example, if the pixel neighborhood size is 3×3, the register outputs every three rows, and within the output row, every three pixels. This allows overlapping pixel neighborhoods to be skipped, reducing overlap within the corresponding 3×3 neighborhood window, thereby improving image processing efficiency.

[0114] In some embodiments, the shift register-based cache further includes a second cache area corresponding to each fluorescence channel. The second cache area is sequentially connected after the corresponding first cache area. The second cache area is configured to store substrate images of multiple fluorescence channels for multiple buffer cycles that occur sequentially after the current cycle and at least one reference cycle, and to shift the substrate images of multiple fluorescence channels for multiple buffer cycles. By configuring multiple buffer cycles, it is possible to reduce the possibility of data not being able to be acquired in a timely manner in the first cache area 10, reduce data transmission interruptions, and thus improve the efficiency and continuity of image processing.

[0115] 2 , the shift register-based buffer 2 further includes a plurality of second buffer regions 20 corresponding to different fluorescence channels. For each fluorescence channel, the second buffer region 20 is sequentially connected after the first buffer region 20 .

[0116] FIG5A is a schematic diagram illustrating the structure of a second cache area according to some embodiments of the present disclosure.

[0117] As shown in FIG5A , the second buffer area 20 includes a plurality of sequentially connected second sub-buffer areas 21. Each second sub-buffer area 21 includes a plurality of sequentially connected shift registers. The number of shift registers in each second sub-buffer area is equal to the number of pixels in the substrate image. The structure of each second sub-buffer area 21 including a plurality of sequentially connected shift registers can be referred to as the first sub-buffer area 11b in FIG4A . Unlike the first sub-buffer area 11b, the shift registers in the second sub-buffer area 21 are configured to shift stored data, and are not capable of outputting fluorescence intensity values ​​within a pixel neighborhood, nor do they need to distinguish between the first and second areas.

[0118] FIG5B is a schematic diagram showing the structure of a second cache area according to other embodiments of the present disclosure.

[0119] As shown in FIG5B , the second buffer area 20 includes a first linear storage area 22 and a ping-pong storage area 24. The first linear storage area 22 is connected after the ping-pong storage area 24. The ping-pong storage area 24 is configured to shift substrate images of multiple fluorescence channels over multiple buffer cycles based on a ping-pong buffering mechanism. The ping-pong storage area 24 shifts the substrate images of multiple fluorescence channels over multiple buffer cycles to the first buffer area.

[0120] FIG6A is a schematic diagram illustrating the structure of a ping-pong storage area according to some embodiments of the present disclosure.

[0121] In some embodiments, the ping-pong storage area 24 is used to implement the second linear storage area 241 and its corresponding first address decoder 241A, and the third linear storage area 243 and its corresponding second address decoder 243A of the ping-pong cache mechanism. By combining the address decoder and the linear storage area, linear storage is utilized as much as possible to replace shift registers with a large number of logic gates, thereby further reducing logic gate resource consumption and improving resource utilization.

[0122] FIG6B is a schematic structural diagram illustrating a ping-pong storage area according to other embodiments of the present disclosure.

[0123] As shown in FIG6B , the ping-pong storage area 24 includes a first shift register area 242 and a second shift register area 244. The first shift register area 242 and the second shift register area 244 each include multiple shift registers for implementing a ping-pong buffering mechanism. The storage area size of each shift register area is equal to the total number of pixels in the substrate image.

[0124] FIG7A is a schematic diagram illustrating storage of a first buffer area and a second buffer area corresponding to one fluorescent channel according to some embodiments of the present disclosure.

[0125] As shown in FIG7A , taking the current cycle as cycle 3 as an example, the storage conditions of cycles 1 to 5 can refer to the description of FIG3 . Taking the buffer cycle including cycles 6 to 9 as an example, the second cache area 20 includes second sub-cache areas 21a, 21b, 21c and 21e, which are used to store the substrate images corresponding to cycles 6, 7, 8 and 9, respectively. The multiple shift register structures of each second sub-cache area can refer to the first sub-cache area 11b in FIG4A . Unlike the first sub-cache area 11b, the shift register in the second sub-cache area 21 is configured to shift the stored data, and is not able to output the fluorescence intensity value within the pixel neighborhood, and there is no need to distinguish between the first area and the second area. The buffer cycle can also be another number of cycles, for example, cycles 6 to cycle N, where N is an integer greater than 6.

[0126] FIG7B is a schematic diagram showing storage of a first buffer area and a second buffer area corresponding to a fluorescent channel according to other embodiments of the present disclosure.

[0127] As shown in Figure 7B , taking the current loop as loop 3 as an example, the storage of loops 1 through 5 can be referred to the description in Figure 3 . Taking the buffered loops including loops 6 through 9 as an example, the second cache area 20 includes a first linear storage area 22 and a ping-pong storage area 24. Ping-pong storage area 24 stores loops 6 and 7. First linear storage area 22 stores loops 8 and 9. The buffered loops can also be any other number of loops, for example, loops 6 through N, where N is an integer greater than 6. This approach can reduce the number of logic gates and improve logic gate resource utilization.

[0128] In some embodiments, the structure of the storage area where loop 6 is located may include the first address decoder 241A and the second linear storage area 241 in Figure 6A, and the structure of the storage area where loop 7 is located may include the second address decoder 243A and the third linear storage area 243 in Figure 6A.

[0129] The data transmission process in the ping-pong storage area will be described in detail below with reference to FIG. 6A and FIG. 7B .

[0130] When the data for loop 6 enters the first sub-cache area 11e, where loop 5 resides, the first address decoder 241A performs a self-increment operation, and the data corresponding to the address in the second linear storage area 241 is also output to the first sub-cache area 11e, where loop 5 resides. This process is equivalent to a shift register. During this process, the data for the next loop is moved from the first linear storage area 22 to the third linear storage area 243, where loop 7 resides.

[0131] When all the data for loop 6 enters the first sub-cache area 11e where loop 5 is located, the storage location pointed to by the first address decoder 241A is reset to the starting location, and the data for the next loop is moved from the first linear storage area 22 to the second linear storage area 241 where loop 6 is located. The roles of the second linear storage area 241 where loop 6 and the third linear storage area 243 where loop 7 are located are swapped. The second address decoder 243A uses a self-increment operation, and the data corresponding to the address in the third linear storage area 243 is also output to the first sub-cache area 11e where loop 5 is located.

[0132] In other embodiments, the structure of the storage area where loop 6 and loop 7 are located may include the first shift register area 242 and the second shift register area 244 in FIG. 6B .

[0133] In some embodiments, taking a pixel neighborhood of 3×3 pixel area and a total number of cycles of 5 for the current cycle and the reference cycle as an example, in a scenario of four fluorescence channels, the size of the vector of fluorescence intensity values ​​required to complete base recognition of a pixel neighborhood is 4×5×3×3=180 dimensions.

[0134] Returning to Figure 1, in step S30, the base at the base position corresponding to the current cycle is identified based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and at least one reference cycle. In the base identification scenario, multiple cycles occur sequentially to identify multiple base positions arranged sequentially on the nucleic acid template. For example, for the nucleic acid template ATCG, the base positions corresponding to the first, second, third, and fourth cycles are A, T, C, and G, respectively.

[0135] In some embodiments, when a pixel neighborhood only covers the pixel area occupied by one nucleic acid nanoball, the result of base recognition is the base at the base position of the nucleic acid nanoball corresponding to the current cycle.

[0136] In other embodiments, when a pixel neighborhood covers pixel areas occupied by multiple nucleic acid nanoballs, the base call result is the base at the base position corresponding to the current cycle of the nucleic acid nanoball located at the neighborhood center of the pixel neighborhood. If the neighborhood center of the pixel neighborhood does not belong to any nucleic acid nanoball, or if the distances between adjacent nucleic acid nanoballs within the pixel neighborhood are equal, it is determined that no nucleic acid nanoball exists within the pixel neighborhood, and the base call result is invalid.

[0137] In some embodiments, a machine learning model can be used to identify the base at the base position corresponding to the current cycle based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and at least one reference cycle.

[0138] In some embodiments, the above step S30 can be implemented as follows.

[0139] First, a linear cache is used to output fluorescence intensity values ​​within a pixel neighborhood corresponding to the current cycle and at least one reference cycle at the same location from a shift register-based cache at a preset rate. In some embodiments, the linear cache may be a ping-pong storage structure. This ping-pong storage structure can reduce linear cache overflows and base call pauses.

[0140] The optical computing chip then identifies the base at the base position corresponding to the current cycle based on the fluorescence intensity values ​​within the pixel neighborhood at the same location corresponding to the current cycle and at least one reference cycle. The preset rate is consistent with the rate or operating frequency at which the optical computing chip identifies bases at the base position corresponding to the current cycle. Using the optical computing chip, it is possible to compare 2 billion images per second. Two billion images per second corresponds to the two billion pixel neighborhoods of the disclosed embodiment, resulting in high detection efficiency.

[0141] In the above embodiment, by configuring the relative rates of the linear cache and the optical computing chip, the output rate of the linear cache is made consistent with the base recognition rate, thereby improving the collaborative efficiency of the linear cache and the optical computing chip, thereby improving the image processing rate and the efficiency of large-scale sequencing.

[0142] In some embodiments, steps S10-S30 can be performed using one or more coprocessors under the control of a central processing unit. By combining the central processing unit and coprocessors to perform image processing, image processing speed can be further improved. Using multiple coprocessors can further enhance parallel processing capabilities. This can improve the efficiency of large-scale sequencing.

[0143] In some embodiments, the base at the base position corresponding to the current cycle can also be identified based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and at least one reference cycle, as well as reference factors. Reference factors include species information of the sample on the sequencing substrate, sample preparation, and testing environment information. In some embodiments, reference factors may also include base call results from other cycles prior to the current cycle. Reference factors reflect the user's requirements for base call.

[0144] FIG8 is a flowchart illustrating an image processing method according to other embodiments of the present disclosure.

[0145] As shown in FIG8 , the image processing method includes steps S10 ′ to S30 ′.

[0146] In step S10', for each of the current time unit and at least one reference time unit adjacent to the current time unit, an image of the object to be identified is obtained for each time unit. The current time unit and the at least one reference time unit are temporally continuous. In some embodiments, the object to be identified can be a scene environment, a tracking target, a person, etc.

[0147] In step S20 ′, the shift register-based cache is used to obtain pixel features within a pixel neighborhood at the same position on each image to be identified corresponding to each time unit.

[0148] In step S30', a target object is identified within the pixel neighborhood at the same location corresponding to the current time unit and at least one reference time unit based on pixel features within the pixel neighborhood at the same location. The target object is at least a portion of the object to be identified. In some embodiments, identifying the target object may include identifying the type or state of the target object. For example, a machine learning model may be used to identify the target object.

[0149] In the above embodiment, using an image to be recognized in a reference time unit that has a continuous temporal relationship with the current time unit to assist in identifying a target object within a pixel neighborhood in the image to be recognized in the current time unit can reduce interference and improve image recognition accuracy. For example, the time unit in which a reference cycle occurs is the reference time unit, and the time unit in which a current cycle occurs is the current time unit.

[0150] In some embodiments, the image to be recognized includes images to be recognized corresponding to a plurality of image channels, each image channel corresponding to one image to be recognized, and the shift register-based buffer includes a first buffer area corresponding to each image channel.

[0151] The first cache area includes a plurality of sequentially connected first sub-cache areas, each of which has a size equal to a pixel size of the image to be recognized. The plurality of first sub-cache areas have the same number of time units as the current time unit and the at least one reference time unit, and the order in which the plurality of first sub-cache areas are connected is consistent with the order in which the current time unit and the at least one reference time unit occur.

[0152] Each first sub-cache area is configured to shift the pixel features of each pixel in the image to be recognized stored therein to the first sub-cache area sequentially connected thereto based on a shift register, and output the pixel features within the pixel neighborhood at the same position during each shift. For example, the pixel features are semantic features represented by the pixel values.

[0153] In some embodiments, each first sub-buffer region includes a first region and a second region sequentially connected to the first region. The first region is configured to shift the pixel features of each pixel based on a shift register and output the pixel features within a neighborhood of pixels at the same position during each shift. The second region is configured to shift the pixel features of each pixel.

[0154] In some embodiments, the first area includes a plurality of shift registers connected in sequence. The number of shift registers in the first area is the product of the number of pixels of the width of the image to be identified and the number of pixels of the width or height of the pixel neighborhood at the same position. Each shift register in the first area is configured to shift the pixel features of each pixel. The shift registers located at a preset sequential position in the first area are further configured to output the pixel features within the pixel neighborhood at the same position during each shift process. The preset sequential position is the arrangement position of the shift registers corresponding to each pixel included in the pixel neighborhood at the same position in the plurality of shift registers.

[0155] In some embodiments, the second area includes a plurality of shift registers connected in sequence. The number of shift registers in the second area is the product of the number of pixels of the width of the image to be identified and the pixel difference. The pixel difference is the difference between the number of pixels of the height of the image to be identified and the number of pixels of the width or height of the pixel neighborhood at the same position. Each shift register in the second area is configured to shift the pixel feature of each pixel.

[0156] In some embodiments, the second region includes a read address decoder, a write address decoder, and a linear storage region. The linear storage region within the second region includes a plurality of storage cells. Each storage cell is configured to store a pixel characteristic of each pixel. The address of the storage cell pointed to by the write address decoder is the address immediately preceding the address of the storage cell pointed to by the read address decoder. The read address decoder and the write address decoder are configured to shift the pixel characteristics of each pixel within the plurality of storage cells.

[0157] In some embodiments, the shift register-based cache further includes a second cache area corresponding to each image channel. The second cache area is sequentially connected after the corresponding first cache area. The second cache area is configured to store images to be recognized for multiple image channels of the object to be recognized for multiple buffer time units that are sequentially arranged after the current time unit and at least one reference time unit, and to shift the images to be recognized for multiple image channels for multiple buffer time units.

[0158] In some embodiments, the second cache area includes a plurality of second sub-cache areas connected sequentially, each second sub-cache area includes a plurality of shift registers connected sequentially, and the number of shift registers in each second sub-cache area is the number of pixels of the image to be identified.

[0159] In some embodiments, the second buffer area includes a first linear storage area and a ping-pong storage area. The first linear storage area is connected after the ping-pong storage area. The ping-pong storage area is configured to shift the images to be recognized of the multiple image channels of the multiple buffering time units based on the ping-pong buffer mechanism.

[0160] In some embodiments, the ping-pong storage area includes a second linear storage area and its corresponding address decoder and a third linear storage area and its corresponding address decoder for implementing a ping-pong cache mechanism.

[0161] In some embodiments, the ping-pong storage area includes a first shift register area and a second shift register area. The first shift register area and the second shift register area each include a plurality of shift registers for implementing a ping-pong buffer mechanism.

[0162] The structures and functions of the various parts of the above-mentioned shift register-based cache can be referred to the relevant descriptions of Figures 2 to 7B, and will not be repeated here.

[0163] In some embodiments, identifying a target object within a pixel neighborhood at the same position corresponding to the current time unit and at least one reference time unit can be achieved by using pixel features within the pixel neighborhood at the same position corresponding to the current time unit and at least one reference time unit.

[0164] First, a linear cache is used to output pixel features within a pixel neighborhood at the same position corresponding to a current time unit and at least one reference time unit from a shift register-based cache at a preset rate.

[0165] The optical computing chip then uses the pixel features within the pixel neighborhood at the same location corresponding to the current time unit and at least one reference time unit to identify a target object within the pixel neighborhood at the same location corresponding to the current time unit. The preset rate is consistent with the rate at which the optical computing chip identifies target objects within the pixel neighborhood at the same location corresponding to the current time unit. This rate matching improves the synergistic efficiency of the linear cache and the optical computing chip, thereby improving image recognition efficiency.

[0166] In some embodiments, steps S10' to S30' in FIG. 8 can be performed using one or more coprocessors under the control of a central processing unit. The synergy between the central processing unit and the coprocessors can improve image processing efficiency. Multiple coprocessors can also improve parallel processing efficiency. For example, the image to be recognized can be multiple images from multiple channels.

[0167] In some embodiments, the image of the object to be identified in each time unit may be acquired in the following manner.

[0168] First, for each time unit, a plurality of field images of the object to be identified are acquired using an imaging device.

[0169] Then, based on the multiple field images of the object to be identified, an image to be identified of the object to be identified in each time unit is obtained.

[0170] In some embodiments, for each time unit, at least one of a registration and stitching operation, a super-resolution reconstruction operation, and an image boundary extension operation is performed on multiple field images of the image to be identified to obtain an image to be identified of the object to be identified in each time unit.

[0171] FIG9 is a block diagram illustrating an image processing apparatus according to some embodiments of the present disclosure.

[0172] As shown in FIG9 , an image processing apparatus 91 includes a memory 911 and a processor 912 coupled to the memory 911. The memory 911 is configured to store instructions for executing the image processing method according to the embodiments. The processor 912 is configured to execute the image processing method according to any of the embodiments of the present disclosure based on the instructions stored in the memory 911.

[0173] FIG10 is a block diagram illustrating an image processing system according to some embodiments of the present disclosure.

[0174] As shown in FIG10 , the image processing system 10 includes an image processing device 11 . The image processing device 11 is configured to execute any image processing method described in the present disclosure, and its function and structure may be the same as those of the image processing device 91 .

[0175] In some embodiments, the image processing device 11 includes a shift register-based buffer 111. The shift register-based buffer 111 is configured to obtain fluorescence intensity values ​​within a pixel neighborhood at the same location on each substrate image corresponding to each cycle, or to obtain pixel features within a pixel neighborhood at the same location on each image to be identified corresponding to each time unit. In some embodiments, the specific structure and function of the shift register-based buffer 111 can be described with reference to Figures 2 to 7B . For example, the shift register-based buffer can be referred to as a multi-cycle image buffer.

[0176] In some embodiments, the image processing device 11 further includes a linear cache 112 and a processing unit 113. For example, the linear cache can be referred to as a neighborhood data cache. The neighborhood data cache can be implemented as an on-chip or off-chip linear storage device such as SRAM (Static Random-Access Memory), GDDR (Graphics Double Data Rate), or HBM (High Bandwidth Memory), or as a customized storage device with a special structure using an FPGA or ASIC (Application Specific Integrated Circuit).

[0177] The linear cache 112 is configured to output fluorescence intensity values ​​within a pixel neighborhood at the same location corresponding to a current cycle and at least one reference cycle from the shift register-based cache at a preset rate, or to output pixel features within a pixel neighborhood at the same location corresponding to a current time unit and at least one reference time unit from the shift register-based cache at a preset rate.

[0178] The processing unit 113 is configured to utilize an optical computing chip to identify the base at the base position corresponding to the current cycle based on the fluorescence intensity value within the pixel neighborhood at the same position corresponding to the current cycle and at least one reference cycle, wherein the preset rate is consistent with the rate at which the optical computing chip identifies the base at the base position corresponding to the current cycle, or to utilize an optical computing chip to identify the target object within the pixel neighborhood at the same position corresponding to the current time unit based on the pixel features within the pixel neighborhood at the same position corresponding to the current time unit and at least one reference time unit, wherein the preset rate is consistent with the rate at which the optical computing chip identifies the target object within the pixel neighborhood at the same position corresponding to the current time unit. In this way, the present disclosure can use multiple cycle images of multiple channels at the same position on the substrate of the optical path as input, output the most likely base at the position, and estimate the accuracy of the judgment, etc.

[0179] In some embodiments, the image processing apparatus 11 further includes an image processing result buffer 114. The image processing result buffer 114 is configured to receive and buffer the image processing result from the processing unit 113.

[0180] In some embodiments, the image processing apparatus 11 further includes an apparatus controller 110 . The apparatus controller 110 is configured to control a shift register-based buffer 111 , a linear cache 112 , a processing unit 113 , and an image processing result buffer 114 within the image processing apparatus 11 .

[0181] In some embodiments, the base call system 10 includes a central processing unit 12 and one or more image processing devices 11. The one or more image processing devices 11 serve as one or more coprocessors. The central processing unit 12 is configured to control the one or more coprocessors. The one or more image processing devices are not shown in FIG10.

[0182] In some embodiments, the central processor 12 includes a system controller 121. The system controller 121 is configured to control one or more image processing devices 11 as co-processors.

[0183] In some embodiments, the central processor 12 further includes an output sequence signal buffer 122. The output sequence signal buffer 122 is configured to obtain image processing results from the image processing result buffer 114 of the image processing device 11 based on the system controller 121 and cache the image processing results.

[0184] In some embodiments, the image processing system 10 further includes an imaging device 13. The imaging device 13 is configured to perform imaging operations on the sequencing substrate or the object to be identified, thereby obtaining multiple field-of-view images. The central processing unit 12 further includes an image reading controller 123. The image reading controller 123 is configured to control the imaging device 13 to perform imaging operations and to control image access operations between different buffers.

[0185] In some embodiments, the image processing system 10 further includes a first cache 124 and a second cache 125. The first cache 124 is configured to cache multiple field of view images obtained by the imaging device 13 through imaging. The second cache 125 is configured to cache images to be recognized or substrate images obtained based on the multiple field of view images. For example, the first cache can be referred to as a FOV image cache, and the second cache can be referred to as a single-cycle image cache.

[0186] In some embodiments, the first cache 125 can be implemented as a global cache of a computer system. The optical computing chip can also be replaced by a GPU (Graphics Processing Unit) or a dedicated FPGA. The shift register-based cache 111 and the linear cache 112 can be implemented using the GPU's internal video memory.

[0187] FIG. 11 is a block diagram illustrating an image processing system according to other embodiments of the present disclosure.

[0188] Figure 11 differs from Figure 10 in that Figure 11 illustrates multiple coprocessor scenarios of image processing systems in some other embodiments. The following will only describe the differences between Figure 11 and Figure 10, and will not repeat the similarities.

[0189] As shown in Fig. 11, the coprocessor 11 is the image processing apparatus 11 in Fig. 10 serving as a coprocessor. In this case, the coprocessor controller 110 in Fig. 11 is equivalent to the apparatus controller 110 in Fig. 10. The image processing system 10 includes a plurality of coprocessors 11.

[0190] In some embodiments, the image processing system 10 further includes a coprocessor 13. The coprocessor 13 includes a second cache 125 and a processing unit 131. The processing unit 131 is configured to perform at least one of a registration and stitching operation, a super-resolution reconstruction operation, and an image boundary extension operation on the multiple field of view images in the first cache 124, and store the operation results in the second cache 125. For example, the coprocessor 13 can be implemented by a CPU (Central Processing Unit), a GPU, or a dedicated FPGA, and the two coprocessors 11 can be implemented by an optical computing chip. The two coprocessors 11 can be executed by dividing the substrate image or the image to be identified in the same cycle or time unit in the second cache 125 into two sub-images of equal size and inputting them into the shift register-based caches 111 of the two coprocessors 11. This approach can reduce the demand on the high-speed linear cache within the coprocessor 11, reduce the load on each coprocessor 11, and improve parallel processing efficiency.

[0191] In some embodiments, when the coprocessor 11 is implemented based on an optical computing chip, the processing unit 113 is a high-throughput processing unit. The processing units 113 in multiple coprocessors 11 are parallel processing units.

[0192] FIG12 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.

[0193] 12 , the computer system 120 may be a general-purpose computing device and may include a memory 1210 , a processor 1220 , and a bus 1200 that connects various system components.

[0194] The memory 1210 may include, for example, system memory, non-volatile storage media, and the like. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs. The system memory may include volatile storage media, such as random access memory (RAM) and / or cache memory. The non-volatile storage media may store, for example, instructions for executing at least one of the corresponding embodiments of the image processing method. Non-volatile storage media include, but are not limited to, disk storage, optical storage, and flash memory.

[0195] The processor 1220 can be implemented as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, or as discrete hardware components such as discrete gates or transistors. Accordingly, each module, such as the judgment module and the determination module, can be implemented by a central processing unit (CPU) executing instructions in a memory that execute corresponding steps, or by a dedicated circuit that executes the corresponding steps.

[0196] The bus 1200 may use any of a variety of bus architectures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0197] The computer system 120 may also include an input / output interface 1230, a network interface 1240, a storage interface 1250, and the like. These interfaces 1230, 1240, and 1250, as well as the memory 1210 and the processor 1220, may be connected via a bus 1200. The input / output interface 1230 provides a connection interface for input / output devices such as a display, mouse, and keyboard. The network interface 1240 provides a connection interface for various networked devices. The storage interface 1250 provides a connection interface for external storage devices such as floppy disks, USB flash drives, and SD cards.

[0198] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks, can be implemented by computer-readable program instructions.

[0199] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0200] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to operate in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0201] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0202] The image processing method, device, system, and computer storable medium in the above embodiments can improve the accuracy of image recognition.

[0203] The image processing method, apparatus, system, and computer-storable medium according to the present disclosure have been described in detail. To avoid obscuring the concepts of the present disclosure, some details known in the art have been omitted. Based on the above description, those skilled in the art will fully understand how to implement the technical solutions disclosed herein.

Claims

1. An image processing method, comprising: For each cycle in a current cycle and at least one reference cycle adjacent to the current cycle, acquiring multiple substrate images of the sequencing substrate in multiple fluorescent channels in each cycle, wherein the current cycle and the at least one reference cycle are multiple consecutive biochemical reaction and imaging cycles occurring sequentially, each fluorescent channel corresponds to a substrate image, and a pixel value of the substrate image of each fluorescent channel represents a fluorescence intensity value under each fluorescent channel; Obtaining fluorescence intensity values ​​within a pixel neighborhood at the same position on each substrate image corresponding to each cycle; and The base at the base position corresponding to the current cycle is identified based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle.

2. The image processing method according to claim 1, wherein: Obtaining the fluorescence intensity value within the pixel neighborhood at the same position on each substrate image corresponding to each cycle includes: The fluorescence intensity values ​​within the pixel neighborhood at the same position on each substrate image corresponding to each cycle are obtained using a shift register-based buffer.

3. The image processing method according to claim 2, wherein: The shift register-based cache includes a first cache area corresponding to each fluorescence channel, the first cache area includes a plurality of first sub-cache areas connected in sequence, the size of each first sub-cache area is the same as the pixel size of the substrate image, the plurality of first sub-cache areas have the same number of cycles as the current cycle and the at least one reference cycle, and the connection order of the plurality of first sub-cache areas is consistent with the occurrence order of the current cycle and the at least one reference cycle, each first sub-cache area is configured to shift the fluorescence intensity value of each pixel on the substrate image stored therein to the first sub-cache area connected in sequence before it based on the shift register, and output the fluorescence intensity value within the pixel neighborhood at the same position during each shift process.

4. The image processing method according to claim 3, wherein: Each first sub-buffer area includes a first area and a second area sequentially connected to the first area, wherein the first area is configured to shift the fluorescence intensity value of each pixel based on a shift register and output the fluorescence intensity value within the pixel neighborhood at the same position during each shift process, and the second area is configured to shift the fluorescence intensity value of each pixel.

5. The image processing method according to claim 4, wherein: The first area includes multiple shift registers connected in sequence, and the number of shift registers in the first area is the product of the number of pixels of the width of the substrate image and the number of pixels of the width or height of the pixel neighborhood at the same position. Each shift register in the first area is configured to shift the fluorescence intensity value of each pixel, and the shift registers located at a preset sequential position in the first area are also configured to output the fluorescence intensity value within the pixel neighborhood at the same position during each shift process. The preset sequential position is the arrangement position of the shift registers corresponding to each pixel included in the pixel neighborhood at the same position in the multiple shift registers. The image processing method according to claim 4 , wherein: The second area includes multiple shift registers connected in sequence, the number of shift registers in the second area is the product of the number of pixels of the width of the substrate image and the pixel difference, the pixel difference is the difference between the number of pixels of the height of the substrate image and the number of pixels of the width or height of the pixel neighborhood at the same position, and each shift register in the second area is configured to shift the fluorescence intensity value of each pixel.

7. The image processing method according to claim 4, wherein: The second area includes a read address decoder, a write address decoder and a linear storage area. The linear storage area within the second area includes multiple storage units, each storage unit is configured to store the fluorescence intensity value of each pixel, the address of the storage unit pointed to by the write address decoder is the previous address of the storage unit pointed to by the read address decoder, and the read address decoder and the write address decoder are configured to shift the fluorescence intensity values ​​of each pixel in the multiple storage units.

8. The image processing method according to claim 3, wherein: The shift register-based cache also includes a second cache area corresponding to each fluorescent channel, and the second cache area is sequentially connected after the first cache area corresponding to it. The second cache area is configured to store substrate images of multiple fluorescent channels of multiple buffer cycles that occur sequentially after the current cycle and the at least one reference cycle, and shift the substrate images of multiple fluorescent channels of the multiple buffer cycles.

9. The image processing method according to claim 8, wherein: The second buffer area includes a plurality of sequentially connected second sub-buffer areas, each second sub-buffer area includes a plurality of sequentially connected shift registers, and the number of shift registers in each second sub-buffer area is the same as the number of pixels of the substrate image.

10. The image processing method according to claim 8, wherein: The second cache area includes a first linear storage area and a ping-pong storage area, the first linear storage area is connected after the ping-pong storage area, and the ping-pong storage area is configured to shift the substrate images of the multiple fluorescence channels of the multiple buffer cycles based on a ping-pong cache mechanism.

11. The image processing method according to claim 10, wherein: The ping-pong storage area includes a second linear storage area and its corresponding address decoder and a third linear storage area and its corresponding address decoder for implementing a ping-pong cache mechanism.

12. The image processing method according to claim 10, wherein: The ping-pong storage area includes a first shift register area and a second shift register area. The first shift register area and the second shift register area each include a plurality of shift registers for implementing a ping-pong buffer mechanism.

13. The image processing method according to claim 2, wherein: Identifying a base at a base position corresponding to the current cycle according to fluorescence intensity values ​​within a pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle includes: Outputting fluorescence intensity values ​​within a pixel neighborhood corresponding to the same position of the current cycle and the at least one reference cycle from the shift register-based cache at a preset rate using a linear cache; and Using an optical computing chip, the base at the base position corresponding to the current cycle is identified based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle, and the preset rate is consistent with the rate at which the optical computing chip identifies the base at the base position corresponding to the current cycle.

14. The image processing method according to claim 1, wherein: Under the control of the central processing unit, using one or more co-processors, acquiring substrate images of the sequencing substrate in multiple fluorescent channels in each cycle; Obtain the fluorescence intensity value within the pixel neighborhood at the same position on each substrate image corresponding to each cycle; The base at the base position corresponding to the current cycle is identified based on the fluorescence intensity values ​​within the pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle.

15. The image processing method according to claim 1, wherein: Acquiring multiple substrate images of the sequencing substrate in multiple fluorescent channels in each cycle includes: For each cycle, using an imaging device, acquiring multiple field images of the sequencing substrate in each fluorescent channel; and According to the multiple field images of the sequencing substrate in each fluorescent channel, a substrate image of each fluorescent channel of the sequencing substrate in each cycle is acquired.

16. The image processing method according to claim 15, wherein: Acquiring, according to the multiple field images of the sequencing substrate in each fluorescent channel, a substrate image of the sequencing substrate in each fluorescent channel in each cycle includes: For each cycle, at least one of a registration and stitching operation, a super-resolution reconstruction operation, and an image boundary extension operation is performed on multiple field-of-view images of the sequencing substrate in each fluorescent channel to obtain a substrate image of the sequencing substrate in each fluorescent channel in each cycle.

17. An image processing method, comprising: For each time unit of a current time unit and at least one reference time unit adjacent to the current time unit, acquiring an image of the object to be identified in each time unit, wherein the current time unit and the at least one reference time unit are continuous in time sequence; Obtaining pixel features within a pixel neighborhood at the same position on each image to be identified corresponding to each time unit using a shift register-based cache; and Based on pixel features in the pixel neighborhood at the same position corresponding to the current time unit and the at least one reference time unit, a target object in the pixel neighborhood at the same position corresponding to the current time unit is identified, wherein the target object belongs to at least a part of the object to be identified.

18. The image processing method according to claim 17, wherein: The image to be identified includes images to be identified corresponding to multiple image channels, each image channel corresponds to an image to be identified, the shift register-based cache includes a first cache area corresponding to each image channel, the first cache area includes multiple first sub-cache areas connected in sequence, the size of each first sub-cache area is the same as the pixel size of the image to be identified, the multiple first sub-cache areas have the same number of time units as the current time unit and the at least one reference time unit, and the connection order of the multiple first sub-cache areas is consistent with the occurrence order of the current time unit and the at least one reference time unit, each first sub-cache area is configured to shift the pixel features of each pixel on the image to be identified stored therein to the first sub-cache area connected in sequence before it based on the shift register, and output the pixel features in the pixel neighborhood at the same position during each shift process.

19. The image processing method according to claim 18, wherein: Each first sub-buffer area includes a first area and a second area sequentially connected to the first area, wherein the first area is configured to shift the pixel features of each pixel based on a shift register and output the pixel features within the pixel neighborhood at the same position during each shift process, and the second area is configured to shift the pixel features of each pixel.

20. The image processing method according to claim 19, wherein: The first area includes multiple shift registers connected in sequence, and the number of shift registers in the first area is the product of the number of pixels of the width of the image to be identified and the number of pixels of the width or height of the pixel neighborhood at the same position. Each shift register in the first area is configured to shift the pixel features of each pixel, and the shift registers located at a preset sequential position in the first area are also configured to output the pixel features in the pixel neighborhood at the same position during each shift process. The preset sequential position is the arrangement position of the shift registers corresponding to each pixel included in the pixel neighborhood at the same position in the multiple shift registers.

21. The image processing method according to claim 19, wherein: The second area includes multiple shift registers connected in sequence, the number of shift registers in the second area is the product of the number of pixels of the width of the image to be identified and the pixel difference, the pixel difference is the difference between the number of pixels of the height of the image to be identified and the number of pixels of the width or height of the pixel neighborhood at the same position, each shift register in the second area is configured to shift the pixel features of each pixel.

22. The image processing method according to claim 19, wherein: The second area includes a read address decoder, a write address decoder and a linear storage area. The linear storage area within the second area includes multiple storage units, each storage unit is configured to store pixel characteristics of each pixel, the address of the storage unit pointed to by the write address decoder is the previous address of the address of the storage unit pointed to by the read address decoder, and the read address decoder and the write address decoder are configured to shift the pixel characteristics of each pixel in the multiple storage units.

23. The image processing method according to claim 18, wherein: The shift register-based cache also includes a second cache area corresponding to each image channel, and the second cache area is sequentially connected after the corresponding first cache area. The second cache area is configured to store the images to be identified of multiple image channels of the object to be identified for multiple buffer time units that are sequentially arranged after the current time unit and the at least one reference time unit in time sequence, and shift the images to be identified of multiple image channels for the multiple buffer time units.

24. The image processing method according to claim 23, wherein: The second buffer area includes a plurality of second sub-buffer areas connected in sequence, each second sub-buffer area includes a plurality of shift registers connected in sequence, and the number of shift registers in each second sub-buffer area is the number of pixels of the image to be recognized.

25. The image processing method according to claim 23, wherein: The second cache area includes a first linear storage area and a ping-pong storage area, the first linear storage area is connected after the ping-pong storage area, and the ping-pong storage area is configured to shift the images to be identified of the multiple image channels of the multiple buffer time units based on the ping-pong cache mechanism.

26. The image processing method according to claim 25, wherein: The ping-pong storage area includes a second linear storage area and its corresponding address decoder and a third linear storage area and its corresponding address decoder for implementing a ping-pong cache mechanism.

27. The image processing method according to claim 25, wherein: The ping-pong storage area includes a first shift register area and a second shift register area. The first shift register area and the second shift register area each include a plurality of shift registers for implementing a ping-pong buffer mechanism.

28. The image processing method according to claim 17, wherein: Identifying a target object in a pixel neighborhood at a same position corresponding to the current time unit and the at least one reference time unit according to pixel features in the pixel neighborhood at the same position includes: Outputting pixel features within a pixel neighborhood corresponding to the current time unit and the at least one reference time unit at the same position from the shift register-based cache at a preset rate using a linear cache; and Using an optical computing chip, a target object within the pixel neighborhood at the same position corresponding to the current time unit and the at least one reference time unit is identified based on pixel features within the pixel neighborhood at the same position corresponding to the current time unit, and the preset rate is consistent with the rate at which the optical computing chip identifies the target object within the pixel neighborhood at the same position corresponding to the current time unit.

29. The image processing method according to claim 17, wherein: Under the control of the central processing unit, using one or more coprocessors, obtaining the image of the object to be identified in each time unit; Obtain pixel features within a pixel neighborhood at the same position on each image to be identified corresponding to each time unit; And based on the pixel features in the pixel neighborhood at the same position corresponding to the current time unit and the at least one reference time unit, the target object in the pixel neighborhood at the same position corresponding to the current time unit is identified.

30. The image processing method according to claim 17, wherein: Acquiring the image of the object to be identified in each time unit includes: For each time unit, using an imaging device, acquiring a plurality of field images of the object to be identified; and An image of the object to be identified in each time unit is acquired according to the multiple field images of the object to be identified.

31. The image processing method according to claim 30, wherein: Acquiring, according to the multiple field images of the image to be recognized, the image to be recognized in each time unit includes: For each time unit, at least one of a registration and stitching operation, a super-resolution reconstruction operation, and an image boundary extension operation is performed on multiple field images of the image to be identified to obtain the image to be identified of the object to be identified in each time unit.

32. An image processing device comprising: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the image processing method according to any one of claims 1 to 31 based on instructions stored in the memory.

33. An image processing system comprising: The image processing apparatus according to claim 32.

34. The image processing system according to claim 33, wherein: The image processing device comprises: The shift register-based cache is configured to obtain the fluorescence intensity value within the pixel neighborhood at the same position on each substrate image corresponding to each cycle or to obtain the pixel features within the pixel neighborhood at the same position on each image to be identified corresponding to each time unit.

35. The image processing system according to claim 34, wherein: The image processing device further includes: a linear cache configured to output fluorescence intensity values ​​within a pixel neighborhood at the same position corresponding to a current cycle and at least one reference cycle from the shift register-based cache at a preset rate, or configured to output pixel features within a pixel neighborhood at the same position corresponding to a current time unit and at least one reference time unit from the shift register-based cache at a preset rate; and A processing unit is configured to utilize an optical computing chip to identify a base at a base position corresponding to the current cycle based on fluorescence intensity values ​​within a pixel neighborhood at the same position corresponding to the current cycle and the at least one reference cycle, wherein the preset rate is consistent with a rate at which the optical computing chip identifies bases at base positions corresponding to the current cycle, or is configured to utilize an optical computing chip to identify a target object within a pixel neighborhood at the same position corresponding to the current time unit and the at least one reference time unit based on pixel features within the pixel neighborhood at the same position corresponding to the current time unit, wherein the preset rate is consistent with a rate at which the optical computing chip identifies a target object within a pixel neighborhood at the same position corresponding to the current time unit.

36. The image processing system according to claim 35, wherein: The base recognition system includes a central processing unit and one or more image processing devices, wherein the one or more image processing devices serve as one or more coprocessors, and the central processing unit is configured to control the one or more coprocessors.

37. A computer storable medium having computer program instructions stored thereon, which, when executed by a processor, implement the image processing method according to any one of claims 1 to 31.

Images