Image analysis system and method for controlling shooting of sample image

Abstract

The invention provides an image analysis system and a method for controlling shooting of a sample image, the system comprises an imaging device, a moving device and an image analysis device, the imaging device comprises an objective lens and a shooting unit, the shooting unit shoots a sample to be tested on a smear to obtain a first image, and first image information of the first image is obtained; the camera shooting unit shoots an area in the visual field on the smear to obtain a second image, and second image information of the second image is obtained; the moving device is used for driving the smear and / or the objective lens to move; the image analysis device is used for analyzing the first image information and the second image information and determining the corresponding position of the second image on the first image; and determining motion information required by the smear and / or the objective lens to move to the target motion position based on the corresponding position. The actual view position when the second image is shot is recognized by determining the corresponding relation between the second image and the first image, so that the smear is controlled to move to the target movement position, the method is simple, and positioning is accurate.

Description

[0001] This application is a divisional application of the Chinese invention patent application with application number "202010232109.3". The application date of the original application is March 27, 2020, and the name of the invention is "Image analysis system and method for controlling the shooting of sample images". Technical Field

[0002] The present invention relates to the field of image analysis systems, and more particularly, to an image analysis system and a method for controlling the shooting of sample images. Background Art

[0003] The current image analysis system (e.g., a blood cell digital image analysis system) performs cell analysis in the following manner: use a low-power objective lens (e.g., 10x) to capture images of target cells to locate cells (e.g., white blood cells), and then use a high-power objective lens (e.g., 100x) to capture images of the cells located by the low-power objective lens one by one. However, after capturing a cell with a high-power objective lens and controlling the movement of the smear to capture another cell with the high-power objective lens, due to mechanical problems such as motor step loss or different thread pitches, there is an inherent problem of inaccurate positioning, resulting in the smear failing to accurately move to the target movement position, thereby failing to capture the target cell. Summary of the invention

[0004] The present invention is proposed to solve the above problems. According to one aspect of the present invention, an image analysis system is provided, the image analysis system comprising an imaging device, a moving device and an image analysis device, wherein: the imaging device comprises an objective lens and a camera unit, wherein the camera unit photographs a sample to be tested on a smear to obtain a first image, and obtains first image information of the first image; the camera unit photographs an area within a field of view on the smear to obtain a second image, and obtains second image information of the second image; the moving device is used to drive the smear and / or the objective lens to move the smear and / or the objective lens; the image analysis device is used to: analyze the first image information and the second image information, and determine the corresponding position of the second image on the first image; based on the corresponding position, determine the motion information required for the smear and / or the objective lens to move to the target motion position.

[0005] According to another aspect of the present invention, there is provided a sample analysis system, comprising a blood analysis device, a smear preparation device, an image analysis device and a control device, wherein the image analysis device comprises the image analysis system as described above.

[0006] According to another aspect of the present invention, there is provided a method for controlling the capturing of sample images by an image analysis system, the method comprising: capturing a sample to be tested on a smear to obtain a first image, and acquiring first image information of the first image; capturing an area within a field of view on the smear to obtain a second image, and acquiring second image information of the second image; analyzing the first image information and the second image information to determine a corresponding position of the second image on the first image; and based on the corresponding position, determining motion information required for the smear and / or the objective lens to move to a target motion position.

[0007] According to another aspect of the present invention, there is provided a computer-readable storage medium, comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, can execute the above method.

[0008] The image analysis system and the method for controlling the shooting of sample images of the embodiments of the present invention identify the actual field of view position when the second image is shot by determining the correspondence between the second image and the first image, thereby controlling the smear to move to the target motion position. The method is simple, the positioning is accurate, and it can be implemented using the existing system without the need for additional devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other purposes, features and advantages of the present invention will become more apparent by describing the embodiments of the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 A schematic structural block diagram of an image analysis system according to an embodiment of the present invention is shown;

[0011] Figure 2 shows an exemplary first image according to one embodiment of the present invention;

[0012] Figure 3 shows an exemplary second image according to one embodiment of the present invention;

[0013] Figure 4 A schematic diagram showing analysis of a first image and a second image using a sliding window method according to an embodiment of the present invention is shown;

[0014] Figure 5 An exemplary integral image according to an embodiment of the present invention is shown, wherein the coordinates of four vertices of the sliding window are shown;

[0015] Figure 6A schematic diagram showing analysis of a first image and a second image when a sliding window is partially located outside the first image according to an embodiment of the present invention;

[0016] Figure 7 A flowchart showing the steps of a method for controlling the image analysis system to capture a sample image according to an embodiment of the present invention is shown;

[0017] Figure 8 A schematic structural diagram of a sample analysis system according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

[0019] In order to solve the above problems, an embodiment of the present invention provides an image analysis system and a method for controlling the shooting of sample images, the image analysis system comprising an imaging device, a moving device and an image analysis device, wherein: the imaging device comprises an objective lens and a camera unit, wherein the camera unit shoots the sample to be tested on the smear to obtain a first image, and obtains first image information of the first image; the camera unit shoots the area within the field of view on the smear to obtain a second image, and obtains second image information of the second image; the moving device is used to drive the smear and / or the objective lens to move the smear and / or the objective lens; the image analysis device is used to: analyze the first image information and the second image information to determine the corresponding position of the second image on the first image; based on the corresponding position, determine the motion information required for the smear and / or the objective lens to move to the target motion position.

[0020] The image analysis system and the method for controlling the shooting of sample images of the embodiments of the present invention identify the actual field of view position when the second image is shot by determining the correspondence between the second image and the first image, thereby controlling the movement of the smear to the target movement position. The method is simple, the positioning is accurate, and it can be implemented using the existing system without the need for additional devices.

[0021] The solution of the present invention is described in detail below in conjunction with specific embodiments.

[0022] Embodiment 1

[0023] This embodiment provides an image analysis system. Figure 1 , Figure 1 FIG. 1 shows a schematic structural block diagram of an image analysis system 10 according to an embodiment of the present invention. Figure 1 As shown, the image analysis system 10 may include an imaging device 100, an image analysis device 200, and a moving device 300. The imaging device 100 is used to capture an image of a sample to be tested smeared on a smear; the image analysis device 200 is used to analyze an image of a sample to be tested (e.g., cells in a sample); and the moving device 300 is used to drive the smear and / or the objective lens of the imaging device 100 to move the smear and / or the objective lens so as to capture an image of a target cell in the sample to be tested in a specific area on the smear. It should be understood that the moving device 300 may only drive the smear to move, or may only drive the objective lens to move, or may simultaneously drive the smear and the objective lens so that the two move simultaneously, and the present invention is not limited to this. When the moving device 300 drives the smear and the objective lens simultaneously, it may include a first moving device and a second moving device, which are used to drive the smear and the objective lens, respectively.

[0024] Exemplarily, the sample to be tested may be a blood sample, etc., which is not limited in the present invention. Exemplarily, the sample to be tested may contain various cells, such as white blood cells, neutrophils, red blood cells, etc., which is not limited in the present invention.

[0025] The imaging device 100 may include an objective lens unit 110 and a camera unit 120. The objective lens unit 110 may include a first objective lens and a second objective lens. Exemplarily, the first objective lens may be a low-power objective lens, such as a 10x objective lens, and the second objective lens may be a high-power objective lens, such as a 100x objective lens. The objective lens unit 110 may also include a third objective lens, etc., and the third objective lens may be, for example, a 40x objective lens. It should be understood that the above-mentioned low-power objective lens and high-power objective lens are relative, and do not specifically refer to certain multiples of objective lenses as low-power objective lenses, or certain multiples of objective lenses as high-power objective lenses. The camera unit 120 may include any camera, camera head, etc. known in the art, such as a CCD camera, a CMOS camera, etc., and the present invention is not limited to this.

[0026] When the imaging device 100 is equipped with an objective lens, the camera unit 120 photographs the sample to be tested on the smear to obtain a first image, and obtains first image information of the first image; then the camera unit 120 photographs the area within the field of view on the smear to obtain a second image, and obtains second image information of the second image. Exemplarily, the camera unit 120 can also obtain the field of view position corresponding to a certain area on the first image that is desired to be photographed, which is referred to as the target shooting position herein. Exemplarily, the image information may include feature information, pixel information, and other information of the image.

[0027] Exemplarily, the first image and the second image can be obtained by photographing with different objective lenses, for example, the first image is photographed with a first objective lens (for example, a 10x objective lens), and the second image is photographed with a second objective lens (for example, a 100x objective lens), for example, a low-power objective lens is used to photograph the sample to be tested on the smear to obtain a first image (low-power objective lens image), and then a high-power objective lens is used to photograph the cells in the sample to be tested one by one to obtain multiple second images (high-power objective lens images). Exemplarily, for example, for a certain photographed object, the first image obtained by the first photograph is not clear, and it is necessary to re-photograph to obtain the second image, the first image and the second image can also be obtained by photographing twice with the same objective lens, for example, by photographing twice with the same high-power objective lens (for example, a 100x objective lens), or by photographing twice with the same low-power objective lens (for example, a 10x objective lens), and the present invention is not limited to this.

[0028] In the example where the first image is taken with a low-magnification objective lens (e.g., a 10x objective lens) and the second image is taken with a high-magnification objective lens (e.g., a 100x objective lens), the field of view of the low-magnification objective lens is all the samples to be tested on the entire smear, so all cells can be included in the first image, such as Figure 2 As shown, the cells in the first image are white blood cells; the field of view of the high-power objective lens is a small area on the smear, so the second image may include one cell, and when two or three cells are close to each other, it may also include two or three cells, or may not include cells, such as Figure 3 A situation involving two cells is shown.

[0029] Exemplarily, for example, when the camera unit 120 has finished photographing a cell and needs to photograph the next cell, or when it is found that the image of the photographed cell is unclear and needs to be re-photographed, the image analysis device 200 can use the image registration method to determine the motion information required for the smear and / or the objective lens to move to the corresponding target motion position, so as to re-photograph after moving to the target motion position. Wherein, the target motion position is the position to which the smear and / or the objective lens needs to be moved when the user wants to take a clear image of an object. Exemplarily, the mobile device 300 can only drive the smear to move, or only drive the objective lens to move, or drive both to move at the same time, and the present invention does not limit this. Wherein, in the case where the mobile device 300 drives both the smear and the objective lens to move at the same time, the mobile device 300 can include a first mobile device and a second mobile device, which are respectively used to drive the smear to move and drive the objective lens to move.

[0030] This article takes the example that the mobile device 300 only drives the smear movement as an example. At this time, the image analysis device 200 can use the image registration method to determine the movement information required for the smear to move to the target movement position, thereby guiding the movement of the smear. The target movement position is the position to which the smear needs to move when an image of the target shooting position is to be captured.

[0031] Specifically, the image analysis device 200 may determine the motion information required for the smear to move to the target motion position by using the image registration method: the image analysis device 200 may analyze the first image information and the second image information, determine the corresponding position of the second image on the first image, and determine the motion information required for the smear to move to the target motion position based on the corresponding position. The corresponding position of the second image on the first image is the area on the first image to which the second image corresponds.

[0032] In one embodiment, the motion information may include the direction and distance of the smear movement, etc. The distance of the movement may be in the form of the number of movement steps, etc. It should be understood that the motion information may also include other information about the smear movement, which is not limited in the present invention.

[0033] In one embodiment, the image analysis device 200 can determine the motion information required for the smear to move to the target motion position based on the corresponding position of the second image on the first image and the target shooting position on the first image. Specifically, the image analysis device 200 can obtain the coordinates of the corresponding position and the coordinates of the target shooting position, subtract the coordinates of the corresponding position and the coordinates of the target shooting position to obtain the coordinate difference, and convert the coordinate difference into the motion information required for the smear. Exemplarily, the coordinates of the corresponding position and the coordinates of the target motion position can both be represented by, for example, the number of moving steps of the mobile device, and the coordinates and distances of other positions involved in this article can also be represented by the number of moving steps of the mobile device.

[0034] Among them, since the corresponding position of the second image on the first image is a small area, the coordinates of a special point on the corresponding position can be selected as the coordinates of the corresponding position, such as the point at the upper left corner, the point at the upper right corner, the point at the lower left corner, the point at the lower right corner, the center point, etc. of the corresponding position, and the present invention is not limited to this. Preferably, the coordinates of the center point of the corresponding position can be selected as the coordinates of the corresponding position.

[0035] Exemplarily, an image registration algorithm may be used to analyze the first image information and the second image information to determine the corresponding position of the second image on the first image. Exemplarily, the image registration algorithm may include a grayscale and template-based registration algorithm, wherein the template-based registration algorithm may be referred to as a template matching method (Blocking Matching), which is to find a sub-image similar to the template image in another image based on a known template image; the grayscale-based matching algorithm may also be referred to as a correlation matching algorithm, which uses a spatial two-dimensional sliding template for matching. Exemplarily, the template-based registration algorithm may include, for example, a mean absolute difference algorithm (MAD), a sum of absolute errors algorithm (SAD), a sum of squared errors algorithm (SSD), etc. Exemplarily, the image registration algorithm may also include a feature-based registration algorithm, which first extracts the features of the image, then generates a feature descriptor, and finally matches the features of the two images according to the similarity of the feature descriptors. Exemplarily, the features of the image may be mainly divided into features such as points, lines (edges), and regions (surfaces), and may also be divided into local features and global features, wherein point features and edge features are mainly used. Exemplarily, the registration algorithm based on point features may include, for example, Harris, Moravec, KLT, Harr-like, HOG, LBP, SIFT, SURF, BRIEF, SUSAN, FAST, CENSUS, FREAK, BRISK, ORB, optical flow method, A-KAZE and other algorithms, and the registration algorithm based on edge features may include, for example, LoG operator, Robert operator, Sobel operator, Prewitt operator, Canny operator, etc., which are not limited in the present invention. It should be understood that other methods may also be used to determine the corresponding position of the second image on the first image, which are not limited in the present invention.

[0036] An exemplary method of using an image registration algorithm to determine the corresponding position of the second image on the first image will be described in detail later.

[0037] In one embodiment, after the smear has moved to the target position according to the movement information, the image analysis device 200 may also determine whether the smear has moved to the target movement position.

[0038] Exemplarily, the image analysis device 200 may use the above-mentioned method for determining the corresponding position to determine whether the smear has moved to the target moving position. Specifically, when the smear has moved to the position, the camera unit 120 may again shoot the area in the current field of view on the smear to obtain another second image (referred to herein as the current second image), and obtain the image information of the current second image (referred to herein as the current second image information), such as pixel information, feature information, etc.; then the image analysis device 200 analyzes the current second image information and the first image information, determines the corresponding position of the current second image on the first image (referred to herein as the current corresponding position), and determines whether the smear has moved to the target moving position based on the current corresponding position and the target shooting position on the first image. Exemplarily, any of the above-mentioned image registration algorithms may be used to determine the current corresponding position of the current second image on the first image, that is, which area on the first image the current second image corresponds to. Specifically, if the determined current corresponding position coincides or substantially coincides with the target shooting position, it can be determined that the smear has moved to the target moving position, otherwise the smear has not moved to the target moving position. It should be understood that other methods may also be used to determine the current corresponding position of the current second image on the first image, and the present invention is not limited thereto. An exemplary method for determining the current corresponding position of the current second image on the first image using an image registration algorithm will be described in detail later.

[0039] When it is determined that the smear has not moved to the target movement position, the image analysis device 200 can determine the movement information required to move the smear to the target movement position again based on the current corresponding position and the target shooting position (referred to as the current movement information herein). Specifically, the coordinates of both the current corresponding position and the target shooting position can be obtained, and the coordinates of the target shooting position and the coordinates of the current corresponding position are subtracted to obtain the coordinate difference (referred to as the current coordinate difference herein), and the current coordinate difference is converted into the current movement information required for the smear to move to the target movement position, for example, the current movement direction and movement distance, etc. Exemplarily, the coordinates of a special point on the current corresponding position can be selected as the coordinates of the current corresponding position, such as the point in the upper left corner, the point in the upper right corner, the point in the lower left corner, the point in the lower right corner, the center point, etc. of the current corresponding position, and the present invention does not limit this. Preferably, the coordinates of the center point of the current corresponding position can be selected as the coordinates of the current corresponding position.

[0040] When the smear moves to the target motion position according to the current motion information, the image analysis device 200 determines again whether the smear moves to the target motion position. This process is similar to the above process of determining whether the smear moves to the target motion position, and will not be repeated here. If it is determined that the smear still has not moved to the target motion position, the above process is repeated, and the image analysis device 200 updates the current motion information required for the smear to move to the target motion position until the smear moves to the target motion position according to the updated current motion information. Among them, the process of the image analysis device 200 determining the updated current motion information is similar to the above process of determining the motion information and the current motion information, and will not be repeated here.

[0041] In one embodiment, the image analysis system 10 further includes an alarm device 400 for issuing an alarm message when certain conditions are met. For example, the alarm message may include a sound alarm, a graphic alarm, a text alarm, etc., which is not limited in the present invention.

[0042] In one embodiment, the image analysis device 200 can obtain the number of times the current motion information is updated, and when the number of times the current motion information is updated is greater than or equal to a preset update number threshold, it indicates that a system failure occurs, and the alarm device 400 is notified to issue an alarm message. Exemplarily, the preset update number threshold can be set by the user as needed, such as 3 times, 4 times, 5 times, etc., and the present invention is not limited to this.

[0043] In one embodiment, if the smear has not moved to the target movement position after the current movement information has been updated multiple times, the image analysis device 200 can obtain the total time spent in the process of updating the current movement information and the smear movement, and if the total time exceeds the preset total time threshold, it indicates that the system has a fault, and the alarm device 400 is notified to issue an alarm message. Exemplarily, the preset total time threshold can be set by the user as needed, such as 3s, 5s, 10s, etc., and the present invention is not limited to this.

[0044] In one embodiment, if the image analysis device 200 determines that the smear has not moved to the target moving position, the distance between the current corresponding position and the target shooting position is calculated. If the distance is greater than a preset distance threshold, it indicates that a system failure has occurred, and the alarm device 400 is notified to issue an alarm message. Exemplarily, the preset distance threshold can be set by the user as needed, such as 1 mm, 2 mm, 3 mm, etc., and the present invention is not limited thereto.

[0045] In one embodiment, the image analysis device 200 can obtain multiple target shooting positions and multiple current corresponding positions corresponding thereto, calculate multiple distances between each target shooting position in the multiple target shooting positions and the current corresponding positions corresponding thereto, and if the number of distances in the multiple distances that are greater than a preset distance threshold exceeds a preset number threshold, indicating that a system failure occurs, the alarm device 400 is notified to issue an alarm message. Exemplarily, the preset number threshold can be set by the user as required, such as 3, 4, 5, etc., and the present invention is not limited thereto.

[0046] In another embodiment, while the smear is moving toward the target motion position according to the motion information, the image analysis device 200 may update in real time the motion information required for the smear to move to the target motion position (referred to herein as real-time motion information), so that the smear can reach the target motion position faster and more accurately according to the real-time motion information.

[0047] Specifically, during the movement of the smear to the target movement position according to the movement information, the camera unit 120 can take real-time photos of the area within the field of view on the smear to obtain multiple second images (referred to herein as real-time second images), and obtain image information of the real-time second images (referred to herein as real-time second image information), such as pixel information, feature information, etc. The image analysis device 200 can analyze the real-time second image information and the first image information of the first image to determine the corresponding position of the real-time second image on the first image (referred to herein as the real-time corresponding position). Exemplarily, any of the above-mentioned image registration algorithms can be used to determine the real-time corresponding position of the real-time second image on the first image, that is, which area on the first image the real-time second image currently corresponds to, and the present invention is not limited to this. It should be understood that other methods can also be used to determine the real-time corresponding position of the real-time second image on the first image, and the present invention is not limited to this. Among them, an exemplary method of using an image registration algorithm to determine the real-time corresponding position of the real-time second image on the first image will be described in detail later.

[0048] Then, the image analysis device 200 can update the real-time motion information required for the smear to move to the target motion position in real time based on the real-time corresponding position, until the smear moves to the target motion position according to the updated real-time motion information. Exemplarily, the image analysis device 200 can update the real-time motion information required for the smear to move to the target motion position in real time based on the real-time corresponding position and the target shooting position on the first image. Specifically, the image analysis device 200 can obtain the coordinates of the real-time corresponding position and the target shooting position, subtract the coordinates of the two to obtain the coordinate difference (referred to as the real-time coordinate difference herein), and convert the real-time coordinate difference into the real-time motion information of the smear, thereby updating the real-time motion information in real time, such as the real-time motion direction and the real-time motion distance, so that the smear moves according to the updated real-time motion information, thereby guiding the smear to move to the target motion position faster and more accurately.

[0049] For example, during the movement of the smear toward the target movement position, if the number of times that the camera unit 120 captures the real-time second image and the image analysis device 200 updates the real-time movement information according to the corresponding position of the real-time second image on the first image exceeds the preset real-time number threshold, the alarm device 400 is notified to issue an alarm message. For example, the preset real-time number threshold can be set by the user as needed, such as 2 times, 3 times, 4 times, 5 times, etc., and the present invention is not limited to this.

[0050] In one embodiment, when the smear does not move to the target moving position within a preset time threshold, the alarm device 400 issues an alarm message. For example, the preset time threshold can be set by the user according to actual needs, such as 3s, 5s, 10s, etc., and the present invention is not limited to this.

[0051] In one embodiment, the process of using an image registration algorithm to determine the corresponding position of the second image on the first image, the current corresponding position of the current second image on the first image, and the real-time corresponding position of the real-time second image on the first image is similar. The following is an example of using an image registration algorithm to analyze the second image information and the first image information to determine the corresponding position of the second image on the first image.

[0052] In one embodiment, the image analysis device 200 may extract pixel information or feature information from the first image information and the second image information, and determine the corresponding position of the second image on the first image based on the pixel information or feature information. It should be understood that the image analysis device 200 may also determine the corresponding position of the second image on the first image based on other methods, such as a method based on domain transformation, etc., and the present invention is not limited to this.

[0053] Exemplarily, when determining the corresponding position of the second image on the first image based on pixel information, the image analysis device 200 can use the pixel information of both the first image and the second image to establish a similarity measurement index, and determine the corresponding position of the second image on the first image based on the similarity measurement index.

[0054] In order to improve the discrimination of the similarity measurement index, a similarity measurement index may be established based on the average pixel value of the second image and the first image. For example, the similarity measurement index γ may be established using the following formula:

[0055]

[0056] Among them, x and y are the coordinates of the pixel points respectively, and f(x, y) is the pixel value of each pixel point in a certain area on the first image. is the average pixel value of all pixels in the area on the first image, g(x,y) is the pixel value of each pixel on the second image, is the average pixel value of all pixels on the second image. It should be understood that f(x, y) can also be used. Represent the second image, using g(x,y), The formula of the present invention can improve the discrimination of the similarity measurement index and better distinguish the second image from the first image.

[0057] In one embodiment, the image analysis device 200 may use a sliding window method to analyze the first image and the second image to determine the value of the similarity measurement index, thereby determining the corresponding position of the second image on the first image.

[0058] Before analyzing the first image and the second image using the sliding window method, the image analyzing device 200 needs to pre-process the first image and the second image.

[0059] Exemplarily, the preprocessing may include resampling the first image and / or the second image so that the resolution of the first image and the second image are the same, that is, the size represented by each pixel on the two images is the same. Resampling may include upsampling and downsampling. Exemplarily, the resolution of the first image may be kept unchanged, and the second image may be downsampled to the same resolution as the first image, or the resolution of the second image may be kept unchanged, and the first image may be upsampled to the same resolution as the second image, or the first image and the second image may be resampled to a certain resolution at the same time, and the present invention is not limited to this. The resampling method may adopt the nearest neighbor interpolation method, bilinear interpolation method, cubic convolution interpolation method, etc., which are well known in the art, and the present invention is not limited to this.

[0060] In order to reduce the amount of calculation, the preprocessing may also include converting the first image and the second image into grayscale images before resampling the first image and / or the second image. The method of converting a color image into a grayscale image is well known and will not be described in detail here. It should be understood that the preprocessing may also include rotating the image, denoising, and other operations, which are not limited in the present invention.

[0061] Specifically, Figure 4 As shown, the sliding window method may include: setting a sliding window so that the sliding window slides along a predetermined path on the first image, wherein the size of the sliding window is the same as the size of the second image; during the sliding process of the sliding window, calculating in real time the value of a similarity measurement index between the second image and the area of ​​the first image located within the sliding window; selecting the maximum value of the similarity measurement index, and the area of ​​the first image within the sliding window corresponding to the maximum value is the corresponding position of the second image on the first image.

[0062] In one embodiment, the predetermined path may include a path from left to right and from top to bottom on the first image. In another embodiment, the predetermined path may include It should be understood that the predetermined path may also be other suitable paths, and the present invention is not limited thereto.

[0063] When the field of view when taking the second image is at the edge of the field of view when taking the first image, there will be an edge problem, which makes the value of the correlation metric index low, so that the correct corresponding position cannot be found. Therefore, in one embodiment, if the sliding window is partially located outside the first image, the image analysis device 200 can also calculate the value of the similarity metric index based on a preset overlap threshold.

[0064] Specifically, if the ratio of the area of ​​the overlapping portion of the sliding window and the first image to the total area of ​​the sliding window is greater than or equal to the overlapping threshold, it indicates that most of the sliding window is in the first image at this time. At this time, the image analysis device 200 can calculate the value of the similarity measurement index between the portion of the first image that overlaps with the sliding window and the corresponding portion of the second image. Figure 6 As shown, at this time, only the value of the similarity metric index between the dotted area in the sliding window on the first image and the dotted area on the second image is calculated. If the ratio of the area of ​​the overlapping part of the sliding window and the first image to the total area of ​​the sliding window is less than the overlap threshold, it means that only a small part of the sliding window is in the first image at this time. At this time, the overlapping area of ​​the field of view of the two images is very small, and the accuracy of the similarity metric index is low. Therefore, the image analysis device 200 can calculate the similarity metric index as zero. Among them, the overlap threshold can be set according to experience, for example, it can be set to 0.6-0.8, etc., and the present invention is not limited to this.

[0065] Since the value of the similarity metric index between the second image and the area of ​​the first image within the sliding window must be recalculated each time the sliding window slides one pixel, many positions need to be traversed, so the amount of calculation is very large. In order to optimize the operation speed, in one embodiment, the image analysis device 200 may first transfer the first image and the second image to the frequency domain, and then calculate the value of the similarity metric index between the second image and the area of ​​the first image within the sliding window. The method of transferring the first image and the second image to the frequency domain may be the FFT (Fast Fourier Transform) method, etc., which will not be described in detail here.

[0066] In order to optimize the operation speed, in another embodiment, the image analysis device 200 may use the integral graph method to calculate the value of the similarity measurement index between the second image and the area of ​​the first image located in the sliding window. First, the image analysis device 200 may first construct an array as an integral image, the width and height of the integral image being the same as those of the first image, and then assign values ​​to the array, with the value of each point being the sum of the pixel values ​​of all the pixels in the coverage area from the upper left corner of the first image to the point; then the integral image may be used to conveniently calculate the sum of the pixel values ​​of all the pixels in the sliding window, and then the value of the similarity measurement index between the second image and the area of ​​the first image located in the sliding window may be conveniently calculated. Figure 5 As shown in the figure, the sum of the pixel values ​​of all pixels in the sliding window is f(x2,y2)+f(x1,y1)-f(x1,y2)-f(x2,y1), where (x1,y1), (x2,y1), (x1,y2) and (x2,y2) are the coordinates of the four vertices of the sliding window respectively.

[0067] In one embodiment, the image analysis system 10 may further include a display interface (not shown) for real-time display of the corresponding position, the current corresponding position, the real-time corresponding position, the target motion position, etc., and may also display in real time the motion path of the smear and / or the imaging device, etc. Exemplarily, the display interface may be presented on the display of the image analysis system 10, or on the display of the image analysis device 200, which is not limited in the present invention.

[0068] In one embodiment, the image analysis system 10 may further include a smear identification device, a smear clamping device, and a smear recovery device, wherein the smear identification device is used to identify the identity information of the smear, the smear clamping device is used to clamp the smear from the identification device to the mobile device 300 for detection, and the smear recovery device is used to place the detected smear. In another embodiment, the image analysis system 10 may also include a smear basket loading device for loading a smear basket containing a smear to be tested, and a smear clamping device for clamping the smear to be tested in the smear basket loaded on the smear basket loading device to the smear identification device for identity information identification. It should be understood that the image analysis system 10 may also include other devices known in the art, and the present invention is not limited thereto.

[0069] Embodiment 2

[0070] This embodiment provides a sample analysis system A, referring to Figure 8 , Figure 8 FIG. 4 shows a schematic diagram of the structure of a sample analysis system A according to an embodiment of the present invention. Figure 8 As shown, the sample analysis system A may include a blood analysis device A100, a smear preparation device A200, an image analysis device A300, and a control device A400, etc. Among them, the blood analysis device A110 is used to perform routine blood tests on the sample to be tested, the smear preparation device A200 is used to prepare a smear of the sample to be tested, the image analysis device A300 is used to take and analyze the image of the sample to be tested on the smear, and the control device A400 is connected to the blood analysis device A100, the smear preparation device A200, and the image analysis device A300 in a communication manner. The sample analysis system A may also include a transmission track connecting each device and a feeding mechanism of each device, etc., which will not be described in detail here to avoid unnecessary ambiguity of the present invention.

[0071] The image analysis device A300 may be the image analysis system 10 according to the first embodiment of the present invention. The structure of the image analysis system 10 is described in detail in the first embodiment and will not be described in detail herein.

[0072] Embodiment 3

[0073] This embodiment provides a method for controlling the image analysis system to capture a sample image. Figure 7 , Figure 7 A flowchart of a method 700 for controlling the capturing of sample images by an image analysis system according to an embodiment of the present invention is shown.

[0074] like Figure 7 As shown, method 700 may include the following steps:

[0075] Step S710: photographing the sample to be tested on the smear to obtain a first image, and acquiring first image information of the first image; photographing the area within the field of view on the smear to obtain a second image, and acquiring second image information of the second image.

[0076] For example, the sample to be tested may be a blood sample, etc., which is not limited in the present invention. For example, the sample to be tested may include various cells, such as white blood cells, neutrophils, red blood cells, etc., which is not limited in the present invention. For example, the image information may include feature information, pixel information, and other information of the image.

[0077] Exemplarily, the first image and the second image can be obtained by photographing with different objective lenses, for example, the first image is photographed with a first objective lens (for example, a 10x objective lens), and the second image is photographed with a second objective lens (for example, a 100x objective lens), for example, a low-power objective lens is used to photograph the sample to be tested on the smear to obtain a first image (low-power objective lens image), and then a high-power objective lens is used to photograph the cells in the sample to be tested one by one to obtain multiple second images (high-power objective lens images). Exemplarily, for example, for a certain photographed object, the first image obtained by the first photograph is not clear, and it is necessary to re-photograph to obtain the second image, the first image and the second image can also be obtained by photographing twice with the same objective lens, for example, by photographing twice with the same high-power objective lens (for example, a 100x objective lens), or by photographing twice with the same low-power objective lens (for example, a 10x objective lens), and the present invention is not limited to this.

[0078] In the example where the first image is taken with a low-power objective lens (e.g., a 10x objective lens) and the second image is taken with a high-power objective lens (e.g., a 100x objective lens), the field of view of the low-power objective lens is all samples to be tested on the entire smear, so all cells can be included in the first image; the field of view of the high-power objective lens is a small area on the smear, so the second image can include one cell, and when two or three cells are close to each other, it can also include two or three cells, or it may not include any cells.

[0079] Step S720: Analyze the first image information and the second image information to determine the corresponding position of the second image on the first image, wherein the corresponding position of the second image on the first image is the area on the first image to which the second image corresponds.

[0080] Exemplarily, an image registration algorithm may be used to analyze the second image information and the first image information to determine the corresponding position of the second image on the first image. Exemplarily, the image registration algorithm may include a grayscale and template-based registration algorithm, wherein the template-based registration algorithm may be referred to as a template matching method (Blocking Matching), which is to find a sub-image similar to the template image in another image based on a known template image; the grayscale-based matching algorithm may also be referred to as a correlation matching algorithm, which uses a spatial two-dimensional sliding template for matching. Exemplarily, the template-based registration algorithm may include, for example, a mean absolute difference algorithm (MAD), a sum of absolute errors algorithm (SAD), a sum of squared errors algorithm (SSD), etc. Exemplarily, the image registration algorithm may also include a feature-based registration algorithm, which first extracts the features of the image, then generates a feature descriptor, and finally matches the features of the two images according to the similarity of the feature descriptors. Exemplarily, the features of the image can be mainly divided into features such as points, lines (edges), and regions (surfaces), and can also be divided into local features and global features, wherein point features and edge features are mainly used. Exemplarily, the point feature-based registration algorithm may include, for example, Harris, Moravec, KLT, Harr-like, HOG, LBP, SIFT, SURF, BRIEF, SUSAN, FAST, CENSUS, FREAK, BRISK, ORB, optical flow method, A-KAZE and other algorithms, and the edge feature-based registration algorithm may include, for example, LoG operator, Robert operator, Sobel operator, Prewitt operator, Canny operator, etc., which are not limited in the present invention. It should be understood that other methods can also be used to determine the corresponding position of the second image on the first image, which are not limited in the present invention. It should be understood that other methods can also be used to determine the corresponding position of the second image on the first image, which are not limited in the present invention.

[0081] Among them, using an image registration algorithm to analyze the second image information and the first image information, and determining the corresponding position of the second image on the first image may include: extracting pixel information or feature information from the first image information and the second image information, and determining the corresponding position of the second image on the first image based on the pixel information or the feature information. It should be understood that the corresponding position of the second image on the first image may also be determined based on other methods, such as a method based on domain transformation, etc., and the present invention is not limited to this.

[0082] Exemplarily, determining the corresponding position of the second image on the first image based on pixel information may include: establishing a similarity measurement index using pixel information of both the first image and the second image, and determining the corresponding position of the second image on the first image based on the similarity measurement index.

[0083] In order to improve the discrimination of the similarity measurement index, the method 700 may include: establishing a similarity measurement index based on the average pixel values ​​of the second image and the first image. For example, the similarity measurement index γ may be established using the following formula:

[0084]

[0085] Among them, x and y are the coordinates of the pixel points respectively, and f(x, y) is the pixel value of each pixel point in a certain area on the first image. is the average pixel value of all pixels in the area on the first image, g(x,y) is the pixel value of each pixel on the second image, is the average pixel value of all pixels on the second image. It should be understood that f(x, y) can also be used. Represent the second image, using g(x,y), The formula of the present invention can improve the discrimination of the similarity measurement index and better distinguish the second image from the first image.

[0086] Exemplarily, a sliding window method may be used to analyze the first image and the second image to determine a value of a similarity measurement index, thereby determining a corresponding position of the second image on the first image.

[0087] Before analyzing the first image and the second image using the sliding window method, the first image and the second image need to be preprocessed.

[0088] Exemplarily, the preprocessing may include resampling the first image and / or the second image so that the resolution of the first image and the second image are the same, that is, the size represented by each pixel on the two images is the same. Resampling may include upsampling and downsampling. Exemplarily, the resolution of the first image may be kept unchanged, and the second image may be downsampled to the same resolution as the first image, or the resolution of the second image may be kept unchanged, and the first image may be upsampled to the same resolution as the second image, or the first image and the second image may be resampled to a certain resolution at the same time, and the present invention is not limited to this. The resampling method may adopt the nearest neighbor interpolation method, bilinear interpolation method, cubic convolution interpolation method, etc., which are well known in the art, and the present invention is not limited to this.

[0089] In order to reduce the amount of calculation, the preprocessing may also include converting both the first image and the second image into grayscale images before resampling the first image and / or the second image. The method of converting a color image into a grayscale image is well known and will not be described in detail here.

[0090] Specifically, the sliding window method may include: setting a sliding window so that the sliding window slides along a predetermined path on the first image, wherein the size of the sliding window is the same as the size of the second image; during the sliding process of the sliding window, calculating in real time the value of a similarity measurement index between the second image and the area of ​​the first image located within the sliding window; selecting the maximum value of the similarity measurement index, and the area of ​​the first image within the sliding window corresponding to the maximum value is the corresponding position of the second image on the first image.

[0091] In one embodiment, the predetermined path may include a path from left to right and from top to bottom on the first image. In another embodiment, the predetermined path may include It should be understood that the predetermined path may also be other suitable paths, and the present invention is not limited thereto.

[0092] When the field of view when taking the second image is at the edge of the field of view when taking the first image, there will be an edge problem, which makes the value of the correlation metric index low, so that the correct corresponding position cannot be found. Therefore, in one embodiment, if the sliding window is partially located outside the first image, the method 700 may also include: calculating the value of the similarity metric index based on a preset overlap threshold.

[0093] Specifically, if the ratio of the area of ​​the overlapping part of the sliding window and the first image to the total area of ​​the sliding window is greater than or equal to the overlap threshold, it indicates that most of the sliding window is in the first image at this time, and the method 700 may include: calculating the value of the similarity measurement index between the part of the first image that overlaps with the sliding window and the corresponding part of the second image. If the ratio of the area of ​​the overlapping part of the sliding window and the first image to the total area of ​​the sliding window is less than the overlap threshold, it indicates that only a small part of the sliding window is in the first image at this time, and the overlapping area of ​​the field of view of the two images is very small, and the accuracy of the similarity measurement index is low, so the method 700 may include: calculating the similarity measurement index as zero. Among them, the overlap threshold can be set according to experience, for example, it can be set to 0.6-0.8, etc., and the present invention is not limited to this.

[0094] Since the value of the similarity metric index between the second image and the area of ​​the first image within the sliding window must be recalculated each time the sliding window slides one pixel, many positions need to be traversed, so the amount of calculation is very large. In order to optimize the operation speed, in one embodiment, method 700 may also include: first converting the first image and the second image to the frequency domain, and then calculating the value of the similarity metric index between the second image and the area of ​​the first image within the sliding window. The method of converting the first image and the second image to the frequency domain may be the FFT (Fast Fourier Transform) method, etc., which will not be repeated here.

[0095] In order to optimize the operation speed, in another embodiment, method 700 may include: using the integral image method to calculate the value of the similarity metric between the second image and the area of ​​the first image located in the sliding window. First, an array may be constructed as an integral image, the width and height of the integral image being the same as those of the first image, and then the array is assigned a value, and the value of each point is the sum of the pixel values ​​of all the pixel points in the coverage area from the upper left corner vertex of the first image to the point; then the integral image can be used to conveniently calculate the sum of the pixel values ​​of all the pixel points in the sliding window, and then the value of the similarity metric between the second image and the area of ​​the first image located in the sliding window can be conveniently calculated.

[0096] Step S730: Based on the corresponding position of the second image on the first image, determine the motion information required for the smear to move to the target motion position.

[0097] The corresponding position of the second image on the first image is the area on the first image to which the second image corresponds, and the target motion position is the position to which the smear needs to be moved when the user wants to take a clear image of a certain area.

[0098] In one embodiment, the motion information may include the direction and distance of the smear movement, etc. The distance of the movement may be in the form of the number of movement steps, etc. It should be understood that the motion information may also include other information about the smear movement, which is not limited in the present invention.

[0099] Exemplarily, the target shooting position on the first image can be obtained, and based on the corresponding position of the second image on the first image and the target shooting position, the motion information required for the smear to move to the target motion position can be determined. Specifically, the coordinates of both the corresponding position and the target shooting position can be obtained, and the coordinates of the corresponding position and the coordinates of the target shooting position can be subtracted to obtain the coordinate difference, and the coordinate difference can be converted into the motion information required for the smear, such as the motion direction and the motion distance. Among them, the coordinates of the corresponding position and the coordinates of the target motion position can both be expressed by, for example, the number of moving steps of the mobile device, and the coordinates of other positions involved in this article can also be expressed by the number of moving steps of the mobile device.

[0100] Among them, since the corresponding position of the second image on the first image is a small area, the coordinates of a special point on the corresponding position can be selected as the coordinates of the corresponding position, such as the point at the upper left corner, the point at the upper right corner, the point at the lower left corner, the point at the lower right corner, the center point, etc. of the corresponding position, and the present invention is not limited to this. Preferably, the coordinates of the center point of the corresponding position can be selected as the coordinates of the corresponding position.

[0101] It should be understood that when photographing target cells, the smear may be moved, the imaging device may be moved, or both may be moved simultaneously to achieve focusing, and the present invention does not limit this. In this embodiment, the smear movement is used as an example for explanation.

[0102] Specifically, in one embodiment, after the smear moves to the target motion position according to the motion information, the method 700 may further include: determining whether the smear moves to the target motion position. Specifically, the current visual field area on the smear may be photographed to obtain the current second image, and the current second image information of the current second image, such as pixel information, feature information, etc., may be obtained; then the current second image information and the first image information are analyzed to determine the current corresponding position of the current second image on the first image, and based on the current corresponding position and the target shooting position on the first image, determine whether the smear moves to the target motion position. Exemplarily, any of the above-mentioned image registration algorithms may be used to determine the current corresponding position of the current second image on the first image, that is, which area on the first image the current second image corresponds to. Specifically, if the determined current corresponding position coincides or substantially coincides with the target shooting position, it may be determined that the smear has moved to the target motion position, otherwise the smear has not moved to the target motion position. It should be understood that other methods may also be used to determine the current corresponding position of the current second image on the first image, and the present invention is not limited thereto. Among them, an exemplary method of using an image registration algorithm to determine the current corresponding position of the current second image on the first image is similar to the above-mentioned method of determining the corresponding position of the second image on the first image, and will not be repeated here.

[0103] When it is determined that the smear has not moved to the target movement position, method 700 may also include: based on the current corresponding position and the target shooting position, re-determining the current movement information required to move the smear to the target movement position. Specifically, the coordinates of both the current corresponding position and the target shooting position can be obtained, and the coordinates of the target shooting position and the coordinates of the current corresponding position are subtracted to obtain the current coordinate difference, and the current coordinate difference is converted into the current movement information required for the smear to move to the target movement position, for example, the current movement direction and movement distance, etc. Exemplarily, the coordinates of a special point on the current corresponding position can be selected as the coordinates of the current corresponding position, such as the point in the upper left corner, the point in the upper right corner, the point in the lower left corner, the point in the lower right corner, the center point, etc. of the current corresponding position, and the present invention is not limited to this. Preferably, the coordinates of the center point of the current corresponding position can be selected as the coordinates of the current corresponding position.

[0104] After the smear moves to the target motion position according to the current motion information, method 700 may further include: determining again whether the smear moves to the target motion position. This process is similar to the above-mentioned process of determining whether the smear moves to the target motion position, and will not be repeated here. If it is determined that the smear still does not move to the target motion position, the above process is repeated to update the current motion information required to move the smear to the target motion position until the smear moves to the target motion position according to the updated current motion information.

[0105] In one embodiment, the method 700 may further include: issuing an alarm message when certain conditions are met. For example, the alarm message may include an audible alarm, a graphic alarm, a text alarm, etc., which is not limited in the present invention.

[0106] In one embodiment, the method 700 may further include: obtaining the number of times the current motion information is updated, and when the number of times the current motion information is updated is greater than or equal to a preset update number threshold, it indicates that a system failure occurs, and an alarm message is issued. Exemplarily, the preset update number threshold can be set by the user as needed, such as 3 times, 4 times, 5 times, etc., and the present invention is not limited to this.

[0107] In one embodiment, if the smear has not moved to the target movement position after the current movement information is updated multiple times, the method 700 may further include: obtaining the total time spent in the process of updating the current movement information and the smear movement, and if the total time exceeds a preset total time threshold, indicating that the system fails, an alarm message is issued. Exemplarily, the preset total time threshold can be set by the user as needed, such as 3s, 5s, 10s, etc., and the present invention is not limited to this.

[0108] In one embodiment, if it is determined that the smear has not moved to the target moving position, the distance between the current corresponding position and the target shooting position is calculated. If the distance is greater than a preset distance threshold, it indicates that the system has a fault, and an alarm message is issued. Exemplarily, the preset distance threshold can be set by the user as needed, such as 1mm, 2mm, 3mm, etc., and the present invention is not limited to this.

[0109] In one embodiment, the method 700 may further include: obtaining multiple target shooting positions and multiple current corresponding positions corresponding thereto, calculating multiple distances between each target shooting position in the multiple target shooting positions and the current corresponding positions corresponding thereto, and issuing an alarm message if the number of distances in the multiple distances that are greater than a preset distance threshold exceeds a preset number threshold. Exemplarily, the preset number threshold may be set by the user as required, such as 3, 4, 5, etc., and the present invention is not limited thereto.

[0110] In another embodiment, while the smear is moving toward the target motion position according to the motion information, method 700 may further include: updating the real-time motion information required for the smear to move to the target motion position in real time, so that the smear can reach the target motion position faster and more accurately according to the real-time motion information.

[0111] Specifically, during the movement of the smear to the target motion position according to the motion information, the area within the field of view on the smear can be photographed in real time to obtain a real-time second image, and the real-time second image information of the real-time second image, such as pixel information, feature information, etc., is obtained, and then the real-time second image information and the first image information of the first image are analyzed to determine the real-time corresponding position of the real-time second image on the first image. Exemplarily, any of the above-mentioned image registration algorithms can be used to determine the real-time corresponding position of the real-time second image on the first image, that is, which area on the first image the real-time second image currently corresponds to, and the present invention is not limited to this. It should be understood that other methods can also be used to determine the real-time corresponding position of the real-time second image on the first image, and the present invention is not limited to this. Among them, an exemplary method of using an image registration algorithm to determine the real-time corresponding position of the real-time second image on the first image is similar to the above-mentioned method for determining the corresponding position of the second image on the first image, and will not be repeated here.

[0112] Then, method 700 may further include: based on the real-time corresponding position, updating the real-time motion information required for the smear to move to the target motion position in real time, until the smear moves to the target motion position according to the real-time motion information. Exemplarily, the real-time motion information required for the smear to move to the target motion position can be updated in real time based on the real-time corresponding position and the target shooting position on the first image. Specifically, the coordinates of the real-time corresponding position and the target shooting position can be obtained, and the real-time coordinate difference can be obtained by subtracting the coordinates of the two, and the real-time coordinate difference is converted into the real-time motion information of the smear, thereby updating the real-time motion information in real time, such as the real-time motion direction and the real-time motion distance, so that the smear moves according to the updated motion information, thereby guiding the smear to move to the target motion position faster and more accurately.

[0113] Exemplarily, during the movement of the smear toward the target movement position, if the number of times the real-time second image is captured and the real-time movement information is updated according to the corresponding position of the real-time second image on the first image exceeds a preset real-time number threshold, an alarm message is issued. Exemplarily, the preset real-time number threshold can be set by the user as needed, such as 2 times, 3 times, 4 times, 5 times, etc., and the present invention is not limited to this.

[0114] In one embodiment, if the smear does not move to the target movement position within a preset time threshold, an alarm message may be issued. For example, the preset time threshold may be set by the user according to actual needs, such as 3s, 5s, 10s, etc., and the present invention is not limited thereto.

[0115] In one embodiment, the method 700 may further include: displaying the corresponding position, current corresponding position, real-time corresponding position and target motion position on a display of the image analysis system 10 or on a display of the image analysis device 200, etc., and may also display the motion route of the smear in real time.

[0116] Embodiment 4

[0117] The present embodiment provides a computer-readable medium having a computer program stored thereon, and the computer program executes the method described in the above embodiment when it is running. Any tangible, non-temporary computer-readable medium can be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray discs, etc.), flash memory and / or the like. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing device to form a machine, so that these instructions executed on a computer or other programmable data processing device can generate a device that implements a specified function. These computer program instructions can also be stored in a computer-readable memory, which can instruct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory can form a manufactured product, including an implementation device that implements a specified function. The computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device can provide steps for implementing the specified function.

[0118] Technical effects of the present invention:

[0119] The image analysis system and the method for controlling the shooting of sample images of the embodiments of the present invention identify the actual field of view position when the second image is shot by determining the correspondence between the second image and the first image, thereby controlling the smear to move to the target motion position. The method is simple, the positioning is accurate, and it can be implemented using the existing system without the need for additional devices.

[0120] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.

[0121] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0122] Similarly, it should be understood that in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

[0123] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0124] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0125] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0126] The above is only a specific embodiment of the present invention or an explanation of a specific embodiment. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An image analysis system, characterized in that: The image analysis system comprises an imaging device, a moving device and an image analysis device, wherein the imaging device comprises an objective lens, wherein: The imaging device is used to photograph the sample to be tested on the smear in a first field of view to obtain a first image, so as to obtain first image information of the first image; and photograph the sample to be tested on the smear in a second field of view to obtain a second image, so as to obtain second image information of the second image; The moving device is used to drive the smear and / or the objective lens to move; after the imaging device photographs the sample to be tested on the smear in the first field of view to obtain the first image, the moving device drives the smear and / or the objective lens to move so that the imaging device photographs the sample to be tested on the smear in the second field of view to obtain the second image; The image analysis device is used for: Analyzing the first image information and the second image information to determine a corresponding position of the second image on the first image; and Determining whether the corresponding position coincides or substantially coincides with the target shooting position on the first image; When it is determined that the corresponding position coincides with or substantially coincides with the target shooting position, determining a target image of the sample to be tested based on the second image; When it is determined that the corresponding position does not overlap or substantially overlaps with the target shooting position, the motion information required for the smear and / or the objective lens to move to the target motion position is determined based on the corresponding position and the target shooting position, so that the moving device drives the smear and / or the objective lens to move to the target motion position according to the motion information, and the imaging device photographs the sample to be tested on the smear under the current second field of view to obtain a current second image, so as to obtain the current second image information of the current second image.

2. The image analysis system according to claim 1, characterized in that: In the process in which the moving device drives the smear and / or the objective lens to move toward the target movement position according to the movement information, the image analysis device is further used for: The real-time motion information required for the smear and / or the objective lens to move to the target motion position is updated in real time, so that the moving device drives the smear and / or the objective lens to move to the target motion position according to the real-time motion information.

3. The image analysis system according to claim 2, characterized in that: In the process that the moving device drives the smear and / or the objective lens to move toward the target movement position according to the movement information, the imaging device is also used to photograph the sample to be tested on the smear in real time under the second field of view to obtain a real-time second image, so as to obtain real-time second image information of the real-time second image; The image analysis device is also used to analyze the first image information and the real-time second image information to determine the real-time corresponding position of the real-time second image on the first image; and based on the real-time corresponding position and the target shooting position, update the real-time motion information in real time.

4. The image analysis system according to claim 1, characterized in that: The image analysis device is also used for: Analyzing the first image information and the current second image information to determine a current corresponding position of the current second image on the first image, and determining whether the smear and / or the objective lens moves to the target movement position according to the current corresponding position; If the smear and / or the objective lens moves to the target movement position, determining the target image of the sample to be tested based on the current second image; If the smear and / or the objective lens does not move to the target movement position, the updated current motion information required for the smear and / or the objective lens to move to the target movement position is determined based on the current corresponding position and the target shooting position, so as to drive the smear and / or the objective lens to move to the target movement position according to the updated current motion information.

5. The image analysis system according to claim 4, characterized in that: The image analysis system also includes an alarm device for issuing an alarm message; Wherein, when the number of updates of the current motion information is greater than or equal to a preset update number threshold, when the total time spent on the update of the current motion information and the movement of the smear and / or the objective lens exceeds a preset total time threshold, or when the distance between the current corresponding position and the target shooting position exceeds a preset distance threshold, the alarm device issues the alarm information.

6. The image analysis system according to claim 1, characterized in that: The objective lens comprises a low-power objective lens and a high-power objective lens; The first image is obtained by taking pictures under the low-power objective lens, and the second image is obtained by taking pictures under the high-power objective lens.

7. The image analysis system according to claim 1, characterized in that: The area of ​​the sample to be tested in the second field of view is a part of the area of ​​the sample to be tested in the first field of view.

8. The image analysis system according to any one of claims 1 to 7, characterized in that: The image analysis system further includes a display interface, wherein the display interface is used to display at least one of the following: The corresponding position, the target shooting position, the target movement position, the movement route of the smear and / or the objective lens, and the alarm information.

9. A method for controlling the shooting of sample images by an image analysis system, characterized in that: The method comprises: photographing the sample to be tested on the smear in a first field of view to obtain a first image, and acquiring first image information of the first image; photographing the sample to be tested on the smear in a second field of view to obtain a second image, and acquiring second image information of the second image; After the imaging device photographs the sample to be tested on the smear in the first field of view to obtain the first image, driving the smear and / or the objective lens to move so that the imaging device photographs the sample to be tested on the smear in the second field of view to obtain the second image; Analyzing the first image information and the second image information to determine a corresponding position of the second image on the first image; and Determining whether the corresponding position coincides or substantially coincides with the target shooting position on the first image; When it is determined that the corresponding position coincides with or substantially coincides with the target shooting position, determining a target image of the sample to be tested based on the second image; When it is determined that the corresponding position does not overlap or substantially overlaps with the target shooting position, the motion information required for the smear and / or the objective lens to move to the target movement position is determined based on the corresponding position and the target shooting position, so as to drive the smear and / or the objective lens to move to the target movement position according to the motion information, and photograph the sample to be tested on the smear under the current second field of view to obtain a current second image, and obtain the current second image information of the current second image.

10. The method according to claim 9, characterized in that The method further comprises: in the process of driving the smear and / or the objective lens to move to the target movement position according to the movement information, updating in real time the real-time movement information required for the smear and / or the objective lens to move to the target movement position; and The smear and / or the objective lens are driven to move to the target movement position according to the real-time movement information.

11. The method according to claim 10, characterized in that The real-time updating of the real-time motion information required for the smear and / or the objective lens to move to the target motion position includes: photographing the sample to be tested on the smear in real time under the second field of view to obtain a real-time second image, and acquiring real-time second image information of the real-time second image; Analyzing the first image information and the real-time second image information to determine a real-time corresponding position of the real-time second image on the first image; and Based on the real-time corresponding position and the target shooting position, the real-time motion information is updated in real time.

12. The method according to claim 9, characterized in that The method further comprises: Analyzing the first image information and the current second image information to determine a current corresponding position of the current second image on the first image, and determining whether the smear and / or the objective lens moves to the target movement position according to the current corresponding position; If the smear and / or the objective lens moves to the target movement position, determining the target image of the sample to be tested based on the current second image; If the smear and / or the objective lens does not move to the target movement position, the updated current motion information required for the smear and / or the objective lens to move to the target movement position is determined based on the current corresponding position and the target shooting position, so as to drive the smear and / or the objective lens to move to the target movement position according to the updated current motion information.

13. The method according to claim 9, characterized in that The analyzing the first image information and the second image information to determine the corresponding position of the second image on the first image includes: An image registration algorithm is used to analyze the first image information and the second image information to determine a corresponding position of the second image on the first image.

14. The method according to claim 13, characterized in that The adopting an image registration algorithm to analyze the first image information and the second image information to determine a corresponding position of the second image on the first image includes: extracting pixel information or feature information from the first image information and the second image information; and A corresponding position of the second image on the first image is determined based on the pixel information or the feature information.

15. The method according to claim 14, characterized in that The determining, based on the pixel information, a corresponding position of the second image on the first image comprises: A similarity measurement index is established using the pixel information of the first image and the pixel information of the second image, and a corresponding position of the second image on the first image is determined based on the similarity measurement index.

16. The method according to claim 15, characterized in that The determining the corresponding position of the second image on the first image based on the similarity measurement index includes: A sliding window method is used to determine the value of the similarity measurement index to determine the corresponding position of the second image on the first image.

17. The method according to claim 16, characterized in that The step of using a sliding window method to determine the value of the similarity measurement index to determine the corresponding position of the second image on the first image includes: Preprocessing the first image and the second image, wherein the preprocessing includes resampling the first image and / or the second image so that the first image has the same resolution as the second image; Setting a sliding window and making the sliding window slide along a predetermined path on the first image, wherein a size of the sliding window is the same as a size of the second image; During the sliding process of the sliding window, calculating in real time the value of the similarity measurement index between the second image and the area of ​​the first image located within the sliding window; Selecting the maximum value among the values ​​of the similarity measurement indicator; and Determine that the area of ​​the first image within the sliding window corresponding to the maximum value is the corresponding position of the second image on the first image.

18. An image analysis system, characterized in that: The image analysis system comprises an imaging device, a mobile device, an image analysis device and an alarm device, wherein the imaging device comprises an objective lens, wherein: The imaging device is used to photograph the sample to be tested on the smear in a first field of view to obtain a first image, so as to obtain first image information of the first image; and photograph the sample to be tested on the smear in a second field of view to obtain a second image, so as to obtain second image information of the second image; The moving device is used to drive the smear and / or the objective lens to move; after the imaging device photographs the sample to be tested on the smear in the first field of view to obtain the first image, the moving device drives the smear and / or the objective lens to move so that the imaging device photographs the sample to be tested on the smear in the second field of view to obtain the second image; The image analysis device is used for: Analyzing the first image information and the second image information to determine a corresponding position of the second image on the first image; and Acquire the distance between the corresponding position and the target shooting position on the first image; When the distance exceeds a preset distance threshold, the alarm device sends out an alarm message.

Images