Image processing method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides an image processing method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring a current image from a microscope, the current image being obtained by, in response to a current adjustment operation on the stage of the microscope, performing image capture on a target object on the stage of the microscope; acquiring a first stitched image; acquiring a plurality of first feature points of the current image and a plurality of second feature points of at least one adjacent historical image adjacent to the current image in the first stitched image; on the basis of the plurality of first feature points and the plurality of second feature points of the at least one adjacent historical image, determining stitching coordinates of the current image relative to the first stitched image; and, on the basis of the stitching coordinates, stitching the current image and the first stitched image, and generating and displaying a second stitched image. Therefore, errors in stitching coordinates of the current image relative to the first stitched image are reduced, and the accuracy in stitching the current image and the first stitched image and the quality of the stitched image are improved.

Description

Image processing method, device, electronic device and storage medium Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, electronic device, and storage medium. Background Art

[0002] The microscope is a widely used optical instrument. It magnifies the object being observed through its optical system, displaying microscopic tissue structures and allowing researchers to understand and study the characteristics of an object from its microscopic morphology. Currently, it is widely used in various fields such as biology, pathology, cytology, and medicinal chemistry.

[0003] Due to the limited field of view of a microscope, only local images can be obtained while ensuring a certain resolution. In order to obtain a microscopic image with a larger field of view and facilitate subsequent observation and analysis, it is necessary to process the local images so that multiple local images can be stitched together to obtain an image with a larger field of view. Therefore, how to process the image is very important. Summary of the Invention

[0004] The present disclosure aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] To this end, the present disclosure proposes an image processing method, device, electronic device and storage medium, which obtains a current image obtained by photographing a target object on the stage of the microscope in response to a current adjustment operation of the stage of the microscope from a microscope, and obtains a first stitched image obtained by stitching together historical images photographed in historical adjustment operations before the current adjustment operation, determines the stitching coordinates of the current image relative to the first stitched image based on multiple first feature points of the current image and multiple second feature points of at least one adjacent historical image in the first stitched image that is adjacent to the current image, stitches the current image with the first stitched image based on the stitching coordinates, and generates and displays a second stitched image. Thus, based on the multiple first feature points of the current image and the multiple second feature points of at least one adjacent historical image in the first stitched image, the stitching coordinates of the current image relative to the first stitched image are determined, thereby improving the accuracy of the stitching coordinate determination and reducing the error of the stitching coordinates of the current image relative to the first stitched image. Based on the stitching coordinates, the current image is stitched with the first stitched image, thereby improving the accuracy of the stitching of the current image and the first stitched image and the quality of the stitched image.

[0006] An embodiment of the first aspect of the present disclosure proposes an image processing method, including: acquiring a current image from a microscope, wherein the current image is obtained by photographing a target object on the stage of the microscope in response to a current adjustment operation of the stage of the microscope; acquiring a first stitched image, wherein the first stitched image is obtained by stitching together historical images photographed according to various historical adjustment operations before the current adjustment operation; acquiring multiple first feature points of the current image and multiple second feature points of at least one adjacent historical image adjacent to the current image in the first stitched image; determining the stitching coordinates of the current image relative to the first stitched image based on the multiple first feature points and the multiple second feature points of the at least one adjacent historical image; and stitching the current image with the first stitched image based on the stitching coordinates to generate and display a second stitched image.

[0007] The second aspect embodiment of the present disclosure proposes an image processing device, including: a first acquisition module, used to acquire a current image from a microscope, wherein the current image is obtained by photographing a target object on the stage of the microscope in response to a current adjustment operation of the stage of the microscope; a second acquisition module, used to acquire a first stitched image, wherein the first stitched image is stitched together according to historical images photographed according to each historical adjustment operation before the current adjustment operation; a third acquisition module, used to acquire multiple first feature points of the current image and multiple second feature points of at least one adjacent historical image adjacent to the current image in the first stitched image; a determination module, used to determine the stitching coordinates of the current image relative to the first stitched image based on the multiple first feature points and the multiple second feature points of the at least one adjacent historical image; a stitching module, used to stitch the current image with the first stitched image according to the stitching coordinates, to generate and display a second stitched image.

[0008] The third aspect embodiment of the present disclosure proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the image processing method described in the first aspect embodiment of the present disclosure.

[0009] The fourth embodiment of the present disclosure proposes a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement the image processing method described in the first embodiment of the present disclosure.

[0010] To achieve the above-mentioned objectives, the fifth embodiment of the present disclosure proposes a computer program product, including a computer program, which implements the image processing method described in the first embodiment of the present disclosure when executed by a processor.

[0011] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and / or additional aspects and advantages of the present disclosure will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0013] FIG1 is a schematic flow chart of an image processing method provided by an embodiment of the present disclosure;

[0014] FIG2 is a flow chart of another image processing method provided by an embodiment of the present disclosure;

[0015] FIG3 is a flow chart of another image processing method provided by an embodiment of the present disclosure;

[0016] FIG4 is a flow chart of another image processing method provided by an embodiment of the present disclosure;

[0017] FIG5 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure;

[0018] FIG6 is a schematic structural diagram of an image processing system provided by an embodiment of the present disclosure;

[0019] FIG7 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0021] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0022] In the related art, only one feature matching is performed between the current image captured by the microscope and the most recently captured image in the stitched image to determine the offset of the current image relative to the stitched image, and the current image and the stitched image are stitched based on the offset. However, the offset of the current image relative to the stitched image determined based on only one feature matching between the current image and the most recently captured image in the stitched image is prone to errors, which may result in the inability to accurately stitch all subsequent images, thereby reducing the quality of the stitched image.

[0023] Therefore, to address the above-mentioned problems, the present disclosure proposes an image processing method, apparatus, electronic device, and storage medium.

[0024] The image processing method, apparatus, electronic device, and storage medium according to embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0025] It should be noted that the embodiments of the present disclosure are described by taking an example where the image processing method is configured in an image processing device. The image processing device can be applied to any electronic device so that the electronic device can perform image processing functions.

[0026] Among them, the electronic device can be any device with computing capabilities, such as a personal computer (PC), a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0027] FIG1 is a flow chart of an image processing method provided by an embodiment of the present disclosure.

[0028] As shown in FIG1 , the image processing method may include the following steps:

[0029] Step 101: Acquire a current image from a microscope.

[0030] The current image is obtained by photographing the target object on the stage of the microscope in response to the current adjustment operation of the stage of the microscope.

[0031] In the embodiment of the present disclosure, the user can move the microscope's stage by moving the microscope's stage moving device, or the user can directly move the microscope's stage. During the movement of the stage, the microscope can photograph the target object on the stage to obtain the current image, so that the image processing device can obtain the current image from the microscope.

[0032] As one possible implementation, in response to a current movement operation on a microscope stage moving device, the stage is moved, and during the movement of the stage, a target object on the stage of the microscope is photographed to obtain a current image. It should be noted that the stage moving device may be, for example, a knob or handle on the microscope for moving the stage.

[0033] It should be understood that the user can move the stage moving device to make the target object move within the shooting field of view according to a set moving trajectory, that is, to shoot the target object according to the set shooting path.

[0034] As an example, a shooting path is obtained, wherein the shooting path is used to indicate the movement trajectory of the target object within the shooting field of view; the moving direction and moving angle of the current moving operation are determined based on the moving direction, moving angle and shooting path of the stage moving device in the previous historical moving operation; the stage moving device is moved according to the moving direction and moving angle of the current moving operation to move the stage; during the movement of the stage, the target object on the stage of the microscope is photographed to obtain the current image.

[0035] That is to say, the microscope can shoot the target object according to the shooting path, wherein the shooting path can be in the shape of a U-shape, L-shape, U-shape, C-shape, straight line, I-shape or rectangle. The microscope responds to the current movement operation of the microscope's stage moving device and can determine the moving direction and moving angle of the current movement operation based on the moving direction, moving angle and shooting path of the stage moving device in the previous historical movement operation of the current movement operation, thereby realizing the movement of the stage moving device based on the moving direction and moving angle of the current movement operation, and shooting the target object on the stage of the microscope during the movement of the stage to obtain the current image.

[0036] As another possible implementation, in response to a current movement operation on the microscope stage, the stage is moved, and a target object on the microscope stage is photographed during the movement to obtain a current image. That is, the user can directly move the stage, and the target object on the microscope stage is photographed during the movement to obtain the current image.

[0037] Step 102: Acquire a first stitched image.

[0038] The first spliced ​​image is obtained by splicing historical images captured by various historical adjustment operations before the current adjustment operation.

[0039] It should be understood that whenever the microscope captures an image, the image processing device can stitch the image with the image obtained by stitching previous historical images when acquiring the image. For example, when the microscope captures image 2, the image processing device can stitch image 2 with the previously captured image 1 to obtain stitched image 1. When the microscope captures image 3, the image processing device can stitch image 3 with stitched image 1 to obtain stitched image 2. When the microscope captures image 4, the image processing device can stitch image 4 with stitched image 2 to obtain stitched image 3. And so on. When the microscope captures image N, the image processing device can stitch image N with stitched image N-2 to obtain stitched image N-1.

[0040] Therefore, after the current image is acquired, a first stitched image may be acquired, wherein the first stitched image is stitched together from historical images captured by various historical adjustment operations before the current adjustment operation.

[0041] Step 103 : Acquire a plurality of first feature points of the current image and a plurality of second feature points of at least one adjacent historical image adjacent to the current image in the first stitched image.

[0042] In order to improve the accuracy of the stitching coordinates of the current image relative to the first stitched image, as a possible implementation method, feature extraction can be performed on the current image to obtain multiple first feature points of the current image. At the same time, feature extraction can be performed on at least one adjacent historical image adjacent to the current image in the first stitched image to obtain multiple second feature points of each adjacent historical image in the at least one adjacent historical image.

[0043] Step 104 : determining the stitching coordinates of the current image relative to the first stitched image based on the plurality of first feature points and the plurality of second feature points of at least one adjacent historical image.

[0044] In order to improve the quality of the stitched image and reduce the error of the stitching coordinates, as an example, the stitching coordinates of the current image relative to the first stitched image can be calculated based on multiple first feature points of the current image and multiple second feature points of at least one adjacent historical image.

[0045] As one possible implementation, the offset between the current image and at least one adjacent historical image can be determined based on multiple first feature points of the current image and multiple second feature points of at least one adjacent historical image. Based on the offset between the current image and at least one adjacent historical image, the stitching coordinates of the current image relative to the first stitched image can be determined. The offset refers to the relative difference in pixel coordinates between the current image and each adjacent historical image, and can be expressed as a difference in pixel position.

[0046] Step 105 : stitching the current image and the first stitched image together according to the stitching coordinates to generate and display a second stitched image.

[0047] Then, based on the stitching coordinates, the current image is stitched with the first stitched image to obtain a second stitched image, and the second stitched image is displayed, wherein the coordinates of the upper left corner of the current image in the second stitched image are the stitching coordinates.

[0048] In summary, the present invention provides a method for obtaining a current image from a microscope, wherein the current image is obtained by photographing a target object on a stage of the microscope in response to a current adjustment operation on the stage of the microscope; obtaining a first stitched image, wherein the first stitched image is obtained by stitching together historical images photographed according to various historical adjustment operations before the current adjustment operation; obtaining a plurality of first feature points of the current image and a plurality of second feature points of at least one adjacent historical image adjacent to the current image in the first stitched image; and determining the position of the current image relative to the first stitched image based on the plurality of first feature points and the plurality of second feature points of at least one adjacent historical image. The invention provides a method for generating and displaying stitching coordinates of a stitched image; stitching a current image with the first stitched image according to the stitching coordinates to generate and display a second stitched image, thereby determining the stitching coordinates of the current image relative to the first stitched image based on multiple first feature points of the current image and multiple second feature points of at least one adjacent historical image in the first stitched image, thereby improving the accuracy of determining the stitching coordinates and reducing the error of the stitching coordinates of the current image relative to the first stitched image; stitching the current image with the first stitched image based on the stitching coordinates, thereby improving the accuracy of stitching the current image with the first stitched image and the quality of the stitched image.

[0049] In order to clearly illustrate how the above embodiment determines the stitching coordinates of the current image relative to the first stitched image based on multiple first feature points and multiple second feature points of at least one adjacent historical image, the present disclosure proposes another image processing method.

[0050] FIG2 is a flow chart of another image processing method provided by an embodiment of the present disclosure.

[0051] As shown in FIG2 , the image processing method may include the following steps:

[0052] Step 201: Acquire a current image from a microscope.

[0053] The current image is obtained by photographing the target object on the stage of the microscope in response to the current adjustment operation of the stage of the microscope.

[0054] Step 202: Acquire a first stitched image.

[0055] The first spliced ​​image is obtained by splicing historical images captured by various historical adjustment operations before the current adjustment operation.

[0056] Step 203 : Acquire a plurality of first feature points of the current image and a plurality of second feature points of at least one adjacent historical image adjacent to the current image in the first spliced ​​image.

[0057] Step 204 : Determine whether there is a matching feature point pair between the at least one adjacent historical image and the current image based on the multiple first feature points of the current image and the multiple second feature points of the at least one adjacent historical image.

[0058] The matching feature point pair includes a first feature point and a second feature point that match each other.

[0059] As a possible implementation method, for any adjacent historical image among at least one adjacent historical image, multiple first feature points of the current image and multiple second feature points of any adjacent historical image can be feature matched to determine whether there is a matching feature point pair between any adjacent historical image and the current image. It should be noted that the matching feature point pair may include a first feature point and a second feature point that match each other.

[0060] Step 205: Determine a matching historical image that matches the current image from at least one adjacent historical image.

[0061] There are matching feature point pairs between the matching historical image and the current image.

[0062] As one possible implementation, in response to a matching feature point pair between any one of at least one adjacent historical image and the current image, the adjacent historical image is used as a matching historical image for the current image. It should be noted that the number of matching historical images can be one or more.

[0063] Step 206 : Obtain the offset between the matching historical image and the current image, and determine the stitching coordinates of the current image relative to the first stitched image based on the offset between the matching historical image and the current image.

[0064] As a possible implementation method, when the number of matched historical images is one, the offset between the matched historical image and the current image can be converted into a global coordinate system to obtain the target position coordinates of the current image relative to the matched historical image, wherein the origin of the global coordinate system is the origin of the pixel coordinate system of the historical image with the earliest shooting time in the first stitched image, and the coordinate axes of the global coordinate system coincide with the coordinate axes of the pixel coordinate system of the historical image with the earliest shooting time in the first stitched image.

[0065] For example, image1 is the earliest historical image in the first stitched image, the origin of the pixel coordinate system of image1 is the origin of the global coordinate system, the coordinate axes of the global coordinate system coincide with the coordinate axes of the pixel coordinate system of image1, the coordinates of the upper left corner of the matching historical image in the global coordinate system are (x1, y1), and the offset of the current image relative to the matching historical image is (dx1, dy1). Then, the offset of the current image relative to the matching historical image is converted to the global coordinate system, and the target position coordinates of the current image relative to the matching historical image are obtained as (x1+dx1, y1+dy1).

[0066] As another possible implementation, when there are multiple matching historical images, the stitching coordinates of the current image relative to the first stitched image may be determined based on the offsets between the multiple matching historical images and the current image.

[0067] As an example, the offset between each matching historical image and the current image is converted to a global coordinate system to obtain the target position coordinates of the current image relative to each matching historical image in the matching historical images. The stitching coordinates of the current image relative to the first stitched image are determined based on the average of the target position coordinates of the current image relative to each matching historical image in the matching historical images. For example, the average of the target position coordinates of the current image relative to each matching historical image in the matching historical images is used as the stitching coordinates of the current image relative to the first stitched image.

[0068] As another example, multiple matching historical images are sorted from latest to earliest according to shooting time to obtain a first sorting sequence; from the first sorting sequence, a first set number of matching historical images ranked first are selected; for any matching historical image in the first set number of matching historical images, the offset between the current image and any matching historical image is determined based on the average of the offsets of each matching feature point pair between the current image and any matching historical image; based on the offsets between each matching historical image in the first set number of matching historical images and the current image, the stitching coordinates of the current image relative to the first stitched image are determined.

[0069] That is, in order to reduce the amount of calculation for stitching coordinates while improving the rationality of the stitching coordinates, a first set number (e.g., two) of matching historical images with the latest shooting time may be selected from multiple matching historical images, and the stitching coordinates of the current image relative to the first stitched image may be determined based on the offset between the first set number of matching historical images and the current image. The method for determining the stitching coordinates of the current image relative to the first stitched image based on the offset between the first set number of matching historical images and the current image may be as follows:

[0070] The offset between each matching historical image in a first set number of matching historical images and the current image is converted into a global coordinate system to obtain the target position coordinates of the current image relative to each matching historical image in the first set number of matching historical images; wherein the origin of the global coordinate system is the origin of the pixel coordinate system of the historical image with the earliest shooting time in the first stitched image; and the stitching coordinates of the current image relative to the first stitched image are determined based on the average of the target position coordinates of the current image relative to each matching historical image in the first set number of matching historical images.

[0071] For example, the current image is image8, and the multiple matching historical images are image5, image6 and image7. There are matching feature point pairs between image5, image6 and image7 and image8. The two matching historical images with the latest shooting time are selected from image5, image6 and image7, that is, image6 and image7 are selected from image5, image6 and image7. The offset between image8 and image6 is converted to the global coordinate system to obtain the target position coordinates of image8 relative to image6 in the global coordinate system. The offset between image8 and image7 is converted to the global coordinate system to obtain the target position coordinates of image8 relative to image7 in the global coordinate system. The average of the target position coordinates of image8 relative to image6 and the target position coordinates of image8 relative to image7 in the global coordinate system is calculated, and the average is used as the stitching coordinates of the current image relative to the first stitched image.

[0072] Step 207 : stitching the current image and the first stitched image together according to the stitching coordinates to generate and display a second stitched image.

[0073] It should be noted that the execution process of steps 201 to 203 and step 207 can be implemented in any way in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0074] In summary, based on multiple first feature points of the current image and multiple second feature points of at least one adjacent historical image, it is determined whether there is a matching feature point pair between at least one adjacent historical image and the current image; from at least one adjacent historical image, a matching historical image that matches the current image is determined; the offset between the matching historical image and the current image is obtained, and based on the offset between the matching historical image and the current image, the stitching coordinates of the current image relative to the first stitched image are determined. Thus, based on the offset between the matching historical image in at least one adjacent historical image that has a matching feature point pair with the current image and the current image, the stitching coordinates of the current image relative to the first stitched image are determined, the accuracy of the stitching coordinates is improved, and the stitching error between the current image and the first stitched image is reduced, so that the current image and the first stitched image are stitched based on the stitching coordinates, thereby improving the accuracy of the stitching of the current image and the first stitched image and the quality of the stitched image.

[0075] In order to clearly illustrate how the above embodiment determines the stitching coordinates of the current image relative to the first stitched image based on multiple first feature points and multiple second feature points of at least one adjacent historical image, the present disclosure proposes another image processing method.

[0076] FIG3 is a flow chart of another image processing method provided by an embodiment of the present disclosure.

[0077] As shown in FIG3 , the image processing method may include the following steps:

[0078] Step 301: Acquire a current image from a microscope.

[0079] The current image is obtained by photographing the target object on the stage of the microscope in response to the current adjustment operation of the stage of the microscope.

[0080] Step 302: Acquire a first stitched image.

[0081] The first spliced ​​image is obtained by splicing historical images captured by various historical adjustment operations before the current adjustment operation.

[0082] Step 303 : Acquire a plurality of first feature points of the current image and a plurality of second feature points of at least one adjacent historical image adjacent to the current image in the first stitched image.

[0083] Step 304 : sort at least one adjacent historical image according to shooting time from latest to earliest to obtain a second sorting sequence.

[0084] It should be understood that each adjacent historical image in the first stitched image has a corresponding shooting time. Therefore, by sorting at least one adjacent historical image in the first stitched image from latest to earliest according to shooting time, a second sorting sequence can be obtained.

[0085] Step 305 : Select a first set number of adjacent historical images that are ranked first from the second sorting sequence.

[0086] In order to improve the rationality of determining the stitching coordinates and reduce the amount of calculation for the stitching coordinates, further, a first set number of adjacent historical images ranked at the front are selected from the second sorting sequence, wherein the first set number can be a number pre-set according to actual needs, for example, the first set number is 2.

[0087] Step 306 : Based on the multiple first feature points of the current image and the multiple second feature points of the first set number of adjacent historical images, determine whether there are matching feature point pairs between the first set number of adjacent historical images and the current image.

[0088] As a possible implementation method, for any adjacent historical image among a first set number of adjacent historical images, based on multiple first feature points of the current image and multiple second feature points of any adjacent historical image, it can be determined whether there is a matching feature point pair between any adjacent historical image and the current image.

[0089] Step 307, in response to the presence of matching feature point pairs between a first set number of adjacent historical images and the current image, for any adjacent historical image in the first set number of adjacent historical images, determine the offset between any adjacent historical image and the current image based on the average of the offsets of each matching feature point pair between any adjacent historical image and the current image.

[0090] As a possible implementation method, when there are matching feature point pairs between a first set number of adjacent historical images and the current image, for any adjacent historical image among the first set number of adjacent historical images, the average of the offsets of each matching feature point pair between any adjacent historical image and the current image can be used as the offset between any adjacent historical image and the current image.

[0091] Step 308 : Determine the stitching coordinates of the current image relative to the first stitched image based on the offset between each adjacent historical image in a first set number of adjacent historical images and the current image.

[0092] As a possible implementation method, when there are matching feature point pairs between a first set number of adjacent historical images and the current image, the offset between each adjacent historical image in the first set number of adjacent historical images and the current image is converted to a global coordinate system to obtain the target position coordinates of the current image relative to each adjacent historical image in the first set number of adjacent historical images; and the stitching coordinates of the current image relative to the first stitched image are determined based on the average of the target position coordinates of the current image relative to each adjacent historical image in the first set number of adjacent historical images.

[0093] As another possible implementation manner, in response to the fact that there are no matching feature point pairs between a second set number of adjacent historical images in the first set number of adjacent historical images and the current image, a second set number of adjacent historical images are selected from the second sorting sequence; wherein the second set number of adjacent historical images are located after the first set number of adjacent historical images in the second sorting sequence, and the second set number is not greater than the first set number; and the stitching coordinates of the current image relative to the first stitched image are determined based on the offset between the second set number of adjacent historical images and the current image, and the offset of the remaining adjacent historical images in the first set number of adjacent historical images except the adjacent historical images for which there are no matching feature point pairs with the current image.

[0094] For example, the current image is image10, and the first set number (e.g., 2) of adjacent historical images selected from at least one adjacent historical image (e.g., image6, image7, image8, and image9) are image9 and image8. There is a matching feature point pair between image9 and image10, but there is no matching feature point pair between image8 and image10, that is, the second set number is 1. Then, image7 is selected from at least one adjacent historical image, and when there is a matching feature point pair between image7 and image10, the offset between image10 and image7 is determined. According to the offset between image10 and image7 The displacement and the offset between image10 and image9 are used to determine the stitching coordinates of the current image relative to the first stitching image. For example, the offset between image10 and image7 is converted to the global coordinate system to obtain the target position coordinates of image10 relative to image7 in the global coordinate system. The offset between image10 and image9 is converted to the global coordinate system to obtain the target position coordinates of image10 relative to image9 in the global coordinate system. Then, the average of the target position coordinates of image10 relative to image9 and the target position coordinates of image10 relative to image7 is used to determine the stitching coordinates of the current image relative to the first stitching image.

[0095] In addition, if there is no matching feature point pair between image7 and image10, image6 is selected from at least one historical image. If there is a matching feature point pair between image6 and image10, the stitching coordinates of the current image relative to the first stitched image are determined based on the offset between image10 and image6 and the offset between image10 and image9; if there is no matching feature point pair between image6 and image10, the stitching coordinates of the current image relative to the first stitched image are determined based on the offset between image10 and image9.

[0096] Step 309 : stitching the current image and the first stitched image together according to the stitching coordinates to generate and display a second stitched image.

[0097] It should be noted that the execution process of steps 301 to 303 and step 309 can be implemented in any way in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0098] In summary, at least one historical image is sorted from latest to earliest according to shooting time to obtain a second sorting sequence; a first set number of adjacent historical images that are sorted first are selected from the second sorting sequence; based on multiple first feature points of the current image and multiple second feature points of the first set number of adjacent historical images, it is determined whether there are matching feature point pairs between the first set number of adjacent historical images and the current image; in response to the presence of matching feature point pairs between the first set number of adjacent historical images and the current image, for any adjacent historical image in the first set number of adjacent historical images, the offset between any adjacent historical image and the current image is determined based on the average of the offsets of each matching feature point pair between any adjacent historical image and the current image; based on the offsets between each adjacent historical image in the first set number of adjacent historical images and the current image, the stitching coordinates of the current image relative to the first stitching image are determined; thereby, based on the offsets between the first set number of adjacent historical images in at least one adjacent historical image and the current image, the stitching coordinates of the current image relative to the first stitching image are determined, thereby improving the accuracy and rationality of determining the stitching coordinates of the current image relative to the first stitching image.

[0099] In order to clearly illustrate how to determine at least one adjacent historical image adjacent to the current image in the first stitched image in the above embodiment, the present disclosure proposes another image processing method.

[0100] FIG4 is a flow chart of another image processing method provided by an embodiment of the present disclosure.

[0101] As shown in FIG4 , the image processing method may include the following steps:

[0102] Step 401: Acquire a current image from a microscope.

[0103] The current image is obtained by photographing the target object on the stage of the microscope in response to the current adjustment operation of the stage of the microscope.

[0104] Step 402: Acquire a first stitched image.

[0105] The first spliced ​​image is obtained by splicing historical images captured by various historical adjustment operations before the current adjustment operation.

[0106] Step 403: Acquire multiple first feature points of the current image.

[0107] Step 404 : Acquire, from the historical images in the first stitched image, a previous historical image captured during an adjustment operation preceding the current adjustment operation.

[0108] As one possible implementation, at least one adjacent historical image in the first stitched image to the current image can be determined based on the previous historical image captured in the previous adjustment operation before the current adjustment operation in the first stitched image and the current image. Therefore, the previous historical image captured in the previous adjustment operation before the current adjustment operation can be obtained from the first stitched image. The previous historical image can be the historical image with the latest capture time in the first stitched image.

[0109] Step 405 : Determine at least one adjacent historical image adjacent to the current image from the historical images of the first spliced ​​image based on the multiple first feature points and the multiple second feature points of the previous historical image.

[0110] In order to improve the quality of the stitched image and avoid the microscope stage moving too fast, which may cause the picture content between the current image and the previous image to be discontinuous, as a possible implementation method, it can be determined whether there is a matching feature point pair between the current image and the previous historical image. If so, it is determined that the picture content between the images is continuous. At this time, based on multiple first feature points and multiple second feature points of the previous historical image, at least one adjacent historical image adjacent to the current image can be determined from the historical images of the first stitched image; if not, it is determined that the picture content between the images is discontinuous. At this time, the current image is discarded, and a prompt message is generated and displayed, wherein the prompt message is used to prompt re-image capture.

[0111] As a possible implementation, a method for determining at least one adjacent historical image adjacent to the current image may be, for example:

[0112] 1. In response to the presence of a matching feature point pair between the current image and the previous historical image, determining the offset between the current image and the previous historical image according to the average offset of the matching feature point pairs between the current image and the previous historical image;

[0113] 2. Obtain the coordinate position of each historical image in the first stitched image in the global coordinate system;

[0114] The origin of the global coordinate system is the origin of the pixel coordinate system of the earliest historical image in the first stitched image, and the coordinate axes of the global coordinate system coincide with the coordinate axes of the pixel coordinate system of the earliest historical image.

[0115] 3. According to the offset between the current image and the previous historical image, obtain the coordinate position of the current image in the global coordinate system;

[0116] For example, the coordinate position of the previous historical image in the global coordinate system is (x2, y2), and the offset between the current image and the previous historical image is (dx2, dy2), then the position of the current image in the global coordinate system can be (x2+dx2, y2+dy2).

[0117] 4. Determine at least one adjacent historical image from the historical images in the first stitched image based on the coordinate position of the current image in the global coordinate system and the coordinate positions of the historical images in the first stitched image in the global coordinate system.

[0118] As an example, by comparing the coordinate position of the current image in the global coordinate system with the coordinate positions of each historical image in the first stitched image in the global coordinate system, at least one adjacent historical image in each historical image of the first stitched image that is adjacent to the current image can be determined.

[0119] Step 406 : Acquire multiple second feature points of at least one adjacent historical image adjacent to the current image in the first stitched image.

[0120] Step 407 : Determine the stitching coordinates of the current image relative to the first stitched image based on the multiple first feature points and the multiple second feature points of at least one adjacent historical image.

[0121] Step 408 : stitching the current image and the first stitched image together according to the stitching coordinates to generate and display a second stitched image.

[0122] It should be noted that the execution process of steps 401 to 403 and steps 406 to 408 can be implemented in any way in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0123] In summary, by obtaining the previous historical image captured in the adjustment operation before the current adjustment operation from the historical images in the first stitched image; and determining at least one adjacent historical image adjacent to the current image from the historical images of the first stitched image based on multiple first feature points and multiple second feature points of the previous historical image, at least one adjacent historical image adjacent to the current image can be accurately determined from the first stitched image, and thus based on the multiple first feature points of the current image and the multiple second feature points of at least one adjacent historical image adjacent to the current image in the first stitched image, the stitching coordinates of the current image relative to the first stitched image can be accurately determined, thereby effectively reducing the stitching coordinate error of the current image relative to the first stitched image.

[0124] Corresponding to the image processing method provided in the embodiments of Figures 1 to 4 above, the present disclosure also provides an image processing device. Since the image processing device provided in the embodiments of the present disclosure corresponds to the image processing method provided in the embodiments of Figures 1 to 4 above, the implementation of the image processing method is also applicable to the image processing device provided in the embodiments of the present disclosure, and will not be described in detail in the embodiments of the present disclosure.

[0125] FIG5 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure.

[0126] As shown in FIG. 5 , the image processing apparatus 500 includes a first acquisition module 510 , a second acquisition module 520 , a third acquisition module 530 , a determination module 540 and a stitching module 550 .

[0127] Among them, the first acquisition module is used to acquire a current image from the microscope, wherein the current image is obtained by photographing the target object on the stage of the microscope in response to the current adjustment operation of the stage of the microscope; the second acquisition module is used to acquire a first stitched image, wherein the first stitched image is stitched together from historical images photographed according to various historical adjustment operations before the current adjustment operation; the third acquisition module is used to acquire multiple first feature points of the current image and multiple second feature points of at least one adjacent historical image adjacent to the current image in the first stitched image; the determination module is used to determine the stitching coordinates of the current image relative to the first stitched image based on the multiple first feature points and the multiple second feature points of at least one adjacent historical image; the stitching module is used to stitch the current image with the first stitched image according to the stitching coordinates to generate and display the second stitched image.

[0128] As a possible implementation method of an embodiment of the present disclosure, the determination module 540 is used to determine whether there is a matching feature point pair between at least one adjacent historical image and the current image based on multiple first feature points of the current image and multiple second feature points of at least one adjacent historical image; wherein the matching feature point pair includes a first feature point and a second feature point that match each other; determine a matching historical image that matches the current image from at least one adjacent historical image, wherein there is a matching feature point pair between the matching historical image and the current image; obtain the offset between the matching historical image and the current image, and determine the stitching coordinates of the current image relative to the first stitching image based on the offset between the matching historical image and the current image.

[0129] As a possible implementation of an embodiment of the present disclosure, there are multiple matching historical images, and the determination module 540 is used to sort the multiple matching historical images from latest to earliest according to the shooting time to obtain a first sorting sequence; from the first sorting sequence, select a first set number of matching historical images that are sorted first; for any matching historical image in the first set number of matching historical images, determine the offset between the current image and any matching historical image based on the average of the offsets of each matching feature point pair between the current image and any matching historical image; and determine the stitching coordinates of the current image relative to the first stitched image based on the offsets between each matching historical image in the first set number of matching historical images and the current image.

[0130] As a possible implementation method of an embodiment of the present disclosure, the determination module 540 is used to convert the offset between each matching historical image in a first set number of matching historical images and the current image into a global coordinate system to obtain the target position coordinates of the current image relative to each matching historical image in the first set number of matching historical images; wherein the origin of the global coordinate system is the origin of the pixel coordinate system of the historical image with the earliest shooting time in the first stitched image; and determine the stitching coordinates of the current image relative to the first stitched image based on the average of the target position coordinates of the current image relative to each matching historical image in the first set number of matching historical images.

[0131] As a possible implementation method of an embodiment of the present disclosure, the determination module 540 is used to convert the offset between each matching historical image in the matching historical image and the current image into a global coordinate system to obtain the target position coordinates of the current image relative to each matching historical image in the matching historical image; and determine the stitching coordinates of the current image relative to the first stitched image based on the average of the target position coordinates of the current image relative to each matching historical image in the matching historical image.

[0132] As a possible implementation method of the embodiment of the present disclosure, the determination module 540 is used to sort at least one adjacent historical image from late to early according to the shooting time to obtain a second sorting sequence; select a first set number of adjacent historical images that are sorted first from the second sorting sequence; determine whether there are matching feature point pairs between the first set number of adjacent historical images and the current image based on multiple first feature points of the current image and multiple second feature points of the first set number of adjacent historical images; in response to the existence of matching feature point pairs between the first set number of adjacent historical images and the current image, determine, for any adjacent historical image in the first set number of adjacent historical images, the offset between any adjacent historical image and the current image based on the average of the offsets of each matching feature point pair between any adjacent historical image and the current image; determine the stitching coordinates of the current image relative to the first stitched image based on the offsets between each adjacent historical image in the first set number of adjacent historical images and the current image.

[0133] As a possible implementation method of the embodiment of the present disclosure, the determination module 540 is used to select a second set number of adjacent historical images from the second sorting sequence in response to the existence of a second set number of adjacent historical images in the first set number of adjacent historical images and no matching feature point pairs between the second set number of adjacent historical images and the current image; wherein the second set number of adjacent historical images are located after the first set number of adjacent historical images in the second sorting sequence; in response to the existence of matching feature point pairs between the second set number of adjacent historical images and the current image, obtain the offset between the second set number of adjacent historical images and the current image; determine the stitching coordinates of the current image relative to the first stitched image based on the offset between the second set number of adjacent historical images and the current image, and the offsets of the remaining adjacent historical images in the first set number of adjacent historical images except the adjacent historical images for which no matching feature point pairs are found with the current image.

[0134] As a possible implementation manner of the embodiment of the present disclosure, at least one adjacent historical image is determined using the following modules: a fourth acquisition module and a processing module.

[0135] Among them, the fourth acquisition module is used to obtain the previous historical image captured in the adjustment operation before the current adjustment operation from the historical images in the first stitched image; the processing module is used to determine at least one adjacent historical image adjacent to the current image from the historical images of the first stitched image based on multiple first feature points and multiple second feature points of the previous historical image.

[0136] As a possible implementation method of an embodiment of the present disclosure, a processing module is used to determine whether there is a matching feature point pair between the current image and the previous historical image based on multiple first feature points and multiple second feature points of the previous historical image; in response to the existence of a matching feature point pair between the current image and the previous historical image, determine the offset between the current image and the previous historical image based on the average offset of the matching feature point pairs between the current image and the previous historical image; obtain the coordinate position of each historical image in the first stitched image in the global coordinate system; wherein the origin of the global coordinate system is the origin of the pixel coordinate system of the historical image with the earliest shooting time in the first stitched image; obtain the coordinate position of the current image in the global coordinate system based on the offset between the current image and the previous historical image; determine at least one adjacent historical image adjacent to the current image from each historical image of the first stitched image based on the coordinate position of the current image in the global coordinate system and the coordinate positions of each historical image in the first stitched image in the global coordinate system.

[0137] As a possible implementation method of the embodiment of the present disclosure, the processing module is also used to discard the current image in response to the absence of a matching feature point pair between the current image and the previous historical image; generate and display prompt information, wherein the prompt information is used to prompt re-image shooting.

[0138] As a possible implementation of the embodiment of the present disclosure, the current image is captured by using the following module: a capturing module.

[0139] Among them, the shooting module is used to move the stage in response to the current movement operation of the microscope's stage moving device, and shoot the target object on the microscope's stage during the movement of the stage to obtain the current image; or, move the stage in response to the current movement operation of the microscope's stage, and shoot the target object on the microscope's stage during the movement of the stage to obtain the current image.

[0140] As a possible implementation method of an embodiment of the present disclosure, a shooting module is used to obtain a shooting path, wherein the shooting path is used to indicate the movement trajectory of the target object within the shooting field of view; the moving direction and moving angle of the current moving operation are determined based on the moving direction, moving angle and shooting path of the stage moving device in the previous historical moving operation of the current moving operation; the stage moving device is moved according to the moving direction and moving angle of the current moving operation to move the stage; during the movement of the stage, the target object on the stage of the microscope is photographed to obtain the current image.

[0141] As a possible implementation method of an embodiment of the present disclosure, the shape of the movement trajectory of the target object indicated by the shooting path within the shooting field of view includes any one of the following shapes: a circular shape, an L shape, a U shape, a C shape, a straight shape, an I shape and a rectangle.

[0142] The image processing device of the embodiment of the present disclosure obtains a current image from a microscope, wherein the current image is obtained by photographing a target object on the stage of the microscope in response to a current adjustment operation on the stage of the microscope; obtains a first stitched image, wherein the first stitched image is obtained by stitching together historical images photographed according to various historical adjustment operations before the current adjustment operation; obtains a plurality of first feature points of the current image and a plurality of second feature points of at least one adjacent historical image adjacent to the current image in the first stitched image; and determines the current image according to the plurality of first feature points and the plurality of second feature points of at least one adjacent historical image. The method comprises the steps of: determining the stitching coordinates of the current image relative to the first stitched image; stitching the current image with the first stitched image according to the stitching coordinates to generate and display the second stitched image, thereby determining the stitching coordinates of the current image relative to the first stitched image based on the multiple first feature points of the current image and the multiple second feature points of at least one adjacent historical image in the first stitched image, improving the accuracy of determining the stitching coordinates, reducing the error of the stitching coordinates of the current image relative to the first stitched image, and stitching the current image with the first stitched image based on the stitching coordinates, thereby improving the accuracy of stitching the current image with the first stitched image and the quality of the stitched image.

[0143] It should be noted that the aforementioned explanations of the image processing method embodiment also apply to the image processing apparatus of this embodiment and will not be repeated here. That is, although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent descriptions of the corresponding method, where blocks or apparatuses correspond to method steps or features of method steps. Similarly, aspects described in the context of method steps also represent descriptions of corresponding blocks, items, or features of the corresponding apparatus.

[0144] In order to implement the above embodiments, the present disclosure also proposes an image processing system.

[0145] FIG6 is a schematic diagram of the structure of an image processing system provided by an embodiment of the present disclosure.

[0146] As shown in FIG6 , the image processing system 600 includes a microscope 610 and a computer system 620 .

[0147] Microscope 610 is configured to capture images and is connected to computer system 620. Computer system 620 is configured to perform the methods described herein. Computer system 620 can be configured to execute a machine learning algorithm. Computer system 620 and microscope 610 can be separate entities, or can be integrated into a common housing. Computer system 620 can be part of a central processing system of microscope 610 and / or can be part of a subassembly of microscope 610, such as a sensor, actuator, camera, or lighting unit of microscope 610.

[0148] The computer system 620 can be a local computer device (e.g., a personal computer, laptop, tablet computer, or mobile phone) having one or more processors and one or more storage devices, or it can be a distributed computer system (e.g., having one or more processors and one or more storage devices distributed across various locations, such as a local client and / or one or more remote server locations and / or data centers). The computer system 620 can include any circuit or combination of circuits. In one embodiment, the computer system 620 can include one or more processors that can be of any type. As used herein, a processor can refer to any type of computing circuit, such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA) such as a microscope or microscope component (e.g., a camera), or any other type of processor or processing circuit. Other types of circuits that can be included in the computer system 620 can be custom circuits, application specific integrated circuits (ASICs), and the like, such as one or more circuits (e.g., communication circuits) used in wireless devices such as mobile phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 620 may include one or more storage devices, which may include one or more storage elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard disk drives, and / or one or more drives for handling removable media such as compact disks (CDs), flash memory cards, digital video disks (DVDs), etc. The computer system 620 may also include a display device, one or more speakers, and a keyboard and / or controller, which may include a mouse, trackball, touch screen, voice recognition device, or any other device that allows a system user to input information to or receive information from the computer system 620.

[0149] Some or all of the method steps may be performed by (or using) a hardware device (e.g., a processor, a microprocessor, a programmable computer or an electronic circuit). In some embodiments, such a device may perform one or more of the most important method steps.

[0150] Depending on certain implementation requirements, embodiments of the present disclosure may be implemented in hardware or software. This implementation may be performed using a non-transitory storage medium (such as a digital storage medium, e.g., a floppy disk, DVD, Blu-ray, CD, ROM, PROM, and EPROM, EEPROM, or FLASH) having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system to perform the corresponding method. Thus, the digital storage medium may be computer-readable.

[0151] Some embodiments of the present disclosure comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0152] Generally, the embodiments of the present disclosure can be implemented as a computer program product having a program code, which can be executed to perform one of the methods when the computer program product runs on a computer. The program code can be stored on a machine-readable carrier, for example.

[0153] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0154] In other words, an embodiment of the present disclosure is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0155] Therefore, another embodiment of the present disclosure is a storage medium (or data carrier or computer-readable medium) comprising a computer program stored thereon, the computer program being configured to perform one of the methods described herein when the computer program is executed by a processor. The data carrier, digital storage medium or recording medium is generally tangible and / or non-transitory. Another embodiment of the present disclosure is an apparatus as described herein, comprising a processor and a storage medium.

[0156] Therefore, another embodiment of the present disclosure is a data stream or a signal sequence representing the computer program for performing one of the methods described in the present disclosure. The data stream or the signal sequence can be configured to be transmitted via a data communication connection, such as via the Internet.

[0157] A further embodiment comprises a processing means, for example a computer or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0158] Yet another embodiment comprises a computer on which is installed the computer program for performing one of the methods described in the present disclosure.

[0159] Yet another embodiment of the present disclosure includes an apparatus or system configured to transmit (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, or the like. The apparatus or system may include, for example, a file server for transmitting the computer program to the receiver.

[0160] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all of the functions of the method described herein. In some embodiments, the field programmable gate array can collaborate with a microprocessor to perform one of the methods described herein. Generally, the method is preferably performed by any hardware device.

[0161] In an exemplary embodiment, an electronic device is also provided.

[0162] Among them, electronic equipment includes:

[0163] processor;

[0164] and a memory communicatively connected to the processor;

[0165] The memory stores computer-executable instructions, and the processor executes the computer-executable instructions stored in the memory to implement the image processing method proposed in any of the aforementioned embodiments.

[0166] As an example, FIG7 is a schematic structural diagram of an electronic device 700 according to an exemplary embodiment of the present disclosure. As shown in FIG7 , the electronic device 700 may further include:

[0167] The memory 710 and the processor 720 , and the bus 730 connecting different components (including the memory 710 and the processor 720 ), the memory 710 stores a computer program, and when the processor 720 executes the program, the image processing method described in the embodiment of the present disclosure is implemented.

[0168] Bus 730 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0169] The electronic device 700 typically includes a variety of electronic device-readable media, which can be any available media that can be accessed by the electronic device 700, including volatile and non-volatile media, removable and non-removable media.

[0170] The memory 710 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 740 and / or cache memory 750. The server 700 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 760 may be used to read and write non-removable, non-volatile magnetic media (not shown in FIG. 7 , commonly referred to as a “hard drive”). Although not shown in FIG. 7 , a disk drive for reading and writing to a removable non-volatile disk (e.g., a “floppy disk”) and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 730 via one or more data media interfaces. The memory 710 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0171] A program / utility 780 having a set (at least one) of program modules 770 may be stored, for example, in memory 710. Such program modules 770 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 770 generally implement the functions and / or methods of the embodiments described herein.

[0172] The electronic device 700 can also communicate with one or more external devices 790 (e.g., a keyboard, a pointing device, a display 791, etc.), one or more devices that enable a user to interact with the electronic device 700, and / or any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication can occur via an input / output (I / O) interface 792. Furthermore, the electronic device 700 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 793. As shown, the network adapter 793 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 700, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0173] The processor 720 executes various functional applications and data processing by running programs stored in the memory 710 .

[0174] It should be noted that the implementation process and technical principles of the electronic device of this embodiment can be found in the aforementioned explanation of the image processing method of the embodiment of the present disclosure, and will not be repeated here.

[0175] In order to implement the above embodiments, the present disclosure further proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the image processing method provided by the above embodiments.

[0176] In order to implement the above embodiments, the present disclosure further provides a computer program product, including a computer program, which implements the image processing method provided by the above embodiments when executed by a processor.

[0177] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this disclosure are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0178] It is important to note that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold beyond these legitimate uses. Furthermore, such collection / sharing should be conducted only after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes the relevant user information before using the feature. Furthermore, any necessary steps must be taken to safeguard and secure access to such personal information and ensure that others with access to personal information comply with its privacy policy and procedures.

[0179] This disclosure contemplates providing implementations that allow users to selectively block the use or access of personal information data. Specifically, this disclosure contemplates providing hardware and / or software to prevent or block access to such personal information data. Risks can be minimized by limiting data collection and deleting data once it is no longer needed. Furthermore, where applicable, such personal information can be de-identified to protect user privacy.

[0180] In the descriptions of the aforementioned embodiments, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are mutually inconsistent.

[0181] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0182] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0183] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0184] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0185] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0186] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0187] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. A person of ordinary skill in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. An image processing method, characterized in that, Including: Obtaining a current image from a microscope, where the current image is obtained by photographing a target object on a stage of the microscope in response to a current adjustment operation on the stage of the microscope; Obtaining a first stitched image, where the first stitched image is stitched from historical images obtained by photographing according to respective historical adjustment operations before the current adjustment operation; Obtaining a plurality of first feature points of the current image and a plurality of second feature points of at least one adjacent historical image adjacent to the current image in the first stitched image; Determining a stitching coordinate of the current image relative to the first stitched image according to the plurality of first feature points and the plurality of second feature points of the at least one adjacent historical image; Stitching the current image and the first stitched image according to the stitching coordinate, and generating and displaying a second stitched image.

2. The method according to claim 1, characterized in that The determining the stitching coordinate of the current image relative to the first stitched image according to the plurality of first feature points and the plurality of second feature points of the at least one adjacent historical image includes: Judging whether there is a pair of matching feature points between the at least one adjacent historical image and the current image according to the plurality of first feature points of the current image and the plurality of second feature points of the at least one adjacent historical image; wherein, the pair of matching feature points includes a mutually matching first feature point and a second feature point; Determining a matching historical image that matches the current image from the at least one adjacent historical image, where there is a pair of matching feature points between the matching historical image and the current image; Obtaining an offset between the matching historical image and the current image, and determining the stitching coordinate of the current image relative to the first stitched image according to the offset between the matching historical image and the current image.

3. The method according to claim 2, wherein The number of the matching historical images is multiple, Obtaining the offset between the matching historical image and the current image, and determining the stitching coordinate of the current image relative to the first stitched image according to the offset between the matching historical image and the current image includes: Sorting the multiple matching historical images in descending order of shooting time to obtain a first sorting sequence; Selecting a first set number of matching historical images that are ranked ahead from the first sorting sequence; For any one of the first set number of matching historical images, determining the offset between the current image and the any one of the matching historical images according to an average value of offsets of respective pairs of matching feature points between the current image and the any one of the matching historical images; Determining the stitching coordinate of the current image relative to the first stitched image according to the offsets between the current image and each of the first set number of matching historical images.

4. The method according to claim 3, wherein The determining the stitching coordinate of the current image relative to the first stitched image according to the offsets between the current image and each of the first set number of matching historical images includes: Convert the offsets between each of the first set number of matching historical images and the current image to the global coordinate system to obtain the target position coordinates of the current image relative to each of the first set number of matching historical images; wherein, the origin of the global coordinate system is the origin of the pixel coordinate system of the historical image with the earliest shooting time in the first stitched image. Determine the stitching coordinates of the current image relative to the first stitched image according to the mean value of the target position coordinates of the current image relative to each of the first set number of matching historical images.

5. The method according to claim 4, wherein Determining the stitching coordinates of the current image relative to the first stitched image according to the offset between the matching historical image and the current image includes: Convert the offsets between each of the matching historical images and the current image to the global coordinate system to obtain the target position coordinates of the current image relative to each of the matching historical images. Determine the stitching coordinates of the current image relative to the first stitched image according to the mean value of the target position coordinates of the current image relative to each of the matching historical images.

6. The method according to claim 1, wherein Determining the stitching coordinates of the current image relative to the first stitched image according to the multiple first feature points and the multiple second feature points of the at least one adjacent historical image includes: Sort the at least one adjacent historical image in descending order of shooting time to obtain a second sorting sequence. Select the first set number of adjacent historical images with the earliest shooting time from the second sorting sequence. Judge whether there are matching feature point pairs between the first set number of adjacent historical images and the current image according to the multiple first feature points of the current image and the multiple second feature points of the first set number of adjacent historical images. In response to the existence of matching feature point pairs between the first set number of adjacent historical images and the current image, for any one of the first set number of adjacent historical images, determine the offset between the any one of the adjacent historical images and the current image according to the mean value of the offsets of each matching feature point pair between the any one of the adjacent historical images and the current image. Determine the stitching coordinates of the current image relative to the first stitched image according to the offsets between each of the first set number of adjacent historical images and the current image.

7. The method according to claim 6, wherein Determining the stitching coordinates of the current image relative to the first stitched image according to the multiple first feature points and the multiple second feature points of the at least one adjacent historical image further includes: In response to the existence of a second set number of adjacent historical images among the first set number of adjacent historical images that have no matching feature point pairs with the current image, select the second set number of adjacent historical images from the second sorting sequence; wherein, the second set number of adjacent historical images are located after the first set number of adjacent historical images in the second sorting sequence. In response to the existence of matching feature point pairs between the second set number of adjacent historical images and the current image, obtain the offset between the second set number of adjacent historical images and the current image; According to the offset between the second set number of adjacent historical images and the current image, and the offset of the remaining adjacent historical images among the first set number of adjacent historical images except for the adjacent historical images that do not have matching feature point pairs with the current image, determine the stitching coordinates of the current image relative to the first stitched image.

8. The method according to claim 1, characterized in that, The at least one adjacent historical image is determined by the following steps: From the historical images in the first stitched image, obtain the previous historical image captured by the previous adjustment operation of the current adjustment operation; According to the multiple first feature points and the multiple second feature points of the previous historical image, determine the at least one adjacent historical image adjacent to the current image from the historical images of the first stitched image.

9. The method according to claim 8, wherein Determining the at least one adjacent historical image adjacent to the current image from the historical images of the first stitched image according to the multiple first feature points and the multiple second feature points of the previous historical image includes: According to the multiple first feature points and the multiple second feature points of the previous historical image, determine whether there are matching feature point pairs between the current image and the previous historical image; In response to the existence of matching feature point pairs between the current image and the previous historical image, determine the offset between the current image and the previous historical image according to the average offset of the matching feature point pairs between the current image and the previous historical image; Obtain the coordinate positions of each historical image in the first stitched image in the global coordinate system; wherein, the origin of the global coordinate system is the origin of the pixel coordinate system of the historical image with the earliest shooting time in the first stitched image; According to the offset between the current image and the previous historical image, obtain the coordinate position of the current image in the global coordinate system; According to the coordinate position of the current image in the global coordinate system and the coordinate positions of each historical image in the first stitched image in the global coordinate system, determine the at least one adjacent historical image adjacent to the current image from each historical image of the first stitched image.

10. The method according to claim 9, characterized in that, The method further includes: In response to the non-existence of matching feature point pairs between the current image and the previous historical image, discard the current image; Generate and display a prompt message, where the prompt message is used to prompt to re-take an image.

11. The method according to claim 1, wherein The current image is captured by the following steps, including: In response to the current movement operation of the stage moving device of the microscope, move the stage, and capture the target object on the stage of the microscope during the movement of the stage to obtain the current image; Or, In response to the current movement operation of the stage of the microscope, move the stage, and capture the target object on the stage of the microscope during the movement of the stage to obtain the current image.

12. The method according to claim 11, wherein In response to a current movement operation of the stage moving device of the microscope, move the stage, and capture a target object on the stage of the microscope during the movement of the stage to obtain the current image, including: Obtain a shooting path, where the shooting path is used to indicate the movement trajectory of the target object within the shooting field of view; Determine the movement direction and movement angle of the current movement operation according to the movement direction, movement angle of the stage moving device and the shooting path of the previous historical movement operation of the current movement operation; Move the stage moving device according to the movement direction and movement angle of the current movement operation to move the stage; During the movement of the stage, capture the target object on the stage of the microscope to obtain the current image.

13. The method according to claim 12, wherein The shape of the movement trajectory of the target object indicated by the shooting path within the shooting field of view includes any one of the following shapes: A square with a hole in the middle; An L shape; A U shape; A C shape; A straight line shape; An I shape; A rectangle.

14. An image processing apparatus, characterized in that, Including: A first acquisition module, configured to acquire a current image from a microscope, where the current image is obtained by capturing a target object on the stage of the microscope in response to a current adjustment operation of the stage of the microscope; A second acquisition module, configured to acquire a first spliced image, where the first spliced image is obtained by splicing historical images captured according to each historical adjustment operation before the current adjustment operation; A third acquisition module, configured to acquire a plurality of first feature points of the current image and a plurality of second feature points of at least one adjacent historical image adjacent to the current image in the first spliced image; A determination module, configured to determine the splicing coordinates of the current image relative to the first spliced image according to the plurality of first feature points and the plurality of second feature points of the at least one adjacent historical image; A splicing module, configured to splice the current image and the first spliced image according to the splicing coordinates to generate and display a second spliced image.

15. An electronic device, characterized in that, Including: A processor and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1-13.

16. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1-13.

Images