Image processing method and device and microscopic camera

Abstract

The invention provides an image processing method and device and a microscopic camera. The target area is determined from the reference image of the target object collected by the image collection device based on the area selection instruction, the translation stage is controlled to move so that the target area can be aligned with the visual field of the image collection device, related personnel do not need to manually adjust the position of the target area, the positioning efficiency of the target area is improved, and the user experience is improved. And errors caused by manual operation are eliminated. Furthermore, by controlling the translation stage to move along the optical axis direction of the image acquisition device, the target position of the translation stage in the optical axis direction of the image acquisition device when the target area is in the focusing state is determined, and the translation stage is controlled to perform automatic scanning based on the target position so as to generate a panoramic depth image of the target area. The panoramic depth image can be automatically obtained only through one-time operation, the operation complexity is simplified, and the generation efficiency of the panoramic depth image is improved.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to image processing methods, apparatus and microscope cameras. Background Technology

[0002] Currently, automated microscope systems are widely used. Taking a microscope camera as an example, a microscope camera typically consists of a translation stage, an image acquisition device, and a controller. However, the degree of automation of microscope cameras in related technologies is limited, and the entire workflow still requires manual operation.

[0003] For example, in the process of locating a target object, the relevant technical solutions require operators to manually adjust the translation stage to position the target object and move it to the center of the field of view. During the focusing process and the generation of depth-of-field images, operators also need to manually control the image acquisition equipment to acquire a series of images at different heights, and then import them into specialized software for fusion. This process is cumbersome and time-consuming, severely reducing the efficiency of generating full depth-of-field images. Summary of the Invention

[0004] In view of this, this application provides an image processing method, apparatus and microscope camera to automatically obtain full depth images and improve the generation efficiency of full depth images.

[0005] The technical solution provided in this application is as follows: According to an embodiment of the first aspect of this application, an image processing method is provided. The method is applied to a control device, which is communicatively connected to a translation stage and an image acquisition device. The translation stage is used to place a target object for image acquisition. The method includes: Based on the region selection command, the target region is determined from the reference image of the target object acquired by the image acquisition device, and the translation stage is controlled to move so that the target region is aligned with the field of view of the image acquisition device. The translation stage is controlled to move along the optical axis of the image acquisition device to determine the target position of the translation stage in the optical axis of the image acquisition device when the target area is in focus. Based on the target position, the translation stage is controlled to move along the optical axis of the image acquisition device, and a panoramic depth image of the target area is generated based on the images acquired by the image acquisition device during the movement.

[0006] According to an embodiment of a second aspect of this application, an image processing apparatus is provided. This apparatus is applied to a control device, which is communicatively connected to a translation stage and an image acquisition device. The translation stage is used to place a target object for image acquisition. The apparatus includes: An alignment unit is used to determine a target region from a reference image of a target object acquired by the image acquisition device based on a region selection instruction, and to control the translation stage to move so that the target region is aligned with the field of view of the image acquisition device. A focusing unit is used to control the translation stage to move along the optical axis of the image acquisition device, and to determine the target position of the translation stage in the optical axis of the image acquisition device when the target area is in a focused state. The generation unit is used to control the translation stage to move along the optical axis of the image acquisition device based on the target position, and to generate a panoramic depth image of the target area based on the images acquired by the image acquisition device during the movement.

[0007] According to an embodiment of the third aspect of this application, a microscope camera is provided, including: a controller, a translation stage, an image acquisition device, and a display; The controller, translation stage, image acquisition device, and display are communicatively connected. The translation stage is used to place the target object for which image acquisition is required. The image acquisition device is used to acquire images of the target object, and the display is used to display the images acquired by the image acquisition device. The controller is used to perform the method as described in the first aspect.

[0008] As can be seen from the above technical solutions, the proposed solution determines the target region from the reference image of the target object acquired by the image acquisition device based on the region selection command, and controls the movement of the translation stage to align the target region with the field of view of the image acquisition device. This eliminates the need for manual centering of the target region, significantly improving the positioning efficiency and eliminating errors caused by manual operation. Furthermore, by controlling the translation stage to move along the optical axis of the image acquisition device, the target position of the translation stage in the optical axis direction of the image acquisition device under focusing state is determined, and the translation stage is controlled to automatically scan based on the target position to generate a panoramic depth image of the target region. This changes the manual acquisition and offline processing working mode of related technologies, allowing for automatic acquisition of a panoramic depth image with only one operation, greatly simplifying the operation complexity and improving the generation efficiency of panoramic depth images. Attached Figure Description

[0009] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the principles of this application.

[0010] Figure 1 A schematic diagram of a microscope camera provided in an embodiment of this application; Figure 2 This is a flowchart of an image processing method provided in an embodiment of this application; Figure 3 A block diagram illustrating the operating principle of a microscope camera provided in this application embodiment; Figure 4 This is a structural diagram of the image processing apparatus provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0012] Currently, in scenarios requiring high-precision and high-efficiency microscopic observation and analysis, such as biomedical research and industrial precision testing, automated microscope systems are often used to acquire full-depth images of the area to be observed (denoted as the target area).

[0013] Specifically, the automated microscope system may include an image acquisition device (such as a microscope), a two-dimensional or three-dimensional precision electric translation stage driven by a stepper motor or servo motor, and a controller to control the movement of the translation stage. The object being photographed (such as a glass slide, wafer, etc.) is placed on the translation stage.

[0014] It should be noted that the automation level of the aforementioned automated microscope system is limited, and the entire workflow still requires manual operation.

[0015] For example, after determining the target area, it is usually necessary to move the target area to the center of the field of view for subsequent image acquisition. This requires locating the target area and moving the translation stage so that the center of the target area is located in the center of the field of view. In related technologies, this process still requires manual positioning by the operator. That is, the operator first manually operates the controller of the translation stage through the eyepiece of the image acquisition device or the real-time image observed on the monitor, and moves the object to be photographed roughly to the center of the field of view by jogging or continuous movement.

[0016] After moving the target area to the center of the field of view, in order to achieve a clear image of the subject, it is necessary to adjust the distance between the image acquisition device and the subject for focusing. During focusing, because the microscope has a very small depth of field at high magnification, the operator needs to manually rotate the focusing wheel of the microscope and repeatedly adjust the distance between the image acquisition device and the subject (referred to as the distance in the Z-axis direction) until the sharpest focal plane is found (i.e., the focal plane corresponding to the distance in the Z-axis direction where the image is sharpest).

[0017] Furthermore, during the depth-of-field fusion process after focusing, if it is necessary to perform full depth-of-field imaging on objects with a certain thickness or uneven surface, the operator must manually adjust the Z-axis height in small steps, take an image at each focal plane, and then import this series of images into professional image processing software (such as ImageJ, Helicon Focus, etc.) for offline fusion through algorithms to finally generate a fully clear image.

[0018] It is evident that automated microscope systems in related technologies still require significant manual processing when generating panoramic depth-of-field images. The process of quickly locating and precisely centering the target of interest (i.e., the aforementioned target region) within the current field of view primarily relies on the operator's visual judgment and manual operation. The centering alignment result is heavily influenced by the operator's subjective experience, introducing errors due to manual operation.

[0019] In addition, the solutions in related technologies usually use manual focusing, which is inefficient. After focusing, the operator needs to manually set the scanning range and step size of the Z-axis based on experience. The process is cumbersome and it is easy to miss the key focal plane.

[0020] Based on this, this application proposes a microscope camera that enables one-stop operation of automatic target centering, automatic focusing, and automatic depth-of-field fusion.

[0021] Please refer to Figure 1 , Figure 1 A schematic diagram of a microscope camera provided in an embodiment of this application.

[0022] like Figure 1 As shown, the microscope camera includes: a control unit, a translation stage, an image acquisition device, and a display.

[0023] In this embodiment, the controller, translation stage, image acquisition device, and display are communicatively connected. The translation stage is used to place the target object for which image acquisition is required; the image acquisition device is used to acquire images of the target object; and the display is used to display the images acquired by the image acquisition device.

[0024] In the proposed solutions of related technologies, the translation stage and the image acquisition device are often two independent hardware units, lacking communication and linkage based on image content. The translation stage can only passively execute preset commands, and the image acquisition device can only passively acquire images; the two do not form a unified whole capable of autonomous perception, decision-making, and execution.

[0025] In this embodiment, the translation stage and the image acquisition device are connected by a controller, which can generate control commands for the translation stage by obtaining the image acquired by the image acquisition device.

[0026] Specifically, image acquisition equipment can consist of a microscope objective and an industrial camera. The microscope objective is responsible for optically magnifying the target object mounted on a translation stage, while the industrial camera is responsible for capturing the magnified image and converting it into a digital signal. The core function of the image acquisition equipment is to acquire a high-resolution real-time image stream of the target object, providing a data source for subsequent image analysis.

[0027] The controller can be an industrial computer or an embedded processor. It integrates vision algorithm software and motion control programs to receive and analyze image streams from the image acquisition device, generating precise motion control commands for the translation stage.

[0028] A translation stage includes a precision translation stage (typically driven by a stepper motor or servo motor) capable of moving along three mutually perpendicular directions (X, Y, and Z), and corresponding motor drivers. The translation stage can receive and execute control commands from a controller to move the target object it carries with micrometer-level or even nanometer-level precision displacement. Movement along the X and Y axes is used for alignment of the target area, while movement along the Z axis is used for focusing.

[0029] The specific image processing methods will be described in detail below, and will not be repeated here.

[0030] Please refer to Figure 2 , Figure 2 This is a flowchart of an image processing method provided in an embodiment of this application.

[0031] In this embodiment, the method can be applied to the control device in the aforementioned microscope camera. The control device is communicatively connected to the translation stage and the image acquisition device. The translation stage is used to place the target object for which image acquisition is required.

[0032] like Figure 2 As shown, the method may include the following steps: Step 201: Based on the region selection command, determine the target region from the reference image of the target object acquired by the image acquisition device, and control the translation stage to move so that the target region is aligned with the field of view of the image acquisition device.

[0033] In this embodiment, the reference image of the target object acquired by the image acquisition device can be displayed on a monitor connected to the image acquisition device. The region selection command here can be triggered by the user performing interactive operations on the reference image displayed on the monitor, such as dragging and drawing a rectangle or polygon on the displayed image using a mouse, stylus, or touch screen, or directly clicking on a specific point.

[0034] The target region is the area selected by the user in the reference image of the target object. The target region may include the entire reference image or a part of the reference image. This application does not impose any restrictions on this.

[0035] Once the target area is identified, it indicates that this target area is the region of interest for generating a full-view depth image. At this point, the panning stage can be controlled to move so that the target area is aligned with the field of view of the image acquisition device, and the target area is moved to a specified position in the field of view for observation and acquisition.

[0036] In this embodiment, the specific method for controlling the movement of the translation stage to align the target area with the field of view of the image acquisition device may include: Determine the pixel coordinate deviation between the first pixel coordinate of the first reference point of the target area in the reference image and the second pixel coordinate of the second reference point in the field of view of the image acquisition device; Based on preset pixel size calibration parameters, pixel coordinate deviation is converted into physical displacement deviation; The translation stage is controlled by physical displacement deviation to move the first reference point of the target area to the second reference point in the field of view of the image acquisition device, so that the first reference point of the target area is aligned with the second reference point in the field of view of the image acquisition device.

[0037] In this embodiment, the first reference point of the target area can be any preset point in the target area, such as the center point, corner point, or edge midpoint of the target area; the second reference point in the field of view of the image acquisition device can also be any preset point in the field of view, such as the center point, corner point, or auxiliary marker point (virtual or actual auxiliary alignment markers in the field of view, such as the intersection of crosshairs or the scale point of a ruler) of the field of view of the image acquisition device. This application does not impose any restrictions on this.

[0038] When aligning the first reference point of the target area with the second reference point in the field of view of the image acquisition device, any combination of the first and second reference points can be aligned, such as aligning the center point of the target area with the center point of the field of view, aligning the corner point of the target area with the center point of the field of view, aligning the corner point of the target area with the auxiliary marker point of the field of view, etc. This application does not impose any restrictions on this.

[0039] It should be noted that during the movement of the translation stage, the target object on the stage moves with it, and the target area also moves along with the target object. During this movement, the pixel distance between the first reference point of the target area and the reference point of the target object (denoted as the third reference point) remains constant in both the X and Y axes, and there is no relative motion such as rotation between the first reference point of the target area and the third reference point of the target object. In other words, the relative positions of the first reference point of the target area and the third reference point of the target object remain unchanged, and the translation stage moves the entire target area to the second reference point within the field of view of the image acquisition device.

[0040] The third reference point of the target object can also be any preset point on the target object, such as the center point of the target object, a specific physical feature point on the target object (such as the corner point of the target object), the center of gravity of the target object, etc. This application does not impose any restrictions on this.

[0041] In this embodiment, taking the first reference point as the center point of the target area (denoted as the center of the target area) and the second reference point as the center point of the field of view (denoted as the center of the field of view) as an example, before controlling the translation stage to move, the first pixel coordinates of the center of the target area in the reference image and the second pixel coordinates of the center of the field of view of the image acquisition device can be obtained, and the pixel coordinate deviation between the two can be determined. This pixel coordinate deviation is the amount of pixel displacement required for the center of the target area to move to the center of the field of view.

[0042] Furthermore, pixel coordinate deviation can be converted into physical displacement deviation based on preset pixel size calibration parameters. Here, pixel size calibration parameters are pre-calibrated parameters used to characterize the conversion relationship between pixel displacement and actual physical displacement. Through these parameters, the determined pixel displacement to be moved can be converted into the actual physical displacement that the translation stage needs to move, denoted as physical displacement deviation.

[0043] The translation stage can then be controlled to move according to the physical displacement deviation, so as to move the center of the target area to the center of the field of view of the image acquisition device.

[0044] The above description is based on the example of the first reference point being the center point of the target area and the second reference point being the center point of the field of view. For other combinations of the first and second reference points, the specific alignment method is similar to that described above, and will not be repeated here.

[0045] In this embodiment, the controller can send control commands to the translation stage via a control bus (such as RS232 / 485 serial port or Ethernet) to control the movement of the translation stage.

[0046] It should be noted that here, the translation stage can be controlled to move directly according to the determined physical displacement deviation in one go. To achieve more precise control of the translation stage, moving the first reference point of the target area to the second reference point in the field of view of the image acquisition device based on the physical displacement deviation can be handled as follows: The translation stage is given a movement command based on the physical displacement deviation; During the movement of the translation stage, the current pixel coordinates of the first reference point of the target area are determined based on the current physical coordinates of the first reference point of the target area fed back by the translation stage, and the current coordinate error is determined based on the current pixel coordinates and the second pixel coordinates. Based on preset pixel size calibration parameters, the current coordinate deviation is converted into the current physical displacement error; the movement command is adjusted based on the current physical displacement error, and the current pixel coordinates of the first reference point of the target area are determined according to the current physical coordinates of the first reference point of the target area fed back by the translation stage, until the current physical displacement error is less than the preset error threshold.

[0047] In this embodiment, the first reference point is still taken as the center point of the target area (denoted as the center of the target area), and the second reference point is taken as the center point of the field of view (denoted as the center of the field of view). During the process of controlling the movement of the translation stage based on the physical displacement deviation, the current physical coordinates of the center of the target area fed back by the translation stage can also be obtained, and converted into the current pixel coordinates according to the above pixel size calibration parameters.

[0048] Furthermore, the error between the current pixel coordinates and the aforementioned second pixel coordinates (i.e., the pixel coordinates corresponding to the center of the field of view) is determined and denoted as the current coordinate error. Here, the current coordinate error is the amount of pixel displacement that moves from the center of the current target area to the center of the field of view during the movement.

[0049] Furthermore, the pixel coordinate error obtained at this time can be converted into a physical displacement error through preset pixel size calibration parameters. The controller can then generate new and more precise motion control commands based on this physical displacement error, and repeat this process.

[0050] If the physical displacement error is less than the preset error threshold, it indicates that the center of the target area has successfully entered the allowable accuracy range. At this point, the alignment operation can be considered complete, and the controller can stop sending movement commands.

[0051] The above description uses the first reference point as the center point of the target area and the second reference point as the center point of the field of view as an example. For other combinations of the first and second reference points, the specific alignment method is similar to that described above, and will not be repeated here. Unless otherwise specified, the following description will continue to use the first reference point as the center point of the target area and the second reference point as the center point of the field of view as an example for subsequent schemes.

[0052] In this embodiment, the user only needs to define the range of the target area, and the target area can be automatically aligned with the field of view of the image acquisition device through the above step 201. There is no need for relevant personnel to manually adjust the position of the target area, which reduces the introduction of human error.

[0053] This concludes the description of step 201. We will now proceed to step 202.

[0054] Step 202: Control the translation stage to move along the optical axis of the image acquisition device to determine the target position of the translation stage in the optical axis of the image acquisition device when the target area is in focus.

[0055] After completing step 201, which involves aligning the target area with the field of view (i.e., moving the translation stage in the X and Y directions), the next step is to adjust the distance between the target area and the image acquisition device to achieve focus and ensure that the target area has a high image clarity in the image acquisition device.

[0056] Specifically, the translation stage can be controlled to move along the optical axis of the image acquisition device. During the movement, when the target area is in focus, the translation stage is positioned at the target location along the optical axis of the image acquisition device.

[0057] As one embodiment, the translation stage is controlled to move along the optical axis of the image acquisition device, and the target focal plane is determined based on the sharpness of the target area in the image acquired by the image acquisition device during the movement of the translation stage; The position of the target is defined by the optical axis corresponding to the target focal plane. The sharpness of the target area on the target focal plane is greater than that on other focal planes.

[0058] In this embodiment, the optical axis direction of the image acquisition device can be denoted as the Z-axis direction. The translation stage can be controlled to move in the Z-axis direction to determine the target focal plane based on the sharpness of the target area.

[0059] Specifically, the sharpness of the target region in each image acquired by the image acquisition device during the movement of the translation stage can be determined based on the sharpness evaluation function.

[0060] In this embodiment, the image acquisition device can be controlled to perform an image acquisition once every specified distance the translation stage moves, according to a specified precision, and the sharpness of the target region in the acquired image can be determined. The sharpness evaluation function here can be the Tenengrad gradient function, the Laplacian energy function, etc., and this application does not impose any limitations on it.

[0061] After determining the sharpness of the target region in each image, a curve showing the change in the sharpness of the target region as a function of its position along the optical axis can be generated based on the sharpness of the target region in each image.

[0062] In this embodiment, the horizontal axis of the change curve can be the position of the translation stage in the optical axis direction of the image acquisition device (i.e., the height in the Z-axis direction), and the vertical axis of the change curve can be the clarity of the target area in the image acquired by the image acquisition device at the corresponding position.

[0063] Furthermore, the focal plane corresponding to the peak point of the sharpness of the target area in the optical axis direction of the variation curve can be determined as the target focal plane.

[0064] After determining the target focal plane, the translation stage can be moved according to the target position (height in the Z-axis direction) corresponding to the target focal plane, so that the target area can be accurately moved to the position of the target focal plane.

[0065] In this embodiment, step 202 enables automatic focusing of the target area without requiring manual adjustment by relevant personnel, thus improving focusing efficiency and accuracy.

[0066] This concludes the description of step 202. We will now proceed to step 203.

[0067] Step 203: Based on the target position, control the translation stage to move along the optical axis of the image acquisition device, and generate a panoramic depth image of the target area based on the images acquired by the image acquisition device during the movement.

[0068] In this embodiment, after determining the target position in the focused state through step 202, considering that a single image cannot clearly record all the clear details of the entire three-dimensional object, in order to obtain a clear image of the target object in the target area, it is necessary to acquire multiple images near the target position, where each image ensures that the target object is clear at a certain height position.

[0069] Specifically, a method for generating a panoramic depth image of the target area by controlling the translation stage to move along the optical axis of the image acquisition device based on the target position, and by generating the panoramic depth image of the target area based on the images acquired by the image acquisition device during the movement, may include: The control translation stage moves along the optical axis of the image acquisition device within a preset scanning range, using the target position as a reference point. Based on the images acquired by the image acquisition device during the movement, a full-view depth image of the target area is generated. The preset scanning range is the scanning range centered on the target position.

[0070] In this embodiment, the target position can be used as a reference point, and the translation stage can be controlled to move along the optical axis of the image acquisition device within a preset scanning range above and below that reference point. Here, the preset scanning range is the scanning range centered on the target position.

[0071] The preset scanning range can include the scanning interval and the scanning step size, i.e., the start and end positions of the scan, and the length of the interval between each scan. Here, "scanning" refers to the process of controlling the translation stage to move to that position and controlling the image acquisition device to acquire an image at that position.

[0072] In this embodiment, the scanning interval and scanning step size can be determined based on the height of the target object, specifically based on the height of the target object within the target area. It is easy to understand that in order to obtain a full depth-of-field image of the target area, a clear image needs to be obtained at every height of the target object within the target area. Therefore, the higher the height of the target object within the target area, the larger the scanning interval needs to be set. The scanning step size can be set according to the accuracy requirements; if the accuracy requirements are high, the scanning step size can be set smaller to obtain images at more heights.

[0073] In this embodiment, the specific method for generating a panoramic depth image of the target area based on images acquired by the image acquisition device during movement may include: For each image captured by the image acquisition device during movement, the clear pixel region included in the target area of ​​the image is extracted, and the location information of the clear pixel region is recorded; the clear pixel region refers to the region with a sharpness greater than a preset sharpness threshold; Based on the location information of the sharp pixel regions included in the target region in each image, the sharp pixel regions included in the target region are fused to obtain a full depth image of the target region.

[0074] In this embodiment, for the image acquired by the image acquisition device at the height corresponding to each scanning step in the above-mentioned scanning area, the clear pixel area in the target area of ​​the image can be extracted, and the position information of the clear pixel area can be recorded. Here, the clear pixel area can be the area where the pixel with a sharpness greater than a preset sharpness threshold is located. The position information of the clear pixel area can be recorded by pixel coordinates or other means, and this application does not limit it.

[0075] For images acquired at different altitudes, the areas of sharp pixels in the target region are also different. After obtaining the areas of sharp pixels included in the target region of each image, the areas of sharp pixels included in the target region can be fused according to the position information of the areas of sharp pixels included in the target region of each image. The areas of sharp pixels at different altitudes are fused together to obtain a full depth image of the target region.

[0076] In this embodiment, the obtained full depth-of-field image can be directly displayed on the aforementioned monitor, allowing the user to immediately determine whether the imaging effect meets the requirements. If not, the parameters can be adjusted immediately for re-acquisition, avoiding the risk of the entire experiment failing due to improper parameter settings, as is common in traditional offline processing methods.

[0077] This concludes the description of step 203.

[0078] This concludes the discussion on... Figure 2 The description.

[0079] The proposed solution determines the target region from the reference image of the target object acquired by the image acquisition device based on the region selection command. It controls the movement of the translation stage to align the target region with the field of view of the image acquisition device, eliminating the need for manual centering of the target region and improving the positioning efficiency while eliminating errors caused by manual operation. Furthermore, by controlling the translation stage to move along the optical axis of the image acquisition device, the target position of the translation stage in the optical axis direction of the image acquisition device is determined under focusing conditions. The translation stage is then automatically scanned based on the target position to generate a panoramic depth image of the target region. This changes the manual acquisition and offline processing mode of related technologies, allowing for automatic acquisition of the panoramic depth image with a single operation, simplifying the operation and improving the generation efficiency of the panoramic depth image.

[0080] The following is through Figure 3 The image processing method proposed in this application is described in its entirety.

[0081] Please refer to Figure 3 , Figure 3 A block diagram illustrating the operating principle of a microscope camera provided in an embodiment of this application.

[0082] like Figure 3 As shown, the microscope camera may include an image acquisition device, a control unit, and a translation stage, on which the target object to be detected is mounted. In this embodiment, the image acquisition device may also be connected to a display to display the acquired images and the generated full-depth-of-view images.

[0083] Specifically, the image processing method proposed in this application may include three specific processing stages, and the underlying operating principle of the microscope camera is similar in each stage.

[0084] Specifically, the three processing stages include the automatic alignment stage, the automatic focus stage, and the automatic depth-of-field fusion stage. These three stages are described below. The automatic alignment process is described using the example of the first reference point as the center point of the target area (denoted as the center of the target area) and the second reference point as the center point of the field of view (denoted as the center of the field of view).

[0085] (1) Automatic alignment stage On the reference image of the target object captured by the image acquisition device displayed on the monitor, the user selects a region of interest (ROI) as the target area by using a mouse or touch screen.

[0086] The controller can acquire images captured in real time by the image acquisition device and information about the target area defined by the user, and calculate the pixel coordinates of the center point of the target area. Simultaneously, the controller can also obtain the pixel coordinates of the center point of the current full-field image.

[0087] The controller calculates the deviation between the center pixel coordinates of the target area and the center pixel coordinates of the field of view (e.g., pixel deviation Δx_pixel in the X-axis direction and pixel deviation Δy_pixel in the Y-axis direction), and combines this with a pre-calibrated "pixel-physical distance" conversion coefficient (i.e., the actual physical size corresponding to each pixel, which can be in μm / pixel) to calculate the physical displacement that the translation stage needs to move (e.g., physical deviation Δx_physical in the X-axis direction and physical deviation Δy_physical in the Y-axis direction).

[0088] The controller can generate control commands based on the aforementioned physical displacement and send them to the translation stage. After receiving the control commands, the translation stage can drive the motors in the X-axis and Y-axis directions, moving the target object a corresponding physical distance, so that the original center of the target area is accurately moved to the center of the field of view, achieving automatic centering.

[0089] Optionally, the method for controlling the translation stage to move the center of the target area to the center of the field of view of the image acquisition device based on the physical displacement deviation can be a closed-loop control method. That is, the translation stage feeds back the current physical position to the controller so that the physical displacement can be re-determined from the current physical position, thereby achieving more accurate closed-loop control. The specific method has been described in detail above and will not be repeated here.

[0090] This concludes the description of the automatic alignment phase.

[0091] In this embodiment, the translation stage in the related technology is an open-loop tool that passively executes control commands, while this application proposes a closed-loop centering scheme of "visual perception-computation decision-motion execution", which can obtain the target area arbitrarily selected by the user in the image and automatically drive the translation stage to accurately move the target area to the center of the field of view.

[0092] The aforementioned automated alignment scheme eliminates subjective positioning errors caused by differences in operator experience and fatigue levels, ensuring that the initial conditions for each test are completely consistent, and greatly improving the reliability and repeatability of the test results.

[0093] (2) Autofocus stage After completing automatic centering, the controller can drive the translation stage to move in the Z-axis direction, and at each Z-axis position, control the image acquisition device to acquire one frame of image.

[0094] The controller evaluates the sharpness of the target region in each frame of the image using a sharpness evaluation function (such as the Tenengrad gradient function, the Laplacian energy function, etc.) to determine the sharpness of the target object in each image.

[0095] Furthermore, the position of each Z-axis and its corresponding sharpness are recorded to generate a "position-sharpness" curve. By finding the peak point of this curve, the Z-axis coordinates corresponding to the target focal plane in the current field of view can be determined.

[0096] As an example, the above process may employ optimization algorithms such as "hill climbing" to improve search speed and accuracy, and this application does not impose any limitations on this.

[0097] This concludes the description of the autofocus stage.

[0098] In this embodiment, the proposed solution can automatically scan along the Z-axis and locate the focal plane corresponding to the peak sharpness of the target area at various heights, achieving fast and accurate focusing on any area and avoiding errors caused by manual focusing. When performing multi-point or large-area scanning, the system can automatically refocus at each new location to adapt to changes in the height of the sample surface.

[0099] (3) Automatic depth-of-field fusion stage After autofocus is completed, a Z-axis scanning range (e.g., ±50μm) and scanning step size (e.g., 1μm) can be automatically set centered on the determined target focal plane. This scanning range can be automatically adjusted according to the estimated thickness of the object being measured, or preset by the user.

[0100] The controller controls the translation stage, using the Z-axis height corresponding to the target focal plane as a reference, to move from the start to the end position of the scanning interval according to a set step size. At each Z-axis position, the controller controls the image acquisition device to acquire and save a high-resolution local focus image, ultimately forming an image sequence.

[0101] While acquiring image sequences, or after all acquisitions are completed, the controller analyzes the target region in each image in the sequence using a specified algorithm, such as a multi-focus fusion algorithm, and extracts the clearest (i.e. most accurately focused) pixel region in the target region of each image.

[0102] By stitching together the clear pixel regions of the target area in all images according to their spatial positions, a panoramic depth image with complete clarity from top to bottom is generated.

[0103] The fused panoramic depth image can be displayed on the user interface in real time, achieving automated depth fusion and being automatically saved without any offline processing software.

[0104] This concludes the description of the automatic depth-of-field fusion stage.

[0105] In this embodiment, the depth-of-field fusion process, which originally required specialized software and manual intervention, is transformed into an automated process executed by control devices. Users can complete the entire process from defining the target area to obtaining the full depth-of-field image with a single click, greatly improving ease of operation.

[0106] This concludes the discussion on... Figure 3 The description.

[0107] Please refer to Figure 4 , Figure 4 This is a structural diagram of an image processing device proposed in an embodiment of this application. The device is applied to a controller, which is communicatively connected to a translation stage and an image acquisition device. The translation stage is used to place the target object for image acquisition. Figure 4 As shown, the device may include a centering unit 401, a focusing unit 402, and a generating unit 403. Specifically, the device includes: Alignment unit 401 is used to determine the target region from the reference image of the target object acquired by the image acquisition device based on the region selection instruction, and control the translation stage to move so that the target region is aligned with the field of view of the image acquisition device. The focusing unit 402 is used to control the translation stage to move along the optical axis of the image acquisition device and determine the target position of the translation stage in the optical axis of the image acquisition device when the target area is in the focusing state. The generation unit 403 is used to control the translation stage to move along the optical axis of the image acquisition device based on the target position, and to generate a panoramic depth image of the target area based on the image acquired by the image acquisition device during the movement.

[0108] Optionally, the alignment unit 401 is specifically used for: Determine the pixel coordinate deviation between the first pixel coordinate of the first reference point of the target area in the reference image and the second pixel coordinate of the second reference point in the field of view of the image acquisition device; Based on preset pixel size calibration parameters, the pixel coordinate deviation is converted into a physical displacement deviation; Based on the physical displacement deviation, the translation stage is controlled to move the first reference point of the target area to the second reference point in the field of view of the image acquisition device, so that the first reference point of the target area is aligned with the second reference point in the field of view of the image acquisition device.

[0109] Optionally, the alignment unit 401 is specifically used for: Based on the physical displacement deviation, a movement command is issued to the translation stage; During the movement of the translation stage, the current pixel coordinates of the first reference point of the target area are determined based on the current physical coordinates of the first reference point of the target area fed back by the translation stage, and the current coordinate error is determined based on the current pixel coordinates and the second pixel coordinates. Based on preset pixel size calibration parameters, the current coordinate deviation is converted into a current physical displacement error; the movement command is adjusted based on the current physical displacement error, and the step of determining the current pixel coordinates of the first reference point of the target area based on the current physical coordinates of the first reference point of the target area fed back by the translation stage is returned, until the current physical displacement error is less than a preset error threshold.

[0110] Optionally, the focusing unit 402 is specifically used for: The translation stage is controlled to move along the optical axis of the image acquisition device, and the target focal plane is determined based on the sharpness of the target area in the image acquired by the image acquisition device during the movement of the translation stage. The position of the target is defined by the optical axis corresponding to the target focal plane. The sharpness of the target area on the target focal plane is greater than that on other focal planes.

[0111] Optionally, the focusing unit 402 is specifically used for: The sharpness of the target region in each image acquired by the image acquisition device during the movement of the translation stage is determined based on the sharpness evaluation function. Generate a curve showing the change in the sharpness of the target region along the optical axis based on the sharpness of the target region in each image. The focal plane corresponding to the peak point of the sharpness of the target area in the optical axis direction of the variation curve is determined as the target focal plane.

[0112] Optionally, the generating unit 403 is specifically used for: The control translation stage moves along the optical axis of the image acquisition device within a preset scanning range, using the target position as a reference point. Based on the images acquired by the image acquisition device during the movement, a full-view depth image of the target area is generated. The preset scanning range is the scanning range centered on the target position.

[0113] Optionally, the preset scanning range includes a scanning interval and a scanning step size, which are determined based on the height of the target object.

[0114] Optionally, the generating unit 403 is specifically used for: For each image captured by the image acquisition device during movement, the clear pixel region included in the target area of ​​the image is extracted, and the location information of the clear pixel region is recorded; the clear pixel region refers to the region with a sharpness greater than a preset sharpness threshold; Based on the location information of the sharp pixel regions included in the target region in each image, the sharp pixel regions included in the target region are fused to obtain a full depth image of the target region.

[0115] This concludes the process. Figure 4 Description of the image processing device.

[0116] This application also provides embodiments that... Figure 4 Hardware structure description of the illustrated device. This hardware structure is... Figure 5 The structure in the illustrated electronic device. Please refer to [link / reference]. Figure 5 , Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application. Figure 5 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0117] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.

[0118] For example, the aforementioned computer-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, computer-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0119] The above are merely preferred embodiments of this application and are not intended to limit the application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image processing method, characterized by, The method is applied to a control device in communication with a translation table and an image acquisition device, the translation table being used to place a target object requiring image acquisition, and the method comprises: determining a target region from a reference image of the target object acquired by the image acquisition device based on a region selection instruction, and controlling the translation table to move so that the target region is aligned with the field of view of the image acquisition device; controlling the translation table to move along the optical axis direction of the image acquisition device, and determining the target position of the translation table in the optical axis direction of the image acquisition device in a focusing state of the target region; controlling the translation table to move along the optical axis direction of the image acquisition device based on the target position, and generating a panoramic depth image of the target region based on images acquired by the image acquisition device during the movement.

2. The method of claim 1, wherein, The control of the translation table to move so that the target region is aligned with the field of view of the image acquisition device comprises: determining a pixel coordinate deviation between a first pixel coordinate of a first reference point of the target region in the reference image and a second pixel coordinate of a second reference point in the field of view of the image acquisition device; converting the pixel coordinate deviation into a physical displacement deviation based on a preset pixel size calibration parameter; controlling the translation table to move the first reference point of the target region to the second reference point in the field of view of the image acquisition device according to the physical displacement deviation, so that the first reference point of the target region is aligned with the second reference point in the field of view of the image acquisition device.

3. The method of claim 2, wherein, The control of the translation table to move the first reference point of the target region to the second reference point in the field of view of the image acquisition device according to the physical displacement deviation comprises: issuing a movement instruction to the translation table based on the physical displacement deviation; during the movement of the translation table, determining a current pixel coordinate of the first reference point of the target region according to a current physical coordinate of the first reference point of the target region fed back by the translation table, and determining a current coordinate error based on the current pixel coordinate and the second pixel coordinate; converting the current coordinate deviation into a current physical displacement error based on a preset pixel size calibration parameter; adjusting the movement instruction based on the current physical displacement error, and returning to the step of determining the current pixel coordinate of the first reference point of the target region according to the current physical coordinate of the first reference point of the target region fed back by the translation table until the current physical displacement error is less than a preset error threshold.

4. The method of claim 1, wherein, The control of the translation table to move along the optical axis direction of the image acquisition device, and determining the target position of the translation table in the optical axis direction of the image acquisition device in a focusing state of the target region comprises: controlling the translation table to move along the optical axis direction of the image acquisition device, and determining a target focal plane based on the sharpness of the target region in images acquired by the image acquisition device during the movement of the translation table; The target position corresponds to a position in an optical axis direction of the target focal plane, and the target region has higher definition in the target focal plane than in other focal planes.

5. The method of claim 4, wherein, The target focal plane is determined based on definition of the target region in images captured by the image acquisition device during movement of the translation stage. The definition of the target region in each image captured by the image acquisition device during movement of the translation stage is determined according to a definition evaluation function. A change curve of the definition of the target region with respect to the position in the optical axis direction is generated according to the definition of the target region in each image. The focal plane to which the target position in the optical axis direction corresponding to the peak point of the definition of the target region in the change curve belongs is determined as the target focal plane.

6. The method of claim 1, wherein, The translation stage is controlled to move along the optical axis direction of the image acquisition device based on the target position, and a full-depth image of the target region is generated based on images captured by the image acquisition device during movement. The translation stage is controlled to move in the optical axis direction of the image acquisition device within a preset scanning range with the target position as a reference point, and a full-depth image of the target region is generated based on images captured by the image acquisition device during movement.

7. The method of claim 6, wherein, The preset scanning range includes a scanning interval and a scanning step, and the scanning interval and the scanning step are determined according to the height of the target object.

8. The method of claim 1 or 6, wherein, The full-depth image of the target region is generated based on images captured by the image acquisition device during movement. For each image captured by the image acquisition device during movement, a clear pixel region included in the target region in the image is extracted, and position information of the clear pixel region is recorded. The clear pixel regions included in the target region are fused according to the position information of the clear pixel regions included in the target region in each image to obtain a full-depth image of the target region.

9. An image processing apparatus characterized by comprising: The device is applied to a controller, which is in communication connection with a translation stage and an image acquisition device. The translation stage is used to place a target object requiring image acquisition. The alignment unit is configured to determine a target region from a reference image of the target object captured by the image acquisition device based on a region selection instruction, and control the translation stage to move so that the target region is aligned with a field of view of the image acquisition device. The focusing unit is configured to control the translation stage to move along an optical axis direction of the image acquisition device, and determine a target position of the translation stage in the optical axis direction of the image acquisition device in a focusing state of the target region.

10. A microscope camera, characterized by The generation unit is configured to control the translation stage to move along the optical axis direction of the image acquisition device based on the target position, and generate a full-depth image of the target region based on images captured by the image acquisition device during movement. The device includes: a controller, a translation stage, an image acquisition device, and a display. The controller, the translation table, the image acquisition device and the display are in communication connection, the translation table is used for placing a target object which needs to be image-acquired; the image acquisition device is used for image-acquiring the target object, and the display is used for displaying the image acquired by the image acquisition device. The controller is used for executing the method in any one of claims 1 to 8.

Images