Fungus picking instrument and petri dish image synthesis processing method

Abstract

The invention discloses a petri dish image synthesis processing method, which comprises the following steps: completing camera internal reference calibration and three-dimensional calibration through a calibration plate image, and obtaining related parameters; shooting a three-dimensional vision image of the culture dish, and obtaining the object distance of the culture dish through binocular vision triangulation based on the three-dimensional calibration parameters; the height of the camera shooting unit is adjusted according to the object distance, local images of a plurality of areas of the culture dish are collected, and after distortion correction, a complete high-resolution image is obtained based on motion vector and internal parameter splicing. The binocular vision function is achieved through the single camera, the hardware cost is reduced, different culture medium thicknesses are adapted through self-adaptive object distance adjustment, high resolution and low cost are both considered through image splicing, the expansion adaptability is high, the positioning precision, the universality and the performance cost performance of the bacterium picking instrument can be remarkably improved, and the application range is wide. The method is suitable for bacterial colony identification and positioning scenes of various microbiological experiments.

Description

A method for image synthesis and processing of a bacterial picker and petri dishes Technical Field

[0001] This invention belongs to the field of petri dish image processing technology, specifically relating to a bacterial picker and a method for synthesizing and processing petri dish images. Background Technology

[0002] In the fields of microbial experimental research and industrial production, the fully automated microbial cloning picker, also known as a microbial picker, is a core piece of laboratory automation equipment. With its integrated machine vision, precision motion control and robotics technology, it can replace manual labor in automatically identifying, locating and picking colonies from petri dishes, and transferring the colonies to target containers for subsequent processing. This greatly improves experimental efficiency and operational accuracy, and has been widely used in many fields such as microbial screening, genetic engineering, and drug development.

[0003] The machine vision module is the core component of a microbial picker, enabling precise colony identification and localization. Its typical structure includes a picking stage for placing the culture dish, an illumination unit for image acquisition, and a camera unit positioned above the stage to capture images of the culture dish. Considering the stringent requirements for reliability, accuracy, and stability in identification and localization scenarios, existing microbial pickers typically use industrial cameras for their camera units. Since industrial cameras are primarily used for precision measurement, they generally lack autofocus. Therefore, during the installation and commissioning phase of the microbial picker system, a precise manual focus must be performed, followed by locking the focus and fixing the relative positions of the stage and camera unit to ensure the stability of image acquisition.

[0004] In existing technologies, the relative position of the camera unit and the stage is fixed, and the focal length is locked. Therefore, to achieve precise colony positioning, the thickness of the culture medium in the petri dish must be strictly controlled. Even slight changes in the thickness of the culture medium directly alter the distance between the colony and the camera, thus introducing significant measurement errors. However, in practical applications, dimensional tolerances are unavoidable in the production and processing of petri dishes. Furthermore, parameters such as the amount of culture medium, its component ratios, and temperature are difficult to standardize completely. These objective factors make it impossible to effectively avoid changes in the distance between the colony and the camera, resulting in uncontrollable measurement errors in the vision system of the colony picking instrument, severely impacting the accuracy of colony positioning and the success rate of picking.

[0005] Meanwhile, the resolution of the petri dish image directly determines the minimum size threshold and positioning accuracy for colony identification. Improving image resolution is one of the key directions for optimizing the performance of the picker. In existing technologies, the main way to improve image resolution is to use higher-resolution industrial cameras, but the procurement cost of high-resolution industrial cameras increases significantly. Furthermore, petri dishes come in various sizes, ranging from 60mm to 150mm. To achieve compatibility with different sizes of petri dishes, existing pickers typically set the object distance and field of view of the camera unit according to the requirements of the largest size petri dish. This results in a significant waste of the camera's effective resolution when photographing the most commonly used small-sized Piper dishes, such as 90mm ones. This means that the camera's performance cannot be fully utilized, and it is difficult to obtain images that meet the high-precision identification requirements within a reasonable cost range using existing solutions. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology of image acquisition and processing of culture dishes in the picker, which is difficult to flexibly align diagonally and has high resolution and cost, so as to provide a method for image synthesis and processing of picker and culture dishes.

[0007] A method for synthesizing petri dish images includes the following steps: S1: Placing an empty petri dish on the stage of a picker, and moving a camera unit above the empty petri dish; placing a calibration plate in the empty petri dish, aligning the center of the calibration plate with the center of the camera's field of view of the camera unit; adjusting the height of the camera and the lens zoom ring to obtain a clear image of the calibration plate; horizontally translating the camera unit to obtain multiple images of the calibration plate; obtaining internal parameter parameters through internal parameter calibration based on the calibration plate images, and obtaining stereo calibration parameters through stereo calibration; S2: Placing a petri dish on the stage of the picker, and moving the camera unit according to... A pair of images with opposite positions are captured by horizontally shifting the azimuth vector, defined as a stereoscopic vision image of the culture dish; based on the stereo calibration parameters, binocular visual triangulation is performed on the stereoscopic vision image of the culture dish to obtain the object distance of the culture dish; S3: based on the object distance of the culture dish, the height of the camera unit is adjusted; the camera unit is horizontally shifted according to the movement vector to acquire local images of multiple regions of the culture dish; distortion correction is performed on the acquired local images based on the intrinsic parameters; based on the movement vector and the intrinsic parameters, the local images of multiple regions of the culture dish are stitched together to obtain a stitched image of the culture dish.

[0008] Furthermore, the method includes the following steps: placing a culture dish on the stage of the picker, and the camera unit horizontally translating and capturing a pair of images with opposite positions according to the shooting orientation vector, defining them as a stereoscopic vision image of the culture dish; performing distortion correction and row alignment processing on the stereoscopic vision image of the culture dish based on the stereo calibration parameters to obtain an alignment-corrected image; extracting feature points from the alignment-corrected image, and obtaining matching feature point pairs between the two images in the alignment-corrected image through a feature descriptor matching algorithm; obtaining filtered feature point pairs by calculating the homography matrix and finding the interior points in the feature point pairs based on polar geometric constraints; calculating the three-dimensional coordinates of the filtered feature point pairs in the camera coordinate system based on the triangulation principle, obtaining the object distance of the feature points based on the three-dimensional coordinates, and calculating the average distance of the object distance of the feature points as the object distance of the culture dish.

[0009] Further, the method for acquiring the calibration board images includes: horizontally translating the camera unit to acquire a total of nine calibration board images; in the fourth calibration board image, the center of the calibration board is aligned with the center of the camera's field of view; using the fourth calibration board image as a reference: the camera unit acquires the zeroth calibration board image at a predetermined distance to the left and backward; the camera unit acquires the first calibration board image at a predetermined distance to the backward; the camera unit acquires the second calibration board image at a predetermined distance to the right and backward; the camera unit... The imaging unit acquires the third calibration plate image at a predetermined distance to the left; the camera unit acquires the fifth calibration plate image at a predetermined distance to the right; the camera unit acquires the sixth calibration plate image at predetermined distances to the left and forward; the camera unit acquires the zeroth calibration plate image at a predetermined distance to the forward; the camera unit acquires the seventh calibration plate image at predetermined distances to the left and backward; and the camera unit acquires the eighth calibration plate image at predetermined distances to the right and forward.

[0010] Furthermore, the method includes the following steps: based on the nine calibration board images, the intrinsic parameters of the camera are obtained through intrinsic parameter calibration, including the camera's focal length, principal point coordinates, pixel distortion coefficient, and object distance.

[0011] Furthermore, the method includes the following steps: obtaining stereo calibration parameters through stereo calibration based on the zeroth calibration plate image, the second calibration plate image, the third calibration plate image, the fifth calibration plate image, the sixth calibration plate image, and the eighth calibration plate image; the stereo calibration parameters include the rotation matrix and translation vector between the left calibration plate image and the corresponding right calibration plate image.

[0012] Furthermore, the method for acquiring a stitched image of a culture dish includes the following steps: adjusting the height of the camera unit based on the distance between the culture dish and the object; horizontally translating the camera unit according to a movement vector to acquire local images of four regions of the culture dish: the left rear, right rear, left front, and right front; adjacent local images partially overlap, and the four local images cover the upper side of the culture dish; performing distortion correction on the acquired local images based on the intrinsic parameters; and stitching the local images of the four regions of the culture dish—the left rear, right rear, left front, and right front—based on the movement vector and the intrinsic parameters to acquire a stitched image of the culture dish.

[0013] Furthermore, by horizontally translating the camera unit according to the movement vector, local images of four regions—left rear, right rear, left front, and right front—of the culture dish are acquired, including the following steps: taking one-quarter of the culture dish diameter as the movement vector; using the alignment of the center of the culture dish with the center of the camera's field of view as the shooting starting point; based on the shooting starting point, translating the camera unit to the left and backward with the movement vector to acquire the left rear local image; based on the shooting starting point, translating the camera unit to the right and backward with the movement vector to acquire the right rear local image; based on the shooting starting point, translating the camera unit to the left and forward with the movement vector to acquire the left front local image; based on the shooting starting point, translating the camera unit to the right and forward with the movement vector to acquire the right front local image.

[0014] Furthermore, obtaining the petri dish stitched image includes the following steps: calculating the corner coordinates required for stitching through affine transformation based on the translation relationship of the movement vectors of the local images and the intrinsic parameters of the camera, and establishing the geometric mapping relationship between the local images; stitching the local images based on the geometric mapping relationship; and smoothing the overlapping areas of the stitched local images using a feather mixer to obtain the petri dish stitched image.

[0015] A microbial picking device is used to acquire a stitched image of a culture dish according to the above-described image synthesis processing method for culture dishes; it includes a camera unit, a microbial picking unit, a culture dish, a stage, and a calibration plate; the microbial picking unit is located on the upper side of the stage and is movable; the camera unit is fixedly connected to the microbial picking unit; the culture dish is placed on the upper side of the stage; the calibration plate is used to place inside the culture dish and obtain internal parameter parameters through internal parameter calibration and obtain stereo calibration parameters through stereo calibration.

[0016] Furthermore, the calibration plate is rectangular; the calibration plate is a dot array calibration plate with white dots arranged in a rectangular dot array on the upper side.

[0017] Beneficial effects: This invention integrates a single industrial camera into a movable bacteria-picking unit. Utilizing the high repeatability of the motion axis, it achieves binocular vision distance perception by capturing images at different positions through a horizontally shifting camera. This eliminates the need for additional binocular cameras and matching synchronization control modules, significantly reducing the hardware procurement and integration costs of the equipment. At the same time, it simplifies the system structure and improves the stability of equipment operation.

[0018] This invention obtains the distance between the culture dish and the colony through stereo vision triangulation and adaptively adjusts the height of the camera unit accordingly to ensure a stable distance between the camera unit and the colony. This effectively avoids measurement errors caused by changes in the distance, significantly improves the accuracy of colony positioning and the success rate of picking, and reduces the requirements for the consistency of culture dish specifications, thus significantly enhancing the versatility and stability of the equipment in practical application scenarios.

[0019] This invention employs a multi-regional local image acquisition and correction stitching technique, which can synthesize complete high-resolution petri dish images without relying on high-resolution industrial cameras. This fully utilizes the image resolution of commonly used small-sized petri dishes such as 90mm, ensuring the minimum size threshold and positioning accuracy for colony identification while reasonably controlling costs, thus achieving a balance between performance and cost. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 is a flowchart illustrating the main method steps of the present invention; Figure 2 is a schematic diagram of the calibration plate image of the present invention; Figure 3 is a schematic diagram of feature point matching of the present invention; Figure 4 is a partial image of the culture dish of the present invention; Figure 5 is a schematic diagram of the spliced ​​image of the culture dish of the present invention. Detailed Implementation

[0022] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0023] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0024] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0025] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0026] Example 1: This example provides a method for synthesizing petri dish images, including the following steps: S1: Place an empty petri dish on the stage of the picker, and move the camera unit above the empty petri dish; place a calibration plate in the empty petri dish, and align the center of the calibration plate with the center of the camera's field of view of the camera unit; adjust the height of the camera and the lens zoom ring to obtain a clear image of the calibration plate; horizontally translate the camera unit to obtain multiple images of the calibration plate; based on the calibration plate images, obtain internal parameter parameters through internal parameter calibration, and obtain stereo calibration parameters through stereo calibration; S2: Place a petri dish on the stage of the picker, The camera unit horizontally shifts and captures a pair of images with opposite positions based on the shooting orientation vector, defining them as a stereoscopic vision image of the petri dish; based on the stereo calibration parameters, binocular visual triangulation is performed on the stereoscopic vision image of the petri dish to obtain the object distance of the petri dish; S3: based on the object distance of the petri dish, the height of the camera unit is adjusted; the camera unit is horizontally shifted according to the movement vector to acquire local images of multiple regions of the petri dish; distortion correction is performed on the acquired local images based on the intrinsic parameters; based on the movement vector and the intrinsic parameters, the local images of multiple regions of the petri dish are stitched together to obtain a stitched image of the petri dish.

[0027] Specifically, in step S1, as some implementations of this embodiment, the method for acquiring the calibration board image includes: referring to Figure 2, horizontally translating the camera unit to acquire a total of nine calibration board images; in the fourth calibration board image, the center of the calibration board is aligned with the center of the camera's field of view; using the fourth calibration board image as a reference: the camera unit acquires the zeroth calibration board image at a predetermined distance to the left and backward; the camera unit acquires the first calibration board image at a predetermined distance to the back; the camera unit acquires the first calibration board image at a predetermined distance to the right and backward. The camera unit acquires a second calibration plate image; at a predetermined distance to the left, it acquires a third calibration plate image; at a predetermined distance to the right, it acquires a fifth calibration plate image; at predetermined distances to the left and forward, it acquires a sixth calibration plate image; at a predetermined distance to the forward, it acquires a zeroth calibration plate image; at predetermined distances to the left and backward, it acquires a seventh calibration plate image; and at predetermined distances to the right and forward, it acquires an eighth calibration plate image.

[0028] In these embodiments, the camera unit is controlled to acquire calibration board images from nine directions at fixed intervals. The shooting azimuth vectors are (d, d, 0), (0, d, 0), (-d, d, 0), (d, 0, 0), (0, 0, 0), (-d, 0, 0), (d, -d, 0), (0, -d, 0), (-d, -d, 0), with a predetermined interval d = 12mm. The calibration board images obtained from the above nine shooting azimuth vectors are respectively the first calibration board image to the eighth calibration board image.

[0029] In this embodiment, the following steps are also included: based on the nine calibration board images, the intrinsic parameters of the camera are obtained through intrinsic parameter calibration, wherein the intrinsic parameters of the camera include the camera's focal length, principal point coordinates, pixel distortion coefficient, and object distance.

[0030] In some embodiments of this example, the intrinsic parameters are calculated by the following methods: using the correspondence between the pixel coordinates of the feature points of the calibration board and the world coordinates under different orientations, a system of equations is constructed, and the system of equations is solved by the least squares method to obtain intrinsic parameters such as focal length and principal point coordinates; by analyzing the distortion deviation of the feature points in different images, the pixel distortion coefficient is calculated; and by combining the actual positional relationship between the calibration board and the camera, the object distance is calculated.

[0031] Among them, focal length Used to describe the scaling ratio by which a camera lens projects points in three-dimensional space onto a two-dimensional image plane. and Corresponding to The focal length of the direction is ideally close to the focal length of the principal point. The coordinates of the intersection point between the camera's optical axis and the image plane, usually located near the center of the image, are used to determine the origin of the image coordinate system. Pixel distortion coefficients. Used to describe the degree of distortion in lens imaging. The radial distortion coefficient is... These are the tangential distortion coefficients, which are used to correct distortions in the image. The object distance is the distance from the camera lens to the calibration plate plane during calibration; it serves as a reference parameter for subsequent adjustments to the imaging distance and ensuring stitching accuracy.

[0032] In this embodiment, the method further includes the following steps: obtaining stereo calibration parameters through stereo calibration based on the zeroth calibration plate image, the second calibration plate image, the third calibration plate image, the fifth calibration plate image, the sixth calibration plate image, and the eighth calibration plate image; the stereo calibration parameters include the rotation matrix and translation vector between the left calibration plate image and the corresponding right calibration plate image.

[0033] In some embodiments of this example, the stereo calibration parameters are calculated using the following methods: Six images to be calibrated are uniformly preprocessed, including grayscale conversion, filtering and denoising, contrast enhancement, etc.; the center feature points of the calibration board are extracted from the six images, including edge detection, contour filtering and fitting, hole center coordinate calculation, etc.; feature point matching is performed on the left and right positions of the six images to be calibrated; for each matching pair, such as the zeroth and second calibration board images, the third and fifth calibration board images, and the sixth and eighth calibration board images, the feature points and world coordinates of the left image are used, combined with intrinsic parameters, to fit the imaging equation using the least squares method to solve for the extrinsic parameters R1 and T1 of the left camera; similarly, the extrinsic parameters R2 and T2 of the right image are solved, and the rotation matrix R and translation vector T of the right camera relative to the left camera are calculated. R and T are calculated for each of the three matching pairs, and then the arithmetic mean method is used to fuse the three results to obtain the final stereo calibration parameters: rotation matrix R and translation vector T.

[0034] In this embodiment, by taking advantage of the high repeatability of the positioning accuracy of the bacterial picking unit's motion axis, a single camera unit moves left and right and captures images to simulate the shooting of left and right cameras, thereby achieving stereo vision capabilities similar to those of a binocular camera without adding additional sensors.

[0035] Specifically, step S2 includes the following steps: placing a culture dish on the stage of the picker, and the camera unit horizontally translating to capture a pair of images (-d, 0, 0) and (d, 0, 0) with opposite positions according to the shooting orientation vector, which are denoted as the left view and the right view, and defined as the stereoscopic visual image of the culture dish; in this embodiment, the shooting orientation vector d is equal to the predetermined spacing d in step S1.

[0036] Based on the stereo calibration parameters, the stereo vision image of the culture dish is subjected to distortion correction and row alignment processing to obtain an aligned and corrected image. In this embodiment, the original left and right images are subjected to distortion correction and row alignment processing by stereo correction methods such as the Bouguet algorithm to generate two strictly aligned corrected images, thereby simplifying subsequent feature matching and improving the accuracy of triangulation.

[0037] Referring to Figure 3, feature points are extracted from the alignment and correction images. A feature descriptor matching algorithm is used to obtain matching feature point pairs between the two images in the alignment and correction images. In this embodiment, a feature detection algorithm is used to extract feature points from the correction images. Then, the feature descriptor matching algorithm is used to find feature point pairs belonging to the same physical point in the left and right images. These matching point pairs form the data basis for subsequent 3D calculations.

[0038] Based on polar geometric constraints, filter feature point pairs are obtained by calculating the homography matrix and finding the interior points in the feature point pairs. In this embodiment, by utilizing the polar geometric constraints that the left and right images should satisfy after stereo correction, feature point pairs with high consistency and extremely high matching quality are selected by calculating the homography matrix and finding its interior points.

[0039] Based on the principle of triangulation, the three-dimensional coordinates of each filtered feature point pair in the camera coordinate system are calculated. The object distance of the feature points is obtained based on these three-dimensional coordinates, and the average distance between the feature points is calculated as the object distance of the culture dish. In this embodiment, the three-dimensional coordinates (X, Y, Z) of each feature point in the camera coordinate system are calculated pair by pair using the principle of triangulation. The Z coordinate is the object distance of the point relative to the camera.

[0040] In this embodiment, a camera coordinate system is first established based on the image from one of the directions. The rotation matrix obtained from stereo calibration is used to describe the attitude rotation relationship of the right camera relative to the left camera, and the translation vector is used to describe the positional offset relationship of the right camera relative to the left camera. For each pair of matched high-quality feature points, the two-dimensional pixel coordinates are first converted into spatial rays in their respective camera coordinate systems, pointing from the optical center of the camera to the true position of the feature point on the petri dish. Using the rotation matrix and translation vector from stereo calibration, the spatial rays in the right camera coordinate system are transformed into the left camera coordinate system, so that the two rays are in the same reference system. The point where the two rays are closest in space is found, and its coordinates are the three-dimensional coordinates (X, Y, Z) of the feature point in the left camera coordinate system.

[0041] Specifically, in step S3, the method for acquiring a stitched image of a culture dish includes the following steps: adjusting the height of the camera unit based on the object distance of the culture dish; thereby ensuring that the actual object distance is consistent with the object distance during system calibration, and ensuring that the acquired image is in the best focus state.

[0042] The camera unit is horizontally translated according to the movement vector to acquire local images of four regions: the rear left, rear right, front left, and front right of the petri dish. Adjacent local images partially overlap, and the four local images cover the upper side of the petri dish. Keeping the Z-axis position constant, the camera unit is controlled to move along a predetermined trajectory in the XY plane. For a 90mm Piper dish, the movement distance s = 90mm / 4 = 22.5mm. When the center of the petri dish is used as the starting point for shooting, the movement vectors are (-s, -s), (s, -s), (-s, s), and (s, s), thus acquiring images that completely cover the upper left, upper right, lower left, and lower right regions of the target.

[0043] In this embodiment, the movement vector is selected as one-quarter of the diameter of the petri dish.

[0044] Distortion correction is performed on the acquired local images based on the intrinsic parameters; distortion correction is performed on the acquired original images using the camera intrinsic parameters obtained in the camera calibration step, eliminating the influence of lens distortion on the image geometry, ensuring spatial consistency between images, and laying the foundation for subsequent accurate stitching.

[0045] Referring to Figure 4, based on the movement vector and the intrinsic parameters, local images of the four regions of the culture dish—left rear, right rear, left front, and right front—are stitched together to obtain a stitched image of the culture dish.

[0046] Based on the translation relationship of the movement vectors of the local images and the intrinsic parameters of the camera, the corner coordinates required for stitching are calculated through affine transformation, and a geometric mapping relationship between the local images is established; the local images are stitched based on the geometric mapping relationship; a feather mixer is used to smooth the overlapping areas of the stitched local images to obtain the petri dish stitched image.

[0047] In this embodiment, the corner coordinates required for stitching are first calculated by affine transformation based on the movement vector during image capture, and the geometric mapping relationship between the images is established. Then, a feather mixer is used to smooth the overlapping areas of the images, eliminate seam traces, and finally generate a stitched image with a natural visual effect.

[0048] Specifically, firstly, an affine transformation is used to determine the corresponding coordinates of the four corner points of each image in the coordinate system of the larger image, thus defining the placement area for each image and achieving precise alignment. A feathering mixer is then used to smoothly transition pixels in overlapping areas from one image to another; for example, the left image has a higher pixel ratio in the overlapping area closer to the left image, and the right image has a higher pixel ratio in the overlapping area closer to the right image, thereby eliminating the seam between the two images. Based on the positions determined by the affine transformation, all local images are placed to obtain a complete petri dish stitched image, as shown in Figure 5.

[0049] Specifically, the camera unit is horizontally translated according to the movement vector to acquire local images of four regions of the petri dish: the left rear, right rear, left front, and right front. This includes the following steps: taking one-quarter of the petri dish diameter as the movement vector; aligning the center of the petri dish with the center of the camera's field of view as the starting point for the shot; based on the starting point, the camera unit is translated to the left and backward by the movement vector to acquire the left rear local image; based on the starting point, the camera unit is translated to the right and backward by the movement vector to acquire the right rear local image; based on the starting point, the camera unit is translated to the left and forward by the movement vector to acquire the left front local image; based on the starting point, the camera unit is translated to the right and forward by the movement vector to acquire the right front local image.

[0050] In this embodiment, the camera unit uses a 12-megapixel (4080×3072) industrial camera with a lens of appropriate focal length, allowing a single shot to cover one-quarter of the area of ​​a 90mm Piper culture dish. By acquiring images of the culture dish from four different angles and stitching these four images together, a high-resolution complete image of the culture dish with 48 megapixels is finally obtained. In terms of imaging resolution, the resolution of a single original image is 90mm / 3072 ≈ 0.029mm / pixel. After stitching four images together, the effective pixel count doubles, resulting in a final image resolution of 90mm / (3072×2) = 0.0146mm / pixel, approximately doubling the spatial resolution.

[0051] Example 2: This example provides a microbial picking device for obtaining stitched images of culture dishes according to the image synthesis processing method for culture dishes described in Example 1; it includes a camera unit, a microbial picking unit, a culture dish, a stage, and a calibration plate; the microbial picking unit is located on the upper side of the stage and is movable; the camera unit is fixedly connected to the microbial picking unit; the culture dish is placed on the upper side of the stage; the calibration plate is placed inside the culture dish and obtains internal parameter parameters through internal parameter calibration and obtains three-dimensional calibration parameters through three-dimensional calibration.

[0052] In this embodiment, the bacteria-picking unit is located above the stage and, as part of the bacteria-picking instrument, can be precisely controlled to move in the X, Y, and Z directions. The camera unit is rigidly connected to the bacteria-picking unit, thus enabling the camera unit to follow the movement of the bacteria-picking unit and achieve precise control.

[0053] As a preferred embodiment, the calibration plate is a Halcon honeycomb calibration plate, which is rectangular in shape and can be used to calibrate camera intrinsic parameters through calibration assistant software; the calibration plate is a dot array calibration plate with white dots arranged in a rectangular dot array on the upper side.

[0054] In this embodiment, the culture dish is a 90mm Piper culture dish.

[0055] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0056] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for synthesizing and processing images of petri dishes, characterized in that, Includes the following steps: Place an empty culture dish on the stage of the picker and move the camera unit above the empty culture dish; place a calibration plate in the empty culture dish and align the center of the calibration plate with the center of the camera field of view of the camera unit. Adjust the height of the camera and the zoom ring of the lens to obtain a clear image of the calibration plate; horizontally shift the camera unit to acquire multiple images of the calibration plate; based on the calibration plate images, obtain internal parameter parameters through internal parameter calibration, and obtain stereo calibration parameters through stereo calibration; place a culture dish on the stage of the picker, and the camera unit horizontally shifts according to the shooting orientation vector to capture a pair of images with opposite positions, defining them as stereo vision images of the culture dish; based on the stereo calibration parameters, perform binocular visual triangulation using the stereo vision images of the culture dish to obtain the distance between the culture dish and the object; adjust the height of the camera unit based on the distance between the culture dish and the object. The camera unit is horizontally translated according to the motion vector to acquire local images of multiple regions of the culture dish; distortion correction is performed on the acquired local images based on the intrinsic parameters; and the local images of multiple regions of the culture dish are stitched together based on the motion vector and the intrinsic parameters to obtain a stitched image of the culture dish.

2. The method for synthesizing petri dish images according to claim 1, characterized in that, It also includes the following steps: A culture dish is placed on the stage of the picker. The camera unit, based on the shooting orientation vector, horizontally shifts to capture a pair of images with opposite positions, which are defined as the stereoscopic vision image of the culture dish. Based on the stereo calibration parameters, the stereoscopic vision image of the culture dish is subjected to distortion correction and row alignment processing to obtain the alignment-corrected image. Feature points in the alignment-corrected image are extracted, and the matching feature point pairs between the two images in the alignment-corrected image are obtained through a feature descriptor matching algorithm. Based on polar geometry constraints, filtered feature point pairs are obtained by calculating the homography matrix and finding the interior points in the feature point pairs. Based on the principle of triangulation, the three-dimensional coordinates of the filtered feature point pairs in the camera coordinate system are calculated one by one. The object distance of the feature points is obtained based on the three-dimensional coordinates, and the average distance of the object distance of the feature points is calculated as the object distance of the culture dish.

3. The method for synthesizing petri dish images according to claim 1, characterized in that, The method for acquiring the calibration plate images includes: horizontally translating the camera unit to acquire a total of nine calibration plate images; in the fourth calibration plate image, the center of the calibration plate is aligned with the center of the camera's field of view; using the fourth calibration plate image as a reference: the camera unit acquires the zeroth calibration plate image at a predetermined distance to the left and backward; the camera unit acquires the first calibration plate image at a predetermined distance to the backward; the camera unit acquires the second calibration plate image at a predetermined distance to the right and backward; the camera unit... The camera unit acquires the third calibration plate image at a predetermined distance to the left; the camera unit acquires the fifth calibration plate image at a predetermined distance to the right; the camera unit acquires the sixth calibration plate image at predetermined distances to the left and forward; the camera unit acquires the zeroth calibration plate image at a predetermined distance to the forward; the camera unit acquires the seventh calibration plate image at predetermined distances to the left and backward; and the camera unit acquires the eighth calibration plate image at predetermined distances to the right and forward.

4. The method for synthesizing petri dish images according to claim 3, characterized in that, It also includes the following steps: Based on the nine calibration board images, the intrinsic parameters of the camera are obtained through intrinsic parameter calibration. The intrinsic parameters of the camera include the camera's focal length, principal point coordinates, pixel distortion coefficient, and object distance.

5. The method for synthesizing petri dish images according to claim 3, characterized in that, It also includes the following steps: Based on the zeroth calibration plate image, the second calibration plate image, the third calibration plate image, the fifth calibration plate image, the sixth calibration plate image, and the eighth calibration plate image, stereo calibration parameters are obtained through stereo calibration; the stereo calibration parameters include the rotation matrix and translation vector between the left calibration plate image and the corresponding right calibration plate image.

6. The method for synthesizing petri dish images according to claim 1, characterized in that, A method for acquiring a stitched image of a culture dish includes the following steps: adjusting the height of the camera unit based on the distance between the culture dish and the object; horizontally translating the camera unit according to a movement vector to acquire local images of four regions of the culture dish: the left rear, right rear, left front, and right front; the adjacent local images partially overlap, and the four local images cover the upper side of the culture dish; performing distortion correction on the acquired local images based on the intrinsic parameters; and stitching the local images of the four regions of the culture dish—the left rear, right rear, left front, and right front—based on the movement vector and the intrinsic parameters to acquire a stitched image of the culture dish.

7. The method for synthesizing petri dish images according to claim 6, characterized in that, The camera unit is horizontally translated according to the movement vector to acquire local images of four regions of the petri dish: the left rear, right rear, left front, and right front. This includes the following steps: taking one-quarter of the petri dish diameter as the movement vector; aligning the center of the petri dish with the center of the camera's field of view as the starting point for the shot; based on the starting point, the camera unit is translated to the left and backward by the movement vector to acquire the left rear local image; based on the starting point, the camera unit is translated to the right and backward by the movement vector to acquire the right rear local image; based on the starting point, the camera unit is translated to the left and forward by the movement vector to acquire the left front local image; based on the starting point, the camera unit is translated to the right and forward by the movement vector to acquire the right front local image.

8. The method for synthesizing petri dish images according to claim 6, characterized in that, Obtaining a stitched image of a petri dish includes the following steps: calculating the corner coordinates required for stitching through affine transformation based on the translation relationship of the movement vectors of the local images and the intrinsic parameters of the camera, and establishing a geometric mapping relationship between the local images; stitching the local images based on the geometric mapping relationship; and smoothing the overlapping areas of the stitched local images using a feather mixer to obtain the stitched image of the petri dish.

9. A bacterial picker, characterized in that, This method is used to obtain stitched images of petri dishes according to any one of claims 1-8; it includes a camera unit, a bacteria-picking unit, a petri dish, a stage, and a calibration plate; the bacteria-picking unit is located on the upper side of the stage and is movable; the camera unit is fixedly connected to the bacteria-picking unit; the petri dish is placed on the upper side of the stage; the calibration plate is used to be placed inside the petri dish and to obtain internal parameter parameters through internal parameter calibration and to obtain stereo calibration parameters through stereo calibration.

10. A bacterial picker according to claim 9, characterized in that, The calibration plate is rectangular; the calibration plate is a dot array calibration plate with white dots arranged in a rectangular dot array on the upper side.

Images