Biological storage library control method and biological storage library

Abstract

The invention relates to the technical field of biomedical treatment, particularly provides a control method of a biological storage library and the biological storage library, and aims to solve the problem that subsequent operation is affected due to the fact that adhesion between a sample tube and a sample box cannot be recognized in time in the prior art. In order to achieve the purpose, the control method comprises the steps that the actual height of each sample tube in a sample box is obtained; comparing the actual height with a preset height threshold value; and when the actual height exceeds a preset height threshold value, judging that the sample tube is an abnormal tube and controlling the biological storage library to execute corresponding operation. According to the invention, the actual height of the sample tube in the sample box is firstly obtained and compared with the preset height threshold value, so that the adhered tube can be accurately and timely found, corresponding operation is carried out, the adhered tube can be found without waiting for grabbing again, and normal proceeding of the subsequent process is ensured.

Description

Technical Field

[0001] The present invention relates to the field of biomedical technology, and specifically provides a control method for a biostorage bank and a biostorage bank. Background Art

[0002] The -196-degree deep cryogenic biological sample storage library is an important basic equipment for current research in the medical and biological fields. Through low-temperature storage in liquid nitrogen, biological tissues such as blood, stem cells and immune cells can remain active for a long time.

[0003] Since the sample box is transferred between a deep cryogenic environment and a normal temperature environment, frost may form on the inner wall of the sample box, which may cause the sample tube to adhere to the wall of the sample box when placed on the sample box.

[0004] The existing operation can only be discovered when the sample tube is grabbed again, which will cause the robot arm to be unable to achieve stable grasping and unable to obtain the true position of the sample tube, thereby affecting subsequent normal use.

[0005] Therefore, the present invention needs to provide a new technical solution to solve the above technical problems. Summary of the invention

[0006] The present invention aims to solve the above technical problem, that is, to solve the problem that the adhesion between the sample tube and the sample box cannot be identified in time, which affects the subsequent operations.

[0007] In a first aspect, the present invention provides a control method for a biological storage repository, wherein the biological storage repository comprises a sample box, wherein a plurality of sample frames are arranged in the sample box, and a plurality of sample tubes are stored in the plurality of sample frames; after the plurality of sample tubes complete the tube placement operation, the control method comprises: obtaining the actual height of each sample tube in the sample box; comparing the actual height with a preset height threshold; when the actual height exceeds the preset height threshold, determining that the sample tube is an abnormal tube and controlling the biological storage repository to perform corresponding operations.

[0008] In a specific implementation of the above-mentioned control method of the biostorage repository, the step of "obtaining the actual height of each sample tube in the sample box" specifically includes: obtaining a 2D image and a 3D image of the sample box; obtaining first feature information based on the 2D image; wherein the first feature information includes position information of the sample frame corresponding to each sample tube; obtaining second feature information based on the 3D image; wherein the second feature information includes height information and corresponding position information of each sample tube; matching the first feature information with the second feature information to determine the position of each sample tube in the sample box and the corresponding actual height.

[0009] In a specific implementation of the control method of the above-mentioned biological storage library, the step of "obtaining first feature information according to the 2D image" specifically includes: inputting the 2D image into a trained image model for comparison to obtain the position information of each sample tube corresponding to the sample frame.

[0010] In a specific implementation of the above-mentioned biological storage repository control method, the step of "obtaining second characteristic information based on the 3D image" specifically includes: obtaining 3D point cloud data based on the 3D image; determining the ROI area based on the 3D point cloud data, and obtaining the height information and corresponding position information of each sample tube in the ROI area.

[0011] In a specific implementation of the above-mentioned biological storage repository control method, the step of "matching the first characteristic information with the second characteristic information to determine the position of each sample tube in the sample box and the corresponding actual height" specifically includes: calibrating the 2D image and the 3D image, and obtaining calibration parameters; mapping the position information of the sample tube to the corresponding position information of the sample box according to the calibration parameters; and determining the position of each sample tube in the sample box and the corresponding actual height according to the mapping result.

[0012] In a specific implementation of the control method of the above-mentioned biological storage library, a calibration point is also provided in the sample box; the step of "calibrating the external parameters of the coordinates of the 2D image and the 3D image, and obtaining the calibration parameters" specifically includes: obtaining the position information of the calibration point in the 2D image and the 3D image; calculating the calibration parameters according to the correspondence between the position information of the calibration point in the 2D image and the 3D image.

[0013] In a specific embodiment of the above-mentioned biological storage repository control method, the biological storage repository also includes a robotic arm, which is configured to perform tube picking or tube placing operations; the step of "controlling the biological storage repository to perform corresponding operations" specifically includes: controlling the robotic arm to grab the abnormal tube and re-place it in the corresponding sample frame.

[0014] In a specific implementation of the above biological storage repository control method, the step of "controlling the biological storage repository to perform corresponding operations" further includes: sending an abnormality reminder to the user.

[0015] In a specific implementation of the control method of the above-mentioned biostorage repository, the control method further comprises: acquiring a 3D image of the sample box while the robotic arm returns to an initial position.

[0016] In a second aspect, the present invention also provides a biological storage bank, comprising a controller, wherein the controller is configured to execute the control method as described above.

[0017] In the case of adopting the above technical solution, the present invention can obtain the actual height of each sample tube in the sample box; compare the actual height with the preset height threshold; when the actual height exceeds the preset height threshold, the sample tube is determined to be an abnormal tube and the biological storage repository is controlled to perform corresponding operations. The present invention first obtains the actual height of the sample tube in the sample box and compares it with the preset height threshold, so that the adhered tube can be accurately and timely found and the corresponding operation can be performed without waiting until it is grabbed again to be found, so as to ensure the normal progress of the subsequent process.

[0018] Furthermore, the present invention first adopts a deep learning method to roughly position the sample tube, and then adopts a laser algorithm to finely position the sample tube and obtain the height of the sample tube. By combining the deep learning algorithm with the laser algorithm, compared with the existing method, the efficiency is improved and the recognition time and recognition cost are shortened.

[0019] Furthermore, the present invention collects the 3D image of the sample box while the robot arm returns to the initial position, and integrates the timing of collecting the 3D image of the sample box into the operation of the equipment, which can improve the problem of a long time spent on separately setting a time for the 3D camera to collect images. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings, in which:

[0021] Figure 1 It is a schematic diagram of the structure of the sample box inside the tank in the biological storage bank of the present invention;

[0022] Figure 2 is a flow chart of the main steps of the control method of the biostorage bank of the present invention;

[0023] Figure 3 is a possible complete step flow chart of the control method of the biostorage bank of the present invention;

[0024] Among them, 1. workbench; 2. sample box; 21. sample frame; 22. sample tube; 3. calibration point. DETAILED DESCRIPTION

[0025] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not used to limit the scope of protection of the present invention. Those skilled in the art can make adjustments to them as needed to adapt to specific application scenarios.

[0026] It should be noted that in the description of the present invention, the terms "inside" and "above" and other terms indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings, which are only for the convenience of description, and do not indicate or imply that the relevant devices or elements must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, ordinal numbers such as "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0027] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0028] In addition, it should be noted that in the description of the present invention, although the various steps of the control method of the present invention are described in this application in a specific order, these orders are not restrictive. Without departing from the basic principles of the present invention, those skilled in the art can perform the steps in a different order.

[0029] See first Figure 1 , which is a schematic diagram of the structure of the sample box inside the tank in the biological storage library of the present invention. Figure 1 As shown, the biological storage repository includes a tank body (not shown in the figure), a sample box 2, a mechanical arm (not shown in the figure) and a gantry (not shown in the figure), a workbench 1 is arranged in the tank body, two sample boxes 2 are arranged and placed side by side on the workbench 1, a plurality of sample frames 21 are arranged in the sample box 2, and a plurality of sample tubes 22 are stored in the plurality of sample frames 21, the gantry is arranged above the tank body, the mechanical arm can be horizontally slidably arranged on the gantry, and the mechanical arm is arranged to be able to slide vertically to extend into the tank body to perform the operations of picking and placing tubes and transporting the sample box 2 in and out of the tank body. It should be noted that the number of sample boxes 2 and the side-by-side arrangement described above are only exemplary, and those skilled in the art can also change the placement of the sample boxes 2 and increase or decrease the number of sample boxes 2 as needed, and the present invention does not impose any restrictions thereon.

[0030] Then see Figure 2 , which is a flow chart of the main steps of the control method of the biological storage bank of the present invention. After the multiple sample tubes 22 are placed, the control method of the present invention includes:

[0031] S1 , obtaining the actual height of each sample tube 22 in the sample box 2 .

[0032] S2, comparing the actual height with a preset height threshold.

[0033] In one embodiment, the preset height threshold is a preset reasonable height threshold of the sample tube in the sample box.

[0034] S3, when the actual height exceeds the preset height threshold, the sample tube 22 is determined to be an abnormal tube and the biological storage library is controlled to perform corresponding operations.

[0035] In one embodiment, when the sample tube 22 and the sample box 2 are adhered to each other due to frost, the height of the sample tube 22 placed in the sample box 2 will be higher than the height of the sample tube 22 placed in the sample box 2 under normal circumstances. For this reason, the present invention first obtains the actual height of the sample tube 22 in the sample box 2 and compares it with the preset height threshold, so as to accurately and timely detect the adhered tube and perform corresponding operations without having to wait until the next grab to detect it, so as to ensure the normal progress of the subsequent process.

[0036] In step S1, the step of “obtaining the actual height of each sample tube 22 in the sample box 2” specifically includes:

[0037] S11, acquiring a 2D image and a 3D image of the sample box 2;

[0038] S12, acquiring first characteristic information according to the 2D image; wherein the first characteristic information includes position information of the sample frame 21 corresponding to each sample tube 22;

[0039] S13, acquiring second characteristic information according to the 3D image; wherein the second characteristic information includes height information of each sample tube 22 and corresponding position information;

[0040] S14, matching the first characteristic information with the second characteristic information to determine the position of each sample tube 22 in the sample box 2 and the corresponding actual height.

[0041] In step S11, a 2D camera is provided on the mechanical arm to collect 2D images, and a 3D laser radar camera is provided on the gantry to collect 3D images. It should be noted that the above-mentioned equipment used for collecting 2D images and 3D images is only exemplary, and those skilled in the art may also use other collection equipment to collect images, as long as it can obtain the 2D image and 3D image of the sample box 2, and the present invention does not impose any limitation on this.

[0042] Further, specifically, each time after the robotic arm grabs a sample tube 22 and places it in the sample box 2, the 2D camera takes a photo to collect a 2D image of the sample box 2. After the last sample tube 22 is placed in the sample box 2, the 3D lidar camera takes a photo to collect a 3D image of the sample box 2 while the robotic arm returns to its initial position. It should be noted that the above timing for collecting 2D images and 3D images is not restrictive, and those skilled in the art can select a suitable timing for image collection according to actual needs. Preferably, the timing for collecting the 3D image of the sample box 2 is integrated into the operation of the device, which can improve the problem of a long time spent on separately setting a time for the 3D lidar camera to collect images.

[0043] In step S12, the step of “obtaining first feature information according to the 2D image” specifically includes:

[0044] S121 , input the 2D image into the trained image model for comparison, and obtain the position information of each sample tube 22 corresponding to the sample frame 21 .

[0045] In some embodiments, the image model is an image model trained by a deep learning algorithm based on sufficient sample images collected in advance, and the acquired sample images are input into the image model for comparison to obtain the position information of the sample frame 21 corresponding to each sample tube 22. A two-dimensional rectangular coordinate system is established with the upper left corner of the 2D image as the origin, the direction from the origin to the lower left corner as the positive direction of the Y axis, and the direction from the origin to the upper right corner as the positive direction of the X axis. The position information of the sample frame 21 specifically refers to the length and width of the sample frame 21 and the position coordinates (X, Y) of the intersection of the frame of the sample frame 21. It should be noted that if the sample frame 21 is a rectangle, the position information is the length and width of the rectangle and the pixel coordinates of the intersection of the rectangular frame. If the sample frame 21 is a circle, the position information is the length and width of the rectangular frame corresponding to the circle and the position coordinates of the intersection of the rectangular frame.

[0046] Furthermore, the sample images described above are sample images of sample tubes 22 stored in different positions on each type of sample box 2, which are collected in advance in a preset number. In the prior art, there are many types of sample boxes 2 for storing sample tubes 22. Sample tubes 22 stored in different positions of each type of sample box 2 are photographed to obtain sample images. Here, different positions refer to the different positions of the sample tubes 22 on the sample box 2 in each sample image. The number of sample tubes 22 in each sample image is not limited in this application and can be set arbitrarily. The placement position of the sample tubes 22 can also be set arbitrarily. Regarding the preset number of sample images recorded above, that is, each type of sample box 2 needs to take a preset number of pictures, and the preset number of pictures can be 100, 200, 500, etc. The specific value of the preset number of pictures is flexibly set according to the accuracy and difficulty of recognition by the trained image model.

[0047] In step S13, the step of “obtaining second characteristic information according to the 3D image” specifically includes:

[0048] S131, acquiring 3D point cloud data within the ROI area according to the 3D image;

[0049] S132, obtaining the height information and corresponding position information of each sample tube 22 according to the 3D point cloud data.

[0050] In some embodiments, the determination of the ROI region (region of interest) in the 3D image is directly set by the 3D camera, that is, the 3D camera pre-sets the ROI region before taking a picture. Figure 1 For example, the ROI area of ​​the present invention is the area of ​​two sample boxes 2. It should be noted that the above-mentioned ROI area is only exemplary, and those skilled in the art can set it according to actual needs, and the present invention does not impose any limitation hereon.

[0051] Furthermore, the 3D point cloud data is a data point set containing coordinate information (X, Y, Z), and a three-dimensional rectangular coordinate system is established with the upper left corner of the 3D image as the origin, the direction from the origin to the lower left corner as the positive direction of the Y axis, the direction from the origin to the upper right corner as the positive direction of the X axis, and the vertical direction from the origin as the positive direction of the Z axis. The corresponding information is obtained based on the above data point set, and the position information of the sample tube 22 is the position coordinate (X mix , Y mix , Z mix ), the height information is the position coordinate (X max , Y max , Z max )The Z-axis coordinate value obtained: Z max .

[0052] In step S14, the step of “matching the first characteristic information with the second characteristic information to determine the position of each sample tube 22 in the sample box 2 and the corresponding actual height” specifically includes:

[0053] S141, calibrating the 2D image and the 3D image, and obtaining calibration parameters;

[0054] S142, mapping the position information of the sample tube 22 to the position information of the corresponding sample frame 21 according to the calibration parameters;

[0055] S143, determining the position of each sample tube 22 in the sample box 2 and the corresponding actual height according to the mapping result.

[0056] Furthermore, a calibration point 3 is also provided in the sample box 2; in step S141, the step of "calibrating the 2D image and the 3D image and obtaining the calibration parameters" specifically includes:

[0057] S1411, obtaining position information of the calibration point 3 in the 2D image and the 3D image;

[0058] S1412: Calculate calibration parameters according to the correspondence between the position information of the calibration point 3 in the 2D image and the 3D image.

[0059] In some embodiments, Figure 1 As shown, a calibration point 3 is set on the workbench 1 between the two sample boxes 2. The calibration point 3 is set as a circular opening with a preset known depth. The depth of the calibration point 3 is the deepest depth in the ROI area. A 2D camera is used to capture a 2D image of the calibration point 3 to obtain the diameter of the calibration point 3 and the position coordinates (X m , Y m ), a 3D laser radar camera is used to collect a 3D image of the calibration point 3 to obtain the height information and position information of the calibration point 3, wherein the position information is the center point position coordinate (X nmix , Y nmix , Z nmix ), the height information is based on the center point position coordinates of the highest point (X nmax , Y nmax , Z nmax )The Z-axis coordinate value obtained: Z nmax Since the depth of calibration point 3 is known, the position coordinate (X m , Y m ) and position coordinates (X nmix , Y nmix , Z nmix ) in the X coordinate value and the Y coordinate value, respectively according to X m With Xnmix , Y m With Y nmix The corresponding calibration parameters are calculated based on the mapping relationship between them. It should be noted that the above-described setting position and setting shape of the calibration point 3 are only exemplary, and those skilled in the art can also set other shapes, such as rectangular openings, and other setting positions according to actual needs, as long as they are set within the ROI area, and the present invention does not impose any limitation on this.

[0060] In addition, the above-mentioned use of a 2D camera to collect a 2D image of the calibration point 3 to obtain relevant information and the use of a 3D lidar camera to collect a 3D image of the calibration point 3 to obtain relevant information are performed simultaneously with the separate collection of the 2D image and 3D image of the sample tube 22, and are the same as the method for obtaining the sample tube related information, and the present invention will not be repeated here.

[0061] In the above steps, the deep learning method is first used to roughly position the sample tube 22, and then the laser algorithm is used to finely position the sample tube 22 and obtain the height of the sample tube 22. By combining the deep learning algorithm with the laser algorithm, compared with the existing method, the efficiency is improved and the recognition time and recognition cost are shortened.

[0062] In step S3, the step of "controlling the biological storage bank to perform corresponding operations" specifically includes:

[0063] S31, controlling the robot arm to grab the abnormal tube and re-place it in the corresponding sample frame 21;

[0064] S32, sending an abnormality reminder to the user.

[0065] In the above steps, the abnormal pipe can be processed in time and the user can be reminded to avoid affecting subsequent operations and use.

[0066] Refer to the following Figure 3 , which is a possible complete step flow chart of the control method of the biological storage bank of the present invention. Figure 3 As shown, after the multiple sample tubes 22 complete the tube placement operation, the control method of the present invention includes:

[0067] S101, acquiring a 2D image and a 3D image of the sample box 2;

[0068] S102, acquiring first characteristic information according to the 2D image; the first characteristic information includes position information of the sample frame 21 corresponding to each sample tube 22;

[0069] S103, acquiring second characteristic information according to the 3D image; the second characteristic information includes height information of each sample tube 22 and corresponding position information;

[0070] S104, matching the first characteristic information with the second characteristic information to determine the position of each sample tube 22 in the sample box 2 and the corresponding actual height;

[0071] S105, comparing the actual height with a preset height threshold;

[0072] S106, when the actual height exceeds the preset height threshold, the robot arm is controlled to grab the abnormal tube and re-place it in the corresponding sample frame 21, and an abnormal reminder is sent to the user.

[0073] In summary, in the preferred technical solution of the present invention, the control method of the present invention includes: obtaining the actual height of each sample tube 22 in the sample box 2; comparing the actual height with a preset height threshold; when the actual height exceeds the preset height threshold, the sample tube 22 is determined to be an abnormal tube and the biological storage repository is controlled to perform corresponding operations. The present invention first obtains the actual height of the sample tube 22 in the sample box 2 and compares it with the preset height threshold, so that the adhered tube can be accurately and timely found and the corresponding operation can be performed without waiting until it is grabbed again to be found, so as to ensure the normal progress of the subsequent process.

[0074] Based on the above embodiments, the present invention further provides a biological storage repository, comprising a controller, wherein the controller is configured to execute the above control method.

[0075] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for controlling a biostorage bank, characterized in that: The biological storage repository comprises a sample box, wherein a plurality of sample frames are arranged in the sample box, and a plurality of sample tubes are stored in the plurality of sample frames; After the plurality of sample tubes complete the tube placing operation, the control method includes: Obtaining the actual height of each sample tube in the sample box; Comparing the actual height with a preset height threshold; When the actual height exceeds a preset height threshold, the sample tube is determined to be an abnormal tube and the biological storage repository is controlled to perform corresponding operations.

2. The control method of the biostorage bank according to claim 1, characterized in that: The step of "obtaining the actual height of each sample tube in the sample box" specifically includes: Acquire a 2D image and a 3D image of the sample box; Acquire first feature information according to the 2D image; wherein the first feature information includes position information of the sample frame corresponding to each sample tube; Acquire second feature information according to the 3D image; wherein the second feature information includes height information and corresponding position information of each sample tube; The first characteristic information is matched with the second characteristic information to determine the position of each sample tube in the sample box and the corresponding actual height.

3. The control method of the biostorage bank according to claim 2, characterized in that: The step of “obtaining first feature information according to the 2D image” specifically includes: The 2D image is input into a trained image model for comparison to obtain the position information of each sample tube corresponding to the sample frame.

4. The control method of the biostorage bank according to claim 2, characterized in that: The step of “obtaining second characteristic information according to the 3D image” specifically includes: Acquire 3D point cloud data according to the 3D image; The ROI area is determined according to the 3D point cloud data, and the height information and the corresponding position information of each sample tube in the ROI area are obtained.

5. The control method of the biostorage bank according to claim 2, characterized in that: The step of "matching the first characteristic information with the second characteristic information to determine the position of each sample tube in the sample box and the corresponding actual height" specifically includes: Calibrate the 2D image and the 3D image, and obtain calibration parameters; Mapping the position information of the sample tube to the corresponding position information of the sample frame according to the calibration parameters; The position of each sample tube in the sample box and the corresponding actual height are determined according to the mapping result.

6. The control method of the biostorage bank according to claim 5, characterized in that: The sample box is also provided with calibration points; The step of “calibrating the external parameters of the coordinates of the 2D image and the 3D image, and obtaining the calibration parameters” specifically includes: Acquire position information of the calibration point in the 2D image and the 3D image; The calibration parameters are calculated according to the corresponding relationship between the position information of the calibration points in the 2D image and the 3D image.

7. The control method of the biostorage bank according to claim 1, characterized in that: The biostorage repository further comprises a robotic arm, which is configured to perform a tube picking or tube placing operation; The step of "controlling the biological storage bank to perform corresponding operations" specifically includes: The robot arm is controlled to grab the abnormal tube and re-place it in the corresponding sample frame.

8. The control method of the biostorage bank according to claim 7, characterized in that: The step of "controlling the biological storage bank to perform corresponding operations" further includes: Send abnormal reminders to users.

9. The control method of the biostorage bank according to claim 2, characterized in that: The control method further comprises: While the robot arm returns to the initial position, a 3D image of the sample box is acquired.

10. A biostorage storage device, characterized in that: The device comprises a controller configured to execute the control method according to any one of claims 1 to 9.

Images