A method for automatically capturing biological particles with optical tweezers based on machine vision

Abstract

The present invention discloses a method for automatically capturing biological particles using optical tweezers based on machine vision, which relates to the field of optical tweezers capturing technology and includes the following steps: obtaining a first threshold value and a second threshold value based on a grayscale image of the biological particles; converting the grayscale image of the biological particles into a binary image of the biological particles based on the first threshold value and the second threshold value; obtaining an independent biological particle region based on the binary image of the biological particles; obtaining the center coordinates of the first particle based on the biological particle region; obtaining the center coordinates of the second particle based on the biological particle region; obtaining the final particle coordinates based on the first particle center coordinates and the second particle center coordinates, and capturing the biological particles based on the final particle coordinates and optical tweezers. The present invention is used to solve the problem that the existing optical tweezers capturing technology fails to identify the exact center position of the biological particles and fails to adjust the diffraction center of the optical tweezers to the exact center position of the biological particles, resulting in the optical tweezers' capturing ability failing to reach the optimal level.

Description

Technical Field

[0001] The present invention relates to the field of optical tweezers capturing technology, and in particular to a method for automatically capturing biological particles with optical tweezers based on machine vision. Background Art

[0002] Optical tweezers use the photopressure or optomechanical effects of laser beams to precisely manipulate tiny objects. In a dual optical trap, the optical trap closer to the optical axis has a greater three-dimensional stiffness than the optical trap slightly further away. This means that the closer to the diffraction center, the higher the diffraction efficiency of light, and the stronger the trapping ability of the resulting optical trap. Therefore, after obtaining the precise center position of a biological particle, the diffraction center of the optical tweezers is adjusted to the corresponding precise center position of the biological particle to ensure a stronger trapping ability.

[0003] When capturing biological particles, the existing optical tweezers are simply moved to the biological particles without obtaining the exact center coordinates of the biological particles. Then, the diffraction center of the optical tweezers is adjusted to the exact center coordinates of the biological particles, so that the optical trap capture ability fails to reach the strongest level. That is, the existing optical tweezers capture technology fails to identify the exact center position of the biological particles, and fails to adjust the diffraction center of the optical tweezers to the exact center position of the biological particles, resulting in the optical tweezers capture ability not reaching the optimal level. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems in the prior art to a certain extent, and obtains an independent biological particle area based on a binary image of biological particles; obtains the center coordinates of a first particle based on the biological particle area; obtains the center coordinates of a second particle based on the biological particle area; obtains the final particle coordinates based on the center coordinates of the first particle and the center coordinates of the second particle, and captures the biological particles based on the final particle coordinates and optical tweezers, so as to solve the problem that the existing optical tweezers capture technology fails to identify the exact center position of the biological particles and fails to adjust the diffraction center of the optical tweezers to the exact center position of the biological particles, resulting in the optical tweezers' capture ability failing to reach the optimal level.

[0005] To achieve the above objectives, the present application provides a method for automatically capturing biological particles using optical tweezers based on machine vision, comprising the following steps:

[0006] Acquire images of biological particles;

[0007] Grayscale processing is performed on the biological particle image to obtain a biological particle grayscale image;

[0008] Obtaining a first threshold and a second threshold based on the biological particle grayscale image;

[0009] Converting the biological particle grayscale image into a biological particle binary image based on a first threshold and a second threshold;

[0010] Obtain independent bioparticle regions based on the bioparticle binary image;

[0011] Obtaining the center coordinates of a first particle based on the biological particle region;

[0012] obtaining the center coordinates of a second particle based on the biological particle region;

[0013] The final particle coordinates are obtained based on the first particle center coordinates and the second particle center coordinates, and the biological particles are captured based on the final particle coordinates and optical tweezers.

[0014] Furthermore, grayscale processing is performed on the biological particle image to obtain a biological particle grayscale image, including the following sub-steps:

[0015] Obtain the RGB value of each pixel in the biological particle image and mark it as the image RGB value;

[0016] All the image RGB values ​​are converted into grayscale values ​​using the grayscale conversion formula to obtain the grayscale image of biological particles;

[0017] The grayscale value of the pixel in the biological particle grayscale image is marked as the biological particle grayscale value.

[0018] Furthermore, obtaining the first threshold and the second threshold based on the biological particle grayscale image includes the following sub-steps:

[0019] Divide the grayscale value from 0 to 255 into a equal ranges, marked as equal ranges;

[0020] In the light guide plate grayscale image, the frequency of the grayscale value of biological particles in each equal division range is counted and marked as equal division frequency;

[0021] A histogram is drawn with the biological particle grayscale value as the X-axis, the equal-division frequency as the Y-axis, and the equal-division range as the histogram interval, and is marked as the particle grayscale histogram.

[0022] Furthermore, obtaining the first threshold and the second threshold based on the biological particle grayscale image further includes the following sub-steps:

[0023] In the particle grayscale histogram, the equal-division frequency that is larger than the two adjacent equal-division frequencies on the left and right sides is marked as the peak frequency;

[0024] Mark the equally divided range corresponding to the peak frequency as the peak range;

[0025] Get the number of peak ranges. If the number of peak ranges is less than 2, increase the specific value of a and rebuild the particle grayscale histogram.

[0026] If the number of peak ranges is greater than or equal to 2, obtain the maximum value of each peak range and mark it as the peak maximum value, obtain the peak range with the smallest peak maximum value and mark it as the first range, and obtain the peak range with the largest peak maximum value and mark it as the second range;

[0027] In the particle grayscale histogram, obtain the equal division range with the smallest equal division frequency between the first range and the second range, and mark it as the third range;

[0028] In the particle grayscale histogram, obtain the equal-division range with the smallest equal-division frequency greater than the second range and mark it as the fourth range. If there is no equal-division range with the smallest equal-division frequency greater than the second range, mark the second range as the fourth range.

[0029] Obtain the mean of the third range, marked as the first threshold;

[0030] The maximum value of the fourth range is obtained and marked as the second threshold.

[0031] Furthermore, converting the biological particle grayscale image into a biological particle binary image based on the first threshold and the second threshold includes the following sub-steps:

[0032] The grayscale values ​​of biological particles in the biological particle grayscale image that are greater than or equal to the first threshold and less than or equal to the second threshold are set to 0, and the grayscale values ​​of biological particles in the biological particle grayscale image that are less than the first threshold or greater than the second threshold are set to 255 to obtain a biological particle binary image.

[0033] Furthermore, obtaining independent biological particle regions based on the biological particle binary image includes the following sub-steps:

[0034] Mark the pixel with gray value 0 in the bioparticle binary image as the initial pixel;

[0035] Get any processing pixel point, get a pixel point in eight directions with the processing pixel point as the center, and mark it as the first search pixel point;

[0036] Determine whether the first search pixel is the initial pixel. If so, obtain a pixel in eight directions with the initial pixel in the first search pixel as the center, and mark it as the second search pixel. Determine whether the second search pixel is a newly appeared initial pixel. If so, obtain a pixel in eight directions with the newly appeared initial pixel in the second search pixel as the center. Repeat the search until no new initial pixel appears. Complete a search. Obtain a set of all initial pixels found in the search and mark it as a separate area.

[0037] Repeat the process of searching multiple times starting from any unsearched processed pixel point until all initial pixel points in the biological particle binary image are searched.

[0038] Furthermore, obtaining independent biological particle regions based on the biological particle binarization image further includes the following sub-steps:

[0039] Acquire a first number of biological particle images, mark them as historical particle images, and acquire the number of pixels occupied by each biological particle in the historical particle images, mark them as historical pixel numbers;

[0040] Get the minimum value of the number of historical pixels and mark it as the particle number threshold;

[0041] Obtain the number of each individual region; determine whether the number of each individual region is less than the particle number threshold; if not, mark the individual region as a biological particle region.

[0042] Furthermore, obtaining the center coordinates of the first particle based on the biological particle region includes the following sub-steps:

[0043] Establishing a plane rectangular coordinate system, marked as a position determination coordinate system, and placing a biological particle area in the position determination coordinate system;

[0044] Obtain the maximum and minimum values ​​of the horizontal coordinate of the bioparticle area, marked as Cmax and Cmin respectively;

[0045] Obtain the maximum and minimum values ​​of the vertical coordinate of the biological particle area, marked as Dmax and Dmin respectively;

[0046] The abscissa of the center coordinate of the first particle is calculated as: E1=(Cmax+Cmin) / 2; E1 is the abscissa of the center coordinate of the first particle;

[0047] The ordinate of the center coordinate of the first particle is calculated as: E2=(Dmax+Dmin) / 2; E2 is the ordinate of the center coordinate of the first particle.

[0048] Furthermore, obtaining the center coordinates of the second particle based on the biological particle region includes the following sub-steps:

[0049] Obtaining coordinate points of edges of a second number of biological particle regions, marking them as edge coordinate points;

[0050] The fitting profile equation is: f 1×T1 2 +f 2×T1×T2+f 3×T2 2 +f4×T1+f5×T2+f6=0; where f1, f2, f3, f4, f5, and f6 are coefficients of the equation, T1 is the abscissa of the edge coordinate point, and T2 is the ordinate of the edge coordinate point;

[0051] Fit all edge coordinate points according to the fitting contour equation to obtain the specific values ​​of the coefficients of the equations f1, f2, f3, f4, f5 and f6;

[0052] The horizontal coordinate of the center coordinate of the second particle is: E3 = (2 × f 3 × f 4 - f 2 × f 5) / (f 2 2 -4×f 1×f 3); E3 is the abscissa of the center coordinate of the second particle;

[0053] The vertical coordinate of the center coordinate of the first particle is: E4 = (2 × f 1 × f 5 - f 2 × f 4) / (f 2 2 -4×f1×f3); E4 is the ordinate of the center coordinate of the second particle.

[0054] Furthermore, obtaining final particle coordinates based on the first particle center coordinates and the second particle center coordinates, and capturing the biological particle based on the final particle coordinates and the optical tweezers includes the following sub-steps:

[0055] The final particle coordinates are obtained as: [(E1+E3) / 2, (E2+E4) / 2]; the optical tweezers are adjusted to the position of the final particle coordinate point to capture the biological particle.

[0056] The beneficial effects of the present invention are as follows: the present invention obtains an independent biological particle region based on a biological particle binary image; obtains the first particle center coordinate based on the biological particle region; obtains the second particle center coordinate based on the biological particle region; obtains the final particle coordinate based on the first particle center coordinate and the second particle center coordinate, and captures the biological particle based on the final particle coordinate and optical tweezers. The advantage of the present invention is that the first particle center coordinate and the second particle center coordinate can be obtained by two methods based on the shape of the biological particle. If the biological particle has a sharp point protrusion, the first particle center coordinate will be biased toward the side of the sharp point of the contour, and the second particle center coordinate will be biased toward the side of the non-sharp point of the contour. The first particle center coordinate and the second particle center coordinate are combined with the center coordinate position to make the obtained center coordinate more accurate;

[0057] The present invention obtains the second particle center coordinates based on the biological particle area. The advantage is that the biological particles are close to being circular or elliptical. The center of the biological particle is obtained by fitting the center of the circle and ellipse through the biological particle outline. The center position of the biological particle is obtained using a method with less calculation data, and the accuracy of the obtained biological particle center position is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a flow chart of the steps of the method of the present invention;

[0059] Figure 2 is a schematic diagram of a particle grayscale histogram of the present invention;

[0060] Figure 3 is a schematic diagram of the bioparticle region of the present invention;

[0061] Figure 4 Schematic diagram of the fitting profile equation of the present invention. DETAILED DESCRIPTION

[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0063] Example 1, please refer to Figure 1 As shown, the present application provides a method for automatically capturing biological particles using optical tweezers based on machine vision, comprising the following steps:

[0064] Step S1, acquiring a biological particle image;

[0065] Step S2, grayscale processing is performed on the biological particle image to obtain a biological particle grayscale image; step S2 also includes the following sub-steps:

[0066] Step S201, obtaining the RGB value of each pixel in the biological particle image, and marking it as the image RGB value;

[0067] Step S202: converting all RGB values ​​in the image into grayscale values ​​using a grayscale conversion formula to obtain a grayscale image of the biological particles; wherein the grayscale conversion formula is to calculate the average of the R value, G value, and B value of the RGB values;

[0068] In actual applications, for example, if the RGB value is (230, 231, 232), the grayscale value is (230+231+232) / 3=231;

[0069] Step S203 : marking the grayscale value of the pixel in the biological particle grayscale image as the biological particle grayscale value.

[0070] Step S3, obtaining a first threshold and a second threshold based on the biological particle grayscale image; Step S3 also includes the following sub-steps:

[0071] Step S301: Divide the grayscale values ​​from 0 to 255 into a equal ranges, marked as equal ranges. The value a is set to distinguish the distribution of grayscale values. Therefore, it should not be too small. At the same time, in order to reduce the amount of calculation, it should not be too large. For example, a is set to 8, which can distinguish the distribution of grayscale values ​​without causing excessive calculation.

[0072] Step S302, counting the frequency of the grayscale value of the biological particles in each equally divided range in the grayscale map of the light guide plate, and marking it as the equally divided frequency;

[0073] Step S303, plotting a histogram with the biological particle grayscale value as the X-axis, the equally divided frequency as the Y-axis, and the equally divided range as the histogram interval, and marking it as a particle grayscale histogram;

[0074] Step S304: Mark the equal-division frequency that is greater than the two adjacent equal-division frequencies on the left and right sides as the peak frequency in the particle grayscale histogram;

[0075] Step S305, marking the equally divided range corresponding to the peak frequency as the peak range;

[0076] Step S306: Obtain the number of peak ranges. If the number of peak ranges is less than 2, increase the specific value of a and rebuild the particle grayscale histogram. Because the particle grayscale histogram includes the background and biological particles, and biological particles appear nearly white under the light source, while the background is black, the grayscale values ​​of the biological particles and the background are not within the same range, resulting in two peak ranges. If no peak ranges appear, it indicates that the setting of a is too small, resulting in the two groups of grayscale values ​​being within the same equally divided range, so a needs to be increased.

[0077] Step S307: If the number of peak ranges is greater than or equal to 2, obtain the maximum value of each peak range and mark it as the peak maximum value; obtain the peak range with the smallest peak maximum value and mark it as the first range; obtain the peak range with the largest peak maximum value and mark it as the second range;

[0078] Step S308: in the particle grayscale histogram, obtain the equally divided range with the smallest equally divided frequency between the first range and the second range, and mark it as the third range;

[0079] Step S309: In the particle grayscale histogram, obtain the equal-division range with the smallest equal-division frequency greater than the second range and mark it as the fourth range. If there is no equal-division range with the smallest equal-division frequency greater than the second range, mark the second range as the fourth range.

[0080] Step S310: Obtain the mean value of the third range, which is marked as the first threshold value; the middle value between the two data is the most appropriate;

[0081] Step S311, obtaining the maximum value of the fourth range, marking it as the second threshold; selecting the maximum value as the unilateral threshold;

[0082] In practical applications, please refer to Figure 2As shown, the peak ranges obtained are 0 to 31 and 223 to 255, that is, the first sub-range is 0 to 31, the second range is 223 to 255, and the sub-range with the smallest sub-frequency between 0 to 31 and 223 to 255 is 127 to 159. In the particle grayscale histogram, there is no sub-range with the smallest sub-frequency greater than 223 to 255. The second range is marked as the fourth range, that is, the fourth frequency is 223 to 255. Therefore, the first threshold is: (127+159) / 2=142, and the second threshold is: 255.

[0083] Step S4, converting the biological particle grayscale image into a biological particle binary image based on the first threshold and the second threshold; Step S4 also includes the following sub-steps:

[0084] Step S401, setting the grayscale values ​​of biological particles in the biological particle grayscale image that are greater than or equal to a first threshold and less than or equal to a second threshold to 0, and setting the grayscale values ​​of biological particles in the biological particle grayscale image that are less than the first threshold or greater than the second threshold to 255, to obtain a binary image of biological particles;

[0085] In practical applications, the grayscale values ​​of biological particles greater than or equal to 142 and less than or equal to 255 in the biological particle grayscale image are set to 0, and the grayscale values ​​of biological particles less than 142 or greater than 255 in the biological particle grayscale image are set to 255 to obtain a biological particle binary image. Since the image only contains the distribution of biological particles and the background, the biological particle image is close to white, and the grayscale value distribution range of the biological particles is relatively large.

[0086] Step S5, obtaining independent biological particle regions based on the biological particle binary image; Step S5 also includes the following sub-steps:

[0087] Step S501, marking pixels with a gray value of 0 in the biological particle binary image as initial pixels;

[0088] Step S502: obtaining any processing pixel point, obtaining a pixel point in eight directions with the processing pixel point as the center, and marking it as the first search pixel point;

[0089] Step S503: Determine whether the first search pixel is an initial pixel. If so, obtain a pixel in eight directions with the initial pixel in the first search pixel as the center, and mark it as a second search pixel. Determine whether the second search pixel is a newly appeared initial pixel. If so, obtain a pixel in eight directions with the newly appeared initial pixel in the second search pixel as the center. Repeat the search until no new initial pixel appears. Complete the search once, obtain a set of all the initial pixels found in the search, and mark it as a separate region. This method can traverse the entire image, and adjacent initial pixels are grouped into one region.

[0090] Step S504, repeating the search multiple times starting from any unsearched processed pixel point until all initial pixel points in the biological particle binary image are searched;

[0091] Step S505: obtaining a first number of biological particle images, marking them as historical particle images, obtaining the number of pixels occupied by each biological particle in the historical particle images, and marking them as the number of historical pixels; the first number is set to the number of pixels that can be found to be occupied by the biological particles, for example, the first number is set to 100;

[0092] Step S506: Obtain the minimum value of the number of historical pixels and mark it as the particle number threshold. Interference points may exist in the binary image, so further screening of individual areas is required. Since interference pixels are usually dispersed, the number of historical pixels in this case will be small. Therefore, screening can be performed based on the number of pixels occupied by the historical biological particle grayscale values, so the particle number threshold is set.

[0093] Step S507, obtaining the number of each individual region; determining whether the number of each individual region is less than a particle number threshold; if not, marking the individual region as a biological particle region; obtaining the biological particle region in the individual region according to the particle number threshold.

[0094] Step S6, obtaining the center coordinates of the first particle based on the biological particle region; Step S6 also includes the following sub-steps:

[0095] Step S601: establishing a plane rectangular coordinate system, marked as a position determination coordinate system, and placing a biological particle area in the position determination coordinate system;

[0096] Step S602, obtaining the maximum and minimum values ​​of the horizontal coordinate of the biological particle area, marked as Cmax and Cmin respectively;

[0097] Step S603, obtaining the maximum and minimum values ​​of the vertical coordinate of the biological particle area, marked as Dmax and Dmin respectively;

[0098] Step S604 , calculating the abscissa of the center coordinate of the first particle as: E1=(Cmax+Cmin) / 2; E1 is the abscissa of the center coordinate of the first particle;

[0099] Step S605 , calculating the vertical coordinate of the center coordinate of the first particle as: E2=(Dmax+Dmin) / 2; E2 is the vertical coordinate of the center coordinate of the first particle;

[0100] In actual application, please participate Figure 3As shown, the maximum and minimum values ​​of the horizontal coordinate of the biological particle area are 60nm and 22nm respectively, and the maximum and minimum values ​​of the vertical coordinate of the biological particle area are 58nm and 38nm respectively. The coordinates of the center of the first particle are: (41, 48).

[0101] Step S7, obtaining the center coordinates of the second particle based on the biological particle region; Step S7 also includes the following sub-steps:

[0102] Step S701: Obtain a second number of coordinate points at the edge of the biological particle region and mark them as edge coordinate points. The second number is set to roughly visualize the biological particle region through the edge coordinate points. The edge coordinate points are randomly distributed on the edge of the biological particle region. For example, the second number is set to 13.

[0103] Step S702: Get the fitting profile equation: f1×T1 2 +f 2×T1×T2+f 3×T2 2 +f4×T1+f5×T2+f6=0; where f1, f2, f3, f4, f5, and f6 are coefficients of the equation, T1 is the abscissa of the edge coordinate point, and T2 is the ordinate of the edge coordinate point; because most biological particles are circular or elliptical, and when biological particles move, the acquired image will be elongated, resulting in a shape closer to an ellipse, so this equation can be used for fitting;

[0104] Step S703: Fit all edge coordinate points according to the fitting contour equation to obtain specific values ​​of the coefficients of the equations f1, f2, f3, f4, f5, and f6;

[0105] Step S704: Obtain the horizontal coordinate of the center of the second particle: E3 = (2 × f3 × f4 - f2 × f5) / (f2 2 -4×f 1×f 3); E3 is the abscissa of the center coordinate of the second particle;

[0106] Step S705: Obtain the vertical coordinate of the center coordinate of the first particle: E4 = (2 × f1 × f5-f2 × f4) / (f2 2 -4×f1×f3); E4 is the ordinate of the center coordinate of the second particle; the center of the ellipse or the center of the circle.

[0107] Step S8, obtaining final particle coordinates based on the first particle center coordinates and the second particle center coordinates, and capturing the biological particle based on the final particle coordinates and the optical tweezers; Step S8 also includes the following sub-steps:

[0108] Step S801, obtaining the final particle coordinates: [(E1+E3) / 2, (E2+E4) / 2]; adjusting the optical tweezers to the final particle coordinate point position to capture the biological particle;

[0109] In actual application, please participate Figure 4 As shown, f1, f2, f3, f4, f5 and f6 are 100, 0, 324, -8400, -31104 and 890496 respectively; then E3 is obtained to be 42, E4 is 48, in order to obtain the final particle coordinates: [(41+42) / 2, (48+48) / 2], that is, the final particle coordinates are (41.5, 48).

[0110] In a second embodiment, the present application further provides an electronic device, which may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The memory stores computer-readable instructions, and the processor can call the instructions in the memory. When the computer-readable instructions are executed by the processor, the steps in the method for automatically capturing biological particles using optical tweezers based on machine vision are executed to implement the following functions: acquiring a biological particle image; grayscale processing the biological particle image to obtain a biological particle grayscale image; obtaining a first threshold and a second threshold based on the biological particle grayscale image; converting the biological particle grayscale image into a biological particle binary image based on the first threshold and the second threshold; acquiring an independent biological particle region based on the biological particle binary image; acquiring a first particle center coordinate based on the biological particle region; acquiring a second particle center coordinate based on the biological particle region; acquiring a final particle coordinate based on the first particle center coordinate and the second particle center coordinate, and capturing the biological particle based on the final particle coordinate and the optical tweezers.

[0111] In addition, the logical instructions in the above-mentioned memory can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0112] Example 3. The present application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the method of automatically capturing biological particles with optical tweezers based on machine vision provided by the above methods, the method including: obtaining a biological particle image; grayscale processing the biological particle image to obtain a biological particle grayscale image; obtaining a first threshold and a second threshold based on the biological particle grayscale image; converting the biological particle grayscale image into a biological particle binary image based on the first threshold and the second threshold; obtaining an independent biological particle area based on the biological particle binary image; obtaining the first particle center coordinates based on the biological particle area; obtaining the second particle center coordinates based on the biological particle area; obtaining the final particle coordinates based on the first particle center coordinates and the second particle center coordinates, and capturing the biological particles based on the final particle coordinates and the optical tweezers.

[0113] Example 4. The present application also provides a computer-readable storage medium. The present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method of automatically capturing biological particles by optical tweezers based on machine vision are executed to achieve the following functions: obtaining a biological particle image; grayscale processing of the biological particle image to obtain a biological particle grayscale image; obtaining a first threshold and a second threshold based on the biological particle grayscale image; converting the biological particle grayscale image into a biological particle binary image based on the first threshold and the second threshold; obtaining an independent biological particle area based on the biological particle binary image; obtaining the first particle center coordinates based on the biological particle area; obtaining the second particle center coordinates based on the biological particle area; obtaining the final particle coordinates based on the first particle center coordinates and the second particle center coordinates, and capturing the biological particles based on the final particle coordinates and the optical tweezers.

[0114] Through the description of the above embodiments, the embodiments of the present invention can be provided as methods, systems, or computer program products. Based on this understanding, the essence of the above technical solutions or the portion that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (such as a personal computer, server, or network device) to execute the methods described in various embodiments or certain portions of the embodiments.

[0115] In the embodiments provided in this application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of systems, modules and units can be electrical, mechanical or other forms.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for automatically capturing biological particles using optical tweezers based on machine vision, characterized in that: The steps include: Acquire images of biological particles; Grayscale processing is performed on the biological particle image to obtain a biological particle grayscale image; Obtaining a first threshold and a second threshold based on the biological particle grayscale image; Converting the biological particle grayscale image into a biological particle binary image based on a first threshold and a second threshold; Obtain independent bioparticle regions based on the bioparticle binary image; Obtaining the center coordinates of a first particle based on the biological particle region; obtaining the center coordinates of a second particle based on the biological particle region; Obtaining final particle coordinates based on the first particle center coordinates and the second particle center coordinates, and capturing the biological particle based on the final particle coordinates and optical tweezers; Acquiring the center coordinates of the first particle based on the biological particle region includes the following sub-steps: Establishing a plane rectangular coordinate system, marked as a position determination coordinate system, and placing a biological particle area in the position determination coordinate system; Obtain the maximum and minimum values ​​of the horizontal coordinate of the bioparticle area, marked as Cmax and Cmin respectively; Obtain the maximum and minimum values ​​of the vertical coordinate of the biological particle area, marked as Dmax and Dmin respectively; The abscissa of the center coordinate of the first particle is calculated as: E1=(Cmax+Cmin) / 2; E1 is the abscissa of the center coordinate of the first particle; The ordinate of the center coordinate of the first particle is calculated as: E2=(Dmax+Dmin) / 2; E2 is the ordinate of the center coordinate of the first particle; Acquiring the center coordinates of the second particle based on the biological particle region includes the following sub-steps: Obtaining coordinate points of edges of a second number of biological particle regions, marking them as edge coordinate points; The fitting profile equation is: f1×T1 2 +f2×T1×T2+f3×T2 2 +f4×T1+f5×T2+f6=0; Where f1, f2, f3, f4, f5 and f6 are equation coefficients, T1 is the horizontal coordinate of the edge coordinate point, and T2 is the vertical coordinate of the edge coordinate point; Fit all edge coordinate points according to the fitting contour equation to obtain the specific values ​​of the equation coefficients f1, f2, f3, f4, f5 and f6; The horizontal coordinate of the center coordinate of the second particle is: E3=(2×f3×f4-f2×f5) / (f2 2 -4×f1×f3); E3 is the abscissa of the center coordinate of the second particle; The vertical coordinate of the center coordinate of the first particle is: E4=(2×f1×f5-f2×f4) / (f2 2 -4×f1×f3); E4 is the ordinate of the center coordinate of the second particle.

2. The method for automatically capturing biological particles using optical tweezers based on machine vision according to claim 1, characterized in that: Grayscale processing of the biological particle image to obtain a biological particle grayscale image includes the following sub-steps: Obtain the RGB value of each pixel in the biological particle image and mark it as the image RGB value; All the image RGB values ​​are converted into grayscale values ​​using the grayscale conversion formula to obtain the grayscale image of biological particles; The grayscale value of the pixel in the biological particle grayscale image is marked as the biological particle grayscale value.

3. The method for automatically capturing biological particles using optical tweezers based on machine vision according to claim 2, characterized in that: Acquiring the first threshold and the second threshold based on the biological particle grayscale image includes the following sub-steps: Divide the grayscale value from 0 to 255 into a equal ranges, marked as equal ranges; In the grayscale image of the light guide plate, the frequency of the grayscale value of the biological particles in each equal division range is counted and marked as the equal division frequency; A histogram is drawn with the biological particle grayscale value as the X-axis, the equal-division frequency as the Y-axis, and the equal-division range as the histogram interval, and is marked as the particle grayscale histogram.

4. The method for automatically capturing biological particles using optical tweezers based on machine vision according to claim 3, characterized in that: Acquiring the first threshold and the second threshold based on the biological particle grayscale image also includes the following sub-steps: In the particle grayscale histogram, the equal-division frequency that is larger than the two adjacent equal-division frequencies on the left and right sides is marked as the peak frequency; Mark the equally divided range corresponding to the peak frequency as the peak range; Get the number of peak ranges. If the number of peak ranges is less than 2, increase the specific value of a and rebuild the particle grayscale histogram. If the number of peak ranges is greater than or equal to 2, obtain the maximum value of each peak range and mark it as the peak maximum value, obtain the peak range with the smallest peak maximum value and mark it as the first range, and obtain the peak range with the largest peak maximum value and mark it as the second range; In the particle grayscale histogram, obtain the equal division range with the smallest equal division frequency between the first range and the second range, and mark it as the third range; In the particle grayscale histogram, obtain the equal-division range with the smallest equal-division frequency greater than the second range and mark it as the fourth range. If there is no equal-division range with the smallest equal-division frequency greater than the second range, mark the second range as the fourth range. Obtain the mean of the third range, marked as the first threshold; The maximum value of the fourth range is obtained and marked as the second threshold.

5. The method for automatically capturing biological particles using optical tweezers based on machine vision according to claim 4, characterized in that: Converting the biological particle grayscale image into a biological particle binary image based on the first threshold and the second threshold includes the following sub-steps: The grayscale values ​​of biological particles in the biological particle grayscale image that are greater than or equal to the first threshold and less than or equal to the second threshold are set to 0, and the grayscale values ​​of biological particles in the biological particle grayscale image that are less than the first threshold or greater than the second threshold are set to 255 to obtain a biological particle binary image.

6. The method for automatically capturing biological particles using optical tweezers based on machine vision according to claim 5, characterized in that: Obtaining independent bioparticle regions based on the bioparticle binary image includes the following sub-steps: Mark the pixel with gray value 0 in the bioparticle binary image as the initial pixel; Get any processing pixel point, get a pixel point in eight directions with the processing pixel point as the center, and mark it as the first search pixel point; Determine whether the first search pixel is the initial pixel. If so, obtain a pixel in eight directions with the initial pixel in the first search pixel as the center, and mark it as the second search pixel. Determine whether the second search pixel is a newly appeared initial pixel. If so, obtain a pixel in eight directions with the newly appeared initial pixel in the second search pixel as the center. Repeat the search until no new initial pixel appears. Complete a search. Obtain a set of all initial pixels found in the search and mark it as a separate area. Repeat the process of searching multiple times starting from any unsearched processed pixel point until all initial pixel points in the biological particle binary image are searched.

7. The method for automatically capturing biological particles using optical tweezers based on machine vision according to claim 6, characterized in that: Obtaining independent biological particle regions based on the biological particle binary image also includes the following sub-steps: Acquire a first number of biological particle images, mark them as historical particle images, and obtain the number of pixels occupied by each biological particle in the historical particle images, mark them as historical pixel numbers; Get the minimum value of the number of historical pixels and mark it as the particle number threshold; Obtain the number of each individual region; determine whether the number of each individual region is less than the particle number threshold; if not, mark the individual region as a biological particle region.

8. The method for automatically capturing biological particles using optical tweezers based on machine vision according to claim 7, characterized in that: Acquiring final particle coordinates based on the first particle center coordinates and the second particle center coordinates, and capturing biological particles based on the final particle coordinates and optical tweezers includes the following sub-steps: The final particle coordinates are obtained as: [(E1+E3) / 2, (E2+E4) / 2]; the optical tweezers are adjusted to the position of the final particle coordinate point to capture the biological particle.

Images