Automated assisted reproduction system based on sound-driven bubble microfluid

Abstract

The invention relates to an automatic assisted reproduction system based on sound-driven bubble microfluid, and belongs to the field of micro-nano operation robots. Comprising the key steps of focusing, capturing, rotating, three-dimensional reconstruction and evaluation and the like, and the problems of high cost and long period in traditional manual operation are solved. The system utilizes secondary radiation force generated by sound wave driving bubble vibration to perform non-contact accurate control on oocytes or embryos. Meanwhile, multi-angle image capture is realized in combination with a three-degree-of-freedom operation platform, and continuous and stable three-dimensional reconstruction and morphological evaluation of the embryo in a long-period culture process are supported. Wherein the deep learning neural network is responsible for analyzing an embryonic development image in real time, and the intelligent algorithm is used for dynamically optimizing an operation path and parameters, so that the system is ensured to efficiently and accurately complete tasks without manual intervention, and the stability and success rate of operation are remarkably improved.

Description

Technical Field

[0001] The present invention relates to an automated assisted reproductive system based on sound-driven bubble microfluidics, belonging to the field of micro-nano manipulation robots, and in particular to a device and method for achieving posture adjustment before oocyte injection and long-term stereoscopic observation of embryos after injection and fertilization by utilizing sound field-driven bubble vibration with the assistance of a robot platform. Background Art

[0002] The widespread application of microfluidic chips in the biomedical field has demonstrated the superiority of microfluidic forces in damaging tiny biological targets such as cells. Conducting experimental analysis in microfluidic devices has obvious advantages. It not only avoids damage to flexible targets due to mechanical contact, but also reduces irreversible damage to biological targets such as cells caused by energy fields such as electricity, magnetism, and light. It also allows for rapid adjustment of experimental parameters for faster and more in-depth research.

[0003] In assisted reproductive technology, the success rate of in vitro manipulation of oocytes and embryos using microfluidic technology is crucial. Cell manipulation primarily involves adjusting the oocyte's position before injection, long-term stereoscopic observation of embryonic development, three-dimensional reconstruction of embryonic and oocyte cells, and assessment of embryonic health indicators. However, current embryo manipulation relies heavily on manual intervention by embryologists, particularly when moving, fixing, and rotating microscale objects such as oocytes and embryos under a microscope. Existing methods suffer from high complexity, low precision, and low efficiency. Furthermore, the training cycle for qualified embryologists is lengthy, and embryo morphology assessment primarily relies on visual observation under a microscope, assessing embryo shape, internal cell number, degree of fragmentation, and surface smoothness based on two-dimensional images. Traditional two-dimensional microscopic image evaluation methods not only easily miss defects outside the microscope's focal plane, but also, due to the lack of long-term observation of embryonic development, make it difficult to detect potential problems such as disrupted mitosis and nuclear division abnormalities that may occur during embryonic development. Furthermore, occlusion poses a significant challenge during three-dimensional reconstruction. Observation of embryonic cells and oocytes under a microscope typically only produces two-dimensional images from a single perspective, making it difficult to fully reflect their three-dimensional structure. Especially in cases where cells are densely packed or overlapping, parts of the cell structure may be obscured by other cells or debris, resulting in incomplete or distorted reconstructions. This affects the accuracy of the three-dimensional model and may mask key defects in embryonic development, such as abnormal cell division or disordered internal structures.

[0004] Therefore, in order to meet the needs of assisted reproduction, it is of great significance to study an acoustically driven micromanipulation robot platform that can realize stereoscopic observation and evaluation of oocytes and long-cycle developing embryonic cells. Summary of the Invention

[0005] The technical problem solved by the present invention is to overcome the shortcomings of the existing technology and propose an automated assisted reproductive system based on acoustic bubble microfluidics. This platform combines the capture and precise posture control technology of micro-targets by acoustic bubbles with visually guided robotic operation, and uses a deep learning network to intelligently analyze image information during embryonic development. It realizes the full-process automation of embryo capture and rotation, and simultaneously completes the three-dimensional reconstruction and health assessment of embryonic cells at each developmental stage. The present invention effectively solves the problems currently faced by the assisted reproductive field, such as heavy workload, large amount of operation and insufficient stability of embryologists, and significantly improves the success rate and operational efficiency of in vitro fertilization.

[0006] The technical solution of the present invention is:

[0007] An automated assisted reproductive system based on acoustically driven bubble microfluidics, comprising an embryo manipulation device, an embryo image acquisition device, an automated manipulation module, a three-dimensional reconstruction module, and an intelligent evaluation module;

[0008] The embryo manipulation device adopts an automated solution combined with a three-degree-of-freedom platform to realize embryo rotation operation before oocyte injection and during long-term multi-angle image capture, comprising a first three-degree-of-freedom operation platform 1, a needle holding device 2, an end effector 3, a cylindrical glass dish 4, a piezoelectric brake 5, and a second three-degree-of-freedom operation platform 6;

[0009] The embryo image acquisition system of the present invention utilizes an inverted microscope optical path to collect and capture embryo image data. This system allows for real-time transmission of images of the embryo's rotation from bottom to top to a computer using a capture device such as a high-speed camera with a frame rate of 200 fps. Subsequently, combined with rotation axis extraction and angle extraction algorithms, three-dimensional reconstruction and evaluation of embryonic cells are performed, significantly improving the success rate of assisted reproduction.

[0010] The embryo image acquisition device includes a light source 7, an aperture 8, a microscope lens 9, a reflector 10, and a camera 11;

[0011] The end effector 3 is a curved micro glass tube with a variable inner diameter, the inner diameter of the curved front end is 0.6 mm, and the inner diameter of the curved rear end is 100 μm;

[0012] The cylindrical glass dish 4 has a height of 15 mm and a diameter of 80 mm;

[0013] The syringe pump is connected to the curved front end of the end effector 3 via a plastic hose. The syringe pump is used to generate a stable bubble column to capture the oocyte. In the end effector with an inner diameter of 100 μm, the bubble diameter usually generated is 80-100 μm, which can meet the requirements of oocyte micromanipulation.

[0014] The first three-degree-of-freedom operating platform 1 is used to clamp the needle holding device 2;

[0015] The needle holder 2 is used to fix the curved front end of the end effector 3. The motion control board PCI1040 controls the movement of the first three-degree-of-freedom operating platform 1, thereby driving the movement of the end effector 3. The PCI1040 board is a high-performance motion control board that can accurately control position, speed, and acceleration according to received computer instructions. It is suitable for application scenarios requiring high-precision motion control.

[0016] The curved rear end of the end effector 3 is located in the cylindrical glass dish 4;

[0017] The piezoelectric actuator 5 is located below the cylindrical glass dish 4. It receives the signal from the PCI8250 analog synchronous output card and generates sound waves parallel to the cylindrical glass dish 4 after power amplification, so that the bubbles in the solution oscillate under the action of the sound waves, thereby driving the oocyte or embryo to rotate around the axis.

[0018] The second three-degree-of-freedom operating platform 6 is also controlled by the PCI1040 control board and is used to adjust the position of the cylindrical glass dish 4 to enable efficient operation and observation of cells in different areas and field of view focusing operations. This integrated design improves operational accuracy and efficiency and is suitable for a variety of applications in the field of micro-nano manipulation.

[0019] The light source 7 is LEDOTB-700, with a maximum flash frequency of 10kHz and a minimum strobe pulse width of 50μs;

[0020] The camera 11 is a Hikvision MV-CH050-10UC with a maximum resolution of 2592x1944 pixels (5MP), which can provide high-definition image acquisition capabilities;

[0021] The light source 7 is used to emit light, which is irradiated to the aperture 8, and then receives image information from the cylindrical glass dish 4 through the microscope lens 9. The image information is then reflected by the reflector 10 and enters the camera 11, and the data is transmitted through the connected computer;

[0022] The automated operation module is integrated into the embryo manipulation device and the image acquisition device. It uses algorithms to drive the hardware to work in coordination, achieving non-destructive capture, multi-angle rotation, and precise positioning of the embryo. The module includes an autofocus unit and an automatic positioning and path planning unit. By working in coordination with these units, the automated operation module can achieve high-precision and high-efficiency embryo manipulation, reduce manual intervention, improve the consistency and stability of embryo manipulation, and provide strong support for the development of assisted reproductive technology.

[0023] The autofocus unit uses a method based on image gradient analysis technology to achieve fast and accurate focusing of the microscope field of view, ensuring image clarity during embryo manipulation and providing high-quality data support for subsequent computer vision analysis. The system quantifies image clarity based on gradient analysis and fuzzy matrix technology. The Sobel operator is used to calculate the gradient amplitude G(x,y) matrix of the image. The specific calculation method is:

[0024]

[0025] This is used to evaluate the imaging quality of different focal planes, and the area with the largest gradient amplitude is selected as the best focus point. The specific calculation method is:

[0026] argmax z ∑ x,y G(x,y,z)

[0027] The calculated results are transmitted to the second 3-DOF operating platform 6 via the PCI1040 control board. This drives the stage to precisely adjust perpendicular to the field of view, moving it along the Z axis in 1μm steps until the desired focal plane is reached. This process enables submicron-level autofocusing in a short period of time, significantly improving the efficiency and accuracy of micromanipulation.

[0028] The automatic positioning and movement unit is responsible for planning the optimal embryo manipulation path, ensuring the system can accurately and stably perform target positioning and motion control. This unit combines deep learning target detection, a rapid randomized tree (RRT) algorithm, and motion control strategies to optimize operation efficiency and minimize interference with the embryo.

[0029] During the automatic positioning stage, the system determines the initial positions of all embryonic cells and the end effector 3 in the cylindrical glass dish 4 based on the preliminary YOLOv5 target detection model, receives images from the microscope lens 9 in real time, and uses a convolutional neural network to detect and select oocytes in the image, calculates the center pixel coordinates (u, v) of each target, and outputs the bounding box coordinates (x_{min}, x_{max}, y_{min}, y_{max}).

[0030] In the path planning stage of the automatic mobile unit, the present invention adopts the RRT algorithm to construct a search tree, gradually expanding nodes from the starting point q_{start} to the target point q_{goal} to generate an optimized collision-free trajectory. This path not only takes into account the spatial constraints of the micromanipulation environment, but also takes into account the mechanical stability during the embryo operation process to avoid unnecessary external interference. The system works in coordination with the three-degree-of-freedom operating platform connected to each other through the PCI1040 control board, and gradually searches for the optimal path to perform precise motion control according to the calculated target point. The platform performs dynamic path adjustment according to real-time image feedback to adapt to the operational requirements of different embryonic individuals. In the capture and movement stage of the automatic mobile unit, the end effector 3 is guided to the target position by the first three-degree-of-freedom operating platform 1, which provides a basis for the various operations of the end effector 3 for the target cell, such as capturing, moving, releasing, rotating, etc.

[0031] The 3D reconstruction module, which includes a target rotation unit and an intelligent image processing system, first performs system posture calibration: the end effector 3 is adjusted to a preset tilt angle, and the radiation force field generated by surface acoustic waves achieves a dynamic balance between gravity and acoustic field forces. This process requires precise control of the gas-liquid interface within the end effector to stabilize at the reference position at the end of the microchannel to ensure zero interference of interfacial tension on the operating system. Based on the 3D environmental model constructed by the computer vision system, a programmable acoustic flow field is generated in the culture medium by adjusting the voltage and frequency input to the piezoelectric actuator 5 via the PCI8250 card. This acoustic flow field generates precise acoustic radiation force gradients in the X, Y, and Z axes, driving the captured cells to spatially displace along a preset path with submicron precision. Once the target cell is transported to the specified coordinates, the acoustic field intensity is gradually attenuated to release the restraining effect of the secondary radiation force on the target. Finally, the end effector 3 is restored to its initial spatial coordinates, completing the entire cell manipulation process.

[0032] The rotation of micro-targets is an important operation in the field of assisted reproduction. For long-term observation of oocytes or embryonic cells, it is necessary to rotate 360° to capture multi-angle images. Figure 3 As shown, facing the target to be operated, the end effector 3 is adjusted to have a central axis of the end effector at an angle of 10° to 30° to the horizontal plane. The displacement platform is controlled to move the end effector 3 downward until its central axis passes through the center of the target cell sphere, and the target cell is kept biased to one side of the central axis. The piezoelectric brake 5 is turned on, and the target cell rotates around the Z axis under the action of the acoustic radiation force and the fluid force. When the target cell reaches the target position or target angle, the piezoelectric brake 5 is turned off.

[0033] The end effector 3 drives the cell to rotate continuously for more than 360° around the vertical axis (Z axis) of the microscope field of view plane under the control of the sound field, and synchronously triggers the high frame rate camera 11 and the microscope lens 9 to collect multi-angle image sequences. In order to solve the problem of cell occlusion that may occur during the rotation process, the present invention adopts deep learning and grayscale mask fusion technology: first, a target mask is generated by a deep learning model and a confidence score is added. When the confidence is lower than the threshold, the grayscale mask generated by Canny edge detection is combined for compensation optimization. After morphological expansion-erosion operation and Gaussian blur processing, a smooth transition of the mask edge is achieved. On this basis, the target bounding box with the largest area is selected based on the Mask R-CNN framework as the main alignment reference, and the inter-frame offset is smoothed by the sliding window algorithm to ensure that the target cell is stably centered.

[0034] In order to calculate the cumulative rotation angle of each frame in the 3D reconstruction, a feature extraction method based on rotation feature points is adopted. By tracking the position of the feature points in the cell rotation, the cumulative rotation angle of the cell at that moment relative to the time origin is calculated.

[0035] For the feature extraction process, the processing mode is dynamically switched according to the number of daughter cells: in the single-daughter cell state, the Lucas-Kanade optical flow method is used, and the feature point trajectory is tracked through Shi-Tomasi corner detection; in the multi-daughter cell state, the Trackor tracking framework is deployed, combined with the Faster R-CNN detector and the ResNet-50 re-identification model, to extract the center coordinates of each daughter cell's bounding box to construct a motion trajectory database.

[0036] Based on the obtained eigenvalues, since the feature points move on an approximate spherical surface, we assume that the rotation surface is a sphere, set the radius and center of the cell, and calculate the rotation angle of the current frame through inverse trigonometric functions. The specific algorithm is:

[0037] First, calculate the offset (the distance between the feature point and the center of the circle) ΔxΔy and calculate the projection radius:

[0038]

[0039] where r newx is the projection radius of the XZ plane, r newy The projection radius corresponding to the YZ plane; then calculate the angle of the frame relative to the center of the circle:

[0040]

[0041]

[0042] Then calculate the difference between the relative angle of this frame and the previous frame to get the angle change of this frame relative to the previous frame, and finally accumulate to get the final xyz angle

[0043] According to the obtained angle, the rotation matrix around the X axis, the Y axis, and the Z axis is calculated. The total rotation matrix is ​​calculated and multiplied by the unit vector to obtain the rotation axis vector. Normalization is performed to obtain the rotation axis, where:

[0044]

[0045] R is the total rotation matrix R x , R y , R z is the rotation matrix corresponding to each axis.

[0046] After obtaining the total rotation matrix, the relative rotation matrix is ​​obtained by multiplying it with the transpose of the rotation matrix of the previous frame, which represents the rotation change from one frame to the current frame. The rotation angle is calculated by the relative rotation matrix to obtain the relative rotation angle of this frame relative to the coordinate axis between the previous frame and the current frame, and finally accumulated.

[0047]

[0048] Current relative to the previous frame

[0049]

[0050] f(x,y) indicates that the angle of rotation of the original image is θ, and the cumulative rotation angle can be calculated by accumulation.

[0051] Import each frame image and the corresponding rotation angle into the system integration platform. First, convert all cells into grayscale images, and perform sinogram transformation on each frame image. The formula is expressed as:

[0052] R f (θ, t) = grayscale value of the image, R f (θ, t) represents the projection function after Radon transformation, which represents the projection intensity at angle θ and distance t, where t is the distance from the projection line to the origin on the image. Θ represents the projection angle, usually from 0° to 180°, and then the δ(·) function is used to internally integrate to obtain the image depth of the embryonic cells in all directions. The next step is back-projection reconstruction.

[0053]

[0054] f ′ (x, y) is the image after backprojection, that is, the final reconstructed image; backproject(θ): the backprojection operation is to convert each projection data R f(θ, t) is back-projected back to the image space along the corresponding angle θ, and the tomogram is finally obtained.

[0055] The obtained tomograms were preprocessed. The images were segmented into cell and background regions by thresholding, and volume data were extracted. The processed two-dimensional slices were stacked to generate a three-dimensional voxel dataset. The cell surface was extracted using the Marching Cubes surface extraction algorithm to generate a three-dimensional network model. A smoothing algorithm was applied to the generated surface network to remove noise and optimize the surface shape.

[0056] The intelligent assessment module automatically quantifies the quality of embryonic development based on a three-dimensional reconstruction model and combines morphological analysis methods to provide feedback for optimizing operational parameters. Specific indicators include:

[0057] Evaluation criterion 1: the number and uniformity of blastomeres. On the third day, the ideal embryo should contain 7-9 blastomeres, and the sizes of these blastomeres should be uniform. This indicator reflects the normal division and development of the embryo. The basic idea of ​​​​quantity detection is to separate the blastomeres from the background through image segmentation methods and calculate the number of separated areas. After the image is binarized (after threshold segmentation, the blastomeres are white and the background is black), the number of blastomeres is calculated by connected domain analysis. Let N be the number of blastomeres, I (x, y) be the pixel value in the image, and R be the segmented area. If I (x, y) = 1, it means the area belonging to the blastomeres:

[0058] N=number of connected components in R

[0059] The uniformity was assessed by calculating the ratio of the standard deviation of the blastomere area to the mean area. i is the area of ​​the i-th blastomere, is the average area of ​​all blastomeres, σ A is the standard deviation of the area, and the uniformity score U can be calculated as:

[0060]

[0061]

[0062] If U is less than the set threshold of 0.25, the blastomeres are considered to be uniform in size.

[0063] Evaluation criterion 2: Fragmentation assessment, which is based on the ratio of the area of ​​the fragment to the area of ​​the entire image. First, the fragment area is extracted through image segmentation, and the area of ​​the fragment area A is obtained after segmentation. frag , the area of ​​the entire image is A total The fragment ratio can be calculated using the following formula:

[0064]

[0065] Embryos are classified into three grades based on the proportion of internal fragments:

[0066] Category 1: The amount of debris is less than 10%;

[0067] Category II: fragmentation amount is 10%-25%;

[0068] Category III: debris volume greater than 25%;

[0069] The amount of fragments directly affects the quality of the embryo. The fewer fragments, the higher the quality of the embryo.

[0070] Evaluation criterion three: The embryonic cytoplasm should be clear, smooth, and free of abnormal particles or color changes. Abnormal cytoplasmic appearance may indicate problems with embryonic development. The examination of cytoplasmic appearance involves color and texture analysis of the image. We use the standard deviation or local contrast of the image to evaluate the smoothness and granularity of the cytoplasm, and define the standard deviation σ of the local area of ​​the image as C , which is used to measure the uniformity of the cytoplasm. Let C(x,y) be the color intensity of the image (such as the pixel value in a certain color space), and the standard deviation of the local area is:

[0071]

[0072] Where M and N are the dimensions of the region, is the average value of the pixels in the region. If the standard deviation is too large, it indicates that abnormal particles may exist in the cytoplasm.

[0073] Evaluation criterion 4: Multinucleation detection. Each blastomere should have only one nucleus. Multinucleation is usually considered a negative indicator that may affect the subsequent development of the embryo. The nuclear region is extracted through image segmentation. If a blastomere contains multiple nuclear regions, it is marked as multinucleation. Let C i is the i-th blastomere region, N i The number of cell nuclei it contains. If N i >1, the blastomere is judged to be multinucleated.

[0074]

[0075] Evaluation Criteria 5: Blastocyst Stage Assessment. For embryos that have reached the blastocyst stage, the degree of expansion of the blastocyst cavity, the quality of the inner cell mass (ICM), and the development of the trophectoderm (TE) need to be assessed. These indicators further reflect the developmental potential of the embryo. The expansion of the blastocyst cavity can be assessed by measuring the area ratio of the blastocyst cavity. Hypothesis A blastocyst is the area of ​​the blastocyst cavity, A totalThe ratio of the blastocyst cavity expansion degree to the area of ​​the entire image can be expressed by the following formula:

[0076]

[0077] A large ratio of the blastocoel to the image area indicates good expansion.

[0078] Based on the performance of the above indicators, a comprehensive score is given to each embryo. The scoring formula is:

[0079]

[0080] S1 to S5 are the scores for blastomere number and uniformity, amount of fragmentation, cytoplasmic appearance, multinucleation, and blastocyst stage assessment, respectively, and w1 to w5 are the corresponding weights. The score ranges from 0 to 1, with 0 representing the worst and 1 representing the best.

[0081] Beneficial effects

[0082] The device proposed in the present invention has significant advantages over the existing technology. Through an automated control system based on deep learning neural networks and intelligent optimization algorithms, it realizes unmanned operation of the entire process from task setting to result output, including key steps such as focusing, capturing, rotating, and three-dimensional reconstruction and evaluation, which completely solves the problems of high cost and long cycle in traditional manual operations. The system innovatively uses the secondary radiation force generated by sound wave-driven bubble vibration to perform non-contact and precise manipulation of oocytes or embryos. At the same time, it combines a three-degree-of-freedom operating platform to achieve multi-angle image capture, supporting continuous and stable three-dimensional reconstruction and morphological evaluation of embryos during long-cycle culture. Among them, the deep learning neural network is responsible for real-time analysis of embryo development images, and the intelligent algorithm dynamically optimizes the operation path and parameters to ensure that the system completes the task efficiently and accurately without human intervention, significantly improving the stability and success rate of the operation.

[0083] The present invention combines a deep learning algorithm with a basic grayscale algorithm to process the collected image data, and can automatically generate a detailed three-dimensional model containing the contents of the embryo, and accurately assess its developmental state, including the number and uniformity of blastomeres, the amount of fragments, the appearance of the cytoplasm, and multiple nuclei. In addition, for the occlusion problem in the three-dimensional reconstruction of the egg cell, the system utilizes multi-angle image capture technology, combines a deep learning algorithm with a grayscale algorithm, and through image fusion and completion technology, effectively solves the problem of incomplete models caused by occlusion in traditional methods, thereby further improving the accuracy and reliability of three-dimensional reconstruction. This highly integrated automated process not only improves operational accuracy and efficiency, but also reduces the possible contamination risk caused by manual intervention, ensures the purity and stability of the embryo culture environment, and provides technical support for improving the success rate of in vitro fertilization. In addition, the system's long-term stereoscopic observation capability enables researchers to more comprehensively understand the subtle changes in the embryonic development process, and automated embryonic development assessment greatly saves labor costs, laying a solid foundation for further optimizing assisted reproductive technology.

[0084] Compared to existing technologies, the device of the present invention significantly reduces hardware and operating costs by integrating automated control with intelligent algorithms. This solution reduces the use of expensive equipment and consumables, while also reducing the frequency and complexity of manual operations, significantly reducing overall costs. This advantage makes the present invention more scalable and practical in clinical and scientific research applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 A schematic diagram of the overall structure of an automated assisted reproductive system based on sound-driven bubble microfluidics provided in an embodiment of the present application;

[0086] Figure 2 A schematic diagram of the internal structure of a cylindrical glass device of an automated assisted reproductive system based on acoustically driven bubble microfluidics provided in an embodiment of the present application;

[0087] Figure 3 The present invention provides an embodiment of an automated assisted reproductive system based on acoustic bubble microfluidics for capturing, moving, and rotating oocytes and embryonic cells, and a schematic diagram of the automated operation process;

[0088] Figure 4 A process and schematic diagram of rotation axis extraction and rotation angle calculation for an automated assisted reproductive system based on acoustically driven bubble microfluidics provided in an embodiment of the present application;

[0089] Figure 5 An initialization automatic operation process and schematic diagram of an automated assisted reproductive system based on acoustically driven bubble microfluidics provided in an embodiment of the present application;

[0090] Figure 6A three-dimensional reconstruction process and schematic diagram of oocytes or embryonic cells in an automated assisted reproductive system based on acoustic bubble microfluidics provided in an embodiment of the present application;

[0091] First three-degree-of-freedom operating platform 1, needle holding device 2, end effector 3, cylindrical glass dish 4, piezoelectric brake 5, second three-degree-of-freedom operating platform 6, light source 7, aperture 8, microscope lens 9, reflector 10, camera 11, droplet 12. DETAILED DESCRIPTION

[0092] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0093] See also Figure 1 and Figure 2 As shown in the figure, this figure is a schematic diagram of the overall structure of an automated embryo operation and intelligent evaluation platform based on sound-driven bubble microfluidics provided in an embodiment of the present application. It has been described in detail above and will not be repeated here.

[0094] First, the system is initialized. The second three-degree-of-freedom operating platform 6 is controlled by the PCI1040 board to move up and down along the Z-axis along the cylindrical glass dish 4. Combined with the recursive gradient calculation method, the clarity of the microscopic image is evaluated in real time to achieve automatic focus. The image clarity gradient recursive algorithm is used to automatically obtain the optimal focal plane, thereby improving the spatial resolution of the cell image and providing a guarantee for subsequent cell identification and positioning. While completing the automatic focus of the microscope, the image edge detection algorithm is combined with the gradient change analysis to enhance the image edge of the target cell and accurately extract the spatial position information of the cell in the Z direction. By detecting the current Z-axis coordinate of the end effector 3, the coordinate synchronization of the microscope system and the end effector 3 in the Z-axis direction is achieved to ensure the spatial consistency of the entire system. Subsequently, the Z-axis coordinate of the end effector 3 is fine-tuned using the first three-degree-of-freedom operating platform 1 so that it is accurately aligned with the target cell in the Z-axis direction. The error between the autofocus and Z-axis calibration is controlled within 1μm.

[0095] After completing the Z-axis calibration, Figure 5As shown, the microscope system is controlled to scan the area along the set broken line path, and the center position of the cell and the end effector 3 is identified and detected in real time by integrating the YOLOv5 deep learning neural network model. Based on the center coordinate information of the cell and the end effector 3, a unified XY coordinate system is established, and the position of the system coordinate origin is determined. This coordinate system serves as the spatial reference system for path planning and motion control of the end effector 3. Subsequently, the rapidly expanding random tree (RRT) path planning algorithm is applied to plan the optimal path from the current position to the vicinity of the target cell for the end effector 3 based on the XY coordinate information. This path planning process ensures that the end effector 3 moves smoothly, avoids potential obstacles, and avoids interference and collision. The end effector 3 moves accurately along the planned path to the vicinity of the target cell. The movement process is completed by the servo motor closed-loop control system to ensure that the position control error is less than ±0.5μm, providing a precise basis for subsequent operations.

[0096] After the end effector 3 completes the initial positioning, the posture of the end effector 3 is adjusted through the first three-degree-of-freedom operating platform 1 so that it remains parallel to the horizontal plane to ensure the uniform distribution of the acoustic wave field during the capture process. For the micro glass tube (inner diameter of about 100μm) used in the experiment, the resonant frequency of the bubble is calculated to be 5kHz through the theoretical model. In order to obtain the best acoustic capture effect, a frequency scanning experiment is carried out near the theoretical resonant frequency. The experimental scanning range is 4.5kHz to 6kHz, and the changes in bubble amplitude under different excitation frequency conditions are recorded. And based on the experimental results, the excitation frequency (such as 5.65kHz) and the driving voltage amplitude (such as 10Vp-p) at which the bubble amplitude reaches the maximum value are determined. At this time, the bubble is in the optimal resonance state.

[0097] At the optimal excitation frequency, the piezoelectric actuator 5 is driven to operate, generating a stable acoustic radiation force field around the end effector 3, forming an acoustic potential well and enabling non-destructive capture of the target cell. After the cell is adsorbed to the end of the end effector 3, a real-time image is captured through a visual feedback system to confirm successful capture of the cell and record its initial position in the XY coordinate system.

[0098] The first three-degree-of-freedom mobile platform 1 is used to fine-tune the end effector 3's posture again, so that it is at a set inclination angle (e.g., 10° to 30°) with the horizontal plane, creating conditions for cell rotation. Excitation signals of varying phases or amplitudes, such as a 6.2kHz, 7Vp-p sine wave, are sent to the piezoelectric actuator 5 to drive the cell to achieve precise rotation around its own axis in the X or Y direction. The cell rotation speed is within the range of 6-8 rpm, and the rotation angle is monitored in real time by a microscopic image analysis algorithm to ensure smooth and accurate cell rotation.

[0099] Based on the image from the microscope lens 9, it is confirmed that the target cell has completed a full rotation (360°). If the rotation requirement is not met, the rotation operation is continued until the rotation condition is met. After the rotation operation is completed, the excitation signal to the piezoelectric actuator 5 is disconnected, the acoustic radiation force is released, and the cell is released to the predetermined target position without external disturbance. During the release process, the end effector 3 remains stable to prevent the cell from drifting or losing control.

[0100] After the cell capture and rotation operations are completed, the cell 3D reconstruction process begins. First, during the cell rotation process, cell images at different angles are synchronously captured by a high-speed camera at a rate of 60 frames per second to ensure full-angle image data covering the 360° rotation process of the cell. Edge detection and watershed segmentation algorithms are used to pre-process the collected cell images to extract the cell surface contours and internal sub-cell structure information. The contour extraction accuracy reaches 1μm, providing precise boundary conditions for 3D modeling. Subsequently, the cell rotation image is analyzed by the image feature point tracking algorithm, the cell rotation axis is calculated, and the rotation angle corresponding to each frame of the image is determined.

[0101] like Figure 4 As shown in the figure, based on the rotation angle information, the Radon transform algorithm is used to generate the tomographic image data of the cell. The tomographic image data is reconstructed through the back-projection algorithm to obtain a high-resolution three-dimensional structural model of the target cell and its daughter cells, with the model reconstruction accuracy reaching the submicron level.

[0102] On this basis, combined with cell image data collected over multiple time periods, a three-dimensional characteristic model of oocytes and embryonic cells at different developmental stages is established, dynamically reflecting the evolution of cell morphology over developmental time. The resulting three-dimensional model is then scored and judged in conjunction with the morphological assessment module. The specific criteria are as follows:

[0103] Evaluation criterion 1: number and uniformity of blastema, score S1.

[0104] Evaluation criterion 2: amount of debris, score S2.

[0105] Evaluation criterion three: cytoplasmic status of the embryo, score S3.

[0106] Evaluation criterion four: multi-core phenomenon detection, score S4.

[0107] Evaluation criterion five: Blastocyst stage assessment, score S5.

[0108] Based on the performance of the above indicators, a comprehensive score is given to each embryo. The scoring formula is:

[0109]

[0110] Among them, w1 to w5 are corresponding weights. The score ranges from 0 to 1, with 0 representing the worst and 1 representing the best. Through this series of evaluation steps, the quality of embryos can be systematically judged and the embryos with the greatest developmental potential can be selected for transplantation, thereby improving the success rate.

[0111] This application example proposes an automated assisted reproductive system based on sound-driven bubble microfluidics, which realizes a comprehensive innovation in the operation of oocytes and embryos by integrating sound-driven bubble manipulation, full-process automated control and deep learning algorithms. The secondary radiation force generated by sound-driven bubble vibration is used to complete the non-destructive capture and precise adjustment of cells, avoiding the damage caused by traditional mechanical operations; combined with a three-degree-of-freedom mobile platform and control system, the full process automation from task setting to result output is realized, which improves operational efficiency and supports long-period stereoscopic observation. At the same time, with the help of deep learning algorithms (such as YOLOv5, MaskR-CNN) and grayscale mask fusion technology, the occlusion problem is solved, the target detection accuracy is optimized, and the three-dimensional reconstruction and morphological evaluation of embryonic cells are completed, providing a scientific basis for embryo selection and significantly improving the success rate of in vitro fertilization. In addition, the device integrates automation and intelligent functions, reduces equipment costs, has high cost performance and wide promotion value, and provides strong support for the clinical application and scientific research development of assisted reproductive technology.

[0112] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An automated assisted reproductive system based on acoustic bubble microfluidics, characterized by: The assisted reproductive system includes an embryo manipulation device, an embryo image acquisition device, an automated operation module, and a three-dimensional reconstruction module; The embryo manipulation device comprises a first three-degree-of-freedom manipulation platform, a needle holding device, an end effector, a cylindrical glass dish, a piezoelectric brake and a second three-degree-of-freedom manipulation platform; The embryo image acquisition device includes a light source, an aperture, a microscope lens, a reflector, and a camera; The automated operation module is integrated into the embryo manipulation device and the embryo image acquisition device to achieve non-destructive capture, multi-angle rotation and precise positioning of the embryo; The three-dimensional reconstruction module includes a target rotation unit and an intelligent image processing system.

2. The automated assisted reproductive system based on acoustic bubble microfluidics according to claim 1, characterized in that: The end effector is a curved micro glass tube with a variable inner diameter, the inner diameter of the curved front end is 0.6 mm, and the inner diameter of the curved rear end is 100 μm; The cylindrical glass dish has a height of 15 mm and a diameter of 80 mm; The syringe pump is connected to the curved front end of the end effector through a plastic hose. The syringe pump is used to generate a stable bubble column to capture the egg cell. The first three-degree-of-freedom operating platform is used to clamp the needle holding device; The needle holding device is used to fix the curved front end of the end effector, and controls the movement of the first three-degree-of-freedom operating platform through the motion control board PCI1040, thereby driving the movement of the end effector; The curved rear end of the end effector is located in a cylindrical glass dish; The piezoelectric actuator is located below the cylindrical glass dish and generates sound waves parallel to the cylindrical glass dish by receiving signals from the PCI8250 analog synchronous output card and generating sound waves parallel to the cylindrical glass dish after power amplification, so that bubbles in the solution oscillate under the action of the sound waves, thereby driving the oocyte or embryo to rotate around the axis; The second three-degree-of-freedom operating platform is also controlled by a PCI1040 control board and is used to adjust the position of the cylindrical glass dish.

3. The automated assisted reproductive system based on acoustic bubble microfluidics according to claim 2, characterized in that: The light source is used to emit light, the light hits the aperture, and then receives image information from the cylindrical glass dish through the microscope lens. The image information is then reflected by the reflector and enters the camera, and data is transmitted through a connected computer.

4. The automated assisted reproductive system based on acoustic bubble microfluidics according to claim 3, characterized in that: The light source is LEDOTB-700, with a maximum flash frequency of 10kHz and a minimum strobe pulse width of 50μs; The camera is a Hikvision MV-CH050-10UC with a maximum resolution of 2592x1944 pixels, which is used to provide high-definition image acquisition capabilities.

5. The automated assisted reproductive system based on acoustic bubble microfluidics according to claim 1, characterized in that: The automated operation module includes an automatic focusing unit and an automatic positioning and path planning unit; The autofocus unit uses a method based on image gradient analysis technology to achieve fast and accurate focusing of the microscope field of view, ensuring image clarity during embryo manipulation.

6. The automated assisted reproductive system based on acoustic bubble microfluidics according to claim 5, characterized in that: Quantify image clarity based on gradient analysis and fuzzy matrix technology; The Sobel operator is used to calculate the gradient magnitude G(x,y) matrix of the image. The specific calculation method is: in, and Represents the grayscale value of the image after horizontal and vertical edge detection respectively. x ) and vertical gradient (G y ) Convolution is performed to obtain G x , G y They are: Where I(x,y) is the pixel value of the image at coordinate (x,y). This is used to evaluate the imaging quality of different focal planes. Collect image sequences of different focal planes {I1,I2,…,I k }, and calculate the gradient magnitude matrix G for each image k (x,y), calculate the clarity evaluation value S of each image k : And select the area with the largest gradient amplitude as the best focus point. The specific calculation method is: argmax k (S k ) The calculation results are transmitted to the second three-degree-of-freedom operating platform through the PCI1040 control board, which drives the stage to precisely adjust in the direction perpendicular to the field of view plane and move the stage along the Z axis in steps of 1 μm until the ideal focal plane is reached.

7. The automated assisted reproductive system based on acoustic bubble microfluidics according to claim 3, characterized in that: The automatic positioning and path planning unit is used to first perform the automatic positioning stage and the path planning stage; In the automatic positioning stage, the initial positions of all embryonic cells and the end effector in the cylindrical glass dish are determined using a preliminary YOLOv5 object detection model. Images from the microscope lens are received in real time, and a convolutional neural network is used to detect and select oocytes in the image. The coordinates (u, v) of the center pixel of each target are calculated and the bounding box coordinates (x_{min}, x_{max}, y_{min}, y_{max}) are output. In the path planning stage, the RRT algorithm is used to construct a search tree, gradually expanding nodes from the starting point q_{start} to the target point q_{goal} to generate an optimized collision-free trajectory.

8. The automated assisted reproductive system based on acoustic bubble microfluidics according to claim 3, characterized in that: The functions of the 3D reconstruction module are: First, the system's posture is calibrated: the end effector is adjusted to a preset tilt angle, achieving a dynamic balance between gravity and acoustic forces through the radiation force field generated by surface acoustic waves. During this process, the gas-liquid interface within the end effector must be precisely controlled to stabilize at the reference position at the end of the microchannel, ensuring zero interference from the interfacial tension on the operating system. Based on a three-dimensional environmental model constructed by a computer vision system, a programmable acoustic flow field is generated in the culture medium by adjusting the voltage and frequency input to the piezoelectric actuator via the PCI8250 card. This acoustic flow field generates precise acoustic radiation force gradients in the X, Y, and Z axes, driving the captured cells to move spatially along a preset path with submicron precision. Once the target cells are transported to the specified coordinates, the acoustic field intensity is gradually attenuated, releasing the restraining effect of the secondary radiation force on the target. Finally, the end effector is restored to its initial spatial coordinates, completing the entire cell manipulation process. Rotation of micro-targets is an important operation in the field of assisted reproduction. For long-term observation of oocytes or embryonic cells, it is necessary to rotate 360° to capture their multi-angle images. Facing the target to be operated, the end effector position is adjusted so that the central axis of the end effector is 10° to 30° with the horizontal plane. The displacement platform is controlled to move the end effector down to the point where its central axis passes through the center of the target cell sphere, and the target cell is kept biased to one side of the central axis. The piezoelectric brake is turned on, and the target cell will rotate around the Z axis under the action of acoustic radiation force and fluid force. When the target cell reaches the target position or target angle, the piezoelectric brake is turned off. Under the control of the acoustic field, the end effector drives the cell to continuously rotate more than 360° around the vertical axis of the microscope's field of view, synchronously triggering the high-frame-rate camera and microscope lens to capture multi-angle image sequences; First, a target mask is generated through a deep learning model and an additional confidence score is added. When the confidence score is lower than the threshold, compensation optimization is performed using the grayscale mask generated by Canny edge detection. A morphological dilation-erosion operation and Gaussian blur processing are used to achieve a smooth transition of the mask edge. On this basis, the target bounding box with the largest area is selected as the primary alignment reference based on the Mask R-CNN framework. The inter-frame offset is smoothed using a sliding window algorithm to ensure that the target cell is stably centered.

9. The automated assisted reproductive system based on acoustic bubble microfluidics according to claim 8, characterized in that: Three-dimensional reconstruction and dynamic tracking are core components of the automated assisted reproductive system based on acoustic bubble microfluidics. To accurately calculate cell rotation angles and reconstruct their three-dimensional structures, the system extracts stable feature points from consecutive image frames and analyzes their motion trajectories. During the feature extraction process, the system dynamically adjusts its algorithm strategy based on the embryonic development stage (single-cell or multi-cell) to ensure robust feature point detection and tracking. For feature extraction, the processing mode is dynamically switched based on the number of daughter cells: in the single-daughter cell state, the Lucas-Kanade optical flow method is used, and the Shi-Tomasi corner point detection is used to track the trajectory of feature points. In the multi-daughter cell state, the Trackor tracking framework is deployed, combining the Faster R-CNN detector and the ResNet-50 re-identification model to extract the center coordinates of each daughter cell's bounding box to construct a motion trajectory database. According to the obtained eigenvalues, since the feature points move on an approximate sphere, it is assumed that the rotation surface is a sphere. The radius and center of the cell are set, and the rotation angle of the current frame is calculated by inverse trigonometric functions. The specific algorithm is as follows: First, calculate the projection radius based on the calculated offset, that is, the distance Δx and Δy of the feature point relative to the center of the circle: Among them, r newx is the projection radius of the XZ plane, r newy Corresponding to the projection radius of the YZ plane, r represents the radius of the cell; then calculate the angle of the frame relative to the center of the circle: angle x ,angle y ,angle z The local rotation angles in the X, Y, and Z axes are calculated respectively. Then the difference between the relative angle of this frame and the previous frame is calculated to get the angle change of this frame relative to the previous frame. Finally, the final xyz angle is accumulated. According to the obtained angle, the rotation matrix around the X axis, the Y axis, and the Z axis is calculated. The total rotation matrix is ​​calculated and multiplied by the unit vector to obtain the rotation axis vector. Normalization is performed to obtain the rotation axis, where: R=R x ·R y ·R z R is the total rotation matrix, R x , R y , R z is the rotation matrix corresponding to each axis, α, β, and γ correspond to the rotation angles of the X, Y, and Z axes respectively.

10. The automated assisted reproductive system based on acoustic bubble microfluidics according to claim 8, characterized in that: After obtaining the total rotation matrix, the relative rotation matrix is ​​obtained by multiplying it with the transpose of the rotation matrix of the previous frame, which represents the rotation change from one frame to the current frame. The rotation angle is calculated by the relative rotation matrix to obtain the relative rotation angle of this frame relative to the coordinate axis between the previous frame and the current frame, and finally accumulated. Among them, R curr is the rotation matrix of the current frame, R prev is the rotation matrix of the previous frame, R rel Indicates the rotation change of the current frame relative to the previous frame. Trace is used to calculate the trace of the matrix. Current relative to the previous frame f(x,y) indicates that the angle of rotation of the original image is θ, and the cumulative rotation angle can be calculated by accumulation; Import each frame image and the corresponding rotation angle into the system integration platform. First, convert all cells into grayscale images, and perform sinogram transformation on each frame image. The formula is expressed as: R f (θ, t) = grayscale value of the image, R f (θ, t) represents the projection function after Radon transformation, which represents the projection intensity at angle θ and distance t, where t is the distance from the projection line to the origin on the image. θ represents the projection angle, usually ranging from 0° to 180°. The δ(·) function is then internally integrated to obtain the image depth of the embryonic cells in all directions. The next step is back-projection reconstruction; f′(x,y) is the image after back projection, that is, the final reconstructed image; N project : The total number of projection angles; backproject(θ): backprojection operation, which is to convert each projection data R f (θ, t) is back-projected back to the image space along the corresponding angle θ, and the tomogram is finally obtained; The obtained tomograms were preprocessed by segmenting the images into cell and background regions through thresholding, and volume data were extracted. The processed two-dimensional slices were stacked to generate a three-dimensional voxel dataset. The cell surface was extracted using the Marching Cubes surface extraction algorithm to generate a three-dimensional network model. A smoothing algorithm was applied to the generated surface network to remove noise and optimize the surface shape. The system also includes an intelligent evaluation module, which automatically quantifies the quality of embryo development based on a three-dimensional reconstruction model and combines morphological analysis methods to provide feedback for optimizing operating parameters. Specific indicators include: Evaluation criterion 1: the number and uniformity of blastomeres. On the third day, the ideal embryo should contain 7-9 blastomeres, and the sizes of these blastomeres should be uniform. This indicator reflects the normal division and development of the embryo. The basic idea of ​​quantity detection is to separate the blastomeres from the background through image segmentation methods and calculate the number of separated areas. After the image is binarized (after threshold segmentation, the blastomeres are white and the background is black), the number of blastomeres is calculated through connected domain analysis. Let N cell is the number of blastomeres, I(x,y) is the pixel value in the image, and P is the connected area after segmentation. If I(x,y)=1, it means the area belongs to the blastomere: The uniformity was assessed by calculating the ratio of the standard deviation of the blastomere area to the mean area. i is the area of ​​the i-th blastomere, ā is the average area of ​​all blastomeres, σ A is the standard deviation of the area, and the uniformity score U can be calculated as: If U is less than the set threshold of 0.25, the blastomeres are considered to be uniform in size. Evaluation criterion 2: Fragmentation assessment, which is based on the ratio of the area of ​​the fragment to the area of ​​the entire image. First, the fragment area is extracted through image segmentation, and the area of ​​the fragment area A is obtained after segmentation. frag , the area of ​​the entire image is A total The fragment ratio can be calculated using the following formula: Embryos are classified into three grades based on the proportion of internal fragments: Category 1: The amount of debris is less than 10%; Category II: fragmentation amount is 10%-25%; Category III: debris volume greater than 25%; The amount of fragments directly affects the quality of the embryo. The fewer fragments, the higher the quality of the embryo. Evaluation criterion three: The embryonic cytoplasm should be clear, smooth, and free of abnormal particles or color changes. Abnormal cytoplasmic appearance may indicate problems with embryonic development. The examination of cytoplasmic appearance involves color and texture analysis of the image. We use the standard deviation or local contrast of the image to evaluate the smoothness and granularity of the cytoplasm, and define the standard deviation σ of the local area of ​​the image as C , which is used to measure the uniformity of the cytoplasm. Let C(x,y) be the color intensity of the image (such as the pixel value in a certain color space), and the standard deviation of the local area is: Among them, M and N are the height and width of the region, is the average value of the pixels in the region. If the standard deviation is too large, it indicates that abnormal particles may exist in the cytoplasm. Evaluation criterion 4: Multinucleation detection. Each blastomere should have only one nucleus. Multinucleation is usually considered a negative indicator that may affect the subsequent development of the embryo. The nuclear region is extracted through image segmentation. If a blastomere contains multiple nuclear regions, it is marked as multinucleation. Let C i is the i-th blastomere region, N i The number of cell nuclei it contains. If N i >1, the blastomere is judged to be multinucleated. Evaluation Criteria 5: Blastocyst Stage Assessment. For embryos that have reached the blastocyst stage, the degree of expansion of the blastocyst cavity, the quality of the inner cell mass (ICM), and the development of the trophectoderm (TE) need to be assessed. These indicators further reflect the developmental potential of the embryo. The expansion of the blastocyst cavity can be assessed by measuring the area ratio of the blastocyst cavity. Hypothesis A blastocyst is the area of ​​the blastocyst cavity, A total The ratio of the blastocyst cavity expansion degree to the area of ​​the entire image can be expressed by the following formula: A large ratio of the blastocoel to the image area indicates good expansion. Based on the performance of the above indicators, a comprehensive score is given to each embryo. The scoring formula is: in, S1 to S5 are scores for blastomere number and uniformity, amount of fragmentation, cytoplasmic appearance, multinucleation, and blastocyst stage assessment, respectively, and w1 to w5 are weights. The scores range from 0 to 1, with 0 representing the worst and 1 representing the best.

Images