Oocyte image material collection method, device and equipment for image recognition

Abstract

This application discloses a method, apparatus, device, and storage medium for acquiring oocyte image materials for image recognition. The method involves acquiring first image data displayed under an oocyte-collecting microscope and corresponding first timestamps for each first image data point; detecting whether an operator is performing oocyte collection; if so, acquiring pipette image data and corresponding second timestamps for each pipette image data point; determining the start time of the oocyte collection operation based on the pipette image data and the second timestamps; and acquiring first image data for a predetermined time period before the start time point, thus obtaining oocyte image materials for image recognition. This method can automatically extract oocyte image materials, facilitating the training of machine learning models to assist in the detection and collection of oocytes. This application has wide applications in the field of image acquisition technology.

Description

Technical Field

[0001] This application relates to the field of image acquisition technology, and in particular to a method, apparatus, device, and storage medium for acquiring oocyte image materials for image recognition. Background Technology

[0002] In assisted reproductive technology (ART), oocytes are surgically retrieved from the patient's ovaries. After being flushed using a tubing system, the oocytes are collected and placed outside the body. Once retrieved, the fluid mixture is manually examined under a microscope, searching for oocytes field by field. When an oocyte is found, it is aspirated using a fine pipette and placed in a culture dish for storage or further processing such as in vitro fertilization and culture. This oocyte collection process is generally referred to as "oocyte retrieval," and it is a demanding task requiring extensive experience and sustained concentration. Furthermore, the large volume of fluid flushed out, along with the presence of granulosa cells and multiple microscopic layers, can lead to missed oocytes.

[0003] Currently, with the rapid development of artificial intelligence technology, various applications have emerged. Among them, image classification is a branch of artificial intelligence applications, which can be used in clinical settings to assist in oocyte retrieval. Image classification is usually implemented using machine learning models. However, before these models can be used, they require a large amount of image data for AI training. For the identification task in the current oocyte collection process, obtaining image data is very difficult and cannot be automated, resulting in low efficiency and accuracy in oocyte image data acquisition.

[0004] In summary, the problems with the relevant technologies urgently need to be addressed. Summary of the Invention

[0005] The purpose of this application is to at least partially solve one of the technical problems existing in the related art.

[0006] Therefore, one objective of this application is to provide a method, apparatus, device, and storage medium for acquiring oocyte image materials for image recognition.

[0007] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of this application include:

[0008] On one hand, embodiments of this application provide a method for acquiring oocyte image materials for image recognition, the method comprising:

[0009] Acquire the first image data displayed in the egg-collecting microscope and the first timestamp corresponding to each of the first image data;

[0010] The system detects whether the operator is performing an egg-collecting operation. If it is determined that the operator is performing an egg-collecting operation, it acquires the straw image data of the straw used by the operator and the second timestamp corresponding to each straw image data.

[0011] Based on the straw image data and the second timestamp, the starting time node at which the operator begins the egg-collecting operation is determined;

[0012] First image data of oocytes for a predetermined time period before the starting time node are obtained by acquiring the first timestamp, and image material of oocytes for image recognition is obtained.

[0013] In addition, the oocyte image material acquisition method for image recognition according to the above embodiments of this application may also have the following additional technical features:

[0014] Furthermore, in one embodiment of this application, the detection of whether an operator is performing an egg-collecting operation includes:

[0015] Acquire image data of the operator's hand movements;

[0016] Based on the hand gesture image data, it is detected whether the operator is performing an egg-collecting operation.

[0017] Furthermore, in one embodiment of this application, the detection of whether an operator is performing an egg-collecting operation includes:

[0018] The liquid level data of the oocyte collection dish or oocyte collection dish corresponding to the operator is detected;

[0019] When the change in the liquid level data exceeds a preset threshold, it is determined that the operator is performing an egg-collecting operation.

[0020] or,

[0021] Check whether the temperature-controlled box containing the oocyte collection dish has been opened;

[0022] When the constant temperature chamber is opened, it is confirmed that the operator is performing an egg-collecting operation.

[0023] Furthermore, in one embodiment of this application, determining the start time node for the operator to begin egg collection based on the straw image data and the second timestamp includes:

[0024] The height of the liquid column in the straw is determined based on the straw image data;

[0025] The pipette image data when the liquid column height first reaches a preset height threshold is determined as the target image data;

[0026] The second timestamp corresponding to the target image data is determined as the starting time node when the operator begins the egg-collecting operation.

[0027] Furthermore, in one embodiment of this application, detecting whether the operator is performing an egg-collecting operation based on the hand gesture image data includes:

[0028] The hand motion image data is input into the trained motion recognition model;

[0029] The action recognition model is used to identify whether the operator is performing an egg-collecting operation, thus obtaining the action recognition result.

[0030] Furthermore, in one embodiment of this application, the action recognition model is trained through the following steps:

[0031] Obtain a batch of sample images and the corresponding action labels for the sample images;

[0032] The sample image is input into the initialized action recognition model, and the action recognition model is used to perform action recognition on the sample image to obtain the sample recognition result;

[0033] Based on the sample recognition results and the action labels, determine the training loss value;

[0034] Based on the loss value, the parameters of the action recognition model are updated to obtain a trained action recognition model.

[0035] Furthermore, in one embodiment of this application, the method further includes:

[0036] Obtain the qualification information of the operator; the qualification information includes at least one of the following: years of service, technical title, and ability level;

[0037] The qualification information and the image materials are stored together.

[0038] On the other hand, embodiments of this application provide an oocyte image acquisition device for image recognition, the device comprising:

[0039] The acquisition unit is used to acquire the first image data displayed in the egg-picking microscope and the first timestamp corresponding to each of the first image data.

[0040] The detection unit is used to detect whether the operator is picking up eggs. When it is determined that the operator is picking up eggs, the unit acquires the straw image data of the straw used by the operator and the second timestamp corresponding to each straw image data.

[0041] The processing unit is used to determine the starting time node when the operator begins the egg-collecting operation based on the straw image data and the second timestamp;

[0042] The acquisition unit is used to acquire first image data for a predetermined time period before the starting time node, and obtain image material of oocytes for image recognition.

[0043] On the other hand, embodiments of this application provide a computer device, including:

[0044] At least one processor;

[0045] At least one memory for storing at least one program;

[0046] When the at least one program is executed by the at least one processor, the at least one processor implements the above-described method for acquiring oocyte image materials for image recognition.

[0047] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to implement the above-described method for acquiring oocyte image materials for image recognition.

[0048] The advantages and beneficial effects of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application:

[0049] This application discloses a method for acquiring oocyte image materials for image recognition. The method includes: acquiring first image data displayed in an oocyte-collecting microscope and a first timestamp corresponding to each of the first image data; detecting whether an operator is performing an oocyte-collecting operation; if it is determined that the operator is performing an oocyte-collecting operation, acquiring pipette image data of the pipette used by the operator and a second timestamp corresponding to each of the pipette image data; determining the start time node when the operator begins the oocyte-collecting operation based on the pipette image data and the second timestamp; acquiring first image data for a predetermined time period before the start time node, thereby obtaining oocyte image materials for image recognition. This method can automatically extract oocyte image materials, which is convenient for training and building machine learning models to assist in the detection and collection of oocytes, helping to reduce the workload of oocyte collection; and it does not require any extra operations from the operator, does not affect the original normal oocyte collection work, and can improve the accuracy of the obtained oocyte image materials through actual operation. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of this application or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0051] Figure 1 This is a schematic diagram of the implementation environment for a method for acquiring oocyte image materials for image recognition provided in this application embodiment;

[0052] Figure 2 This is a flowchart illustrating a method for acquiring oocyte image materials for image recognition provided in an embodiment of this application;

[0053] Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0054] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0055] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0057] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0058] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence. Its applications span all areas of artificial intelligence. Machine learning (deep learning) typically includes techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0059] In assisted reproductive technology (ART), oocytes are surgically retrieved from the patient's ovaries. After being flushed using a tubing system, the oocytes are collected and placed outside the body. Once retrieved, the fluid mixture is manually examined under a microscope, searching for oocytes field by field. When an oocyte is found, it is aspirated using a fine pipette and placed in a culture dish for storage or further processing such as in vitro fertilization and culture. This oocyte collection process is generally referred to as "oocyte retrieval," and it is a demanding task requiring extensive experience and sustained concentration. Furthermore, the large volume of fluid flushed out, along with the presence of granulosa cells and multiple microscopic layers, can lead to missed oocytes.

[0060] Currently, with the rapid development of artificial intelligence technology, various applications have emerged. Among them, image classification is a branch of artificial intelligence applications, which can be used in clinical settings to assist in oocyte retrieval. Image classification is usually implemented using machine learning models. However, before these models can be used, they require a large amount of image data for AI training. For the identification task in the current oocyte collection process, obtaining image data is very difficult and cannot be automated, resulting in low efficiency and accuracy in oocyte image data acquisition.

[0061] In view of this, this application provides a method for acquiring oocyte image materials for image recognition. During the oocyte collection process, based on the operator's oocyte-picking operation, the method determines the starting time point at which the operator discovers the oocyte using an oocyte-picking microscope. Then, the first image data displayed under the oocyte-picking microscope within a certain period before the starting time point is used as the oocyte image material for image recognition. On the one hand, this method can automatically extract oocyte image materials, facilitating the training of machine learning models to assist in the detection and collection of oocytes, thus helping to reduce the workload of oocyte collection. On the other hand, this method requires no additional operation from the operator, has no impact on the normal oocyte collection process, and can improve the accuracy of the obtained oocyte image materials through actual operation.

[0062] Figure 1 This is a schematic diagram illustrating the implementation environment of a method for acquiring oocyte image materials for image recognition, as provided in an embodiment of this application. (Refer to...) Figure 1 The main hardware and software components of this implementation environment include a user terminal 101 and a server 102, which are communicatively connected. The method for acquiring oocyte image materials for image recognition can be executed based on the interaction between the user terminal 101 and the server 102. For example, in some embodiments, the user terminal 101 can be responsible for acquiring oocyte images for image recognition and sending the acquired image materials to the server 102 for storage and training of machine learning models, thus assisting in the oocyte acquisition task. Of course, appropriate selections can be made according to the actual application, and this embodiment does not impose specific limitations on this.

[0063] The user terminal 101 may include, but is not limited to, smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, and in-vehicle terminals. The server 102 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The user terminal 101 and server 102 can establish a communication connection via a wireless or wired network. This wireless or wired network uses standard communication technologies and / or protocols. The network can be the Internet or any other network, including but not limited to any combination of Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wired or wireless networks, private networks, or virtual private networks.

[0064] Of course, this is understandable. Figure 1 The implementation environment described in this application is only one of the optional application scenarios for the oocyte image material acquisition method for image recognition provided in this embodiment. The actual application is not fixed. Figure 1 The software and hardware environment is shown. The methods provided in this application can be applied to various technical fields, and this application does not impose specific limitations on them.

[0065] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a method for acquiring oocyte image materials for image recognition, provided in an embodiment of this application. (Refer to...) Figure 2 This method for acquiring oocyte image materials for image recognition includes, but is not limited to:

[0066] Step 110: Obtain the first image data displayed in the egg retrieval microscope and the first timestamp corresponding to each of the first image data;

[0067] When collecting oocytes, operators need to use a microscope for observation; this application refers to it as an oocyte-collecting microscope. Generally, during oocyte collection, the fluid containing oocytes that has been rinsed out is stored in an oocyte-collecting dish. The operator needs to use a pipette to aspirate the oocytes and transfer them to an incubator containing an oocyte collection dish. The fluid containing oocytes aspirated from the pipette is blown out and injected into the oocyte collection dish.

[0068] In this step, during the oocyte collection process, image data displayed on the oocyte collection microscope can be acquired and recorded as the first image data. The first timestamp corresponding to each first image data point is also obtained. Specifically, a manufacturer-supplied or compatible image data acquisition accessory can be installed in the optical path channel of the oocyte collection microscope, and external hardware devices such as a video capture card can be connected to acquire the first image data. It can be understood that the acquired first image data represents the observation data taken by the operator during the oocyte collection process.

[0069] Step 120: Detect whether the operator is performing an egg-collecting operation. If it is determined that the operator is performing an egg-collecting operation, obtain the straw image data of the straw used by the operator and the second timestamp corresponding to each straw image data.

[0070] In this step, during the process of collecting oocytes, it is possible to detect whether the operator is performing an oocyte retrieval operation. If the operator is not performing an oocyte retrieval operation, it means that no oocytes are likely found at this time. However, if the operator is performing an oocyte retrieval operation, it means that an image of an oocyte appears in the first image data observed by the operator. Therefore, in this embodiment, when it is determined that the operator is performing an oocyte retrieval operation, the pipette image data used by the operator and the second timestamp corresponding to each pipette image data are obtained. It is understood that when the operator performs the oocyte retrieval operation, it needs to use a pipette to aspirate the oocytes and some liquid in the oocyte retrieval dish. During this process, the liquid column in the pipette will change, and a corresponding liquid level line will appear in the pipette image data. In this embodiment, when acquiring each pipette image, their corresponding timestamps are also acquired and recorded as the second timestamp. Subsequently, the time point when the operator started the liquid aspiration operation can be determined based on the second timestamp.

[0071] Step 130: Determine the starting time node when the operator begins the egg-collecting operation based on the straw image data and the second timestamp;

[0072] In this step, after obtaining the straw image data, the straw image data at the time when the operator begins the egg collection operation can be detected and determined based on the straw image data. Then, based on the second timestamp corresponding to the straw image data, the starting time node when the operator begins the egg collection operation can be determined.

[0073] Specifically, for example, in some embodiments, determining the starting time point at which the operator begins the egg-collecting operation based on the straw image data and the second timestamp includes:

[0074] The height of the liquid column in the straw is determined based on the straw image data;

[0075] The pipette image data when the liquid column height first reaches a preset height threshold is determined as the target image data;

[0076] The second timestamp corresponding to the target image data is determined as the starting time node when the operator begins the egg-collecting operation.

[0077] In this embodiment, when determining the starting time point, the height of the liquid column in the pipette can be determined based on the pipette image data. Specifically, it can be understood that the tube inside the pipette is generally smooth and without boundaries. In this embodiment, after image recognition of the pipette image data, edge detection processing can be performed on the liquid column inside the pipette. By analyzing whether there is a clear liquid surface line inside the pipette, it can be determined whether there is liquid in the pipette and the position of the liquid column edge, thus determining the height of the liquid column. Furthermore, since the follicle flushing fluid being processed is generally bloody, the color contrast is relatively obvious, making it easy to enhance the color contrast and smoothness using computer image processing technology, thereby reducing the difficulty of recognition and obtaining a more accurate liquid column height.

[0078] It is understandable that in this embodiment, when the operator uses a pipette to collect oocytes, if no oocytes are found, they generally will not aspirate or will only gently aspirate to identify them, and no significant liquid column will appear. Therefore, in this embodiment, a height threshold can be preset. When the liquid column height does not reach the preset height threshold, it indicates that the operator has not yet started the oocyte collection operation and is still in the observation and search stage; conversely, when the liquid column height exceeds the preset height threshold, it indicates that the operator has aspirated and started the oocyte collection operation. Therefore, the pipette image data at which the liquid column height first reaches the preset height threshold can be determined and recorded as the target image data. Then, the second timestamp corresponding to the target image data can be determined as the starting time node when the operator begins the oocyte collection operation.

[0079] Step 140: Obtain first image data for a predetermined time period before the starting time node, using the first timestamp, to obtain image material of oocytes for image recognition.

[0080] In this step, after determining the starting time node, first image data within a predetermined time period prior to the starting time node can be acquired. It is understood that since the operator begins collecting oocytes at the starting time node, it means that before the starting time node, the operator likely located oocytes in the field of view of the oocyte-collecting microscope. Therefore, within a certain period before the starting time node, the first image data is likely to contain oocytes, making it suitable as image material containing oocytes for training a machine learning model for image recognition. Specifically, in this embodiment, the length of the predetermined time period before the starting time node can be flexibly set as needed. For example, in some embodiments, the predetermined time period can be set to 2 seconds, allowing the acquisition of first image data within 2 seconds before the starting time node as image material. The specific time period can be determined based on the starting time node and the first timestamp, which will not be elaborated upon here.

[0081] It is understood that the oocyte image acquisition method provided in this application, during the oocyte collection process, determines the starting time point of the oocyte discovered by the operator through the oocyte-collecting microscope based on the operator's oocyte-collecting operation. Then, the first image data displayed in the oocyte-collecting microscope within a period of time before the starting time point is used as the oocyte image material for image recognition. On the one hand, this method can automatically extract oocyte image materials, which is convenient for training and building machine learning models to assist in the detection and collection of oocytes, and helps to reduce the workload of oocyte collection. On the other hand, this method does not require any extra operations from the operator, has no impact on the original normal oocyte collection work, and can improve the accuracy of the obtained oocyte image materials through actual operation.

[0082] In some embodiments, detecting whether an operator is performing an egg-collecting operation includes:

[0083] Acquire image data of the operator's hand movements;

[0084] Based on the hand gesture image data, it is detected whether the operator is performing an egg-collecting operation.

[0085] In this embodiment, when detecting whether an operator is performing an egg-collecting operation, the operator's hand gesture images can be acquired using a camera device. Then, based on the hand gesture image data, the operation is detected. Specifically, for example, the hand gesture image data can be input into a trained action recognition model. The action recognition model can then identify whether the operator is performing an egg-collecting operation, yielding the corresponding action recognition result. Here, the action recognition model can be built using any machine learning algorithm; this application does not impose any restrictions. Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and many other disciplines. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent; its applications span all areas of artificial intelligence.

[0086] In some embodiments, the action recognition model is trained through the following steps:

[0087] Obtain a batch of sample images and the corresponding action labels for the sample images;

[0088] The sample image is input into the initialized action recognition model, and the action recognition model is used to perform action recognition on the sample image to obtain the sample recognition result;

[0089] Based on the sample recognition results and the action labels, determine the training loss value;

[0090] Based on the loss value, the parameters of the action recognition model are updated to obtain a trained action recognition model.

[0091] In this embodiment, before the action recognition model is put into use, it needs to be trained to adjust its internal parameters and achieve better prediction results. Specifically, when training the model, a batch of sample images can be acquired. Each sample image may include image data of the hand movements of relevant personnel, and the corresponding action label for the sample image is also acquired. This label is used to characterize the true type of the hand movements of relevant personnel in the sample image, such as the category of egg-picking. Then, each sample image and its corresponding label can be used as a set of training data. The input data of the model is the sample image, and the model predicts the sample images. The output data of the model is the sample recognition result. After obtaining the sample recognition result output by the model, the accuracy of the model prediction can be evaluated based on the sample recognition result and the action label, thereby updating the model parameters.

[0092] Specifically, for machine learning models, the accuracy of model predictions can be measured by a loss function. The loss function is defined on a single training data point and measures the prediction error of that data point. Specifically, the loss value is determined by the label of the individual training data point and the model's prediction result for that data point. However, in actual training, a training dataset contains many data points. Therefore, a cost function is generally used to measure the overall error of the training dataset. The cost function is defined on the entire training dataset and calculates the average prediction error of all training data points, providing a better measure of the model's prediction performance. For general machine learning models, the aforementioned cost function, plus a regularization term to measure model complexity, serves as the training objective function. Based on this objective function, the loss value of the entire training dataset can be calculated. Many types of loss functions are commonly used, such as 0-1 loss, squared loss, absolute loss, logarithmic loss, and cross-entropy loss, which will not be elaborated upon here. In this embodiment, any one of these loss functions can be selected to determine the training loss value, such as the cross-entropy loss function. Based on the training loss value, the backpropagation algorithm is used to update the model parameters. After several iterations, a well-trained action recognition model can be obtained. The specific number of iterations can be preset, or training can be considered complete when the accuracy requirement is met on the test set.

[0093] In some embodiments, detecting whether an operator is performing an egg-collecting operation includes:

[0094] The liquid level data of the oocyte collection dish or oocyte collection dish corresponding to the operator is detected;

[0095] When the change in the liquid level data exceeds a preset threshold, it is determined that the operator is performing an egg-collecting operation.

[0096] or,

[0097] Check whether the temperature-controlled box containing the oocyte collection dish has been opened;

[0098] When the constant temperature chamber is opened, it is confirmed that the operator is performing an egg-collecting operation.

[0099] In this embodiment, it is understood that if an operator performs an egg-collecting operation, they need to draw some liquid from the egg-collecting dish and transfer it to the oocyte collection dish. During this process, the liquid level in the corresponding egg-collecting dish or oocyte collection dish will change. Therefore, in this embodiment, the liquid level data of the egg-collecting dish or oocyte collection dish can be detected, the changes in the liquid level data can be observed, and a corresponding threshold can be set. If the change in the liquid level data is greater than the preset threshold, it can be determined that the operator has performed an egg-collecting operation; conversely, if the change in the liquid level data is less than or equal to the preset threshold, it can be determined that the operator has not performed an egg-collecting operation. In some embodiments, pipette image data can also be acquired first, and the change in the liquid column in the pipette image data can be used to determine whether the operator has performed an egg-collecting operation. If the operator has performed an egg-collecting operation, the start time of the operator's egg-collecting operation can be further determined based on the pipette image data. Conversely, if the operator has not performed an egg-collecting operation, the pipette image data can be deleted to reduce the memory space occupied by image storage.

[0100] In some embodiments, the system can also detect whether the operator has opened the incubator containing the oocyte collection dish to determine if the operator has performed an oocyte retrieval operation. For example, the incubator can be monitored for open and closed states using magnetic materials or RFID sensors. Furthermore, it can be equipped with a sensor switch, such as an infrared sensor. When the operator needs to open or close the incubator, they simply need to bring their hand near the sensor switch. This greatly improves the convenience and efficiency of the operation.

[0101] Of course, this application does not limit whether the specific detection operator uses egg-collecting techniques. For example, in some embodiments, multiple detection methods can be combined to comprehensively determine the method, thereby improving the accuracy of the detection.

[0102] In some embodiments, the method further includes:

[0103] Obtain the qualification information of the operator; the qualification information includes at least one of the following: years of service, technical title, and ability level;

[0104] The qualification information and the image materials are stored together.

[0105] In this embodiment, a central server can be set up. After the collection of image materials is completed, the distributed servers can automatically upload all the acquired image materials, along with the operator's qualification information, to the central server for associated storage. Here, the qualification information may include at least one of the following: years of experience, technical title, and ability level, to facilitate the selection of suitable image materials for subsequent model training.

[0106] This application embodiment also provides an oocyte image material acquisition device for image recognition, the device comprising:

[0107] The acquisition unit is used to acquire the first image data displayed in the egg-picking microscope and the first timestamp corresponding to each of the first image data.

[0108] The detection unit is used to detect whether the operator is picking up eggs. When it is determined that the operator is picking up eggs, the unit acquires the straw image data of the straw used by the operator and the second timestamp corresponding to each straw image data.

[0109] The processing unit is used to determine the starting time node when the operator begins the egg-collecting operation based on the straw image data and the second timestamp;

[0110] The acquisition unit is used to acquire first image data for a predetermined time period before the starting time node, and obtain image material of oocytes for image recognition.

[0111] Understandable Figure 2 The content of the oocyte image material acquisition method embodiment for image recognition shown is applicable to the oocyte image material acquisition device embodiment for image recognition. The specific functions implemented by the oocyte image material acquisition device embodiment for image recognition are the same as those shown in the embodiment. Figure 2 The embodiment shown is the same as the method for acquiring oocyte image materials for image recognition, and the beneficial effects achieved are the same. Figure 2 The beneficial effects achieved by the embodiment of the oocyte image material acquisition method for image recognition shown are also the same.

[0112] Reference Figure 3 This application also discloses a computer device, including:

[0113] At least one processor 301;

[0114] At least one memory 302 is used to store at least one program;

[0115] When at least one program is executed by at least one processor 301, such that at least one processor 301 performs as follows: Figure 2 An embodiment of a method for acquiring oocyte image materials for image recognition is shown.

[0116] It is understandable that, such as Figure 2The content of the oocyte image material acquisition method embodiment for image recognition shown is applicable to the embodiment of this computer device. The specific functions implemented by the embodiment of this computer device are the same as those shown below. Figure 2 The embodiment shown is the same as the one for acquiring oocyte image materials for image recognition, and the beneficial effects achieved are the same as those described above. Figure 2 The beneficial effects achieved by the embodiment of the oocyte image material acquisition method for image recognition shown are also the same.

[0117] This application also discloses a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to implement, for example... Figure 2 An embodiment of a method for acquiring oocyte image materials for image recognition is shown.

[0118] It is understandable that, such as Figure 2 The content of the oocyte image material acquisition method embodiment for image recognition shown is applicable to the embodiment of this computer-readable storage medium. The specific functions implemented by the embodiment of this computer-readable storage medium are the same as those shown below. Figure 2 The embodiment shown is the same as the one for acquiring oocyte image materials for image recognition, and the beneficial effects achieved are the same as those described above. Figure 2 The beneficial effects achieved by the embodiment of the oocyte image material acquisition method for image recognition shown are also the same.

[0119] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0120] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features may be integrated into a single physical system and / or software module, or one or more functions and / or features may be implemented in a separate physical system or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the system disclosed herein, the actual implementation of the module will be understood within the scope of ordinary skill of an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0121] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, system, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, system, or device). For the purposes of this specification, "computer-readable medium" can mean any system that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, system, or device.

[0123] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections with one or more wires (electronic systems), portable computer disk drives (magnetic systems), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic systems, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0124] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0125] In the foregoing description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0126] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0127] The foregoing has provided a detailed description of the preferred embodiments of this application. However, this application is not limited to these embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

[0128] In the description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0129] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for acquiring oocyte image materials for image recognition, characterized in that, The method includes: Acquire the first image data displayed in the egg-collecting microscope and the first timestamp corresponding to each of the first image data; The system detects whether the operator is performing an egg-collecting operation. If it is determined that the operator is performing an egg-collecting operation, it acquires the straw image data of the straw used by the operator and the second timestamp corresponding to each straw image data. Based on the straw image data and the second timestamp, the starting time node at which the operator begins the egg-collecting operation is determined; First image data of oocytes for a predetermined time period before the starting time node is obtained by acquiring the first timestamp; The step of determining the starting time point for the operator to begin egg collection based on the straw image data and the second timestamp includes: The height of the liquid column in the straw is determined based on the straw image data; The pipette image data when the liquid column height first reaches a preset height threshold is determined as the target image data; The second timestamp corresponding to the target image data is determined as the starting time node when the operator begins the egg-collecting operation.

2. The method for acquiring oocyte image materials for image recognition according to claim 1, characterized in that, The detection of whether the operator was engaged in egg collection includes: Acquire image data of the operator's hand movements; Based on the hand gesture image data, it is detected whether the operator is performing an egg-collecting operation.

3. The method for acquiring oocyte image materials for image recognition according to claim 1, characterized in that, The detection of whether the operator was engaged in egg collection includes: The liquid level data of the oocyte collection dish or oocyte collection dish corresponding to the operator is detected; When the change in the liquid level data exceeds a preset threshold, it is determined that the operator is performing an egg-collecting operation. or, Check whether the incubator containing the oocyte collection dish has been opened; When the constant temperature chamber is opened, it is confirmed that the operator is performing an egg-collecting operation.

4. The method for acquiring oocyte image materials for image recognition according to claim 2, characterized in that, The step of detecting whether the operator is performing an egg-collecting operation based on the hand gesture image data includes: The hand motion image data is input into the trained motion recognition model; The action recognition model is used to identify whether the operator is performing an egg-collecting operation, thus obtaining the action recognition result.

5. A method for acquiring oocyte image materials for image recognition according to claim 4, characterized in that, The action recognition model is trained through the following steps: Obtain a batch of sample images and the corresponding action labels for the sample images; The sample image is input into the initialized action recognition model, and the action recognition model is used to perform action recognition on the sample image to obtain the sample recognition result; Based on the sample recognition results and the action labels, determine the training loss value; Based on the loss value, the parameters of the action recognition model are updated to obtain a trained action recognition model.

6. The method for acquiring oocyte image materials for image recognition according to claim 1, characterized in that, The method further includes: Obtain the qualification information of the operator; the qualification information includes at least one of the following: years of service, technical title, and ability level; The qualification information and the image materials are stored together.

7. A device for acquiring oocyte image data for image recognition, characterized in that, The device includes: The acquisition unit is used to acquire the first image data displayed in the egg-picking microscope and the first timestamp corresponding to each of the first image data. The detection unit is used to detect whether the operator is performing an egg-collecting operation. When it is determined that the operator is performing an egg-collecting operation, the unit acquires the straw image data of the straw used by the operator and the second timestamp corresponding to each straw image data. The processing unit is used to determine the starting time node when the operator begins the egg-collecting operation based on the straw image data and the second timestamp; The acquisition unit is used to acquire first image data for a predetermined time period before the starting time node, and obtain image material of oocytes for image recognition; The step of determining the starting time point for the operator to begin egg collection based on the straw image data and the second timestamp includes: The height of the liquid column in the straw is determined based on the straw image data; The pipette image data when the liquid column height first reaches a preset height threshold is determined as the target image data; The second timestamp corresponding to the target image data is determined as the starting time node when the operator begins the egg-collecting operation.

8. A computer device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a method for acquiring oocyte image materials for image recognition as described in any one of claims 1-6.

9. A computer-readable storage medium storing a processor-executable program, characterized in that: The processor-executable program, when executed by the processor, is used to implement a method for acquiring oocyte image materials for image recognition as described in any one of claims 1-6.

Images