Data processing apparatus, data processing method, and data processing program

Abstract

To provide a data processing device, a data processing method, and a program capable of easily evaluating an embryo obtained by in vitro fertilization.SOLUTION: A data processing device includes an acquisition unit that acquires microscope image data of an embryo obtained by in vitro fertilization and request sentence data representing an evaluation request for an embryo shown in the microscope image data, an evaluation unit that inputs the microscope image data and the request sentence data to a data generation model and acquires evaluation data of the embryo shown in the microscope image data output from the data generation model and corresponding to the request sentence data, and an output unit that outputs the evaluation data of the embryo.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a data processing device, a data processing method, and a data processing program. [Background technology]

[0002] US Patent No. 6,299,649 discloses an apparatus, method and system for image-based human germ cell classification.

[0003] Patent Document 2 discloses an artificial intelligence (AI) calculation system for generating an embryo viability score from a single image of an embryo to contribute to the selection of embryos for implantation in in vitro fertilization (IVF) methods.

[0004] U.S. Patent Application Publication No. 2009 / 0129999 discloses a system for predicting the viability of one or more embryos. The system disclosed in U.S. Patent Application Publication No. 2009 / 0129999 may include receiving a single image of an embryo via a real-time communication link with an image capture device, and generating a viability score for the embryo by classifying the single image via at least one convolutional neural network. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2022-087297 [Patent Document 2] Special Publication No. 2022-528961 [Patent Document 3] Special Publication No. 2024-513659 Summary of the Invention [Problem to be solved by the invention]

[0006] Incidentally, when a system evaluates embryos obtained through in vitro fertilization, it is preferable for the system to perform the evaluation as desired by the user. For example, if the user wants to know the predicted results of the embryo implantation rate, it is preferable for the system to immediately output the implantation rate. Also, if the user wants to know the embryo grade, it is preferable for the system to immediately output the embryo grade. Furthermore, if there is any supplementary information regarding the embryo being evaluated, it is preferable for the system to evaluate the embryo taking that supplementary information into account. Such a system requires dialogue with the user.

[0007] In this regard, there is room for improvement in the prior art. Specifically, the input data in the technologies disclosed in the above Patent Documents 1 to 3 is only image data of embryos, and text data input by users cannot be handled. Therefore, the technologies disclosed in the above Patent Documents 1 to 3 have the problem that they cannot evaluate embryos obtained by in vitro fertilization while having a dialogue with the user. [Means for solving the problem]

[0008] A first aspect of the technology disclosed herein is a data processing device comprising: an acquisition unit that acquires microscopic image data of an embryo obtained by in vitro fertilization and request text data representing a request for evaluation of the embryo depicted in the microscopic image data; an evaluation unit that inputs the microscopic image data and the request text data into a data generation model and acquires evaluation data of the embryo depicted in the microscopic image data output from the data generation model, the evaluation data corresponding to the request text data; and an output unit that outputs the evaluation data of the embryo.

[0009] A second aspect of the technology disclosed herein is a data processing method that includes acquiring microscopic image data of an embryo obtained by in vitro fertilization and request text data representing a request for evaluation of the embryo depicted in the microscopic image data, inputting the microscopic image data and the request text data into a data generation model, acquiring evaluation data of the embryo depicted in the microscopic image data output from the data generation model that corresponds to the request text data, and outputting the evaluation data of the embryo.

[0010] A third aspect of the technology disclosed herein is a program for causing a computer to execute a process including acquiring microscopic image data of an embryo obtained by in vitro fertilization and request text data representing a request for evaluation of the embryo depicted in the microscopic image data, inputting the microscopic image data and the request text data into a data generation model, acquiring evaluation data of the embryo depicted in the microscopic image data output from the data generation model, which evaluation data corresponds to the request text data, and outputting the evaluation data of the embryo. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a conceptual diagram illustrating an example of a configuration of a data processing system. [Figure 2] FIG. 2 is a conceptual diagram showing an example of main functions of a data processing device and a user terminal. [Figure 3] FIG. 1 is a diagram illustrating an embryo obtained by in vitro fertilization. [Figure 4] 2 shows a schematic functional configuration of a specific processing unit of the data processing device. [Figure 5A] 10 is an example of a screen displayed on a display of a user terminal. [Figure 5B] 10 is an example of a screen displayed on a display of a user terminal. [Figure 5C] 10 is an example of a screen displayed on a display of a user terminal. [Figure 6] 10 is a diagram illustrating an example of an operational flow of specific processing by a data processing device. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an example of an embodiment of a data processing device, a data processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0013] First, the terms used in the following description will be explained.

[0014] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), or an APU (Accelerated Processing Unit).

[0015] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0016] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0017] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0019] FIG. 1 shows an example of the configuration of a data processing system 10 according to the embodiment.

[0020] As shown in Fig. 1, the data processing system 10 includes a data processing device 12 and a user terminal 14. An example of the data processing device 12 is a server. An example of the user terminal 14 is a personal computer or a smartphone. In this embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of the present disclosure, and the user terminal 14 is an example of a "terminal" according to the technology of the present disclosure.

[0021] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0022] The user terminal 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0023] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0024] The output device 40 includes a display 40A and a speaker 40B, and presents data to the person 20 by outputting the data in a form of expression that the person 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0025] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0026] FIG. 2 shows an example of the main functions of the data processing device 12 and the user terminal 14.

[0027] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0028] The storage 32 stores a data generation model 58. The data generation model 58 is used by the specific processing unit 290.

[0029] In the user terminal 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 62. The reception output program 62 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 62 from the storage 50 and executes the read reception output program 62 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 62 executed on the RAM 48.

[0030] Next, the processing of the specific processing unit 290 when the data processing device 12 performs specific processing for evaluating an embryo obtained by in vitro fertilization will be described.

[0031] FIG. 3 is a diagram illustrating an embryo obtained by in vitro fertilization. As shown in FIG. 3, an embryo obtained by in vitro fertilization progresses through the 2-cell, 4-cell, and 8-cell stages to become a morula, and then a blastocyst. The blastocyst is returned to the uterus to promote implantation. The embryos shown in FIG. 3 are visually inspected, for example, by a doctor, and only embryos in good condition are selected and returned to the uterus. However, the evaluation criteria for embryos by visual inspection are vague, and the evaluation results may vary. For example, one doctor may evaluate an embryo in a microscopic image as "good," while another doctor may evaluate the embryo as "poor."

[0032] Therefore, in this embodiment, the embryos obtained by in vitro fertilization are evaluated using a data generation model 58, which will be described later. This makes it possible to reduce the variability in the results of embryo evaluation by visual inspection. Furthermore, as will be described later, the data generation model 58 can also handle text data, so that the target embryos can be evaluated in the way that the user desires while having a dialogue with the user.

[0033] As shown in FIG. 4, the specification processing unit 290 includes an acquisition unit 292, an evaluation unit 294, and an output unit 296.

[0034] The acquisition unit 292 acquires user input accepted by the user terminal 14. Specifically, the acquisition unit 292 acquires at least one of text, voice, and image data of the user accepted by the user terminal 14. Note that the user in this embodiment is, for example, a doctor.

[0035] Specifically, the acquisition unit 292 acquires, input by the user, microscopic image data of an embryo obtained by in vitro fertilization and request sentence data expressing a request for evaluation of the embryo captured in the microscopic image data.

[0036] 5A to 5C are examples of screens displayed on the display 40A of the user terminal 14. The acquisition unit 292 of the data processing device 12 causes the display 40A of the user terminal 14 to display the screens shown in FIGS.

[0037] The left side of the screen shown in FIG. 5A shows the source data used to train the data generation model 58. The right side of the screen shown in FIG. 5A provides a field for inputting microscopic image data of the embryo to be analyzed (the "Upload Image" section in FIG. 5A). The right side of the screen shown in FIG. 5A also provides a field for inputting supplemental information about the embryo to be analyzed. This field for inputting supplemental information can accept text data input by the user.

[0038] 5B, when the microscopic image data to be analyzed is input, the request text data "Please analyze the implantation rate of this embryo" is input in the field for inputting supplemental information, and the "Start analysis" button is clicked, the acquisition unit 292 acquires the microscopic image data and the request text data. The text data "Please analyze the implantation rate of this embryo" also serves as a prompt for the data generation model 58.

[0039] The evaluation unit 294 performs a specification process using the data generation model 58. Specifically, the evaluation unit 294 inputs the microscopic image data and the requested sentence data acquired by the acquisition unit 292 into the data generation model 58, and obtains evaluation data of the embryo as a result of the generation. More specifically, the evaluation unit 294 obtains evaluation data of the embryo depicted in the microscopic image data output from the data generation model 58, which corresponds to the requested sentence data.

[0040] The output unit 296 transmits the evaluation data that is the result of the specific processing to the user terminal 14. In the user terminal 14, the control unit 46A causes the display 40A of the output device 40 to output the evaluation data that is the result of the specific processing.

[0041] Fig. 5C is a diagram showing an example of evaluation data displayed on display 40 A. The evaluation data shown in Fig. 5C includes text data indicating the implantation rate of the embryos shown in the microscopic image data, the grade of the embryos, and the evaluation of the embryos.

[0042] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generative AI models. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0043] 5A, the user can perform additional training and fine-tune the data generation model 58. In this case, for example, the acquisition unit 292 acquires training data that is a combination of training microscope image data that shows an embryo different from the embryo shown in the microscopic image data to be analyzed and that shows an embryo provided by the patient who provided the embryo shown in the microscopic image data to be analyzed, and training evaluation data for the training microscope image data.

[0044] Next, the evaluation unit 294 generates a new data generation model 58 for the patient by fine-tuning the data generation model 58 using a known machine learning algorithm based on the learning data. The evaluation unit 294 then inputs the microscopic image data to be analyzed into the new data generation model 58 for the patient, and obtains evaluation data of the embryos depicted in the microscopic image data output from the new data generation model 58 for the patient. In this way, the evaluation unit 294 can generate evaluation data that is more suited to the target patient by fine-tuning the data generation model 58 for the patient.

[0045] In addition, the user can input various text data as prompt request text data in the "Supplementary Information" field on the screen shown in Figure 5A. For example, it is possible to input text data relating to the specific circumstances of the patient who provided the embryo to be analyzed.

[0046] Furthermore, in the "Upload Image" field on the screen shown in FIG. 5A, the user may upload time-series data (growth data) of microscopic image data showing the growth process of one embryo, rather than a single piece of microscopic image data. In this case, the acquisition unit 292 acquires growth data, which is time-series data of microscopic image data showing embryos provided by the same patient and showing the growth process of one embryo. The evaluation unit 294 inputs the growth data and the request sentence data acquired by the acquisition unit 292 to the data generation model 58. The data generation model 58 outputs evaluation data corresponding to the input growth data. The evaluation unit 294 acquires the evaluation data output from the data generation model 58. Note that the evaluation unit 294 may extract feature data from the microscopic image data using a known image processing technique and input the feature data to the data generation model 58.

[0047] 5C shows an example in which the evaluation data includes text data representing the implantation rate, embryo grade, and embryo evaluation of the embryos captured in the microscopic image data. However, only the evaluation data corresponding to the requested text data may be output. For example, when requested text data such as "Please calculate only the implantation rate" is input, the data generation model 58 outputs only the implantation rate as evaluation data. Furthermore, when requested text data such as "Please output only the grade" is input, the data generation model 58 outputs only the embryo grade as evaluation data. In this way, by using the data generation model 58 capable of handling text data when evaluating embryos captured in microscopic image data, the user's desired evaluation can be performed more flexibly.

[0048] Next, the operation of the data processing system 10 will be described.

[0049] An example of the flow of the identification process will be described with reference to Fig. 6. The flow of the identification process shown in Fig. 6 is an example of a "data processing method" according to the technique of the present disclosure.

[0050] In step S300, the acquisition unit 292 acquires the microscopic image data of the embryo to be analyzed and the request sentence data input by the user.

[0051] In step S301, the evaluation unit 294 inputs the microscopic image data of the embryo and the request sentence data acquired in step S300 to the data generation model 58.

[0052] In step S302, the evaluation unit 294 acquires evaluation data of the embryo captured in the microscopic image data output from the data generation model 58 and that corresponds to the requested sentence data.

[0053] In step S303, the output unit 296 outputs the evaluation data to the user terminal 14, and the identification process ends.

[0054] In this way, the data processing device of this embodiment acquires microscopic image data of an embryo obtained by in vitro fertilization and request text data representing a request for evaluation of the embryo depicted in the microscopic image data. The data processing device inputs the microscopic image data and the request text data into a data generation model, and acquires evaluation data of the embryo depicted in the microscopic image data output from the data generation model, which evaluation data corresponds to the request text data. The data processing device outputs the embryo evaluation data. This makes it possible to easily evaluate an embryo obtained by in vitro fertilization. It also reduces the variability in embryo evaluation results obtained by visual inspection by a doctor or the like. Furthermore, by using a data generation model capable of handling text data, it is possible to conduct a dialogue with the user and evaluate the target embryo in the way the user desires.

[0055] The data processing device of this embodiment is realized by an in vitro fertilization embryo evaluation technology that utilizes AI (Artificial Intelligence). In this embodiment, AI analyzes microscopic images of embryos or embryo growth data to identify embryos with a high probability of implantation. This can improve the success rate of treatment and the birth rate of healthy babies. It can also reduce the burden on infertility patients, improve their satisfaction, and make efficient use of medical resources. It can also provide patients with personalized treatment plans and support.

[0056] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 290, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[0057] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0058] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0059] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0060] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0061] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0062] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[0063] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0064] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0065] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0066] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0067] 10 Data Processing System 12 Data Processing Device 14 User terminal 290 Special Processing Department 292 Acquisition Department 294 Evaluation Department 296 Output Section< / url:>

Claims

1. an acquisition unit that acquires microscopic image data of an embryo obtained by in vitro fertilization and request sentence data that represents a request for evaluation of the embryo captured in the microscopic image data; an evaluation unit that inputs the microscopic image data and the requested sentence data into a data generation model and acquires evaluation data of an embryo captured in the microscopic image data output from the data generation model, the evaluation data corresponding to the requested sentence data; an output unit that outputs evaluation data of the embryo; 2. A data processing device comprising:

2. the acquisition unit acquires training data that is a combination of training microscope image data that shows an embryo different from the embryo shown in the microscope image data and that shows an embryo provided by a patient who provided the embryo shown in the microscope image data, and training evaluation data for the training microscope image data; the evaluation unit generates a new data generation model for the patient by fine-tuning the data generation model based on the learning data; inputting the microscopic image data into a new data generation model for the patient, and obtaining evaluation data of the embryo depicted in the microscopic image data output from the new data generation model for the patient; 2. The data processing device according to claim 1.

3. The evaluation data includes at least one of an implantation rate of the embryos shown in the microscopic image data, a grade of the embryos, and text data representing an evaluation of the embryos.

3. The data processing device according to claim 1.

4. the acquisition unit acquires growth data, which is time-series data of microscopic image data showing embryos provided by the same patient and which represents the process of growth of one embryo; the evaluation unit inputs the growth data and the request sentence data into the data generation model, and acquires the evaluation data output from the data generation model; 3. The data processing device according to claim 1.

5. Obtaining microscopic image data of an embryo obtained by in vitro fertilization and request sentence data expressing a request for evaluation of the embryo shown in the microscopic image data; inputting the microscopic image data and the requested sentence data into a data generation model, and obtaining evaluation data of the embryo depicted in the microscopic image data output from the data generation model, the evaluation data corresponding to the requested sentence data; outputting the evaluation data of the embryo; A data processing method in which a computer executes a process including:

6. Obtaining microscopic image data of an embryo obtained by in vitro fertilization and request sentence data expressing a request for evaluation of the embryo shown in the microscopic image data; inputting the microscopic image data and the requested sentence data into a data generation model, and obtaining evaluation data of the embryo depicted in the microscopic image data output from the data generation model, the evaluation data corresponding to the requested sentence data; outputting the evaluation data of the embryo; A program that causes a computer to execute processes including the above.

Images