Image processing method, and predictive model based on results of nematode taxis behavior with respect to cancer patient urine

Abstract

The present disclosure provides a method for processing an image in which a distribution of nematodes after motion in a taxis behavior assay of nematodes with respect to a urine sample obtained from a subject is recorded. The method of the present disclosure involves, for example, performing one or more processes selected from the group consisting of: (i) a process for cutting out, from the image, an image of a urine sample placement portion and a surrounding portion around the entire circumference thereof, including the nematodes, and (ii) a process for emphasizing the nematodes in the image.

Description

Image processing method and predictive model based on the results of nematode taxis toward cancer patient urine

[0001] The present disclosure relates to a method for processing images showing the distribution of nematodes after migration in a nematode migration assay for the urine of cancer patients. The present disclosure also relates to a predictive model based on the results of the nematode migration to the urine of cancer patients, wherein the predictive model can predict the presence or absence of cancer in the patient, and preferably the type of cancer the cancer patient has. The present disclosure further relates to a method for predicting the type of cancer a cancer patient has using the predictive model, and a diagnostic support system for implementing the method.

[0002] This method is constructed as a cancer screening test that utilizes the characteristics of nematodes that are attracted to cancer-specific odors in biological substances or processed products thereof. Patent Document 1 discloses a method for determining whether a subject from which a sample is derived is likely to have cancer by utilizing differences in the responsiveness of nematodes to urine samples.

[0003] WO2015 / 088039

[0004] The present disclosure provides a method for processing an image showing the distribution of nematodes after their migration in a nematode migration assay for the urine of a cancer patient. The present disclosure also provides a predictive model based on the results of the nematode's migration to the urine of a cancer patient. The predictive model can predict the presence or absence of cancer in the patient, and preferably the type of cancer the cancer patient has. The present disclosure further provides a method for predicting the type of cancer a cancer patient has using the predictive model, and a diagnostic support system for implementing the method.

[0005] The present inventors have invented a method for processing images showing the distribution of nematodes after their migration to the urine of cancer patients in an assay of their migration to the urine of cancer patients. The present inventors have demonstrated that the presence or absence of cancer in a patient, and preferably the type of cancer the cancer patient has, can be predicted from the distribution of nematodes after their migration to the urine of cancer patients in an assay of their migration to the urine of cancer patients. They have also found that preprocessing images using the image processing method improves prediction accuracy. Based on this finding, the present inventors have further demonstrated that the image processing method can be useful as preprocessing for obtaining training data for training a prediction model.

[0006] The present disclosure provides the following inventions, for example: (1) A method for processing an image recording the distribution of nematodes after behavior in a taxis behavior assay of the nematodes with respect to a urine sample obtained from a subject, the method comprising: obtaining an image recording the distribution of nematodes after behavior in a taxis behavior assay of the nematodes with respect to a urine sample obtained from a subject; and performing one or more steps selected from the group consisting of: (i) cropping from the image an image showing the distribution of nematodes gathered in the urine sample placement area; and (ii) highlighting the nematodes in the image to obtain a processed image, the method being implemented on a computer. (2) The method according to (1) above, which includes the cropping process defined in (i) above. (3) The method according to (2) above, which further includes highlighting the nematodes in each image obtained by the cropping process. (4) The method according to (2) or (3) above, which does not include connecting multiple images obtained by the cropping process on a specimen-by-specimen basis. (5) The method according to any one of (1) to (4) above, which does not include image inversion. (6) The method according to any one of (1) to (5) above, which does not perform image selection processing based on the number of nematodes attracted to the urine sample. (7) The method according to any one of (1) to (6) above, which performs the clipping processing defined in (i) above, a processing to emphasize nematodes in each image obtained by the clipping processing, and a processing to remove images obtained by the clipping processing in which the area ratio of nematodes does not meet a standard. (8) The method according to any one of (1) to (7) above, wherein the processed image is used to generate a predictive model for predicting the type of cancer a subject has from training data including the processed image and the type of cancer the subject has linked to the image, or the processed image is used to predict the type of cancer the subject from whom the urine sample that generated the processed image was derived has, using a predictive model generated from other processed images and training data including the type of cancer the subject has linked to the image.(9) A cancer diagnosis support system for estimating the type of cancer a subject has, comprising: a first memory unit that stores an image or a portion thereof recording the distribution of nematodes after behavior in a nematode taxis behavior assay for a urine sample obtained from the subject; wherein the image and the portion thereof stored in the first memory unit include an image showing the distribution of nematodes gathered in a urine sample placement area; a second memory unit that stores a prediction model obtained by deep learning using as training data an image or a portion thereof recording the distribution of nematodes after behavior in a nematode taxis behavior assay for each of urine samples obtained from a plurality of cancer patients and the cancer types of each of the plurality of cancer patients linked to the image; wherein the image and the portion thereof in the training data include an image showing the distribution of nematodes gathered in the urine sample placement area; an arithmetic unit that uses the prediction model stored in the second memory unit to predict the type of cancer from the image stored in the first memory unit to obtain a prediction result; and an output unit that outputs the obtained prediction result. (10) A cancer diagnosis support system for estimating the type of cancer a subject has, comprising: a server; and a terminal device connected to the server via a network, wherein the server comprises: a first memory unit that stores images recording the distribution of nematodes after behavior in a nematode taxis behavior assay for a urine sample obtained from the subject; a second memory unit that stores images recording the distribution of nematodes after behavior in a nematode taxis behavior assay for each of urine samples obtained from a plurality of cancer patients, and a prediction model obtained by deep learning using as training data the cancer types of each of the plurality of cancer patients linked to the images; and an arithmetic unit that uses the prediction model stored in the second memory unit to predict the type of cancer from the image stored in the first memory unit and obtain a prediction result; wherein the terminal device comprises an output device, and the prediction result is displayed on a display device of the terminal device by the server transmitting the prediction result to the terminal device.

[0007] 1 shows an example of a system for evaluating nematode taxis behavior. Nematodes are typically subjected to taxis behavior assays on a flat surface. 1 shows an example of nematode enhancement processing in an image. In FIG. 2, the nematode is enlarged by increasing its width. 1 shows an example of image patching processing. In the figure, (1) and (2) are regions containing urine sample placement areas, and (3) and (4) are regions containing control placement areas. 1 shows an example of patching processing of patched images. In the figure, (1) and (2) are regions containing urine sample placement areas, and (3) and (4) are regions containing control placement areas. 1 shows a histogram of the percentage of pixels occupied by nematodes in each patched or connected image, along with examples of images where the percentage is 1%, 2%, 3%, and 4%, respectively. 1 shows prediction results using a prediction model. 1 shows prediction results for an accuracy of approximately 0.4 and approximately 0.55. 1 shows an example of a cancer diagnosis support system equipped with a server and a terminal device; 2 shows an example of a process of an image processing method of the present disclosure; 3 shows an example of an image processing process (S2); 4 shows an example of a further image processing process (S5); 5 shows an example of a prediction model generation process (S10); 6 shows an example of a prediction process (S20); 7 shows an example of a computer, system, or server of the present disclosure.

[0008] <Definition of Terms> As used herein, the term "nematode" refers to Caenorhabditis elegans. Nematodes are popular organisms that are widely kept and studied around the world as model organisms in biological research, and are characterized by their ease of rearing and excellent sense of smell.

[0009] As used herein, the term "cancer" refers to a malignant tumor, including, but not limited to, solid cancers such as melanoma, stomach cancer, colorectal cancer, esophageal cancer, pancreatic cancer, prostate cancer, bile duct cancer, lung cancer, kidney cancer, and bladder cancer, as well as carcinomas such as hematopoietic tumors, including blood cancer, leukemia, and lymphoma.

[0010] As used herein, "subject" means a mammal, for example, a human.

[0011] As used herein, "taxis behavior" refers to attractive behavior or repulsive behavior. Attractive behavior refers to behavior that shortens the physical distance from a substance, and repulsive behavior refers to behavior that increases the physical distance from a substance. A substance that induces attractive behavior is called an attractant, and a substance that induces repulsive behavior is called a repellent.

[0012] C. elegans has the ability to be attracted to attractants and repelled by repellents through its sense of smell (see WO 2015 / 88039). The behavior of attracting to attractants is called "attractive behavior," and the behavior of repelling from repellents is called "avoidance behavior." The combination of attractant and avoidance behavior is called "taxis behavior."

[0013] As used herein, the term "taxis behavior assay" refers to an evaluation system that utilizes the fact that nematodes are attracted to the urine of cancer patients and repelled by the urine of healthy individuals, and observes the attraction and repulsion behavior of nematodes toward urine. Taxis behavior assays can be performed by placing a urine sample or a diluted solution of the urine sample at a distance from the nematodes (but within the range of their odor) and allowing the nematodes to move freely. Taxis behavior assays can be performed using an evaluation system such as the taxis behavior evaluation system shown in Figure 1. The urine sample can be diluted undiluted or at a specific dilution ratio, for example, within a range of 1:10 to 1:1000 (e.g., 1:10 to 1:1000). The dilution ratio can be appropriately determined by those skilled in the art depending on the purpose. An anesthetic solution that stops nematode behavior may be added to the urine sample placement area and the control placement area, preferably to prevent nematodes from moving away again after approaching. Taxis behavior assays are typically performed on solid media (e.g., agar media).

[0014] As used herein, "machine learning" refers to the technique of training a computer to learn from data and build a model to perform a specific task. This learning process requires large amounts of data and refines algorithms to perform specific tasks based on patterns in the data. Learning algorithms can take a variety of approaches, including supervised learning, unsupervised learning, and reinforcement learning.

[0015] As used herein, "deep learning" refers to a type of machine learning, an approach that uses multi-layered neural networks for learning. Deep learning models are capable of automatically extracting highly abstract features from input data, allowing them to efficiently handle complex tasks such as image recognition, natural language processing, and speech recognition. While machine learning (non-deep learning) requires manual feature design to extract advanced features, deep learning allows feature extraction to be performed automatically.

[0016] In this specification, a "predictive model" is an algorithm obtained by machine learning or deep learning and constructed on a computer, which is capable of returning a prediction result for an input.

[0017] <Image Processing Method of the Present Disclosure> According to the present disclosure, an image processing method (also referred to as an image generation method, hereinafter the same) is provided. The image processing method of the present disclosure can be used to obtain a prediction model that predicts and outputs a cancer type based on an input image, or to input data into the obtained prediction model. Therefore, the image processing method of the present disclosure can be image preprocessing to obtain training data or data for prediction. The image processing method of the present disclosure can also be a method implemented on a computer.

[0018] The image processing method of the present disclosure includes obtaining an image recording the distribution of nematodes after behavior in a nematode taxis assay for a urine sample obtained from a subject (e.g., the relative positions of the nematodes relative to one another and the shape of the nematodes after behavior) (see, for example, S1 in FIG. 9 ). Testing a urine sample using the taxis assay allows the distribution of the nematodes (e.g., the relative positions of the nematodes relative to one another and the shape of the nematodes after behavior) to be obtained as an image. While the positional relationship between the urine sample placement area and the nematodes may not be included as a feature in the examples described below, it may also be possible to further consider the positional relationship between the urine sample placement area and the nematodes. Images such as these are the target of processing by the image processing method of the present disclosure. The image processing method of the present disclosure may include processing the image and obtaining a processed image (see, for example, S2 and S3 in FIG. 9 ). While the taxis assay does not necessarily have to be performed on a solid medium, it is typically performed on a solid medium and the image includes the distribution of nematodes on the solid medium.

[0019] The image processing method of the present disclosure may include performing one or more steps selected from the group consisting of: (i) cutting out an area from the image showing the distribution of nematodes gathered in the urine sample placement area (also referred to as patch processing, and the same applies below); and (ii) enhancing the nematode image in the image (also referred to as enhancement processing, and the same applies below) to obtain a processed image (see, for example, S2-2L and S2-2R in Figure 10).

[0020] Description of Patching Process Images recording the distribution of nematodes after behavior in a taxis behavior assay may include the distribution of nematodes near the urine sample placement area after behavior. For example, if the urine sample is derived from a cancer patient, the nematodes may be attracted to the urine sample placement area. In the present disclosure, a region (i.e., a portion of the image) showing the distribution of nematodes attracted to the urine sample placement area can be cut out from the image (see, for example, Figure 3). While a prediction model can be obtained by training using the entire image, a prediction model can also be obtained by training using only the region in question, and predictions can be made by inputting only the region into the obtained prediction model. In a preferred embodiment, the region may include the urine sample placement area and its surrounding areas in all directions. The region cut out by patching may include only one urine sample placement area, or may include two or more urine sample placement areas. Multiple patched regions derived from the same urine sample may be linked into a single image (linking process; see, for example, Figure 4).

[0021] Description of Enhancement Processing Nematodes are very small creatures. Although the presence of nematodes is clearly visible in the image without enhancement, enhancing the nematodes can improve the accuracy of image recognition. Enhancement processing can include, for example, increasing the brightness of the nematode and enlarging the nematode (see, for example, Figure 2). Enhancement processing can improve the accuracy of image recognition by increasing the brightness of the nematode and / or enlarging the nematode without changing its shape or size. Enlarging the nematode can be achieved, for example, by enlarging the nematode evenly in all directions from its center or by increasing its width and thickness. The nematode is stretched, either straight or bent, and it is preferable to perform the enlargement processing so as to maintain information about this bending and stretching, its orientation, and / or its relative distance from the urine sample placement site.

[0022] Both patch processing and enhancement processing may be performed on an image (see (ii) and (iii) in FIG. 10). Enhancement processing may be performed before patch processing (see (iv) in FIG. 10) or after patch processing (see (i) in FIG. 10). This is because both are equivalent. After performing one or more processes selected from the group consisting of patch processing and enhancement processing, further image processing (see S2-5 in FIG. 10) may be performed, or the processed image may not be performed and may be obtained as a final preprocessed image (S2-6 in FIG. 10).

[0023] Description of Inversion Processing Images may be inverted vertically or horizontally to achieve line symmetry before use. This allows all input images to have the same orientation. Description of Rotation Processing Images may also be rotated. The rotation may be within ±180°, ±90°, ±45°, ±30°, ±10°, or ±5°, for example. The rotation angle may be common between images. In a preferred embodiment, the image may be rotated so that the line connecting the nematode placement area to the urine sample placement area is horizontal or vertical. However, in a preferred embodiment of the present disclosure, the image is used without being inverted and / or rotated.

[0024] Measurement of Nematode Area Ratio and Image Selection Based on Area Ratio The area ratio of nematodes in an image or a cropped region can affect the accuracy of predictions made by a prediction model. A reference value can be established, and images showing an area ratio exceeding the reference value can be selected (see, for example, FIG. 5 ). These images can be used as training data or as input images for prediction into a prediction model. The images can preferably be cropped images, and more preferably, images that have been cropped and enhanced. In this embodiment, the method can include measuring the nematode area ratio (see S2-5-1 in FIG. 11 ), comparing the nematode area ratio with the reference value (see S2-5-2 in FIG. 11 ), and selecting and acquiring images that exceed the reference value (see S2-5-3 in FIG. 11 ). The reference value can be, for example, 1% or more, 1.5% or more, 2% or more, 2.5% or more, 3% or more, 3.5% or more, or 4% or more. The reference value can be, for example, a value of approximately 2% to 4%. The threshold may also be set to include the top 25%, top 50%, or top 75% of pixels occupied by nematodes, and any percentages in between.

[0025] <Prediction Model Creation Method of the Present Disclosure> The present disclosure provides a method for obtaining a prediction model. The prediction model can be obtained by using images obtained by the above-described image processing method (see S11 in FIG. 12 ) as training data. The training data includes, or can include, information indicating the type of cancer the patient associated with the image has (see S12 in FIG. 12 ).

[0026] Creating a Predictive Model by Learning Images (See S13 in FIG. 12) In deep learning, images can be trained using a variety of architectures. Representative architectures include, but are not limited to, convolutional neural networks (CNNs) such as LeNet, AlexNet, VGGNet, and ResNet; transformer architectures such as Vision Transformer (ViT); multi-scale architectures such as Inception (GoogleLeNet); and architectures involving object detection such as R-CNN, YOLO, and SSD. Transformer architectures use a self-attention mechanism to learn how each element of the input data is related to all other elements. In ViT, an image is divided into small patches, and each patch is transformed into a format that can be processed by a transformer model. The resulting vector is then embedded into a fixed-size vector through a series of linear transformations and fed to a transform encoder. Features are extracted in the encoder, and the image is classified using an MLP-Head. Those skilled in the art will be able to train an appropriately selected architecture with information on images and cancer types to obtain a prediction model.

[0027] When there are multiple urine sample placement areas, the above-mentioned areas including each urine sample placement area can be obtained by clipping. The multiple obtained areas may be trained separately to obtain a prediction model, or the multiple obtained areas may be linked into a single image and trained as a whole to obtain a prediction model. In a preferred embodiment, the present disclosure allows the multiple obtained areas to be trained separately to obtain a prediction model. It is preferable that all training data be provided in the same format.

[0028] In machine learning, feature values ​​can be set appropriately. This example demonstrates that the distribution, orientation, shape, etc. of nematodes can be used to predict the presence or absence of cancer and the type of cancer, and a prediction model can be constructed by appropriately combining such indicators. Those skilled in the art will be able to obtain a prediction model by appropriately setting feature values. The model to be trained is not particularly limited, but for example, a model that performs classification using feature values ​​extracted from an image (e.g., Random Forest, LightGBM, etc.) can be used. In one embodiment, the feature amounts are not particularly limited, but include, for example, 1: area (area of ​​the nematode), 2: box area (area of ​​the smallest rectangle surrounding the nematode), 3: axis major length (major axis of the smallest ellipse surrounding the nematode), 4: axis minor length (minor axis of the smallest ellipse surrounding the nematode), 5: eccentricity (nematode shape) (index showing whether the nematode is close to a circle or a line), 6: Euler number (the number of connected components minus the number of holes)), 7: extent (ratio of the number of pixels of the nematode to the box area), 8: orientation (orientation of the nematode (angle between the major axis and the axis of the ellipse)), 9: perimeter (perimeter of the nematode), 10: solidity (ratio of pixels within the area to pixels in the convex hull image), 11: distance to urine control spot 1 (distance between the nematode and the spot in the upper right where the control is dropped), 12: distance to urine control spot Examples of such measurements include 1: distance to urine target spot 1 (the distance between the nematode and the spot on the lower right where the control is dropped), 13: distance to urine target spot 1 (the distance between the nematode and the spot on the upper left where urine is dropped), and 14: distance to urine target spot 2 (the distance between the nematode and the spot on the lower left where urine is dropped). If accuracy does not improve when using the image as is, components other than nematodes can be removed from the image as noise. For example, it may be effective to identify nematodes as those with a bounding box size equal to or larger than a standard value.The reference value may vary depending on the resolution of the captured image, but a person skilled in the art would be able to set it appropriately taking into account the size of the nematode on the image.

[0029] <Prediction Method Using a Prediction Model> In the present disclosure, by inputting an image into the obtained prediction model (see S21 in FIG. 13 ), it is possible to predict the type of cancer a patient may have (see S22 in FIG. 13 ). The prediction model typically extracts features related to the distribution of nematodes from the input image and predicts the type of cancer from the image based on the features. While not particularly limited, the image is preferably subjected to the same image processing as the images used as training data. This is because it is generally believed that this improves prediction accuracy. Since this method is implemented on a computer, the prediction result is output from the computer (see S23 in FIG. 13 ). A single test can be performed to predict the type of cancer with the highest probability as the type of cancer the patient has. Alternatively, multiple tests can be performed to predict the type of cancer with the highest predicted frequency as the type of cancer the patient has. The prediction result is intended for use in cancer risk testing services, particularly cancer type risk (probability of having a specific cancer type) testing services, or to assist physicians in making diagnoses, but is not a diagnosis in itself. A doctor can diagnose cancer, differentiate between cancer types, etc. based on the prediction results and, if necessary, other diagnostic information.

[0030] <Implementation of the Method on a Computer> The computer may have a computing unit (e.g., a CPU and / or a GPU), and volatile and / or non-volatile memory. The computer may also have an input device (keyboard) and output devices (display, speaker, and printer) (see FIG. 14).

[0031] In the method of the present disclosure, images recording the distribution of nematodes after behavior in a nematode taxis behavior assay toward a urine sample obtained from a subject (e.g., the relative positions of the nematodes and the shape of the nematodes after behavior) can be stored in a first memory unit (volatile memory or non-volatile memory) included in a computer.

[0032] The image stored in the first storage unit is subjected to one or more processes selected from the group consisting of patch processing and enhancement processing on the computing unit to obtain a processed image. In the method of the present disclosure, the processed image can be stored in a second storage unit (volatile memory or non-volatile memory).

[0033] The arithmetic unit may perform image processing such as patch processing, enhancement processing, connection processing, inversion processing, calculation of nematode area ratio, and image selection based on the area ratio, etc. The processed images may be stored in the second storage unit or another storage unit each time.

[0034] When each image obtained by processing is used as training data, it can be linked to information indicating the type of cancer the subject has and stored in the same memory unit.

[0035] In the method disclosed herein, the computer may include a third memory unit that stores the architecture and a fourth memory unit that stores the prediction model. The first to fourth memory units may be physically located at different addresses on the same recording device, or may be located on different physical devices. In the computer, the images stored in the first memory unit or the second memory unit and information indicating the type of cancer the subject has linked to the images are input into the architecture in a computing unit, and a prediction model is constructed.

[0036] Once the prediction model is obtained, the prediction model can be stored in a first memory unit of another computer (second computer), images or processed images showing the distribution of nematodes after their migration behavior in urine obtained from a patient whose type of cancer is to be predicted can be saved in a second memory unit, and the images can be input into the prediction model on a computing unit to store the prediction results in a third memory unit. The first to third memory units in the second computer may each be located at different addresses on the same physical recording device, or may be located on physically different devices.

[0037] <Cancer Diagnosis Support System> The present disclosure provides a cancer diagnosis support system equipped with the above-described prediction model. The cancer diagnosis support system of the present disclosure can predict the type of cancer based on the results of a nematode migration behavior assay.

[0038] The present disclosure provides the following system, as illustrated in FIG. 14 , for example. A cancer diagnosis support system for predicting the type of cancer a subject has, the system comprising: a first memory unit that stores images or portions thereof recording the distribution of nematodes after behavior in a nematode taxis behavior assay for a urine sample obtained from the subject, wherein the portions stored in the first memory unit include images showing the distribution of nematodes gathered in a urine sample placement area; a second memory unit that stores a prediction model obtained by deep learning using as training data images or portions thereof recording the distribution of nematodes after behavior in the nematode taxis behavior assay for each of urine samples obtained from a plurality of cancer patients and the cancer types of each of the plurality of cancer patients linked to each of the images, wherein the images and portions thereof in the training data include images showing the distribution of nematodes gathered in the urine sample placement area; a calculation unit that uses the prediction model stored in the second memory unit to predict the type of cancer from the images or portions thereof stored in the first memory unit to obtain a prediction result; and an output unit (output device) that outputs the obtained prediction result. The output unit may be, for example, a display device or a printer, and may display or print the prediction results. The output unit may also be, for example, a data transmission device, and may transmit the obtained prediction results to an external computer. Transmission may be via a wired or wireless connection such as a wireless LAN or Bluetooth.

[0039] The present disclosure also provides the following system: The system is illustrated, for example, in FIG. 8, and the server is illustrated, for example, in FIG. a first memory unit that stores an image or a portion thereof recording the distribution of nematodes after behavior in a nematode taxis behavior assay for urine samples obtained from the subject, wherein the portion stored in the first memory unit includes an image showing the distribution of nematodes gathered in a urine sample placement area; a second memory unit that stores a prediction model obtained by deep learning using as training data images recording the distribution of nematodes after behavior in a nematode taxis behavior assay for urine samples obtained from each of a plurality of cancer patients and the cancer types of each of the plurality of cancer patients linked to the image, wherein the image or a portion thereof in the training data includes an image showing the distribution of nematodes gathered in the urine sample placement area; and a calculation unit that uses the prediction model stored in the second memory unit to predict the type of cancer from the image or a portion thereof stored in the first memory unit to obtain a prediction result; and the terminal device is equipped with a display device, the server distributes the prediction result to the terminal device, and the prediction result is displayed on a display device of the terminal device.

[0040] An image or a portion thereof recording the distribution of nematodes after behavior in a nematode taxis behavior assay toward a urine sample obtained from a subject may be transmitted from the terminal device to a server, and generation of the portion thereof from the image may be performed on the terminal device or on the server.

[0041] In the training data, there is a one-to-one correspondence between images and the type of cancer the subject has.

[0042] The computing unit may be, for example, a CPU and / or a GPU. The storage unit may be a volatile memory (e.g., RAM such as SRAM, DRAM, PRAM, SDRAM, or VRAM), or a non-volatile memory (e.g., flash memory, magnetic memory, optical memory (e.g., optical disk, CD, and DVD)), or freeze-change memory. The computing unit may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), or a neural network computing unit. The computing unit and each storage unit are electrically connected. The connection may be via a memory bus (e.g., a front-side bus, double data rate (DDR), or the like), a chipset, a northbridge, or the like. The other components may be as described above.

[0043] In this example, a nematode taxis assay for urine samples from cancer patients was performed using a nematode taxis behavior evaluation system (see Figure 1) according to the method described in Patent Document 1. In Figure 1, a urine sample and anesthetic solution (containing sodium azide) were placed in urine sample placement areas 1 and 2 on a 9-cm dish containing solid medium. Nematodes were placed in nematode placement area 3, and anesthetic solution was placed in control placement areas 4 and 5 to evaluate taxis behavior. The urine sample placement area and control placement area each contained the same amount of anesthetic solution at the same concentration, which served to stop the behavior of nematodes that gathered nearby. Mark 6 was printed on the dish and was used as a marker to adjust the dish's rotation angle to a constant value during image processing.

[0044] Urine samples were obtained from the patient groups shown in Table 1 .

[0045]

[0046] Urine samples obtained from each patient were diluted 10-fold and 100-fold, respectively, and the above-described migration assay was performed. Images of the nematodes on the dish were photographed and imported into a computer. Image preprocessing included removing the edges of the dish, correcting for tilt, and adjusting the contrast. Furthermore, the strip-shaped area in the center of the dish containing the nematode placement area 3 and the marker 6 was removed from the image, yielding images containing urine sample placement areas 1 and 2, and control placement areas 4 and 5.

[0047] Calculation of the taxis index The number of nematodes (A) in the images containing urine sample areas 1 and 2 and the number of nematodes (B) in each of the images containing control areas 4 and 5 were counted. The taxis index was calculated by (A - B) / (A + B).

[0048] Nematode Enhancement Processing We performed a process to enhance the nematodes in the image. The nematode enhancement process was performed to help the trained model more clearly identify the nematodes. Here, we performed an operation to enhance the nematodes in the image while maintaining their position, shape, and orientation (see Figure 2). In Figure 2, the width of the nematodes has been increased to the point where they are easily visible.

[0049] Inversion and rotation: Images were inverted by randomly flipping them upside down or rotating them left or right within a range of ±5 degrees.

[0050] Patching The image was subjected to patching, which involves cutting out areas from the image including urine sample placement areas 1 and 2, and control placement areas 4 and 5 and their surrounding areas (see Figure 3).

[0051] The patch-processed images were combined. This patch combining process involves directly connecting the images containing urine sample placement areas 1 and 2 and the images containing control placement areas 4 and 5 to form a single image (see Figure 4).

[0052] Images were selected based on the area ratio of nematodes. Specifically, images containing a certain number of nematodes or less were removed from the analysis. For example, only images in which the ratio of nematode pixels to the total number of pixels in the patched image was greater than 3% were selected (see Figure 5).

[0053] Image selection based on the taxis index Images were selected based on the taxis index. That is, images with non-positive taxis indices were removed so as not to be used in subsequent analysis. In addition, the taxis assay was repeated multiple times for each specimen, and specimens with non-positive average taxis indices were not used in analysis. This is because there is a high possibility that the characteristics of cancer are not apparent in the images.

[0054] Creating a deep learning model The Vision Transformer (ViT) model is a high-performance image recognition model released by Google in 2020. The Vision Transformer (ViT) model was trained on images obtained from a taxis assay associated with cancer type. Training was performed under the 10 conditions shown in Table 2.

[0055]

[0056]

[0057] The predictive model was used to predict the type of cancer in the patient from each image. The results are shown in Table 3. As shown in Table 3, patch processing, nematode enhancement, and image selection based on nematode area ratio improved prediction accuracy. However, data augmentation reduced prediction accuracy. Furthermore, patch merging did not significantly affect prediction accuracy. Even with an accuracy of 0.4, the predictability of the results was not negated; prediction was sufficient (see the lower panel of Figure 7). In Figure 7, for example, the most frequently predicted cancer type can be predicted as the patient's cancer type. Image selection based on the taxis index may reduce prediction accuracy (see Condition 6 vs. Conditions 7–9). These results suggest that including images with a negative taxis index is advantageous. This suggests that the presence and type of cancer can be assessed solely based on the distribution of nematodes around the urine.

[0058] When predictions were made on an image-by-image basis using the training data and validation data, the best predictive model was the model under condition No. 6.

[0059] Figure 6 shows the results of evaluating images by applying the prediction model under condition No. 6 to the training data. Multiple images were obtained from the specimen, and prediction results were obtained for each image. For each specimen, the prediction results for each image were tallied, and the most frequent prediction result was used as the prediction result for that specimen. The true label indicates the actual type of cancer, and the predicted label indicates the cancer type estimated by the prediction model.

[0060] As shown in Figure 6, the type of cancer could be accurately identified from images, both on an image-by-image basis (left panel) and on a specimen-by-specimen basis (right panel). It appears that the dispersion, density, and shape of nematodes are affected by the type of cancer. These results suggest that it may be possible to identify the type of cancer based on images of nematodes in a taxis assay. Further tuning is expected to improve prediction accuracy.

[0061] <Generation of a predictive model using machine learning (non-deep learning)> Nematodes were segmented from the image under condition 6, and the distribution of nematodes was extracted as a feature. For segmentation, a binarization method was used to separate the image into the background and nematodes. To remove noise other than nematodes from the binarized image, a threshold was set for the size of the segmented objects, and objects smaller than that threshold were excluded as noise. The specific feature values ​​were as follows: 1: area (area of ​​the nematode), 2: box area (area of ​​the smallest rectangle surrounding the nematode), 3: axis major length (major axis of the smallest ellipse surrounding the nematode), 4: axis minor length (minor axis of the smallest ellipse surrounding the nematode), 5: eccentricity (nematode shape) (index showing whether the nematode is close to a circle or a line), 6: Euler number (the number of connected components minus the number of holes)), 7: extent (ratio of the number of pixels in the nematode to the box area), 8: orientation (the angle between the major axis of the ellipse and the axis)), 9: perimeter (perimeter of the nematode), 10: solidity (ratio of pixels in the area to pixels in the convex hull image), 11: distance to urine control spot 1 (distance between the nematode and the spot in the upper right where the control is dropped), 12: distance to urine control spot 2 (distance between the nematode and the spot in the lower right where the control is dropped), 13: distance to urine target spot 1 (distance between the upper left spot where urine is dropped and the nematode), 14: distance to urine target spot 2 (distance between the lower left spot where urine is dropped and the nematode). Nos. 1 to 10 were calculated using a Python package, while the others were calculated using custom code. Random Forest and LightLGBM models were trained using the extracted features to obtain predictive models. The sample-level predictive accuracy of Random Forest and LightLGBM, respectively, was 0.5423 and 0.5721. In this way, a predictive model could be established using machine learning (non-deep learning).Even when adding features related to nematode variability (density) to the above features, we were able to establish a prediction model with a prediction accuracy of over 0.5. In the above model, nematode variability was defined as the average distance between all nematodes.

[0062] 1: Urine sample placement section 2: Urine sample placement section 3: Nematode placement section 4: Control placement section 5: Control placement section 6: Marker 7: Solid medium surface 10: Nematode tactic behavior evaluation system 100: Cancer diagnosis support system equipped with server and terminal device 101: Server 102: Terminal device 103: Network

Claims

1. A method for processing an image recording the distribution of nematodes after their behavior in a taxis behavior assay for a urine sample obtained from a subject, the method comprising: obtaining an image recording the distribution of nematodes after their behavior in a taxis behavior assay for a urine sample obtained from a subject; and performing one or more processes selected from the group consisting of: (i) cropping out from said image an image showing the distribution of nematodes gathered in the urine sample placement area; and (ii) highlighting the nematodes in said image, the method being implemented on a computer.

2. The method of claim 1, including the cropping process defined in (i) above.

3. The method of claim 2, further comprising a process for highlighting nematodes in each image obtained by the cropping process.

4. The method according to claim 2 or 3, which does not include linking multiple images obtained by the cropping process on a specimen-by-specimen basis.

5. The method according to any one of claims 1 to 4, which does not include image inversion.

6. The method according to any one of claims 1 to 5, wherein the image is not sorted based on the number of nematodes attracted to the urine sample.

7. The method according to any one of claims 1 to 6, further comprising carrying out the cutout process defined in (i) above, a process of emphasizing nematodes in each image obtained by the cutout process, and a process of removing images obtained by the cutout process in which the area ratio of nematodes does not meet a standard.

8. A method according to any one of claims 1 to 7, wherein the processed image is used to generate a predictive model that predicts the type of cancer a subject has from training data including the processed image and the type of cancer the subject associated with the image has, or the processed image is used to predict the type of cancer a subject from whom the urine sample that produced the processed image was derived has, using a predictive model generated from training data including the other processed image and the type of cancer the subject associated with the other processed image has.

9. A cancer diagnosis support system for predicting the type of cancer a subject has, comprising: a first memory unit that stores images or portions thereof recording the distribution of nematodes after behavior in a nematode taxis behavior assay for a urine sample obtained from the subject; wherein the portions stored in the first memory unit include images showing the distribution of nematodes gathered in a urine sample arrangement; a second memory unit that stores images or portions thereof recording the distribution of nematodes after behavior in a nematode taxis behavior assay for each of urine samples obtained from a plurality of cancer patients and a prediction model obtained by deep learning using as training data the cancer types of each of the plurality of cancer patients linked to the images; wherein the images and portions thereof in the training data include images showing the distribution of nematodes gathered in a urine sample arrangement area; an arithmetic unit that uses the prediction model stored in the second memory unit to predict the type of cancer from the images stored in the first memory unit to obtain a prediction result; and an output unit that outputs the obtained prediction result.

10. A cancer diagnosis support system for predicting the type of cancer a subject has, comprising: a server; and a terminal device connected to the server via a network; wherein the server comprises: a first memory unit for storing images or portions thereof recording the distribution of nematodes after behavior in a nematode taxis behavior assay for urine samples obtained from the subject, wherein the portions stored in the first memory unit include images showing the distribution of nematodes gathered in a urine sample placement area; a second memory unit for storing a prediction model obtained by deep learning using as training data images recording the distribution of nematodes after behavior in a nematode taxis behavior assay for urine samples obtained from each of a plurality of cancer patients and the cancer types of each of the plurality of cancer patients linked to each of the images; wherein the images and portions thereof in the training data include images showing the distribution of nematodes gathered in the urine sample placement area; and a calculation unit for predicting the type of cancer from the images or portions thereof stored in the second memory unit using the prediction model stored in the first memory unit to obtain a prediction result; wherein the terminal device comprises a display device; the prediction result is distributed from the server to the terminal device, and the prediction result is displayed on a display device of the terminal device.

Images