Information processing system, method for processing information, and recording medium

Abstract

The invention discloses an information processing system, a method for processing information, and a recording medium. According to the present disclosure, an information processing system includes a data acquisition unit, a feature acquisition unit, and a similarity acquisition unit. The data acquisition unit is configured to acquire first medical data including a plurality of items related to a subject or different types of data. The feature acquisition unit is configured to acquire a feature representing a characteristic of the medical data. The feature acquisition unit acquires at least a subset of the feature by inputting medical data included in the first medical data into at least one of an encoder of a large language model and an encoder of a large multi-mode model. The similarity acquisition unit is configured to acquire a similarity between the feature and a feature of another piece of medical data.

Description

Technical Field

[0001] This disclosure relates to an information processing system, a method for processing information, and a recording medium. Background Technology

[0002] A technique for searching similar data is known, in which data such as text or images are converted into a digital representation to perform the search.

[0003] For example, Japanese Patent Application Publication No. 2008-027131 discloses a technique for calculating features from a partial image of a document image and searching for another document with similar features.

[0004] However, the method described in Japanese Patent Application Publication No. 2008-027131 does not take into account situations where multiple types of data, including text, are used to search for similar data, or where the user's intent is considered when searching for similar data. Therefore, in some cases, it may be difficult to properly calculate the similarity. Summary of the Invention

[0005] This disclosure aims to provide an information processing system capable of appropriately calculating similarity, a method for processing information, and a recording medium.

[0006] According to one aspect of this disclosure, an information processing system includes a data acquisition unit, a feature acquisition unit, and a similarity acquisition unit. The data acquisition unit is configured to acquire first medical data comprising multiple items or different types of data related to a subject. The feature acquisition unit is configured to acquire features representing characteristics of the medical data. The feature acquisition unit acquires at least a subset of the features by inputting medical data included in the first medical data into at least one of an encoder of a large language model and an encoder of a large multimodal model. The similarity acquisition unit is configured to acquire the similarity between the features and features of another piece of medical data.

[0007] The features of this disclosure will become apparent from the following description of embodiments with reference to the accompanying drawings. The following description of embodiments is by way of example. Attached Figure Description

[0008] Figure 1 An example of the functional configuration of the information processing system according to the first embodiment is shown.

[0009] Figure 2 This is a flowchart of the process performed by the information processing system according to the first embodiment.

[0010] Figure 3 This is a flowchart of a process stored in a database performed by an information processing system according to a variant of the first embodiment, Example 2.

[0011] Figure 4 This is a flowchart of the process performed by the information processing system according to Variation Example 2 of the first embodiment.

[0012] Figure 5 This is a flowchart of the process performed by the information processing system according to Variation Example 3 of the first embodiment.

[0013] Figure 6 This is a flowchart of the process performed by the information processing system according to Variation 5 of the first embodiment.

[0014] Figure 7 This is a configuration example of medical record information processed by the information processing system.

[0015] Figure 8 This is an example of a SOAP document processed by an information processing system.

[0016] Figure 9 This is a configuration example of a database processed by an information processing system.

[0017] Figure 10 An example of the functional configuration of an information processing system according to a second embodiment is shown.

[0018] Figure 11 This is a flowchart of the process performed by the information processing system according to the second embodiment. Detailed Implementation

[0019] In the following description, embodiments of the present disclosure will be illustrated with reference to the accompanying drawings. It should be understood that the present disclosure is not limited to the following embodiments. The present disclosure may be appropriately modified without departing from its spirit.

[0020] In the accompanying drawings described below, the same elements are represented by the same reference numerals, and redundant descriptions may be omitted or simplified.

[0021] First Embodiment

[0022] Figure 1 An example of the functional configuration of the information processing system 100 according to the first embodiment is shown. The information processing system 100 is configured to acquire medical data and perform information processing on the medical data. Here, the target data is not limited to medical data. The target data can be content data that includes the data described above.

[0023] Here, the medical data relevant to this embodiment refers to data containing multiple data items. These data items may include different media formats (numerical values, categories, text, sound, still images, moving images, etc.). For example, medical record information related to a single patient, managed by electronic medical records, is an example of medical data including multiple items relevant to this embodiment.

[0024] Specifically, medical record information as medical data can include data items in digital form, such as the patient's (subject's) age, height, weight, and blood test data. Medical record information as medical data can also include data items in categorized form, such as gender (male, female, other), smoking history, family medical history, and consent to the patient's (subject's) diagnosis and hospitalization.

[0025] In addition, medical record information may include the patient's name, all documentation of the SOAP (Subjective, Objective, Assessment, and Planning) approach in the medical care records (hereinafter referred to as SOAP documentation), diagnosis, and links to external systems (such as departmental systems and websites). Medical record information may also include data items in text form, such as statements from examination reports that can be accessed based on linked information.

[0026] Medical record information may also include image-based data items, such as scanned images of referral letters and consent forms, examination images such as X-ray images and computed tomography (CT) images that can be obtained based on the aforementioned linked information, and key images in examination reports. Medical record information may also include waveform data items such as electrocardiograms, sound-based data items such as body sounds acquired by a digital stethoscope, and motion image and coordinate-based data items related to walking data during rehabilitation.

[0027] As a specific example Figure 7 The diagram illustrates exemplary data items included in medical record information. Medical Digital Imaging and Communication (DICOM) image data associated with a single examination or single photograph managed by a Picture Archiving and Communication System (PACS) is an example of medical data including data corresponding to multiple data items relevant to this embodiment. Specifically, DICOM image data may include data items in digital form, such as information about the camera location and pixel configuration. Furthermore, DICOM image data may also include data items in text form, such as the name of the imaging device, the name of the institution, the patient's name, the identifier of the examination, and the name of the area. Additionally, DICOM image data may include data items in image form, such as image data and overlay data. Furthermore, DICOM image data may include specific forms of data items used only by a subset of DICOM image viewers. Depending on the imaging device, etc., that outputs the DICOM image data, the image data may be a two-dimensional or three-dimensional image, or a two-dimensional or three-dimensional moving image.

[0028] Furthermore, for example, SOAP documents in medical care records can be considered as an example of medical data where multiple data items coexist in relation to this embodiment. SOAP documents are medical data comprising four textual data items: "Document about subjective information," "Document about objective information," "Document about assessment," and "Document about treatment plan." Depending on the use of electronic medical records, the group of data items included in a SOAP document can be recorded as a single data item. As a specific example, an example of a SOAP document is as follows: Figure 8 As shown, SOAP document A101 has a structure comprising four data item groups. SOAP document A102 is an example of an unstructured document with only one data item. In this embodiment, for ease of understanding, it is assumed that the SOAP document comprises four data items. When the SOAP document is unstructured, it can be processed by dividing it into four data items, or the entire SOAP document can be processed as a single item.

[0029] The information processing system 100 can be implemented using computer equipment such as a server or workstation. The information processing system 100 can display the results of information processing, store the results of information processing, and output the results of information processing to external devices. For example, the information processing system 100 can search for other medical data similar to any given medical data. Link information and general information corresponding to subsets of similar medical data groups that appear as search results can be displayed by a display 105 or a display device (not shown) connected to the information processing system 100.

[0030] The information processing system 100 can be communicatively connected to a medical data management device (not shown) to acquire medical data via a network 200 or, for example, a communication cable (not shown) or a communication circuit (not shown). As a component of the information processing system 100, the medical data management device can be integrated with the information processing system 100.

[0031] The medical data management device is a database device configured to store medical data and send / receive medical data to / from another device (such as information processing system 100) that can communicate with it. The medical data management device can receive search queries for medical data from other devices (such as information processing system 100) and send search results. Specifically, for example, the medical data management device can receive queries using a Structured Query Language (SQL) in which numerical values ​​or strings included in data items of the medical data are specified as search criteria, and can send medical data sets matching the criteria and identifier sets for those medical data sets, etc.

[0032] A medical data management device can associate features in vector (numerical group) form with stored medical data and store these features, wherein the vector-form features indicate the characteristics of at least a subset of data items included in the medical data. Furthermore, the medical data management device can receive queries specifying features as search criteria and send medical data groups or groups of identifiers of medical data groups that match or are related to similar features.

[0033] Information processing system 100 can be connected to another device to acquire features in vector form that indicate characteristics of at least a subset of data items included in medical data. For example, information processing system 100 can be communicatively connected to a feature acquisition device (not shown) via network 200 or, for example, a communication cable (not shown) or a communication circuit (not shown). As one of the components of information processing system 100, the feature acquisition device can be integrated with information processing system 100.

[0034] The feature acquisition device is configured to acquire at least a subset of data item groups included in medical data, compute features based on the acquired data item groups, and send the computed features to another device (e.g., information processing system 100) that can communicate with the feature acquisition device. The feature acquisition device includes one or more encoders configured to compute features in vector form, whereby the vector features represent the characteristics of the data corresponding to the data item, when inputting data corresponding to one or more data items. As the encoder, an encoder included in a large language model or a large multimodal model can be used.

[0035] The encoder included in a large language model is a text encoder that can input text prompts in the form of text.

[0036] For example, Meta AI (Llama), a large-scale language model from Meta Platforms Inc., is an example of a large-scale language model. The encoders contained in a large-scale multimodal model are any of the encoders for different media formats that can receive cues in different media formats (numerical encoders, category encoders, text encoders, sound encoders, image (still and motion) encoders).

[0037] For example, OpenAI's Contrastive Language Image Pretraining (CLIP) is an example of a large-scale multimodal model. The encoder receives data in the corresponding media format and outputs features in vector form representing characteristics, such as content or concepts related to the data. Data in numerical or categorical form can also be input into a text encoder when it is converted into strings (text).

[0038] As an encoder, it can be used as part of a neural network configured to perform a predetermined image processing task (such as image classification, image generation, image segmentation, metric learning, or contrastive learning). That is, the intermediate features computed when data is input into the neural network can be regarded as features representing the characteristics of the data.

[0039] An encoder can be trained such that feature sets obtained by inputting similar data sets have high similarity, while feature sets obtained by inputting dissimilar data sets have low similarity.

[0040] Specifically, suppose that, for example, among images A through C, only image C is dissimilar to the other two images. In this case, for features A through C obtained by inputting the images into an encoder, the encoder can be trained such that the similarity between features A and B is high, and the similarity between features A and C, as well as the similarity between B and C, is low.

[0041] In other words, the encoder is characterized as follows: the encoder is trained such that the similarity of the output feature sets in response to inputs of check data sets in which the characteristics of predetermined data item sets are similar is higher than the similarity of the output feature sets in response to inputs of dissimilar check data sets.

[0042] This training can be achieved by fine-tuning the pre-trained encoder. For fine-tuning, general learning methods such as supervised learning, reinforcement learning, training only the output layer, or low-rank adaptive learning (LoRA) can be used.

[0043] Here, when calculating the similarity between features using the similarity acquisition function, the distance difference between vectors decreases as the dimensionality of the features in vector form increases. This leads to a problem where similarity recognition becomes difficult. Therefore, the similarity acquisition function can apply dimensionality reduction techniques to features as a preprocessing step for calculating feature similarity. Examples of dimensionality reduction techniques include, for instance, Principal Component Analysis (PCA), t-random neighbor embedding (t-SNE), and Uniform Manifold Approximation and Projection (UMAP).

[0044] One approach involves using a similarity extraction function to perform statistical analysis on feature sets obtained by inputting similar and dissimilar data sets into the encoder. This process retains only dimensions useful for assessing the similarity between features and removes values ​​from other dimensions, thereby reducing dimensionality. Specifically, for example, the similarity extraction function can remove values ​​from dimensions with high variance obtained by inputting similar data sets into the encoder.

[0045] Whether data groups are defined as similar or dissimilar may vary depending on the user's perspective. For example, in an image group, images could be grouped into dark image groups and bright image groups based on brightness, and into image groups showing the upper body and image groups showing the lower body based on the camera area.

[0046] From a certain perspective, it could be said that each group is similar. However, for example, not all images in the dark image group depict the upper body (or lower body). From the perspective of the shooting area, the dark image group may contain dissimilar images.

[0047] Therefore, when performing a similar data search, the characteristics used to evaluate similarity can be changed according to the user's intent. That is, the dimensions to be removed in the aforementioned dimensionality reduction can be changed, and the combination of data used to evaluate similarity can be altered, depending on the user's intent. In this embodiment, to allow the user to perform a similar data search as expected, the aforementioned changes can be performed based on the search information described later.

[0048] Figure 1 An example configuration of the information processing system 100 according to this embodiment is shown. For example... Figure 1 As shown, the information processing system 100 includes a communication interface 101, a storage circuit 102, a processing circuit 103, an input interface 104, and a display 105. The information processing system 100 can be communicatively connected to the network 200 via the communication interface 101.

[0049] Communication interface 101 is configured to communicate with another device for medical data, search results, etc. Communication interface 101 is implemented using a network communication interface such as a network adapter or network interface controller (NIC). Alternatively, communication interface 101 can be implemented via a device connectivity interface such as Universal Serial Bus (USB), PCI Express, Serial Advanced Technology Accessories (SATA, Serial ATA), or M.2.

[0050] The storage circuit 102 is configured to store various types of data and various types of programs to be used for processing performed by the information processing system 100 according to this embodiment. Specifically, the storage circuit 102 is connected to the processing circuit 103 and operates under the control of the processing circuit 103. The storage circuit 102 also has the function of a working memory for temporarily storing various types of data for processing performed by the processing circuit 103. The storage circuit 102 is implemented by a semiconductor storage device such as random access memory (RAM) or flash memory, a hard disk drive, an optical disk, etc.

[0051] The processing circuit 103 is configured to control the operation of the aforementioned components of the information processing system 100. For example, the processing circuit 103 performs various types of processing according to instructions received from the user via the input interface 104 connected to the information processing system 100. Alternatively, for example, the processing circuit 103 can perform various types of processing according to instructions received from the user via the communication interface 101. The processing circuit 103 is implemented, for example, by a central processing unit (CPU).

[0052] The processing circuit 103 includes, for example, a data acquisition function 103a that implements a data acquisition unit configured to acquire medical data and a search information acquisition function 103b that implements a search information acquisition unit configured to acquire search information. The processing circuit 103 also includes a feature acquisition function 103c that implements a feature acquisition unit configured to acquire features and a similarity acquisition function 103d that implements a similarity acquisition unit configured to acquire similarities between medical data. The processing circuit 103 also includes a search function 103e that implements a search unit configured to acquire search results for similar medical data.

[0053] When the information processing system 100 has the above-mentioned functional configuration, it can fully acquire the similarity to the medical data desired by the user. When the information processing system 100 performs a search using similarity, it can obtain search results that match the user's search intent.

[0054] Here, for example, by Figure 1 Each processing function implemented by the corresponding element of the processing circuit 103 shown is stored in the storage circuit 102 in the form of a program that can be executed by a computer.

[0055] The processing circuit 103 reads a program from the storage circuit 102 and executes the read program to realize the function corresponding to the program. That is, the processing circuit 103, having read the program, has functions implemented by the data acquisition function 103a, the search information acquisition function 103b, the feature acquisition function 103c, the similarity acquisition function 103d, and the search function 103e. Specifically, the information processing system 100 according to this embodiment has a data acquisition function 103a, which is configured to acquire first medical data related to the subject and including multiple items. The information processing system 100 also has a search information acquisition function 103b, which is configured to acquire search information as information for searching for medical data (second medical data) similar to the first medical data. Furthermore, the information processing system 100 has a feature acquisition function 103c, which is configured to acquire features representing the characteristics of the medical data based on the search information. The feature acquisition function 103c is characterized in that it is configured to acquire at least a subset of features by inputting medical data included in the first medical data into at least one of the encoder of a large language model and the encoder of a large multimodal model. The information processing system 100 is characterized in that it also has a similarity acquisition function 103d, which is configured to acquire the similarity between a feature and a feature of another piece of medical data. The information processing system 100 also has a search function 103e, which is configured to acquire search results for medical data (second medical data) similar to the first medical data based on similarity. Here, the search information includes the user's search intent and modifies the features acquired by using the similarity acquisition function 103d, via at least one of the encoder of a large language model and the encoder of a large multimodal model. Alternatively, the search information includes the user's search intent and acquires features from the first features acquired by using the similarity acquisition function, via at least one of the encoder of a large language model and the encoder of a large multimodal model. As described above, the feature acquisition function 103c may include various means. When feature calculations are performed by different devices, feature acquisition function 103c can acquire the calculated features. Alternatively, information processing system 100 can have the function of a feature acquisition device.

[0056] Input interface 104 is configured to accept input operations of various types of instructions and various types of information from users to information processing system 100.

[0057] Specifically, the input interface 104 is connected to the processing circuit 103, which converts the input information received from the user into an electrical signal and sends the signal to the processing circuit 103. For example, the input interface 104 can be implemented as a trackball, a switch button, a mouse, a keyboard, or a touchpad that allows input operations to be performed when its operating surface is touched. Alternatively, the input interface 104 can be implemented as a touch screen formed by integrating the display surface and the touchpad, a contactless input interface using an optical sensor, a voice input interface, etc.

[0058] Input interface 104 is not limited to interfaces that include physical operating components such as a mouse or keyboard. For example, an example of input interface 104 includes electrical signal processing circuitry configured to receive electrical signals corresponding to input operations from external input devices provided separately from information processing system 100, and to send such electrical signals to processing circuitry 103.

[0059] Display 105 is configured to display various types of data via a graphical user interface (GUI), such as medical data and search results data processed by information processing system 100. Specifically, display 105 is connected to processing circuitry 103 and displays various types of data received from processing circuitry 103. For example, display 105 displays textual and image data items based on medical data, and also displays link information and general information corresponding to subsets of similar medical data groups that are search results. Specifically, display 105 is implemented using, for example, a liquid crystal display monitor, a cathode ray tube (CRT) monitor, a touch panel, etc.

[0060] The above description is an example of the functional configuration of the information processing system 100 according to this embodiment.

[0061] The following describes an example of obtaining medical data that is highly similar to the user's search intent. This acquisition is performed during a search process for medical data similar to first medical data, which includes multiple data items, executed by the information processing system 100 according to this embodiment. In other words, examples are presented that are unlikely to include medical data with low relevance to the user's search intent.

[0062] In this embodiment, for ease of description, the medical data refers to the patient's examination data in the example. Specifically, the medical data is a SOAP document comprising four data items in text form described in the medical record information of the electronic medical record. Furthermore, the medical data management device is an electronic medical record system that serves as a database for managing SOAP documents.

[0063] The following is for reference Figure 2The flowchart shown illustrates an example of the processing flow performed by the information processing system 100 according to this embodiment. The order of steps and the sequence of processes within those steps can be changed without causing contradiction.

[0064] In the following description, for ease of understanding, it is assumed that a user operates input interface 104 and selects a single SOAP document from the electronic medical record system. The user is currently editing the selected SOAP document. It is also assumed that the user is performing a similar data search on the electronic medical record system to search for other SOAP documents similar to this SOAP document for reference. In the following description, the selected SOAP document is referred to as "search condition data," and the SOAP documents included in the electronic medical record system other than the "search condition data" are referred to as "searched data."

[0065] In step S101, the search information acquisition function 103b acquires the search purpose selected by the user via the GUI displayed on the display 105 as search information.

[0066] Assume that in the description of this embodiment, a purpose can be selected from three examples of search purposes: “evaluation”, “planning”, and “no specific”.

[0067] In step S102, data acquisition function 103a acquires first medical data including multiple items. Specifically, data acquisition function 103a acquires all SOAP document groups stored in the electronic medical record system. SOAP documents are medical data including four data items in text form (text form namely "document about subjective information", "document about objective information", "document about assessment", and "document about treatment plan").

[0068] In step S103, the feature acquisition function 103c selects data items included in the multiple medical data (SOAP documents) acquired in step S102 based on the search purpose as the search information. For example, when the search information is "assessment", two data items are selected: "documents about subjective information" and "documents about objective information". When the search information is "planning", two data items are selected: "documents about objective information" and "documents about assessment". When the search information is "no specific", all data items are selected, i.e., four data items.

[0069] In step S104, feature acquisition function 103c performs data item processing based on search information. Specifically, for a single SOAP document, the data items selected in step S103 are applied to a template document selected based on search information to generate single text data. When this is done for each SOAP document using search information, a single set of text data is obtained. In the following description, it is assumed that in the description of a SOAP document, "document about subjective information" is text T1, "document about objective information" is text T2, "document about assessment" is text T3, and "document about treatment plan" is text T4.

[0070] For example, when a template document is provided for the search information to be used for "evaluation," the template document takes the form of: "The document about subjective information is '<placeholder P1>'. The document about objective information is '<placeholder P2>'." When the actual selected data item is applied to the template document, the operation is as follows: the <placeholder P1> portion is replaced with text T1, and the <placeholder P2> portion is replaced with text T2. That is, a single text data "The document about subjective information is 'text T1'. The document about objective information is 'text T2'" is generated in a similar manner when the search information is "plan." The single text data "The document about objective information is 'text T2'. The document about evaluation is 'text T3'" is generated in a similar manner when the search information is "no specific." The single text data "The document about subjective information is 'text T1'. The document about objective information is 'text T2'. The document about evaluation is 'text T3'. The document about treatment plan is 'text T4'" is generated in a similar manner. Here, an example of generating simple single text data is described. To improve performance, the template document can be modified. Alternatively, single text data can be generated by simply concatenating selected data items without using a template document.

[0071] In step S105, feature acquisition function 103c sends the single text data group generated in step S104 to feature acquisition device (not shown). The feature acquisition device inputs each element of the received single text data group into an encoder provided in the feature acquisition device, and sends the calculated feature group to feature acquisition function 103c. Feature acquisition function 103c receives and acquires the feature group. In other words, the characteristic of feature acquisition function 103c is that it inputs prompts matching the user's search intent into the encoder to acquire features matching the user's search intent.

[0072] In step S106, the similarity acquisition function 103d calculates the similarity set between the features corresponding to the search condition data and the feature set corresponding to the searched data group using the feature set obtained in step S105. The features are in vector form. Therefore, quantifying and comparing the similarity is sufficient to evaluate the similarity between two feature differences.

[0073] To quantify similarity, one can use general techniques for evaluating vector similarity, such as Manhattan distance, Euclidean distance, cosine similarity, Jaccard coefficient, or a neural network trained to compute similarity.

[0074] In step S107, search function 103e sorts the searched data groups corresponding to the similarity values ​​calculated in step S106. Therefore, even with a large amount of searched data, user accessibility to SOAP documents similar to the search criteria data can be improved. Specifically, for example, when the similarity calculated in step S106 is the Euclidean distance between features, the similarity increases as the value approaches zero. Therefore, the similarity values ​​are sorted in ascending order, and the searched data groups are sorted according to the order of the sorted similarity values. In addition to the similarity between features, search function 103e can also obtain search results for similar medical data based on the values ​​of items included in the medical data.

[0075] In step S108, the search function 103e displays the sorted searched data group as the search results in the GUI of the display 105. At this time, in order to limit the number of SOAP documents included in the search results, the similarity from the highest to the Nth (N is a predetermined number) can be set as the search results.

[0076] When every SOAP document in the search results is displayed in its entirety in the GUI, the results may not be easy to navigate due to the sheer volume of information. Instead, search function 103e can display links that allow the screen to navigate to the patient's medical record information corresponding to the SOAP document in the search results.

[0077] Instead of displaying the entirety of every SOAP document in the search results in the GUI, search function 103e can display a summary of SOAP documents using a large language model. Instead of displaying the entirety of every SOAP document in the search results in the GUI, only a subset of the data items included in the SOAP document can be displayed. For example, specifically, when the search information is "evaluation," only "documents about subjective information" and "documents about objective information" can be displayed, or only "documents about evaluation" can be displayed. Instead of displaying the entirety of every SOAP document in the search results in the GUI, only the first X characters (X is a predetermined number) of the document corresponding to each data item included in the SOAP document can be displayed.

[0078] Based on the above description, the information processing system 100 of this embodiment can calculate an appropriate similarity score in the search processing of medical data similar to medical data including multiple data items, taking into account the user's search intent. By using the similarity score for searching through the search function 103e, the possibility of including medical data that is not highly relevant to the user's search intent can be reduced.

[0079] For example, suppose a user is describing a SOAP document as "a document about an assessment" and refers to another case of a "document about an assessment." In this situation, the "document about an assessment" or "document about a treatment plan" in the SOAP document has not yet been described. In this case, prior art similar data searches might include data in the search results that also do not describe "documents about assessments" or "documents about treatment plans" as similar data. However, using the information processing system 100 according to this embodiment, the user sets search information. Therefore, for example, a similar data search can be performed using only the similarity between "documents about subjective information" and "documents about objective information." That is, the search information is based on instructions from the user. Compared to prior art, this reduces the likelihood that search results will include data that the user is not interested in.

[0080] Hereinafter, variations of the processing of the information processing system 100 according to the first embodiment described above are described. In the following description, configurations and processes similar to those of the information processing system 100 are indicated by the same reference numerals, and thus detailed descriptions are appropriately omitted.

[0081] Variation 1 of the first embodiment

[0082] Although in step S108 according to the first embodiment, the search function 103e displays the search results in the GUI of the display 105, according to Variation 1, the search results can be sent to an external system (not shown). In this case, the search results may or may not be displayed in the GUI of the display 105.

[0083] In this variant, the information processing system 100 can send search results to an external system. Therefore, the external system can display the search results on a display device (not shown) and apply the search results to information analysis.

[0084] Variation 2 of the first embodiment

[0085] The computational cost of feature acquisition in step S105 of the first embodiment is typically high, and the time spent obtaining search results may increase when there is a large amount of search data. To address this issue, for each SOAP document stored in the electronic medical record system, features corresponding to multiple search objectives can be pre-calculated before performing similar data searches. Furthermore, these features can be associated with SOAP documents and search objectives and stored in a database to reduce the time spent obtaining search results. Here, the database is a medical data management device (not shown) communicatively connected to the information processing system 100. The database may be a separately provided database system or a database included in the electronic medical record system.

[0086] First, refer to Figure 3 The flowchart shown illustrates an example of the processing flow performed by the information processing system 100 in this variant. These processes are performed to pre-store the characteristics of SOAP documents from the electronic medical record system in a database. Figure 9 An example of the database table processed in this variant is shown. In this variant, a single type of feature is used in the similarity calculation. Therefore, the table also manages a single type of functionality. When multiple types of features are used in the similarity calculation, the number of feature types to be managed in the table can be changed accordingly. The order of steps and the sequence of processes described herein can be changed without causing inconsistencies.

[0087] Preferably, the process of performing the following steps is carried out during a period when similar data searches are not performed by user operation. For example, the execution of the process can be triggered by instructions from the administrator of the information processing system 100, a scheduled time and date (midnight on a weekday), or a user's scheduled operation (e.g., an operation to complete the editing of a SOAP document).

[0088] In step S1101, data acquisition function 103a acquires SOAP document groups in which features based on SOAP documents and search purposes should be stored in the database. For example, if no features based on any search purpose for SOAP documents exist in the database, the SOAP document group is acquired. For example, when updating a SOAP document after storing existing features of the SOAP document, the SOAP document group is acquired. When it is desired to update all features stored in the database due to, for example, a change in the feature acquisition device, all SOAP document groups are acquired.

[0089] In step S1102, the search information acquisition function 103b selects a search target that has not been selected even once in a series of processes (steps S1101 to S1106). When no search target exists or has not been selected even once, the process ends.

[0090] In steps S1103 to S1104, a process similar to that in steps S101 to S104 of the first embodiment is performed.

[0091] In step S1105, a process similar to that in step S105 of the first embodiment is performed.

[0092] In step S1106, the feature acquisition function 103c associates the acquired feature groups with the corresponding SOAP documents in the electronic medical record and the search purpose selected in step S1102, and stores the acquired feature groups in the database. The process returns to step S1102.

[0093] Based on the above description, the information processing system 100 of this variant can associate features corresponding to the search purpose with SOAP documents for each SOAP document group stored in the electronic medical record, and store the features in the database.

[0094] Next, the following state is described as an example: As a result of executing the above series of processing steps, one or more features associated with the SOAP document in the electronic medical record have been stored in the database (steps S1101 to S1106). Reference Figure 4 The flowchart shown illustrates an example of a process for performing similar data search processing. The order of steps and procedures described herein can be altered without causing inconsistencies.

[0095] The following processing flow describes an example in which, as in the first embodiment, the user operates the input interface 104 to begin searching for similar data from the status of a single SOAP document that has been selected and retrieved from the electronic medical record.

[0096] In step S1201, a process similar to that in step S101 of the first embodiment is performed.

[0097] In step S1202, data acquisition function 103a retrieves the SOAP document group that was added or updated after the last execution of step S1101 from the SOAP documents stored in the electronic medical record.

[0098] In steps S1203 and S1204, a process similar to that in steps S103 and S102 of the first embodiment is performed.

[0099] In step S1205, a process similar to that in step S105 of the first embodiment is performed. Furthermore, the feature acquisition function 103c associates the acquired feature sets with the corresponding SOAP documents and inspection objectives, and stores the acquired feature sets in the database. This aims to reduce the computational cost of acquiring features when performing a similar data search next time.

[0100] In step S1206, feature acquisition function 103c acquires feature groups associated with the remaining SOAP document groups not acquired in step S1202 and the search purpose selected in step S1201, and stores these feature groups in the database.

[0101] In step S1207, the similarity acquisition function 103d calculates the similarity using the feature set obtained from the feature acquisition device in step S1205 and the feature set obtained from the database in step S1206. Specifically, the similarity acquisition function 103d calculates the similarity set between the features corresponding to the search condition data and the feature set corresponding to the searched data set.

[0102] In steps S1208 and S1209, a process similar to that in steps S107 and S108 of the first embodiment is performed.

[0103] As described above, the information processing system 100 in this variant acquires feature groups that have been previously calculated, associated with SOAP document sets and search objectives, and stored in a database. Therefore, the time spent retrieving search results can be reduced.

[0104] Variation 3 of the first embodiment

[0105] In a variant of the first embodiment, step S1207 uses a feature set obtained from the feature acquisition device and a feature set obtained from the database. In variant 2, a similarity set is calculated between the features corresponding to the search condition data and the feature set corresponding to the searched data set.

[0106] In this variant, when the database is a vector database, the feature groups acquired from the feature acquisition device are stored in the database, and the searched data groups that are similar to the search condition data can be identified by using the vector search function provided in the database.

[0107] Specifically, refer to Figure 5 The flowchart shown illustrates an example of the processing flow performed by the information processing system 100 in this variant, corresponding to the series of processing flows described in the first embodiment. The order of steps and the sequence of processes described herein can be changed without causing contradiction.

[0108] In steps S1301 to S1304, a process similar to that in steps S101 to S104 of the first embodiment is performed.

[0109] In step S1305, a process similar to that in step S105 of the first embodiment is performed. Furthermore, the feature acquisition function 103c associates the acquired feature sets with the corresponding SOAP documents and inspection objectives, and stores the acquired feature sets in a database.

[0110] In step S1306, firstly, search function 103e sends a query to the database, where features corresponding to the search criteria data are specified as search criteria. The database sends groups of SOAP documents associated with features that match or are similar to the sent features as search results to search function 103e. At this point, the search results corresponding to similarity can be sorted using the sorting function provided in the database.

[0111] In step S1307, a process similar to that in step S108 of the first embodiment is performed.

[0112] As described above, in this variant, the information processing system 100 retrieves previously calculated feature groups associated with SOAP document sets and search objectives, and stores them in a database. Therefore, the time spent retrieving search results can be reduced.

[0113] Variation 4 of the first embodiment

[0114] The feature acquisition device may include multiple encoder groups to select and switch encoders for feature computation based on a single type of input data or media format based on a single medical data set. Encoders exhibit varying performance. Each encoder can obtain relatively good features when the input is suitable for its data type. Here, encoders that obtain good features are characterized by outputting highly similar feature sets in response to input data sets with similar features, and outputting low-similarity feature sets in response to input data sets with dissimilar features. This performance difference can be adjusted, for example, by changing the configuration of the dataset used to train the encoder, as well as general training conditions such as model architecture, model size, and hyperparameters.

[0115] Specifically, in the first embodiment, the data items selected based on search information and the template documents are different. Therefore, the characteristics of the generated single text data vary significantly.

[0116] To address this issue, in the processing of this variant corresponding to step S105 of the first embodiment, the feature acquisition function 103c sends identification information (e.g., search information) used to identify the characteristics of individual text data, along with the generated group of individual text data, to the feature acquisition device. Based on the received identification information, the feature acquisition device selects and switches the encoder used to calculate the features.

[0117] As described above, in this variant, the information processing system 100 can select and switch encoders by acquiring the type and media format of the input data of the target feature based on the feature acquisition device, thereby obtaining search results that better match the search intent.

[0118] Variation 5 of the first embodiment

[0119] The feature acquisition apparatus may include multiple encoder groups to select the encoder for feature computation based on the type or media format of each of multiple input data based on a single medical data set. Encoders exhibit different performance characteristics. Each encoder can obtain relatively good features when the input is suitable for the data type of the encoder. Here, the meaning of an encoder capable of obtaining good features is similar to that described in Variation 4 of the first embodiment. In this case, the multiple feature groups obtained by inputting multiple input data groups into the encoder can be processed as is or integrated into a single feature. In the case of integration, the features are concatenated, added, or simply averaged, or weighted averaged with predetermined weights. In this variation, an example is described where multiple feature groups are processed as is without integration.

[0120] Specifically, refer to Figure 6 The flowchart shown illustrates an example of the processing flow performed by the information processing system 100 in this variant, corresponding to the series of processing flows described in the first embodiment. The order of steps and the sequence of processes within those steps can be changed without causing contradiction.

[0121] In steps S1401 to S1403, a process similar to that in steps S101 to S103 of the first embodiment is performed.

[0122] In step S1404, feature acquisition function 103c sends the data item combination groups selected in step S103 to the feature acquisition device (not shown). The feature acquisition device inputs each data item included in each group of the received data item combination groups into any one of the multiple encoder groups provided in the feature acquisition device to calculate multiple feature combination groups, and sends the calculated feature combination groups to feature acquisition function 103c. Feature acquisition function 103c receives and acquires the multiple feature combination groups.

[0123] In step S1405, the similarity acquisition function 103d uses the multiple feature combination groups acquired in step S105. The similarity acquisition function 103d calculates the similarity groups between the combination of features corresponding to the search condition data and each feature combination group corresponding to the searched data group. In this variant, the similarity between feature combinations is calculated. However, when the i-th feature in the feature combination is feature Fi (“i” is the number of data items selected from 1), the similarity Si is calculated for the feature corresponding to the same index. Specifically, when the search information is “evaluation”, two data items, “documents about subjective information” and “documents about objective information”, are selected. In this case, for each SOAP document, a combination of features F1 based on “documents about subjective information” and features including F2 based on “documents about objective information” is calculated and acquired. Regarding similarity, the similarity S1 between the feature F1_Q corresponding to the search condition data and the feature F1_V corresponding to a single searched data group is calculated. Furthermore, the similarity S2 between the feature F2_Q corresponding to the search condition data and the feature F2_V corresponding to a single searched data point is calculated. The combined value of S1 and S2, which uses the calculated similarity of the combination of features corresponding to the search condition data and the combination of features corresponding to a single searched data point, can be obtained. Moreover, the combination of similarities S1 and S2 can be obtained as a scalar value, which can be compared with other similarities, for example, by adding them together or by performing a simple or weighted average.

[0124] In steps S1406 and S1407, a process similar to that in steps S107 to S108 of the first embodiment is performed.

[0125] As described above, in this variant, the information processing system 100 can use multiple different encoders to obtain search results that better match the search intent by acquiring the type and media format of the input data of the target based on the feature acquisition device.

[0126] Variation 6 of the first embodiment

[0127] In step S101 of the first embodiment, the search information acquisition function 103b acquires the search information selected by the user. However, search information can be automatically acquired based on medical data without requiring user selection. That is, the characteristic is that the search information is determined based on the analysis results of medical data.

[0128] Specifically, let's describe an example where a user is editing a SOAP document. For instance, suppose the input for the "Document on Subjective Information" and "Document on Objective Information" sections of this SOAP document is complete, or the "Evaluation Document" is being edited. In this case, it's conceivable that the user will complete the "Document on Evaluation" as their next task. Therefore, the search information retrieval function 103b automatically retrieves "Evaluation" as the search objective, i.e., searches for information. For instance, suppose the input for the "Document on Subjective Information," "Document on Objective Information," and "Document on Evaluation" sections of this SOAP document is complete, or the user is editing a "Document on Treatment Plan." In this case, it's conceivable that the user will complete the "Document on Treatment Plan" as their next task. Therefore, the search information retrieval function 103b automatically retrieves "Plan" as the search objective, i.e., searches for information. For instance, suppose the input for the entire SOAP document is complete. In this case, it's difficult to predict which SOAP document the user wants to refer to. However, the user is likely to want to refer to SOAP documents whose entire content (all data items) is similar. Therefore, the search information retrieval function 103b automatically retrieves "No Specific" as the search objective, i.e., searches for information. After the search information acquisition function 103b has automatically acquired the search information, the search information acquisition function 103b can display the string "evaluation" or "plan" in the GUI so that users can refer to the automatically set search information.

[0129] Furthermore, after the search information acquisition function 103b automatically acquires the search information, user operations can cause the information processing system 100 to execute step S102 and subsequent processing in the first embodiment to perform a similar data search. Here, user operations refer to, for example, pressing a button control in the GUI that indicates "start similar data search".

[0130] When the automatically acquired search information does not match the user's search intent, the user can perform the operation described in step S101 of the first embodiment, and then the process of this variant can switch to the series of processing flows in the first embodiment.

[0131] As another example, when the medical data is, for example, medical record information, the search information acquisition function 103b can automatically acquire search information based on the diagnosis recorded in the medical record information. In this case, for example, the search information is a string based on a diagnosis such as "breast cancer" or "lung cancer". Before acquiring the entire medical record information stored in the electronic medical record in step S102 of the first embodiment, only the medical record entries that record the diagnosis can be acquired. Then, in step S103 of the first embodiment, the feature acquisition function 103c selects data items that are useful for diagnosing the disease corresponding to the search information. For example, when the search information is "breast cancer", data related to the presence / absence of family medical history in the form of categories and mammographic images in the form of images are selected as data items. Then, in step S104 of the first embodiment, the feature acquisition function 103c processes the data items as needed to allow the data items to be input into various types of encoders provided in the feature acquisition device. For example, by using rule-based processing, etc., data related to the presence / absence of family medical history is converted into text data, such as "mother has a history of breast cancer", to allow the data to be input into a text encoder. For example, a mammogram image is converted into a set of pixel values ​​to allow the image to be input into an image encoder. Subsequent steps can be handled by step S1405 and the subsequent process of a variant of the first embodiment 5, which performs a similarity search considering the similarity associated with multiple sets of data items.

[0132] As another example, when the medical data is, for example, medical record information, the search information acquisition function 103b acquires CT images based on link information to examination data recorded in the medical record information. The search information acquisition function 103b can automatically acquire search information based on image analysis results of the CT images. In this case, for example, the search information is a string such as "lung cancer" or "liver cancer". Then, in step S103 of the first embodiment, the feature acquisition function 103c selects data items useful for diagnosing the disease corresponding to the search information. For example, when the search information is "lung cancer", data related to smoking history in category form and CT images in image form are selected as data items. Then, in step S104 of the first embodiment, the feature acquisition function 103c processes the data items as needed to allow input of the data items into various encoders provided in the feature acquisition device. For example, by using rule-based processing, data related to smoking history is converted into text form, such as "smoking history; Brinkman index is 40 × 20 = 800", so that the data can be input into the text encoder. For example, CT images that can be input into the image encoder as is are not processed. The subsequent steps can be handled by step S1405 and the subsequent process of variant 5 of the first embodiment, which takes into account the similarity associated with multiple groups of data items to perform a similar data search.

[0133] As described above, in this variant, by analyzing the status of medical data, the information processing system 100 can automatically obtain search information and perform similar data searches.

[0134] Variation 7 of the first embodiment

[0135] In step S104 of the first embodiment, the feature acquisition function 103c generates text data by using a template document to process the data item, allowing the feature acquisition device to input the data into the encoder. During the processing of this data item, the information processing system 100 can add a predetermined string (text). The process of adding the string can be performed regardless of whether the feature acquisition function 103c uses a template document.

[0136] For example, when the search term is "evaluation," you could add a string such as "checking the above information." Similarly, when the search term is "planning," you could add a string such as "what to do next in the above situation."

[0137] The string to be added can be determined based on information beyond the search results. For example, the string to be added can be determined by using predetermined analysis processing, based on the results of processing data items included in the medical data. Specifically, when disease X is predetermined to be positive as a result of analysis processing of blood tests and image examinations included in the medical data, a string such as "Note, disease X is positive" can be added.

[0138] The string to be added can be a pre-defined string recorded in the information processing system 100.

[0139] As described above, in this variant, when the feature acquisition device further processes the textual data to input it into the encoder, the information processing system 100 can guide the trend of features calculated by the encoder. That is, it emphasizes the characteristics of medical data among the features and reduces features that would become noise and reduce search accuracy in similar data searches. This reduces the likelihood of including medical data that is not highly relevant to the user's search intent.

[0140] Second Embodiment

[0141] In the following, in the second embodiment, reference is made to Figure 10 and Figure 11The description describes the search processing performed by the information processing system 1000 on medical data similar to medical data that includes multiple types of data. Here, multiple types of data refer to data items that include data in different media formats. In this embodiment, for ease of description, medical data refers to a patient's examination data in the medical field. Specifically, medical data is medical record information from an electronic medical record. Furthermore, the medical data management device is an electronic medical record system that serves as a database for managing medical record information.

[0142] Reference Figure 10 This section describes the functional configuration of the information processing system 1000 according to this embodiment. The information processing system 1000 includes a communication interface 1010, a storage circuit 1020, a processing circuit 1030, an input interface 1040, and a display 1050. The information processing system 1000 can be communicatively connected to a network 2000 via the communication interface 1010. The functional configurations of the communication interface 1010, storage circuit 1020, input interface 1040, and display 1050 are similar to those in the first embodiment. Therefore, a description of these components is omitted.

[0143] The processing circuit 1030 according to this embodiment includes a data acquisition function 1030a that implements a data acquisition unit configured to acquire medical data and a feature acquisition function 1030c that implements a feature acquisition unit configured to acquire features. The processing circuit 1030 also includes a similarity acquisition function 1030d, which implements a similarity acquisition unit configured to acquire the similarity between multiple pieces of medical data. The processing circuit 1003 also includes a search function 1030e, which implements a search unit configured to acquire search results for similar medical data.

[0144] When the information processing system 1000 has the above-described functional configuration, it can fully calculate the similarity to medical data, which includes different types of data. By using the similarity calculation, medical data (second medical data) similar to the first medical data can be obtained with high accuracy.

[0145] Here, for example, by Figure 10 Each processing function implemented by the corresponding element of the processing circuit 1030 shown is stored in the storage circuit 1020 in the form of a computer-executable program. The elements include a data acquisition function 1030a, a feature acquisition function 1030c, a similarity acquisition function 1030d, and a search function 1030e.

[0146] The processing circuit 1030 reads the program from the storage circuit 1020 and executes the read program, thereby realizing the function corresponding to the program. That is, the processing circuit 1030 that has read the program has the functions implemented by the data acquisition function 1030a, the feature acquisition function 1030c, the similarity acquisition function 1030d, and the search function 1030e.

[0147] Specifically, the information processing system 1000 according to this embodiment has a data acquisition function 1030a configured to acquire first medical data related to a subject and including different types of data. The information processing system 1000 also has a feature acquisition function 1030c configured to represent the characteristics of the medical data. The feature acquisition function 1030c is characterized in that it is configured to acquire at least a subset of features by inputting medical data including the different types of data included in the first medical data into an encoder. Here, the encoder is at least one of a large language model encoder and a large multimodal model encoder. Furthermore, the information processing system 1000 has a similarity acquisition function 1030d configured to acquire the similarity between a feature and a feature of another piece of medical data. The information processing system 1000 also has a search function 1030e configured to acquire search results for medical data (second medical data) similar to the first medical data based on similarity. Through the above configuration, appropriate similarity can be calculated for medical data including different types of data, and high-precision search results can be obtained by using this similarity. The following describes an example of the processing flow performed by the information processing system 1000 according to this embodiment. The order of steps and the sequence of processes described herein may be changed without causing contradiction.

[0148] In the following description, for ease of understanding, it is assumed that the current user operates input interface 104 to select patient medical record information from the electronic medical record system, view the selected information, and use it to create a treatment plan. It is also assumed that the user is performing a similar data search on the electronic medical record system to find other medical record information similar to the selected information for reference. In the following description, the selected medical record information is referred to as "search condition data," and the other medical record information is referred to as "searched data."

[0149] In step S201, the data acquisition function 1030a acquires information about the start of an examination of similar data similar to the first medical data. The user operates the GUI displayed on the monitor 105 to indicate that the examination should begin. For example, this operation refers to pressing the button control in the GUI that indicates "Start similar data search".

[0150] In step S202, the data acquisition function 1030a acquires all medical record information stored in the electronic medical record. Medical record information is medical data that includes data items in various media formats (such as digital, text, and image formats).

[0151] In step S203, the feature acquisition function 1030c sends the data item combination group included in the medical record information acquired in step S202 to the feature acquisition device (not shown). The feature acquisition device inputs each data item included in the received data item combination group into various types of encoders provided in the feature acquisition device, and sends the calculated feature combination group to the feature acquisition function 1030c. The feature acquisition function 1030c receives and acquires the feature group.

[0152] Various types of encoders include numerical encoders that can accept data in digital form, text encoders that can accept data in text form, and image encoders that can accept data in image form. In other words, data items in various media formats included in medical record information are input into various types of encoders, and various types of encoders calculate features.

[0153] In step S204, the similarity acquisition function 1030d uses the feature combination groups acquired in step S203. The similarity acquisition function 1030d calculates the similarity groups between the combination of features corresponding to the search condition data and each feature combination group corresponding to the searched data group. Here, each similarity can be calculated using the method described in step S1406 of variant 5 of the first embodiment.

[0154] In step S205, the search function 1030e sorts the searched data groups based on the similarity values ​​calculated in step S204. When the above process is performed, even if there is a large amount of searched data, the user's accessibility to medical record information similar to the search criteria data is improved.

[0155] In step S206, the search function 1030e displays the sorted searched data group as the search result in the GUI of the display 105.

[0156] Based on the above description, the information processing system 1000 according to this embodiment can search for medical data (second medical data) that is similar to first medical data, which includes multiple types of data.

[0157] The similarity can be appropriately calculated using the techniques disclosed herein.

[0158] While this disclosure has been described with reference to embodiments, it should be understood that this disclosure is not limited to the disclosed embodiments. The scope of the following claims should be given the broadest interpretation to cover all such variations and equivalent structures and functions.

Claims

1. An information processing system, comprising: The data acquisition unit is configured to acquire first medical data, which includes multiple items or different types of data related to the subject. A feature acquisition unit is configured to acquire features representing the characteristics of medical data, wherein the feature acquisition unit acquires at least a subset of the features by inputting medical data included in the first medical data into at least one of an encoder of a large language model and an encoder of a large multimodal model. as well as The similarity acquisition unit is configured to acquire the similarity between the feature and a feature of another piece of medical data.

2. The information processing system according to claim 1, in, The first medical data includes the aforementioned multiple items. The information processing system further includes: a search information acquisition unit, configured to acquire search information for searching for second medical data similar to the first medical data, and The feature acquisition unit acquires at least a subset of the features based on the search information.

3. The information processing system according to claim 1 further includes: The search unit is configured to obtain search results for second medical data that are similar to the first medical data based on the similarity.

4. The information processing system according to claim 1, in, The first medical data includes the aforementioned different types of data. The information processing system further includes: a search information acquisition unit, configured to acquire search information for searching for second medical data similar to the first medical data, and The feature acquisition unit acquires the features based on the search information.

5. The information processing system according to claim 2, in, The search information includes the user's search intent, and the search information modifies the features obtained by using the similarity acquisition unit, the encoder of the large language model, and the encoder of the large multimodal model.

6. The information processing system according to claim 4, in, The search information includes the user's search intent, and the search information modifies the features obtained by using the similarity acquisition unit, the encoder of the large language model, and the encoder of the large multimodal model.

7. The information processing system according to claim 2, in, The search information includes the user's search intent, and the search information is obtained from a first feature acquired by using the similarity acquisition unit, the encoder of the large language model, and the encoder of the large multimodal model.

8. The information processing system according to claim 4, in, The search information includes the user's search intent, and the search information is obtained from a first feature acquired by using the similarity acquisition unit, the encoder of the large language model, and the encoder of the large multimodal model.

9. The information processing system according to claim 2, in, The feature acquisition unit inputs a prompt, including the user's search intent, into the encoder to obtain features that match the user's search intent.

10. The information processing system according to claim 4, in, The feature acquisition unit inputs a prompt, including the user's search intent, into the encoder to obtain features that match the user's search intent.

11. The information processing system according to claim 2, in, The search information is set based on the user's instructions.

12. The information processing system according to claim 4, in, The search information is set based on the user's instructions.

13. The information processing system according to claim 2, in, The search information is determined based on the analysis results of the medical data.

14. The information processing system according to claim 4, in, The search information is determined based on the analysis results of the medical data.

15. The information processing system according to claim 1, in, The encoder is trained such that the similarity of the output feature set in response to an input of a medical data set in which the characteristics of a predetermined data item group are similar is higher than the similarity of the output feature set in response to an input of a medical data set in which the characteristics of the predetermined data item group are dissimilar.

16. The information processing system according to claim 3, in, The search unit causes the search results to be displayed by the display unit.

17. The information processing system according to claim 3, in, The search unit obtains the search results for the second medical data based on the similarity and the values ​​of one or more items included in the medical data.

18. The information processing system according to claim 2, in, At least one of the large language model and the large multimodal model includes multiple encoders, and The feature acquisition unit selects the encoder based on the search information.

19. The information processing system according to claim 2, in, At least one of the large language model and the large multimodal model includes multiple encoders, and The feature acquisition unit selects the plurality of encoders based on the search information.

20. The information processing system according to claim 19, in, The feature acquisition unit acquires the features by integrating the features acquired using the plurality of selected encoders.

21. A method for processing information, the method comprising the following steps: Acquire first medical data, which includes multiple items or different types of data related to the subject; The step of acquiring features, which represent the characteristics of medical data, is to acquire at least a subset of the features by inputting medical data included in the first medical data into at least one of an encoder of a large language model and an encoder of a large multimodal model. as well as Obtain the similarity between the feature and a feature of another piece of medical data.

22. The method according to claim 21, in, The first medical data includes the aforementioned multiple items. The method further includes acquiring search information for searching second medical data similar to the first medical data, and In the step of obtaining features, at least a subset of the features is obtained based on the search information.

Images