Information processing systems, information processing methods, and programs

Abstract

The objective is to provide an information processing system that can appropriately calculate similarity. [Solution] The information processing system according to the present invention is characterized by comprising: data acquisition means for acquiring first medical data comprising a plurality of items relating to a subject; search information acquisition means for acquiring search information which is information for searching for medical data similar to the first medical data; feature quantity acquisition means for acquiring feature quantities that represent the characteristics of the medical data based on the search information, wherein the feature quantity acquisition means acquires at least a part of the feature quantities by inputting the medical data included in the first medical data into at least one of a large-scale language model encoder and a large-scale multimodal model encoder; and similarity acquisition means for acquiring the similarity between the feature quantities and the feature quantities of other medical data.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] A similar data search technique for searching for similar data by converting data such as text and images into numerical representations is known.

[0003] For example, Patent Document 1 describes a technique for calculating a feature amount from a partial image in a document image and searching for another document having a similar feature amount.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the method described in Patent Document 1 has not been considered for cases where it is desired to search for similar data using a plurality of types of data including text, or cases where it is desired to search for similar data in consideration of the user's intention, and appropriate similarity calculation may be difficult in some cases.

[0006] An object of the present invention is to provide an information processing system, an information processing method, and a program capable of appropriately calculating a similarity.

Means for Solving the Problems

[0007] An information processing system according to one aspect of this disclosure is characterized by comprising: data acquisition means for acquiring first medical data comprising a plurality of items relating to a subject; search information acquisition means for acquiring search information which is information for searching for medical data similar to the first medical data; feature quantity acquisition means for acquiring feature quantities that represent the characteristics of the medical data based on the search information, wherein the feature quantity acquisition means acquires at least a portion of the feature quantities by inputting the medical data included in the first medical data into at least one of a large-scale language model encoder and a large-scale multimodal model encoder; and similarity acquisition means for acquiring the similarity between the feature quantities and the feature quantities of other medical data.

[0008] Furthermore, an information processing system according to one aspect of the present disclosure includes: a data acquisition means for acquiring medical data, which acquires first medical data comprising different types of data relating to a subject; a feature quantity acquisition means for representing the characteristics of the medical data, which acquires at least a portion of the feature quantity by inputting the medical data, which includes different types of data contained in the first medical data, into at least one of a large-scale language model encoder and a large-scale multimodal model encoder; and a similarity acquisition means for acquiring the similarity between the feature quantity and the feature quantity of other medical data. [Effects of the Invention]

[0009] The technology described herein allows for the accurate calculation of similarity. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of the functional configuration of the information processing system according to the first embodiment. [Figure 2] This is a flowchart of the process performed by the information processing system according to the first embodiment. [Figure 3] This is a flowchart of the database storage process performed by the information processing system according to Modification 2 of the First Embodiment. [Figure 4] This is a flowchart of the process performed by the information processing system according to Modification 2 of the First Embodiment. [Figure 5] This is a flowchart of the process performed by the information processing system according to Modification 3 of the First Embodiment. [Figure 6] This is a flowchart of the process performed by the information processing system according to Modification 5 of the First Embodiment. [Figure 7] This is an example of the structure of medical record information handled by an information processing system. [Figure 8] This is an example of a SOAP document handled by an information processing system. [Figure 9] This is an example of a database configuration handled by an information processing system. [Figure 10] This figure shows an example of the functional configuration of an information processing system according to the second embodiment. [Figure 11] This is a flowchart of the process performed by the information processing system according to the second embodiment. [Modes for carrying out the invention]

[0011] Embodiments of this disclosure will be described below with reference to the drawings. However, this disclosure is not limited to the embodiments described below, and may be modified as appropriate without departing from its essence. Furthermore, in the drawings described below, components having the same function are denoted by the same reference numerals, and their descriptions may be omitted or simplified.

[0012] <First Embodiment> Figure 1 shows an example of the functional configuration of the information processing system 100 according to the first embodiment. The information processing system 100 acquires medical data and performs information processing on the medical data. Here, the target data is not limited to medical data, but may also be content data including the above data.

[0013] Here, the medical data according to this embodiment refers to data in which a plurality of data items are mixed. Each data item may be composed of different media formats (numerical values, categories, text, audio, still images, moving images, etc.). For example, the medical record information regarding a single patient managed in an electronic medical record is an example of medical data including a plurality of items related to this embodiment.

[0014] Specifically, the medical record information, which is medical data, can include data items in numerical format such as the age, height, weight, and blood test data of a patient (subject). Also, as medical data, it may include data items in category format such as the gender (male, female, other) of the patient (subject), the presence or absence of a smoking history, the presence or absence of each disease history in the family, and the presence or absence of consent regarding medical treatment and hospitalization.

[0015] Furthermore, it may include the patient's name, the entire text in the SOAP (Subjective Objective Assessment Plan) format in the medical and nursing record (hereinafter referred to as the SOAP text), the diagnosis name, and link information to external systems (department systems, websites, etc.). Also, it may include data items in text format such as the text in the inspection report that can be obtained based on the link information.

[0016] In addition, it can include data items in image format such as scanned images of referral letters and consent forms, inspection images such as X-ray images and CT images that can be obtained based on the above link information, and key images in the inspection report. Also, it may include data items in waveform format such as electrocardiograms, data items in audio format such as body sounds obtained by a digital stethoscope, and data items in video format or coordinate format related to walking data in rehabilitation.

[0017] As a specific example, examples of data items included in medical record information are shown in FIG. 7. DICOM image data related to one examination or one imaging, which is managed by a PACS (Picture Archiving and Communication System), is also an example of medical data including data corresponding to a plurality of data items related to the present embodiment. Specifically, the DICOM image data can include data items in numerical format such as information regarding imaging position and pixel configuration. Further, it can include data items in text format such as imaging device name, facility name, patient name, examination identifier, and site name. Also, it includes data items in image format such as image data and overlay data. Moreover, it can include data items in a proprietary format that can be used only by some DICOM image viewers. Note that the image data can be a 2D image, a 3D image, or a video of a 2D or 3D image depending on an imaging device that outputs the DICOM image data.

[0018] Also, for example, the SOAP note in a medical nursing record can also be regarded as an example of medical data in which a plurality of data items related to the present embodiment are mixed. The SOAP note is medical data including four data items: a text-form "note on subjective information", "note on objective information", "note on evaluation", and "note on treatment plan". The group of data items constituting the SOAP note may be recorded as one data item by using an electronic medical record. As a specific example, an example of the SOAP note is shown in FIG. 8. The SOAP note data A101 is an example structured into four data item groups. The SOAP note data A102 is an example that is not structured and has only one data item. In the present embodiment, for simplicity, it is assumed that the SOAP note is composed of four data items. Note that when it is not structured, a process of dividing it into four data items may be used, or the entire text may be treated as one item.

[0019] The information processing system 100 may be implemented by computer equipment such as a server or workstation, and may display the results of information processing, save the results of information processing, or output the results of information processing to an external device. For example, the information processing system 100 may search for other medical data similar to any medical data. Link information and summary information corresponding to a portion of the search results, which are a group of similar medical data, may be displayed on a display 105 or an unshown display device connected to the information processing system 100.

[0020] The information processing system 100 may be connected to a medical data management device (not shown) via a network 200 or a communication cable or communication circuit (not shown) to acquire medical data. The medical data management device may be configured as an integral part of the information processing system 100.

[0021] The medical data management device is a database device that stores medical data and can send and receive medical data to and from other devices such as an information processing system 100 that can communicate with the medical data management device. Furthermore, the medical data management device may also receive search queries for medical data from other devices such as the information processing system 100 that can communicate with the medical data management device and transmit the search results. Specifically, for example, it may receive queries using SQL (Structured Query Language) or similar methods, using numerical values ​​or strings within data items contained in the medical data as search conditions, and transmit sets of medical data matching the conditions, or their identifiers.

[0022] Furthermore, the medical data management device may store feature quantities in vector (numerical group) format that represent the characteristics of at least some of the data item groups contained in each stored medical data. The medical data management device may also receive queries using feature quantities as search criteria and transmit medical data sets or their identifiers associated with feature quantities that match or are similar to those specified.

[0023] The information processing system 100 may be connected to other devices in order to acquire vector-form feature quantities that represent the characteristics of at least some of the data item groups included in the medical data. For example, the information processing system 1000 may be connected to a feature quantity acquisition device (not shown) via a network 200, or via a communication cable or communication circuit (not shown). The feature quantity acquisition device may be configured integrally with the information processing system 100 as one of its components.

[0024] The feature acquisition device is a device that acquires at least a portion of the data items contained in medical data, calculates features based on those data items, and transmits those features to other devices such as an information processing system 100 that can communicate with the feature acquisition device. The feature acquisition device includes one or more encoders that, upon input of data corresponding to one or more data items, calculate vector-format feature quantities that represent the characteristics of the data corresponding to those data items. Encoders included in large-scale language models or large-scale multimodal models can be used as encoders.

[0025] An encoder included in a large-scale language model is a text encoder that can accept text-formatted text prompts as input.

[0026] For example, an example of a large-scale language model is Meta AI's "Llama (Large Language Model Meta AI)". Furthermore, the encoders included in a large-scale multimodal model are media-specific encoders (numerical encoders, categorical encoders, text encoders, audio encoders, and image (still image and video) encoders) that can accept prompts for each media format.

[0027] For example, an example of a large-scale multimodal model is OpenAI's "CLIP (Contrastive Language - Image Pre-Training)". The encoder takes data in a corresponding media format as input and outputs vector-based feature quantities that represent the content, concepts, and other features related to that data. Text encoders can also take numerical and categorical data as input by converting them into strings (text).

[0028] Furthermore, the encoder may utilize a portion of a neural network configured to perform predetermined image processing tasks such as image classification, image generation, image segmentation, distance learning, and comparative learning. In other words, the intermediate features calculated when data is input to that neural network may be used as features that represent the characteristics of that data.

[0029] Furthermore, the encoder may be trained to obtain a high similarity score for feature sets when similar data sets are input, and a low similarity score for feature sets when dissimilar data sets are input.

[0030] Specifically, consider a case where, among images A through C, only image C is similar to the other two images. In this case, the encoder may be trained so that the similarity between feature A and feature B is high, while the similarity between feature A and feature C, and between feature B and feature C, is low.

[0031] In other words, the encoder is characterized by being trained such that when a set of test data with similar features is input, the similarity of the output feature set is higher than when a set of test data with dissimilar features is input. This training may be achieved by fine-tuning a pre-trained encoder. For fine-tuning, common learning methods such as supervised learning, reinforcement learning, training of only the output layer, or LoRA (Low-Rank Adaptation) can be used.

[0032] Here, when the similarity retrieval function calculates the similarity between features, a problem arises as the number of dimensions of vector-form features increases, causing the difference in distance between vectors to decrease and making it difficult to distinguish similarity. Therefore, the similarity retrieval function may apply a dimensionality reduction method to the features as a preprocessing step before calculating the similarity of the features. Examples of dimensionality reduction methods include principal component analysis (PCA), t-SNE (t-Stochastic Neighbor Embedding), and UMAP (Uniform Manifold Approximation and Projection).

[0033] Another method of dimensionality reduction involves using a similarity acquisition function to input similar or dissimilar data sets into an encoder, statistically analyzing the resulting feature set, and removing the values ​​of other dimensions while retaining only those dimensions useful for evaluating the similarity between features. Specifically, the similarity acquisition function may, for example, remove the values ​​of dimensions with large variance from the feature set obtained by inputting similar data sets into an encoder.

[0034] It should be noted that whether data sets are considered similar or dissimilar may vary depending on the user's perspective. For example, consider a case where a set of images can be grouped by brightness into dark images and bright images, and also by imaging location into images showing the upper body and images showing the lower body.

[0035] While each group may be similar in some respects, for example, not all of the dark images necessarily depict the upper (or lower) body; the dark images may be a collection of images that are not similar in terms of the captured body part.

[0036] Therefore, when performing a similar data search, it is desirable to change the features used for similarity evaluation according to the user's intent. In other words, it is desirable to change the dimensions to be reduced or the combination of data used to evaluate similarity, according to the user's intent. In this embodiment, the above changes are made according to the search information described later, so that the similar data search intended by the user can be performed.

[0037] Figure 1 shows an example of the configuration of the information processing system 100 according to this embodiment. As shown in Figure 1, the information processing system 100 includes a communication interface 101, a storage circuit 102, a processing circuit 103, an input interface 104, and a display 105. The information processing system 100 can also be connected to the network 200 via the communication interface 101.

[0038] The communication interface 101 is an interface for communicating medical data, search results, etc., with other devices. The communication interface 101 can be implemented by a network communication interface such as a network adapter or a NIC (Network Interface Controller). Alternatively, the communication interface 101 may be implemented by a device connection interface such as USB (Universal Serial Bus), PCI Express, SATA (Serial ATA), or M.2.

[0039] The memory circuit 102 stores various data and programs used in the processing performed by the information processing system 100 according to this embodiment. Specifically, the memory circuit 102 is connected to the processing circuit 103 and operates under the control of the processing circuit 103. The memory circuit 102 also functions as a work memory for temporarily storing various data used in the processing performed by the processing circuit 103. The memory circuit 102 can be implemented, for example, by semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or by hard disks, optical discs, etc.

[0040] The processing circuit 103 controls the operation of each of the above-mentioned parts of the information processing system 100. For example, the processing circuit 103 performs various processes in response to instructions received from the user via the input interface 104 connected to the information processing system 100. Alternatively, for example, the processing circuit 103 may perform various processes in response to instructions received from the user via the communication interface 101. The processing circuit 103 is implemented, for example, by a CPU (Central Processing Unit).

[0041] The processing circuit 103 includes, for example, a data acquisition function 103a that implements a data acquisition means for acquiring medical data, and a search information acquisition function 103b that implements a search information acquisition means for acquiring search information. The processing circuit 103 also includes a feature acquisition function 103c that implements a feature acquisition means for acquiring feature quantities, and a similarity acquisition function 103d that implements a similarity acquisition means for acquiring the similarity between medical data. Furthermore, the processing circuit 103 includes a search function 103e that implements a search means for acquiring search results for similar medical data.

[0042] The information processing system 100, with the above-described functional configuration, can appropriately acquire the degree of similarity with the medical data intended by the user. Furthermore, by performing a search using this similarity, the information processing system 100 can acquire search results that align with the user's search intent.

[0043] Here, for example, each processing function realized by the components of the processing circuit 103 shown in Figure 1 is stored in the memory circuit 102 in the form of a program that can be executed by a computer.

[0044] The processing circuit 103 reads each program from the storage circuit 102 and executes each read program to realize the function corresponding to each program. That is, the processing circuit 103 that reads each program has functions realized by the data acquisition function 103a, the search information acquisition function 103b, the feature quantity acquisition function 103c, the similarity acquisition function 103d, and the search function 103e. Specifically, the information processing system 100 according to this embodiment has a data acquisition function 103a that acquires first medical data consisting of multiple items relating to a subject. The information processing system 100 also has a search information acquisition function 103b that acquires search information, which is information for searching for medical data similar to the first medical data. In addition, the information processing system 100 has a feature quantity acquisition function 103c that acquires feature quantities that represent the characteristics of the medical data based on the search information. The feature quantity acquisition function 103c is characterized by acquiring at least a part of the feature quantities by inputting the medical data contained in the first medical data into at least one of a large-scale language model encoder and a large-scale multimodal model encoder. Furthermore, the information processing system 100 is characterized by having a feature quantity and a similarity acquisition means 103d for acquiring the similarity between the feature quantity and the feature quantity of other medical data. The information processing system 100 also further has a search function 103e for acquiring search results for medical data similar to the first medical data based on the similarity. Here, the search information is information that includes the user's search intent and is information for changing the feature quantity acquired by the similarity acquisition function 103d via at least one of the encoder of the large-scale language model and the encoder of the large-scale multimodal model. Alternatively, the search information is information that includes the user's search intent and is information for acquiring the feature quantity from the first feature quantity acquired by the similarity acquisition function via at least one of the encoder of the large-scale language model and the encoder of the large-scale multimodal model. As described above, the feature quantity acquisition function 103c may be composed of different devices, and if the calculation of the feature quantity is performed by different devices, the function may acquire the calculated feature quantity. Alternatively, the information processing system 100 may have the function of a feature quantity acquisition device.

[0045] The input interface 104 receives various instructions and input operations for various types of information from the user to the information processing system 100.

[0046] Specifically, the input interface 104 is connected to the processing circuit 103 and converts input operations received from the user into electrical signals and transmits them to the processing circuit 103. For example, the input interface 104 can be implemented by a trackball, switch buttons, mouse, keyboard, or touchpad, which allows input operations by touching the operating surface. Alternatively, the input interface 104 may be implemented by a touchscreen that integrates a display screen and a touchpad, a non-contact input interface using an optical sensor, or an audio input interface, etc.

[0047] Furthermore, the input interface 104 is not limited to those equipped with physical operating components such as a mouse or keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device located separately from the information processing system 100 and transmits these electrical signals to the processing circuit 103 is also included as an example of the input interface 104.

[0048] The display 105 displays various data, such as medical data and data based on search results, processed by the information processing system 100, along with a GUI (Graphical User Interface). Specifically, the display 105 is connected to the processing circuit 103 and displays various data received from the processing circuit 103. For example, the display 105 displays text-format data items and image-format data items based on medical data, as well as link information and summary information corresponding to some of the similar medical data groups that are search results. Specifically, for example, the display 105 can be implemented using an LCD monitor, a CRT (Cathode Ray Tube) monitor, a touch panel, etc.

[0049] The above is a description of an example of the functional configuration of the information processing system 100 according to this embodiment.

[0050] In this embodiment, an example is described of a search process for medical data similar to a first medical data set, which consists of multiple data items, performed by the information processing system 100, in which medical data with a high degree of similarity to the user's search intent is obtained. In other words, it is an example of making it less likely to include medical data that is not related to the user's search intent.

[0051] In this embodiment, for ease of explanation, an example will be described in which the medical data is patient test data. Specifically, the medical data will be a SOAP document consisting of four text-format data items recorded in the medical record information of the electronic medical record. The medical data management device will be an electronic medical record system, which is a database for managing SOAP documents.

[0052] Below, an example of the processing flow executed by the information processing system 100 according to this embodiment will be explained using the flowchart in Figure 2. Note that the order of the steps and the order of procedures within each step described here may be changed as long as no inconsistencies arise.

[0053] For the sake of clarity, the following explanation assumes that the user has selected a SOAP document from the electronic medical record system by operating the input interface 104 and is currently editing it. It also assumes that the user is attempting to find other SOAP documents similar to the selected one within the electronic medical record system for reference. In the following explanation, the selected SOAP document will be referred to as "search criteria data," and SOAP documents other than the "search criteria data" included in the electronic medical record system will be referred to as "searched data."

[0054] In step S101, the search information acquisition function 103b acquires the search purpose selected by the user via the GUI displayed on the display 105 as search information. In this description of the embodiment, it is assumed that three search purposes are available for selection: "Assessment," "Planning," and "None in particular."

[0055] In step S102, the data acquisition function 103a acquires the first medical data, which consists of multiple items. Specifically, the data acquisition function 103a acquires all SOAP documents stored in the electronic medical record system. A SOAP document is medical data that includes four data items in text format: "subjective information," "objective information," "evaluation," and "treatment plan."

[0056] In step S103, the feature acquisition function 103c selects data items that make up each medical data (SOAP document) acquired in step S102, based on the search objective, which is the search information. For example, if the search information is "assessment," it selects two data items: "subjective information" and "objective information." Alternatively, if the search information is "planning," it selects two data items: "objective information" and "evaluation." Or, if the search information is "none in particular," it selects all four data items.

[0057] In step S104, the feature acquisition function 103c performs a data item processing procedure based on the search information. Specifically, for a single SOAP document, the data items selected in the procedure in step S103 are applied to a template document selected based on the search information to generate single text data. By performing this operation for each SOAP document, a set of single text data is obtained. The following explanation assumes that in a certain SOAP document, the "subjective information" is text T1, the "objective information" is text T2, the "evaluation" is text T3, and the "treatment plan" is text T4.

[0058] For example, if the search information is "assessment," the template text used might look like this: "The subjective information is '<placeholder P1>'. The objective information is '<placeholder P1>'." When the selected data items are actually applied to the template text, the <placeholder P1> part is replaced with text T1, and the <placeholder P2> part is replaced with text T2. In other words, a single text data is generated: "The subjective information is 'text T1'. The objective information is 'text T2'." Similarly, if the search information is "planning," a single text data is generated: "The objective information is 'text T2'. The evaluation is 'text T3'." Similarly, if the search information is "none in particular," a single text data is generated: "The subjective information is 'text T1'. The objective information is 'text T2'. The evaluation is 'text T3'. The treatment plan is 'text T4'." This generates a single text data item. Here, we have shown a simple example of generating a single text data item, but you can modify the template text to get better performance. Alternatively, you can generate a single text data item by simply concatenating the selected data items without using a template text.

[0059] In step S105, the feature acquisition function 103c transmits the single-text data set generated in the procedure of step S104 to a feature acquisition device (not shown). The feature acquisition device inputs each of the received single-text data sets into an encoder provided by the feature acquisition device and transmits the calculated feature set to the feature acquisition function 103c. The feature acquisition function 103c receives and acquires the feature set. In other words, the feature acquisition function 103c is characterized by inputting a prompt in line with the user's search intent to the encoder in order to acquire features that match the user's search intent.

[0060] In step S106, the similarity acquisition function 103d uses the feature set obtained in step S105 to calculate similarity sets between the feature set corresponding to the search condition data and the feature set corresponding to the searched data set. Since the feature sets are in vector form, to evaluate the similarity between two feature sets, it is necessary to quantify the similarity so that it can be compared. For quantifying the similarity, common vector similarity evaluation methods such as Manhattan distance, Euclidean distance, cosine similarity, Jaccard coefficient, or a neural network trained to calculate similarity can be used.

[0061] In step S107, the search function 103e sorts the search data sets corresponding to the similarity values ​​calculated in step S106. This improves user access to SOAP documents similar to the search criteria data, even when there is a large amount of search data. Specifically, for example, if the similarity calculated in step S106 is the Euclidean distance between features, a similarity value closer to 0 indicates greater similarity. Therefore, the similarity values ​​are sorted in ascending order, and the search data sets are also sorted according to the sorted similarity order. In addition to the similarity between features, the search function 103e may also obtain search results for similar medical data based on the values ​​of items included in the medical data.

[0062] In step S108, the search function 103e displays the sorted data set as search results on the GUI of the display 105. At this time, in order to limit the number of SOAP documents included in the search results, only the top N similarity results (where N is a predetermined number) may be included as search results.

[0063] The search function 103e may display links to the patient's medical record information corresponding to each SOAP document in the search results instead, because displaying the entire SOAP document in the GUI may result in an excessive amount of information that is difficult to review.

[0064] Furthermore, instead of displaying the entire SOAP text of each search result in the GUI, the search function 103e may display a summary of the SOAP text using a large-scale language model. Alternatively, instead of displaying the entire SOAP text of each search result in the GUI, only a portion of the data items constituting the SOAP text may be displayed. For example, if the search information is "assessment," only the "subjective information text" and the "objective information text" may be displayed, or only the "evaluation text" may be displayed. Alternatively, instead of displaying the entire SOAP text of each search result in the GUI, only the first X characters (where X is a predetermined number) of the text corresponding to each data item constituting the SOAP text may be displayed.

[0065] As described above, the information processing system 100 according to this embodiment can calculate an appropriate similarity score that takes into account the user's search intent when searching for medical data similar to medical data composed of multiple data items. Furthermore, by performing a search using the search function 103e with this similarity score, it is possible to reduce the inclusion of medical data that is not highly related to the user's search intent.

[0066] For example, consider a situation where a user is trying to write an "evaluation-related statement" for a SOAP document and is looking to refer to other "evaluation-related statements." In this situation, the "evaluation-related statement" and "treatment plan-related statement" sections of the SOAP document are not yet filled in. In such a case, with conventional similar data searches, there was a possibility that data that also lacked "evaluation-related statements" and "treatment plan-related statements" would be included in the search results as similar data. However, in the information processing system 100 according to this embodiment, by setting search information, the user can perform a similar data search based, for example, only on the similarity between "subjective information-related statements" and "objective information-related statements." In other words, because the search information is based on user instructions, data that is not of interest to the user is less likely to be included in the search results compared to conventional methods.

[0067] The following describes a modified version of the processing of the information processing system 100 in the first embodiment described above. In the following description, the same reference numerals are used for components and processes that are the same as those of the information processing system 100 described above, and detailed explanations are omitted as appropriate.

[0068] (Modification 1 of the first embodiment) In step S108 of the first embodiment, the search function 103e displayed the search results on the GUI of the display 105, but in this modified example, the search results may be transmitted to an external system (not shown). In this case, the search results may be displayed on the GUI of the display 105, or they may not be displayed.

[0069] According to this modified example, the information processing system 100 can transmit the search results to an external system, which can then display the search results on a display device (not shown) or use the information for analysis.

[0070] (Modification 2 of the first embodiment) The feature acquisition process in step S105 of the first embodiment is generally computationally expensive, and if there is a large amount of data to be searched, the time it takes to obtain search results may be long. Therefore, for each SOAP document stored in the electronic medical record system, the feature quantities corresponding to each of the multiple search objectives may be calculated in advance before the similar data search is performed. Furthermore, the time it takes to obtain search results may be shortened by associating SOAP documents with search objectives and storing them in a database. Here, the database is a medical data management device (not shown) that is communicably connected to the information processing system 100, and can be an individual database system or a database provided by the electronic medical record system.

[0071] First, an example of the process flow for saving the feature quantities of SOAP documents from the electronic medical record system to a database in advance, as performed by the information processing system 100 in this modified example, will be explained using the flowchart in Figure 3. Figure 9 shows an example of a database table used in this modified example. In this modified example, since only one type of feature quantity is used when calculating the similarity, only one type of feature quantity is managed in the table. If there are multiple feature quantities when calculating the similarity, the feature quantities managed in the table may be changed accordingly. Also, the order of the steps and the order of procedures within each step described here may be changed as long as no inconsistencies arise.

[0072] The following processing flow is preferably executed during times when no similar data search is being performed by user operations. For example, it may be triggered by instructions from the administrator of the information processing system 100, a predetermined date and time (such as midnight on a weekday), or a predetermined operation by the user (such as the completion of editing a SOAP document).

[0073] In step S1101, the data acquisition function 103a acquires a group of SOAP documents that are in a state where it is meaningful to save SOAP documents and features based on the search objective to the database. For example, it acquires a SOAP document if any of the features based on the search objective do not exist in the database. Also, for example, it acquires a SOAP document if it has been updated since the time when existing features were saved for that SOAP document. Furthermore, if it is desired to update all features stored in the database due to reasons such as a change in the feature acquisition device, it acquires all SOAP document groups.

[0074] In step S1102, the search information acquisition function 103b selects a search objective that has not been selected at any point in this series of processing steps (steps S1101 to S1106). If there are no unselected search objectives, the process terminates.

[0075] In steps S1103 to S1104, the same processing as in steps S101 to S104 of the first embodiment is performed.

[0076] In step S1105, the same process as in step S105 of the first embodiment is performed.

[0077] In step S1106, the feature acquisition function 103c stores each of the acquired feature sets in the database, associating them with the corresponding SOAP text of the electronic medical record and the search objective selected in step S1102. The process returns to step S1102.

[0078] As described above, the information processing system 100 according to this modified example can store in the database, for each SOAP document stored in the electronic medical record, feature quantities corresponding to each search purpose associated with the SOAP document.

[0079] Next, we will explain an example where, as a result of executing the above series of processes (steps S1101 to S1106), one or more feature quantities associated with the SOAP text of the electronic medical record are stored in the database. Below, we will explain an example of the process of performing a similar data search using the flowchart in Figure 4. Note that the order of the steps and the order of procedures within each step described here may be changed as long as no inconsistencies arise.

[0080] The following processing flow describes an example of searching for similar data, starting from a state in which the user operates the input interface 104 to select and retrieve a SOAP document from the electronic medical record, similar to the first embodiment.

[0081] In step S1201, the same process as in step S101 of the first embodiment is performed.

[0082] In step S1202, the data acquisition function 103a acquires SOAP documents stored in the electronic medical record that have been added or updated since the last execution of step S1101.

[0083] In steps S1203 and S1204, the same processing as in steps S103 and S102 of the first embodiment is performed.

[0084] In step S1205, the same processing as in step S105 of the first embodiment is performed. Furthermore, the feature acquisition function 103c may attempt to reduce the computational cost of feature acquisition processing when a similar data search is performed next time by associating the acquired set of features with the corresponding SOAP document and inspection purpose and saving them in the database.

[0085] In step S1206, the feature acquisition function 103c acquires the remaining SOAP texts that were not acquired in step S1202 and the feature sets stored in the database associated with the search objective selected in step S1201.

[0086] In step S1207, the similarity acquisition function 103d calculates the similarity using the feature set obtained from the feature acquisition device in step S1205 and the feature set obtained from the database in step S1206. Specifically, the similarity acquisition function 103d calculates the similarity between the feature corresponding to the search condition data and the feature set corresponding to the searched data set.

[0087] In steps S1208 and S1209, the same processing as in steps S107 and S108 of the first embodiment is performed.

[0088] As described above, the information processing system 100 according to this modified example can shorten the time it takes to obtain search results by acquiring a set of previously calculated features stored in the database in association with SOAP document sets and search objectives.

[0089] (Modification 3 of the first embodiment) In Modification 2 of the First Embodiment, an example was described in which a set of features obtained from a feature acquisition device and a set of features obtained from a database are used in step S1207. In Modification 2, similarity sets are calculated between the features corresponding to the search condition data and the set of features corresponding to the data being searched.

[0090] In this modified example, if the database is a vector database, the set of features obtained from the feature acquisition device may be stored in the database, and the vector search function provided by the database may be used to identify the set of searched data that is similar to the search condition data.

[0091] Specifically, an example of the processing flow executed by the information processing system 100 according to this modified example will be explained using the flowchart in Figure 5, corresponding to the series of processing flows described in the first embodiment. Note that the order of the steps and the order of procedures within each step described here may be changed as long as no inconsistencies arise.

[0092] Steps S1301 to S1304 perform the same processing as steps S101 to S104 of the first embodiment.

[0093] In step S1305, the same processing as in step S105 of the first embodiment is performed. Furthermore, the feature acquisition function 103c associates the acquired feature set with the corresponding SOAP document and inspection purpose and stores it in the database.

[0094] In step S1306, the search function 103e first sends a query to the database using the features corresponding to the search condition data as search conditions. The database sends a group of SOAP sentences associated with features that match or are similar to the sent features back to the search function 103e as search results. At this time, the search results may be sorted according to similarity using the sorting function provided by the database.

[0095] In step S1307, the same process as in step S108 of the first embodiment is performed.

[0096] As described above, according to this modified example, the information processing system 100 can shorten the time it takes to obtain search results by acquiring a set of previously calculated features stored in the database in association with SOAP document sets and search objectives.

[0097] (Modification 4 of the first embodiment) A feature acquisition device may be equipped with multiple encoder groups, and the encoder used to calculate features may be selected and switched based on the type of input data or media format based on a single medical dataset. Each encoder has different performance characteristics and is designed to acquire relatively good features when inputting the type of data it excels at. Here, an encoder that can acquire good features is one that produces high similarity in the output feature set when inputting data sets with similar features, and low similarity in the output feature set when inputting data sets with dissimilar features. This performance difference can be adjusted by changing general training conditions, such as the configuration of the dataset used when training the encoder, the model architecture, the model capacity, and each hyperparameter.

[0098] Specifically, in the first embodiment, the characteristics of the generated single text data change significantly because the data items and template sentences selected based on the search information are different. Therefore, in the process corresponding to step S105 of the first embodiment according to this modified example, the feature acquisition function 103c transmits identification information (e.g., search information) for identifying the characteristics of the single text data, along with the generated single text data set, to the feature acquisition device. The feature acquisition device selects and switches the encoder to be used to calculate the features based on the received identification information.

[0099] As described above, according to this modified version, the information processing system 100 can obtain search results that are more in line with the search intent by selecting and switching encoders based on the type and media format of the input data from which the feature acquisition device is acquiring features.

[0100] (Modification 5 of the first embodiment) The feature acquisition device may be equipped with multiple encoder groups, and the encoder to be used to calculate features may be selected based on the type and media format of each of the multiple input data based on a single medical data set. Each encoder has different performance characteristics, and it is assumed that each encoder is capable of acquiring relatively good features when the type of data it excels at is input. Here, an encoder capable of acquiring good features is the same as described in Modification 4 of the first embodiment. In this case, the multiple feature sets acquired by inputting each of the multiple input data sets into the encoder may be used as they are, or they may be combined into a single feature. When combining, there are methods such as concatenating, adding, simply averaging, or weighting average by predetermined weights. In this modification, an example is described in which the multiple feature sets are treated as they are without being combined.

[0101] Specifically, an example of the processing flow executed by the information processing system 100 according to this modified example will be explained using the flowchart in Figure 6, corresponding to the series of processing flows described in the first embodiment. Note that the order of the steps and the order of procedures within each step described here may be changed as long as no inconsistencies arise.

[0102] Steps S1401 to S1403 perform the same processing as steps S101 to S103 of the first embodiment.

[0103] In step S1404, the feature acquisition function 103c transmits the set of data item combinations selected in the procedure of step S103 to a feature acquisition device (not shown). The feature acquisition device inputs each of the data items constituting each of the received set of data item combinations into one of the multiple encoder groups provided by the feature acquisition device to calculate a set of multiple feature combinations and transmits it to the feature acquisition function 103c. The feature acquisition function 103c receives and acquires the set of multiple feature combinations.

[0104] In step S1405, the similarity acquisition function 103d uses the set of feature combinations obtained in step S105. The similarity acquisition function 103d calculates a similarity set between the set of feature combinations corresponding to the search condition data and the set of feature combinations corresponding to the searched data. In this modified example, it is necessary to calculate the similarity between the feature combinations, but if the i-th feature in the feature combination is the feature Fi (i: 1 to the number of selected data items), the similarity Si is calculated between features corresponding to the same ordinal number i. Specifically, when the search information is "assessment", two data items are selected: "text related to subjective information" and "text related to objective information". In such cases, a combination of feature consisting of feature F1 based on "text related to subjective information" and F2 based on "text related to objective information" is calculated and obtained for each SOAP text. The similarity S1 is calculated between the feature F1_Q corresponding to the search condition data and the feature F1_V corresponding to one of the searched data. Furthermore, a similarity score S2 is calculated between the feature F2_Q corresponding to the search condition data and the feature F2_V corresponding to one of the searched data items. Values ​​can be obtained using the similarity combinations S1 and S2 of the calculated combination of features corresponding to the search condition data and the combination of features corresponding to one of the searched data items. Alternatively, the similarity combinations S1 and S2 can be obtained as scalar values ​​that can be compared in magnitude with other similarity scores by adding them together, taking a simple average, or taking a weighted average.

[0105] Steps S1406 and S1407 perform the same processing as steps S107 to S108 of the first embodiment.

[0106] As described above, according to this modified example, the information processing system 100 can obtain search results that better match the search intent by using multiple encoders based on the type and media format of the input data from which the feature acquisition device acquires features.

[0107] (Modification 6 of the first embodiment) In step S101 of the first embodiment, the search information acquisition function 103b acquires search information selected by the user, but it may also acquire search information automatically based on medical data without user selection. That is, the search information is determined based on the results of the analysis of medical data.

[0108] Let's explain this using a specific example where the user is editing a SOAP document. For example, consider a case where the input of the "subjective information" and "objective information" sections of the SOAP document is complete, or where the user is editing the "evaluation section." In this case, the user is likely to try to complete the "evaluation section" as their next task, so the search information acquisition function 103b automatically acquires "assessment" as the search objective. Also, consider a case where the input of the "subjective information," "objective information," and "evaluation section" sections of the SOAP document is complete, or where the user is editing the "treatment plan section." In this case, the user is likely to try to complete the "treatment plan section" as their next task, so the search information acquisition function 103b automatically acquires "planning" as the search objective. Furthermore, consider a case where the input of the entire SOAP document is complete. In this case, it is difficult to predict what kind of SOAP document the user will want to refer to, but there is a good possibility that they will want to refer to a SOAP document with similar overall text (all data items). Therefore, the search information acquisition function 103b automatically acquires "None in particular" as the search purpose, which is the search information. After the search information acquisition function 103b automatically acquires the search information, the search information acquisition function 103 may display the strings "Assessment" or "Planning" in the GUI so that the user can refer to the automatically set search information.

[0109] Furthermore, after the search information acquisition function 103b automatically acquires the search information, the information processing system 100 may execute the processes from step S102 onward in the first embodiment and perform a similar data search, triggered by user actions or other means. Here, user actions include pressing a button control in the GUI that is intended to start a similar data search.

[0110] Furthermore, if the automatically acquired search information does not match the user's search intent, the user may perform the operation described in step S101 of the first embodiment, and thereafter the procedure of this modified example may proceed to the series of processing steps of the first embodiment.

[0111] As another example, if the medical data is medical record information, the search information acquisition function 103b may automatically acquire search information based on the diagnosis name recorded in the medical record information. In this case, for example, the search information is a string based on a diagnosis name such as "breast cancer" or "lung cancer." Then, in step S102 of the first embodiment, before acquiring all medical record information stored in the electronic medical record, it may be possible to acquire only the medical record entries in which the diagnosis name is recorded. Then, in step S103 of the first embodiment, the feature acquisition function 103c selects data items that are considered useful for diagnosing the disease corresponding to the search information. For example, if the search information is "breast cancer," data regarding the presence or absence of each disease history in the family in category format, or mammography images in image format, may be selected as data items. Then, in step S104 of the first embodiment, each data item is processed by the feature acquisition function 103c as necessary so that it can be input into the various encoders provided by the feature acquisition device. For example, data regarding the presence or absence of each family member's disease history is converted into text data such as "Mother has a history of breast cancer" through rule-based processing or the like, so that it can be input into a text encoder. Similarly, mammography images are converted into pixel value group data so that they can be input into an image encoder. Subsequent steps can be processed by the procedure shown from step S1405 onwards in Modification 5 of the first embodiment, which performs a similar data search considering the similarity of multiple data item groups.

[0112] As another example, if the medical data is medical record information, the search information acquisition function 103b acquires CT images based on link information to the examination data recorded in the medical record information. The search information acquisition function 103 may also automatically acquire search information based on the results of image analysis of the CT images. In this case, for example, the search information is a string such as "lung cancer" or "liver cancer". Then, in step S103 of the first embodiment, the feature acquisition function 103c selects data items that are considered useful for diagnosing the disease corresponding to the search information. For example, if the search information is "lung cancer", data regarding the presence or absence of smoking history in category format and CT images in image format are selected as data items. Then, in step S104 of the first embodiment, each data item is processed by the feature acquisition function 103c as necessary so that it can be input into various encoders provided by the feature acquisition device. For example, data regarding smoking history is converted into text format data such as "smoking history, Brinkman index is 40 × 20 = 800" by rule-based processing so that it can be input into a text encoder. Furthermore, for example, CT images can be input directly into the image encoder and therefore do not require processing. Subsequent steps can be processed by the procedure shown from step S1405 onwards in Modification 5 of the First Embodiment, which involves performing a similar data search considering the similarity of multiple data item groups.

[0113] As described above, according to this modified version, the information processing system 100 can automatically acquire search information and perform similar data searches by analyzing the state of medical data.

[0114] (Modification 7 of the first embodiment) In step S104 of the first embodiment, the feature acquisition function 103c generates text data for input to the encoder by the feature acquisition device through a procedure for processing data items using a template document. In this procedure for processing data items, the information processing system 100 may further add a predetermined string (text). Note that this procedure for adding the string may be performed regardless of whether or not the feature acquisition function 103c uses a template document.

[0115] For example, if the search term is "assessment," you may add a string such as "Considerations based on the above information." Similarly, if the search term is "planning," you may add a string such as "What should be done in the future given the above situation."

[0116] Furthermore, the string to be added may be determined by information other than search information. For example, it may be the result of processing data items included in medical data through a predetermined analysis process. Specifically, if the analysis results of blood tests or imaging tests included in medical data show that a predetermined disease X is positive, a string such as "It is important to note that disease X is positive" may be added.

[0117] Furthermore, the string to be added may be a predetermined string recorded in the information processing system 100.

[0118] As described above, according to this modified version, the information processing system 100 can guide the trend of the features calculated by the encoder by further processing the text-format data that the feature acquisition device inputs to the encoder. In other words, features of medical data are emphasized in the features, and features that become noise and worsen the search accuracy of similar data searches are reduced. As a result, medical data that is not highly relevant to the user's search intent is less likely to be included.

[0119] <Second Embodiment> The following describes, using Figures 10 and 11, the search process for medical data similar to medical data, which is performed by the information processing system 1000 in this embodiment and consists of multiple types of data. Here, multiple types of data refer to data that includes data items in different media formats. In this embodiment, for ease of explanation, an example will be given in which the medical data is patient examination data in the medical field. Specifically, the medical data will be medical record information from an electronic medical record. The medical data management device will be an electronic medical record system, which is a database for managing medical record information.

[0120] The functional configuration of the information processing system 1000 in this embodiment will be explained using Figure 10. The information processing system 1000 includes a communication interface 1010, a memory circuit 1020, a processing circuit 1030, an input interface 1040, and a display 1050. The information processing system 1000 can also be connected to the network 2000 via the communication interface 1010. The communication interface 1010, memory circuit 1020, input interface 1040, and display 1050 are similar in function to those of the first embodiment, so their explanation will be omitted.

[0121] The processing circuit 1030 in this embodiment includes a data acquisition function 1030a that implements a data acquisition means for acquiring medical data, and a feature acquisition function 1030c that implements a feature acquisition means for acquiring feature quantities. The processing circuit 1030 also includes a similarity acquisition function 1030d that implements a similarity acquisition means for acquiring the similarity between medical data. Furthermore, the processing circuit 1003 includes a search function 1030e that implements a search means for acquiring search results for similar medical data.

[0122] The information processing system 100, having the above-described functional configuration, can appropriately calculate similarity for medical data containing different types of data, and by using this similarity, it can acquire medical data similar to the first medical data with high accuracy.

[0123] Here, for example, each processing function realized by the components of the processing circuit 1030 shown in Figure 10 is stored in the memory circuit 1020 in the form of a program that can be executed by a computer. The components are the data acquisition function 1030a, the feature acquisition function 1030c, the similarity acquisition function 1030d, and the search function 1030e.

[0124] The processing circuit 1030 reads each program from the memory circuit 1020 and executes each read program to realize the function corresponding to each program. In other words, the processing circuit 1030 that reads each program has functions realized by the data acquisition function 1030a, the feature acquisition function 1030c, the similarity acquisition function 1030d, and the search function 1030e.

[0125] Specifically, the information processing system 1000 according to this embodiment has a data acquisition function 1030a that acquires first medical data comprising different types of data relating to a subject. The information processing system 1000 also has a feature acquisition function 1030c that expresses the characteristics of the medical data. The feature acquisition function 1030c is characterized by acquiring at least a portion of the feature by inputting the medical data comprising different types of data contained in the first medical data into an encoder. Here, the encoder is at least one of a large-scale language model encoder and a large-scale multimodal model encoder. In addition, the information processing system 1000 is configured to include a similarity acquisition function 1030d that acquires the similarity between the feature and the feature of other medical data. The information processing system 1000 further includes a search function 1030e that acquires search results for medical data similar to the first medical data based on the similarity. With this configuration, an appropriate similarity can be calculated even for medical data comprising different types of data, and highly accurate search results can be obtained by using this similarity. The following describes an example of the processing flow executed by the information processing system 1000 according to this embodiment. Note that the order of the steps and the order of procedures within each step described here may be changed as long as no inconsistencies arise.

[0126] For the sake of clarity, the following explanation assumes that a user is operating the input interface 104 to select and confirm a patient's medical record information from the electronic medical record system and to formulate a treatment plan. Furthermore, the user is attempting to search the electronic medical record system for similar data, seeking other medical record information that is similar to the selected medical record. In the following explanation, the selected medical record information will be referred to as search condition data, and the other medical record information as searched data.

[0127] In step S201, the data acquisition function 1030a acquires information to start a similar data search for the first medical data, as instructed by the user by operating the GUI displayed on the display 105. This operation may include, for example, pressing a button control in the GUI that is intended to start a similar data search.

[0128] In step S202, the data acquisition function 1030a acquires all medical record information stored in the electronic medical record. This medical record information includes data items in various media formats, such as numerical, text, and image formats.

[0129] In step S203, the feature acquisition function 1030c transmits a set of combinations of data items included in the medical record information acquired in step S202 to a feature acquisition device (not shown). The feature acquisition device inputs each data item included in the received set of data item combinations into various encoders provided by the feature acquisition device, and transmits the calculated set of feature combinations to the feature acquisition function 1030c. The feature acquisition function 1030c receives and acquires this set of features.

[0130] The various encoders refer to numerical encoders that can accept numerical data, text encoders that can accept text data, and image encoders that can accept image data. In other words, they are encoders that calculate feature quantities by inputting data items in various media formats that contain medical record information.

[0131] In step S204, the similarity acquisition function 1030d uses the set of feature combinations acquired in step S203. The similarity acquisition function 1030d calculates a set of similarities between the set of feature combinations corresponding to the search condition data and the set of feature combinations corresponding to the searched data. Here, each of the similarities can be calculated by the method described in step S1406 of Modification 5 of the first embodiment.

[0132] In step S205, the search function 1030e sorts the search data sets corresponding to the similarity scores calculated in step S204. This process improves user access to medical record information similar to the search criteria data, even when there is a large amount of search data.

[0133] In step S206, the search function 1030e displays the sorted data set as search results on the GUI of the display 105.

[0134] As described above, the information processing system 1000 according to this embodiment can search for medical data similar to the first medical data which is composed of multiple types of data. [Explanation of Symbols]

[0135] 100 Information Processing Systems 101 Communication Interface 102 Memory circuit 103 Processing Circuit 103a Data acquisition function 103b Function to retrieve search details 103c Feature Acquisition Function 103d similarity acquisition function 103e Search function 104 Input Interfaces 105 displays 200 Networks

Claims

1. A data acquisition means for acquiring first medical data consisting of multiple items related to a subject, A means for obtaining search information, which is information for searching for medical data similar to the first medical data, A feature acquisition means for acquiring feature quantities that represent the characteristics of medical data based on the search information, the feature acquisition means for acquiring at least a portion of the feature quantities by inputting the medical data included in the first medical data into at least one of a large-scale language model encoder and a large-scale multimodal model encoder, A means for obtaining similarity between the aforementioned feature quantity and the feature quantity of other medical data, An information processing system characterized by having the following features.

2. A data acquisition means for acquiring first medical data consisting of different types of data relating to a subject, A feature acquisition means for acquiring feature quantities that represent the characteristics of medical data, wherein the feature acquisition means acquires at least a portion of the feature quantities by inputting medical data, which includes different types of data contained in the first medical data, into at least one of a large-scale language model encoder and a large-scale multimodal model encoder, A means for obtaining similarity between the aforementioned feature quantity and the feature quantity of other medical data, An information processing system characterized by having the following features.

3. The information processing system according to claim 1 or 2, further comprising a search means for obtaining search results for medical data similar to the first medical data based on the aforementioned similarity.

4. The system further includes means for obtaining search information, which is information for searching for medical data similar to the first medical data mentioned above. The information processing system according to claim 2, characterized in that the means for acquiring the feature quantity acquires the feature quantity based on the search information.

5. The information processing system according to claim 1 or 4, characterized in that the search information includes the user's search intent and is information for changing the feature quantities obtained by the similarity acquisition means via at least one of the encoder of the large-scale language model and the encoder of the large-scale multimodal model.

6. The information processing system according to claim 1 or 4, characterized in that the search information includes the user's search intent and is information for obtaining the first feature quantity obtained by the similarity acquisition means via at least one of the encoder of the large-scale language model and the encoder of the large-scale multimodal model.

7. The information processing system according to claim 1 or 4, wherein the feature acquisition means is characterized by inputting a prompt including the user's search intent to the encoder in order to acquire features that are in line with the user's search intent.

8. The information processing system according to claim 1 or 4, characterized in that the search information is set based on user instructions.

9. The information processing system according to claim 1 or 4, characterized in that the search information is determined based on the results of the analysis of the medical data.

10. The information processing system according to claim 1 or 2, characterized in that the encoder is trained such that the similarity of the feature set output when a medical data set in which the features of a predetermined data item set are similar is higher than the similarity of the feature set output when a medical data set in which the features of the predetermined data item set are not similar is input.

11. The information processing system according to claim 3, characterized in that the search means displays the search results on the display means.

12. The information processing system according to claim 3, characterized in that the search means obtains search results for medical data similar to the first medical data based on the similarity and the values ​​of items included in one or more medical data.

13. The information processing system according to claim 1, wherein at least one of the large-scale language model and the large-scale multimodal model comprises a plurality of encoders, and the feature acquisition means selects the encoders based on the search information.

14. The information processing system according to claim 1, wherein at least one of the large-scale language model and the large-scale multimodal model comprises a plurality of encoders, and the feature acquisition means selects a plurality of encoders based on the search information.

15. The information processing system according to claim 14, characterized in that the feature acquisition means acquires the feature by integrating the feature acquired using a plurality of selected encoders.

16. A data acquisition step to obtain first medical data consisting of multiple items related to the subject, A search information acquisition step, which involves acquiring search information that is information for searching for medical data similar to the first medical data, A feature acquisition step for acquiring feature quantities that represent the characteristics of medical data based on the search information, comprising: a feature acquisition step for acquiring at least a portion of the feature quantities by inputting the medical data included in the first medical data into at least one of a large-scale language model encoder and a large-scale multimodal model encoder; A similarity acquisition step to obtain the similarity between the aforementioned feature quantity and the feature quantity of other medical data, An information processing method characterized by having the following features.

17. A data acquisition step to obtain first medical data consisting of different types of data concerning the subject, A feature acquisition step for obtaining feature quantities that represent the characteristics of medical data, comprising: inputting medical data, which includes different types of data contained in the first medical data, into at least one of a large-scale language model encoder and a large-scale multimodal model encoder, thereby obtaining at least a portion of the feature quantities; A similarity acquisition step to obtain the similarity between the aforementioned feature quantity and the feature quantity of other medical data, An information processing method characterized by having the following features.

18. A program for a computer to execute the information processing method described in claim 16 or 17.

Images