Data collection device, data collection method, and computer program product

Abstract

According to the data collection device, the data collection method and the computer program product, medical data effective for training or testing of an artificial intelligence model can be efficiently collected. A data collection device for collecting medical data for training or testing an artificial intelligence model, the data collection device being provided with: a first collection unit for collecting the medical data as a first data set on the basis of a result of evaluation by a user with respect to a result of reasoning of the medical data based on the artificial intelligence model; a second collection unit that collects the medical data that does not belong to the first data set as a second data set on the basis of the degree of matching between the medical data and the artificial intelligence model; a third collection unit that generates medical data of a third data set by extracting image features of target regions of the medical data of the first data set and the second data set; and a target collection unit for collecting, as a target data set, the medical data of the second data set corresponding to the medical data of the third data set on the basis of the reasoning result of the medical data of the third data set.

Description

Technical Field

[0001] This invention relates to data collection devices, data collection methods, and computer program products. Background Technology

[0002] In the medical field, artificial intelligence (AI) models are widely used in tasks such as biometric recognition, medical diagnosis, and medical image processing. An AI model comprises a training process and an inference process. The training process is the first stage of the AI ​​model, referring to the process of learning from existing data to acquire certain patterns or abilities. The inference process is the application stage of the AI ​​model, referring to the process of applying its abilities to new data to obtain the expected output results.

[0003] Training an artificial intelligence model is a computationally intensive operation, typically requiring a large amount of data. During training, backpropagation is used to optimize the model's parameters, gradually reducing the error between the model's predictions and actual values. Thus, the trained AI model can accurately predict input data.

[0004] Robustness of an AI model can be understood as its ability to maintain stability and performance when faced with changes in input data; that is, the degree to which the AI ​​model tolerates changes in input data. Generalization ability refers to the predictive ability of an AI model when faced with new data, or in other words, whether the prediction results of an AI model meet expectations when faced with new data of the same type and independently distributed.

[0005] Robustness and generalization ability are commonly used to evaluate the performance of artificial intelligence (AI) models. A highly robust AI model can still achieve good inference results even with significant variations in input data. A highly generalizable AI model can learn well on limited training data and accurately predict new data. Increasing the quantity and diversity of training data (features) during the training process can effectively improve the robustness and generalization ability of an AI model. Furthermore, since low-quality or biased training data negatively impacts AI model performance, improving the quality of training data also plays a crucial role in enhancing the model's capabilities. Therefore, a large amount of diverse, high-quality training data is essential for training AI models.

[0006] In the medical field, doctors and technicians review and annotate patients' medical image data, recording diagnoses in medical reports. Therefore, medical data management systems typically contain a large amount of medical data that meets the demand for high-quality, diverse, and comprehensive data. Using such medical data to train artificial intelligence models in the medical field can effectively improve the robustness and generalization ability of predictive AI models for medical data.

[0007] However, existing technologies present the following problems. First, medical data is characterized by its large volume and dispersion, making manual collection of this scattered data extremely time-consuming and labor-intensive. Second, medical data typically encompasses various types. When collecting medical data for use as training data in an artificial intelligence model, if the collected data possesses characteristics readily available to or already obtained by the AI ​​model and is sufficiently abundant, blindly adding such data as training data would increase the computational load and resources required for the AI ​​model's training process, resulting in waste.

[0008] Therefore, a data collection technology is needed to determine whether medical data has characteristics that an artificial intelligence model has not learned or has not learned sufficiently from a large amount of diverse medical data, and to collect medical data with such characteristics as training data for the training process of the artificial intelligence model. Summary of the Invention

[0009] The present invention was made in view of the above problems, and aims to provide a data collection device, a data collection method, and a computer program product that can efficiently collect medical data that is effective for training or testing artificial intelligence models.

[0010] Furthermore, the data collection device, data collection method, and computer program product according to the present invention can efficiently collect medical data that is effective for training or testing artificial intelligence models when the medical data contains an overall structure.

[0011] According to the present invention, a data collection apparatus is used to collect medical data for training or testing an artificial intelligence model. The apparatus comprises: a first collection unit that collects the medical data into a first dataset based on a user's evaluation of the inference results of the medical data based on the artificial intelligence model; a second collection unit that collects medical data not belonging to the first dataset into a second dataset based on the matching degree between the medical data and the artificial intelligence model; a third collection unit that extracts image features of target regions from the medical data in the first dataset and the second dataset respectively to generate medical data for a third dataset; and a target collection unit that collects medical data from the second dataset corresponding to the medical data in the third dataset into a target dataset based on the inference results of the medical data in the third dataset.

[0012] Therefore, through the data collection device of the present invention, medical data is differentiated into those whose inference results from the artificial intelligence model meet or do not meet the requirements, and those required for training or testing the artificial intelligence model and those not required. Only medical data whose inference results from the artificial intelligence model do not meet the requirements and are required for training or testing the artificial intelligence model, and which are either unlearned or insufficiently learned, is collected as the target dataset for training or testing the artificial intelligence model. Thus, effective medical data for training or testing the artificial intelligence model can be collected efficiently.

[0013] Specifically, the medical data in the target dataset possesses the image features required for training or testing an artificial intelligence (AI) model, and the AI ​​model's inference results based on the target dataset's medical data do not meet the requirements. In other words, the medical data in the target dataset is medical data that the AI ​​model has not learned or has not learned sufficiently. Therefore, it is possible to efficiently collect effective medical data for training or testing AI models.

[0014] According to the data collection apparatus of the present invention, the third collection unit calculates the difference between the image features of the medical data in the first dataset and the image features of the medical data in the second dataset, obtains the image features of the medical data in the second dataset with a large difference, and uses the image features of the medical data in the second dataset with a large difference to process the medical data in the first dataset to generate the medical data in the third dataset.

[0015] Therefore, the medical data in the third dataset contains image features that the inference results of the artificial intelligence model do not meet the requirements for, and at the same time, it has the corresponding medical data from the first dataset as ground truth, which can be used for training or testing the artificial intelligence model. Thus, it is possible to further efficiently collect effective medical data for training or testing the artificial intelligence model.

[0016] According to the data collection apparatus of the present invention, the third collection unit processes the medical data of the first dataset by using the histogram distribution characteristics of the medical data of the second dataset as the image features through histogram specification.

[0017] Therefore, the medical data in the third dataset can be appropriately endowed with image features from the medical data in the second dataset—image features that the inference results of the artificial intelligence model do not meet and require. This enables the more efficient collection of effective medical data for training or testing the artificial intelligence model.

[0018] According to the data collection device of the present invention, the image feature is at least one of the histogram distribution feature, morphological feature, and texture feature of the target region of the medical data.

[0019] Therefore, the medical data in the third dataset can more appropriately incorporate the image features of the medical data in the second dataset—that is, image features that the inference results of the artificial intelligence model do not meet the requirements for and are needed. This allows for the more efficient collection of effective medical data for training or testing the artificial intelligence model.

[0020] According to the data collection apparatus of the present invention, the data collection apparatus further comprises an extraction unit, which extracts lesion keywords from medical reports corresponding to medical data that do not belong to the first dataset, compares the training data features of the artificial intelligence model with the lesion keywords, and determines whether the lesion keywords contain the training data features that are missing from the artificial intelligence model. When the lesion keywords contain the training data features that are missing from the artificial intelligence model, the second collection unit collects the medical data corresponding to the medical report in which the lesion keywords are extracted as a second dataset.

[0021] Therefore, the second collection unit gathers medical data that does not belong to the first dataset and contains training data features lacking in the artificial intelligence model, forming a second dataset. The target dataset collected from this second dataset is what the artificial intelligence model needs for training. Thus, it becomes possible to more efficiently collect medical data effective for training or testing the artificial intelligence model.

[0022] According to the data collection apparatus of the present invention, the data collection apparatus further comprises a detection unit, the extraction unit extracts key images from the medical report and extracts sequences from the medical data, the detection unit detects feature points in the key images and the sequences respectively, and the second collection unit collects the sequence into a second dataset based on the consistency between the feature points of the key images and the feature points of the sequences.

[0023] According to the data collection apparatus of the present invention, the data collection apparatus further comprises a detection unit, an extraction unit extracting key images from the medical report and extracting sequences from the medical data, the detection unit detecting region of interest keywords in the key images and detecting feature points in the sequences, and a second collection unit collecting the sequences into a second dataset based on the consistency between the region of interest keywords and the feature points of the sequences.

[0024] Therefore, the second collection unit only collects sequences whose feature points match the feature points of the key images in the medical report, or whose feature points match the region of interest of the key images in the medical report, as the second dataset. This reduces the amount of invalid data in the target dataset and improves the collection efficiency of the target dataset. Consequently, it enables the more efficient collection of effective medical data for training or testing artificial intelligence models.

[0025] According to the data collection apparatus of the present invention, the target collection unit calculates the similarity between the inference result of the medical data in the third dataset and the inference result of the corresponding medical data in the first dataset, scores the inference result of the medical data in the third dataset based on the similarity, and collects the medical data of the second dataset corresponding to the medical data in the third dataset as the target dataset based on the score.

[0026] Therefore, the medical data from the first dataset is used as the ground truth. The inference results from the third dataset are evaluated to see if they closely approximate the ground truth. Based on this result, the medical data from the second dataset corresponding to the third dataset is collected as the target dataset. Thus, the medical data in the target dataset used for training or testing the AI ​​model consists of medical data that the AI ​​model has not learned or has learned insufficiently, resulting in poor inference accuracy. This allows for the efficient collection of effective medical data for training or testing the AI ​​model.

[0027] According to the data collection device of the present invention, the inference result modified by the user and the corresponding medical data are collected as the target dataset.

[0028] Therefore, medical data that users modify because they deem the inference results unsatisfactory is collected as the target dataset, and the modified inference results from users become the truth values ​​of that medical data. This target dataset contains medical data that the AI ​​model's inference results do not meet the requirements for, and which is needed and can be used for training or testing. This allows for the efficient collection of effective medical data for training or testing AI models.

[0029] According to the data collection apparatus of the present invention, the first dataset is a benchmark dataset containing the medical data as a reference, the second dataset is a dataset to be collected containing the medical data to be collected, and the third dataset is a scoring dataset containing the medical data as a scoring object.

[0030] According to the data collection apparatus of the present invention, the data collection apparatus further comprises a classification unit, which classifies the medical data of the target dataset by means of clustering.

[0031] Therefore, medical data in the target dataset can be classified. When training or testing artificial intelligence models, appropriate medical data can be selected and used according to the classification as needed, thereby improving the efficiency of AI model training or testing. This, in turn, enables the efficient collection of effective medical data for training or testing AI models.

[0032] According to the data collection apparatus of the present invention, the data collection apparatus further comprises: a receiving unit for receiving the medical data; an inference unit for inferring the medical data using the artificial intelligence model; and a training and testing unit for training or testing the artificial intelligence model based on the medical data of the target dataset.

[0033] According to the data collection apparatus of the present invention, the data collection apparatus further comprises a division unit, which divides the medical data of the first dataset and the medical data of the second dataset into multiple blocks respectively; a third collection unit extracts local image features of each block of the medical data of the first dataset and each block of the medical data of the second dataset respectively, calculates the difference between the local image features of each block of the medical data of the first dataset and the local image features of each block of the medical data of the second dataset, obtains the local image features of the medical data of the second dataset with large differences, and uses the local image features of the medical data of the second dataset with large differences to process the medical data of the first dataset to generate the medical data of the third dataset; a target collection unit calculates the similarity between the overall inference result and / or block inference result of the medical data of the third dataset based on the artificial intelligence model and the inference result of the corresponding medical data of the first dataset, scores the inference result of the medical data of the third dataset according to the similarity, and collects the medical data of the second dataset corresponding to the medical data of the third dataset as the target dataset according to the score.

[0034] Therefore, the medical data in the target dataset contains overall and / or local structures that artificial intelligence models cannot reason about well. Such medical data is the medical data needed for training or testing artificial intelligence models. Thus, even when medical data packages contain varying overall and / or local structures, it is possible to efficiently collect medical data effective for training or testing artificial intelligence models.

[0035] According to the data collection method of the present invention, medical data is collected for training or testing an artificial intelligence model. The method comprises: a first dataset collection step, in which the medical data is collected as a first dataset based on a user's evaluation of the inference results of the medical data based on the artificial intelligence model; a second dataset collection step, in which medical data not belonging to the first dataset is collected as a second dataset based on the matching degree between the medical data and the artificial intelligence model; a third dataset collection step, in which image features of target regions of the medical data in the first dataset and the second dataset are extracted to generate medical data for a third dataset; and a target dataset collection step, in which the medical data of the second dataset corresponding to the medical data in the third dataset is collected as a target dataset based on the inference results of the medical data in the third dataset.

[0036] The computer program product according to the present invention includes a computer program, which, when executed by a processor, is used to collect medical data for training or testing an artificial intelligence model. The computer program, when executed, includes the following steps: a first dataset collection step, collecting the medical data as a first dataset based on a user's evaluation of the inference results of the medical data based on the artificial intelligence model; a second dataset collection step, collecting medical data from medical data not belonging to the first dataset, based on the matching degree between the medical data and the artificial intelligence model, as a second dataset; a third dataset collection step, extracting image features of target regions from the medical data in the first dataset and the second dataset respectively to generate medical data for a third dataset; and a target dataset collection step, collecting the medical data of the second dataset corresponding to the medical data in the third dataset as a target dataset based on the inference results of the medical data in the third dataset.

[0037] Invention Effects

[0038] The data collection apparatus, data collection method, and computer program product according to the present invention can efficiently collect medical data that is effective for training or testing artificial intelligence models.

[0039] Furthermore, the data collection device, data collection method, and computer program product according to the present invention can efficiently collect medical data that is effective for training or testing artificial intelligence models when the medical data contains an overall structure. Attached Figure Description

[0040] Figure 1 This is a block diagram illustrating a configuration example of the data collection system according to the first embodiment.

[0041] Figure 2 This is a block diagram illustrating a configuration example of the data collection apparatus according to the first embodiment.

[0042] Figure 3 This is a schematic diagram illustrating an example of disease keywords in a medical report of the present invention.

[0043] Figure 4 This is a schematic diagram illustrating an example of the training data features of the artificial intelligence model of the present invention.

[0044] Figure 5 This is a schematic diagram illustrating an example of a key image in a medical report of the present invention.

[0045] Figure 6 This is a schematic diagram illustrating medical data generated by the third collection unit of the first embodiment in the generation of a third dataset.

[0046] Figure 7 This is a schematic diagram showing the classification performed by the classification section of the first embodiment.

[0047] Figure 8 This is a flowchart illustrating the data collection method of the first embodiment.

[0048] Figure 9 This is a flowchart illustrating the process of determining the matching degree in the data collection method of the first embodiment.

[0049] Figure 10 This is a block diagram illustrating an example of the configuration of the data collection device according to the second embodiment.

[0050] Figure 11 This is a schematic diagram showing the division of blocks in the second embodiment. Detailed Implementation

[0051] Hereinafter, embodiments of the data collection apparatus, data collection method, and computer program product of the present invention will be described with reference to the accompanying drawings.

[0052] (First Implementation)

[0053] In the description of the first embodiment, an example is given of an artificial intelligence model performing a segmentation task on a medical image containing an aortic valve. The medical data collected by the data collection device 1 of the first embodiment is used to train or test the artificial intelligence model, enabling the artificial intelligence model to perform the aortic valve segmentation task.

[0054] In the training process for aortic valve segmentation tasks, training data typically includes data on normal aortic valves. However, with changes in human function, the aortic valve often develops lesions such as calcification and stenosis. Artificial intelligence models using only data containing normal aortic valves as training data perform poorly when segmenting data containing diseased aortic valves. To improve the robustness and generalization ability of artificial intelligence models, it is necessary to efficiently collect medical data containing abnormal structures that are effective for training or testing the AI ​​model.

[0055] However, this invention is not limited to the collection of training data for an artificial intelligence model that performs segmentation tasks on the aortic valve. It can also be applied to the collection of other types of medical data, including other parts, organs, or tissues, and can also be applied to artificial intelligence models that perform other image processing tasks.

[0056] Reference Figure 1 An example of the configuration of the data collection system 100 will be described. Figure 1 This is a block diagram illustrating a configuration example of the data collection system 100 according to the first embodiment.

[0057] The data collection system 100 includes a data collection device 1, an operating terminal 2, a medical data storage device 3, and a database DB. The devices are connected to each other via a shared signal transmission line, such as a bus or network, enabling them to communicate with each other.

[0058] Data collection device 1 is a device for collecting medical data for training or testing artificial intelligence models. Data collection device 1 functions, for example, as a server of data collection system 100.

[0059] Operating terminal 2 is a terminal operated by a user (e.g., a doctor or technician). Operating terminal 2 functions as a client of the data collection system 100. Operating terminal 2 can be a computer, laptop, smartphone, tablet computer, wearable device, mobile terminal, or handheld terminal, etc.

[0060] In addition, the operating terminal 2 has an input interface for receiving various operations from the user. The input interface converts the various operations received by the operator into electrical signals and transmits the electrical signals to the processing circuit of the operating terminal 2. The input interface of the operating terminal 2 can be a mouse, keyboard, button, panel switch, slider switch, trackball, operation panel, or touch panel.

[0061] The operating terminal 2 may also have a display for showing various data or information. The display may be a liquid crystal display, a plasma display, an organic EL display, or an LED display. The display may also be a touch panel display that also functions as an input interface.

[0062] Specifically, the operating terminal 2 accepts the user's operation to evaluate the inference results of the artificial intelligence model and sends the evaluation results to the data collection device 1. Additionally, the operating terminal 2 can also accept the user's operation to modify the inference results of the artificial intelligence model, sending the modified medical data to the target dataset (described later) in the database DB, and sending a message indicating that the user has made the modification to the data collection device 1.

[0063] Medical data storage device 3 is a device for storing medical data. For example, it stores raw medical data, medical reports, etc. Medical data storage device 3 can be a storage medium (e.g., magnetic storage medium, electromagnetic storage medium, optical storage medium, semiconductor memory) or a drive device for reading and writing information between itself and the storage medium. Medical data storage device 3, for example, stores raw medical data sent from an image data generation device (not shown). The image data generation device can be a medical image diagnostic device (e.g., X-ray diagnostic device, X-ray CT device, MRI device, ultrasound diagnostic device, nuclear medicine examination device, etc.).

[0064] The database DB is used to store the distribution characteristics of the training data of the artificial intelligence model (such as the distribution of features like aortic calcification and aortic stenosis) and the data in each dataset described later. The database DB can be a database such as MySQL or MongoDB. In this embodiment, the database DB and the data collection device 1 are described as separate entities, but the database DB can also be incorporated into the data collection device 1.

[0065] Reference Figure 2 An example of the configuration of data collection device 1 will be described. Figure 2 This is a block diagram illustrating a configuration example of the data collection device 1 according to the first embodiment.

[0066] The data collection device 1 includes a processing circuit 11, a memory 12, and a communication IF 13. The components are connected to each other in a communicable manner via a bus that serves as a shared signal transmission line.

[0067] The memory 12 is a device for storing various data or information. The memory 12 can be a storage medium that can be read by a processor (e.g., magnetic storage medium, electromagnetic storage medium, optical storage medium, semiconductor memory), or a drive device for reading and writing data or information between itself and the storage medium. For example, the memory 12 stores computer programs that enable the processing circuit 11 to perform its various functions.

[0068] Communication IF13 is an interface for communicating various data or information with the various components included in the data collection system 100. Communication IF13 communicates various data or information with the operating terminal 2 or the medical data storage device 3.

[0069] The processing circuit 11 is a circuit that controls the overall operation of the data collection device 1. The processing circuit 11 includes at least one processor. A processor can be a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), or a programmable logic device (such as a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), or a Field Programmable Gate Array (FPGA)). When the processor is a CPU, the CPU implements each function by reading and executing the computer programs stored in the memory 12. When the processor is an ASIC, each function is directly embedded as a logic circuit within the ASIC's circuitry. The processor can be configured as a single circuit or by combining multiple independent circuits. The processing circuit 11 implements the functions of the functional units described later.

[0070] In this embodiment, the processing circuit 11 includes a receiving unit 110, an inference unit 111, a first collection unit 112, a second collection unit 113, a third collection unit 114, a target collection unit 115, a classification unit 116, and a training and testing unit 117.

[0071] The receiving unit 110 receives, for example, medical data stored in the medical data storage device 3 via communication IF 13. The medical data includes, for example, clinical image data and corresponding medical reports (described later). In this embodiment, the medical data is, for example, a medical image obtained by scanning the aortic valve.

[0072] The inference unit 111 uses an artificial intelligence model to infer the medical data received by the receiving unit 110 and obtains the inference result. In this embodiment, for example, the inference unit 111 uses an artificial intelligence model to infer the medical data by segmenting the aortic valve in the aortic valve medical data, and sends the segmented aortic valve as the inference result to the first collection unit 112. In addition, the inference unit 111 also sends the inference result to the third collection unit 114 and the target collection unit 115, which will be described later.

[0073] The first collection unit 112 collects medical data into a first dataset based on the user's evaluation of the reasoning results of the medical data based on the artificial intelligence model. Furthermore, the first collection unit 112 stores the storage location of the medical data and image information of the first dataset in a database DB.

[0074] Specifically, the first collection unit 112 receives the reasoning results obtained by the reasoning unit 111 using an artificial intelligence model to reason about medical data. The first collection unit 112 displays the reasoning results to the user, for example, through the operation terminal 2.

[0075] Next, the user evaluates the displayed reasoning result through the operating terminal 2, for example, whether the user can accept the reasoning result. In this embodiment, the user evaluates the reasoning result that the segmented aortic valve has clear boundaries, accurate regions, or high precision as "acceptable," and evaluates the reasoning result that the segmented aortic valve has blurry boundaries, contains errors, or has low precision as "unacceptable." The operating terminal 2 sends the user's evaluation result to the first collection unit 112.

[0076] Next, the first collection unit 112 collects medical data whose evaluation result is "acceptable" based on the user's assessment results into a first dataset. The first collection unit 112 then sends medical data that does not belong to the first dataset (medical data whose evaluation result is "unacceptable") to the second collection unit 113.

[0077] Here, the first dataset is, for example, a benchmark dataset, containing medical data that serves as the ground truth (GT) as described later. The medical data in the benchmark dataset is medical data whose inference results are deemed "acceptable" by the user. The medical data in the first dataset can be considered as data from which the artificial intelligence model can perform inference well. In this embodiment, the medical data in the first dataset is, for example, medical data whose inference results regarding the aortic valve are highly accurate and acceptable to the user.

[0078] Additionally, the user can modify the inference results evaluated as "unacceptable" via the operation terminal 2. The operation terminal 2 sends the modified inference results, the corresponding medical data, and a message indicating that the user has made the modification to the target collection unit 115. The first collection unit 112 sends the medical data for which the inference results have not been modified by the user to the second collection unit 113.

[0079] The target collection unit 115 collects the received inference results modified by the user and the corresponding medical data into a target dataset, which will be described later.

[0080] The second collection unit 113 collects medical data that does not belong to the first dataset into a second dataset based on the matching degree between the medical data and the artificial intelligence model. Furthermore, the second collection unit 113 stores the storage location and image information of the collected second dataset's medical data in a database DB.

[0081] Here, the second dataset is, for example, a dataset to be collected, containing medical data that needs further evaluation and is yet to be collected. Specifically, the medical data in the dataset to be collected is medical data whose inference results have been evaluated as "unacceptable" by the user, have not been modified by the user, and contain training data features that the artificial intelligence model lacks. Furthermore, the medical data in the second dataset can be considered data that the artificial intelligence model has not learned or has not learned sufficiently to perform inference effectively. In this embodiment, the medical data in the second dataset is, for example, medical data whose inference results regarding the aortic valve are of low accuracy and have not been accepted by the user, whose inference results have not been modified by the user, and which contain training data features such as aortic valve lesions that the artificial intelligence model lacks.

[0082] Alternatively, the second collecting unit 113 may also include an extraction unit 1131.

[0083] The following is for reference Figures 3-5 The extraction unit 1131 and the detection unit 1132 will be described. Figure 3 This is a schematic diagram illustrating an example of disease keywords in a medical report of the present invention. Figure 4 This is a schematic diagram illustrating an example of the training data features of the artificial intelligence model of the present invention. Figure 5 This is a schematic diagram illustrating an example of a key image in a medical report of the present invention. Figure 4 The horizontal axis represents the category of the training data features, and the vertical axis represents the number of training data features.

[0084] In medical data that does not belong to the first dataset, the extraction unit 1131 extracts disease keywords from the medical reports corresponding to the medical data, compares the training data features of the artificial intelligence model with the disease keywords, and determines whether the disease keywords contain training data features that the artificial intelligence model lacks. When the disease keywords contain training data features that the artificial intelligence model lacks, the second collection unit 113 collects the medical data corresponding to the medical report in which the disease keywords were extracted as the second dataset.

[0085] Alternatively, the extraction unit 1131 may extract key images from the medical report and extract sequences from the medical data, the detection unit 1132 may detect feature points in the key images and sequences respectively, and the second collection unit 113 may collect the sequence as a second dataset based on the consistency between the feature points of the key images and the feature points of the sequences.

[0086] Alternatively, the extraction unit 1131 extracts key images from the medical report and extracts sequences from the medical data; the detection unit 1132 detects keywords in the key images and detects feature points in the sequences; and the second collection unit 113 collects the sequences into a second dataset based on the consistency between the keywords in the regions of interest and the feature points in the sequences.

[0087] Extraction unit 1131 extracts disease keywords from the medical reports corresponding to the medical data received by second collection unit 113 that do not belong to the first dataset.

[0088] like Figure 3 As shown, medical reports are stored, for example, in medical data storage device 3. Disease keywords are, for example,... Figure 3 The medical report shown contains keywords such as "aortic valve," "calcium," and "AVC>130," indicating the location, mode, and indicators of the lesion.

[0089] like Figure 4 As shown, the training data features of an artificial intelligence model refer, for example, to the features contained in the existing training data of the artificial intelligence model. In Figure 4 In the example shown, the artificial intelligence model has more training data features of "normal data" but lacks training data features of "aortic valve calcification", "aortic valve insufficiency" and "aortic valve stenosis".

[0090] Next, the extraction unit 1131 compares the training data features of the artificial intelligence model with the extracted lesion keywords to determine whether the lesion keywords contain training data features that are missing from the artificial intelligence model. Figure 3 , 4 In the example shown, the extraction unit 1131 determines that the lesion keywords "aortic valve," "calcium," and "AVC>130" contain the training data feature "aortic valve calcification" that is missing from the artificial intelligence model. At this time, the second collection unit 113 collects the medical data corresponding to the medical report containing the extracted lesion keywords as a second dataset. Furthermore, "the matching degree between medical data and the artificial intelligence model" refers to whether the lesion keywords in the medical report corresponding to the medical data contain the training data feature that is missing from the artificial intelligence model. Additionally, "high matching degree" means, for example, that the lesion keywords in the medical report corresponding to the medical data contain at least the training data feature that is missing from the artificial intelligence model.

[0091] Additionally, the second collection unit 113 may also include a detection unit 1132.

[0092] like Figure 5As shown, the extraction unit 1131 extracts key images from a medical report (e.g., CT Aortic Valve Analysis). Next, the extraction unit 1131 extracts sequences from the medical data corresponding to the medical report.

[0093] Here, "medical report" refers, for example, to the report corresponding to a single medical examination (Study). A single medical examination typically includes multiple "sequences" used for the examination, and within a single "sequence," multiple medical images are usually captured. A "key image" refers, for example, to a representative medical image among the multiple medical images captured in a single "sequence."

[0094] Next, the detection unit 1132 detects landmarks in the key images and sequences extracted by the extraction unit 1131. In the example of the present invention, the landmarks are, for example, "sino tubular junction", "aortic annulus", "ascending aorta", and "sinus of Valsalva".

[0095] Next, the second collection unit 113 determines the consistency between the feature points of the key image and the feature points of the sequence. For example, if at least one feature point in the key image matches the feature points in the sequence, the consistency is determined to be high; if none match, the consistency is determined to be low. Figure 5 In the example shown, the detection unit 1132 detects feature points of the aortic sinus in the key image and the sequence, respectively. At this time, the second collection unit 113 determines that the feature points of the key image and the feature points of the sequence have a high degree of consistency. The second collection unit 113 collects the sequence as a second dataset.

[0096] Alternatively, the detection unit 1132 may detect region of interest (ROI) keywords in the key image extracted by the extraction unit 1131. Here, the detection unit 1132 may extract ROI keywords, for example, based on ROI information in a medical report of the key image. In the example of this invention, "ROI keywords" may be, for example, "sinotubular junction," "ascending aorta," or "sinus of Valsalva."

[0097] Next, the second collection unit 113 determines the consistency between the keywords in the region of interest of the key image and the feature points of the sequence. For example, if at least one keyword in the region of interest of the key image matches the feature points of the sequence, the consistency is determined to be high; if none of them match, the consistency is determined to be low. If the second collection unit 113 determines that the consistency between the region of interest of the key image and the feature points of the sequence is high, the second collection unit 113 collects the sequence as a second dataset.

[0098] Therefore, the medical data in the second dataset can be all the medical data corresponding to multiple sequences obtained from a single medical examination, or it can be a subset of those sequences.

[0099] The third collection unit 114 extracts image features of target regions from the medical data in the first dataset and the medical data in the second dataset to generate medical data for the third dataset. Furthermore, the third collection unit 114 stores the storage location and image information of the generated medical data in the third dataset, along with the corresponding storage location of the medical data in the second dataset, in the database DB.

[0100] The third collection unit 114 calculates the difference between the image features of the medical data in the first dataset and the image features of the medical data in the second dataset, obtains the image features of the medical data in the second dataset with large differences, uses the image features of the medical data in the second dataset with large differences to process the medical data in the first dataset, and collects the processed medical data in the first dataset as the medical data in the third dataset.

[0101] Furthermore, the third collection unit 114 processes the medical data of the first dataset using the histogram distribution characteristics of the medical data in the second dataset as image features through histogram specification.

[0102] Reference Figure 6 The medical data generated by the third collection unit 114 for the third dataset is explained. Figure 6 This is a schematic diagram illustrating the medical data generated by the third collection unit 114 in the first embodiment, which generates a third dataset.

[0103] The third collection unit 114 receives the reasoning results of the medical data sent by the reasoning unit 111.

[0104] Next, as Figure 6As shown, the third collection unit 114 extracts image features of the target region from the medical data of the second dataset. Specifically, the third collection unit 114 obtains the inference result of the medical data of the second dataset. The third collection unit 114 performs dilation processing on the inference result of the medical data of the second dataset. The third collection unit 114 extracts the target region corresponding to the dilated inference result from the medical data of the second dataset, and extracts the image features of the target region. In addition, the third collection unit 114 similarly extracts image features of the target region (not shown) from the medical data of the first dataset.

[0105] In this embodiment, the "target area" is, for example, the area of ​​the aortic valve.

[0106] Here, the image features extracted by the third collection unit 114 include, for example, histogram distribution features, morphological features, and texture features. Alternatively, image features can also be other features that reflect the properties of the target region of the medical data.

[0107] Next, the third collection unit 114 calculates the differences between the image features of the medical data in the first dataset and the image features of the medical data in the second dataset. The third collection unit 114 obtains the image features of the medical data in the second dataset with the largest differences. In this embodiment, the example with the largest difference in histogram distribution features between the two datasets will be described. At this time, the third collection unit 114 extracts the histogram distribution features of the medical data in the second dataset. However, the third collection unit 114 may also extract other image features with large differences; the image features with large differences extracted by the third collection unit 114 only need to be able to be used to generate the medical data of the third dataset described later.

[0108] Next, the third collection unit 114 processes the medical data of the first dataset using the histogram distribution characteristics of the medical data in the second dataset through histogram specification. Specifically, the third data collection unit 114 equalizes the histogram distribution characteristics of the medical data in the second dataset and obtains a grayscale mapping table between the grayscale values ​​and the equalized values ​​in the medical data. In a manner consistent with the equalization results of the histogram distribution characteristics of the target regions of the medical data in the first and second datasets, the histogram distribution characteristics of the target regions of the medical data in the first dataset are applied to process the medical data in the target regions of the medical data in the first dataset, thereby generating the medical data in the third dataset.

[0109] Here, the third dataset is, for example, a rating dataset, containing medical data that serves as the rating object, as described later. The medical data in the third dataset is formed by appending image features of the target regions of the medical data in the second dataset to the target regions of the medical data in the first dataset. Therefore, the medical data in the third dataset possesses image features of the target regions of data that artificial intelligence models cannot reason about effectively.

[0110] Furthermore, the example described is of the third collection unit 114 processing the medical data of the first dataset using histogram specification. However, the present invention is not limited to this; the third collection unit 114 may also use methods such as texture transfer to process the medical data of the first dataset as image features of the medical data of the second dataset.

[0111] In this embodiment, the medical data of the third dataset is, for example, medical data of the aortic valve region of the medical data of the first dataset with the image features of the aortic valve after lesion from the medical data of the second dataset appended.

[0112] Based on the inference results of the medical data in the third dataset, the target collection unit 115 collects the medical data in the second dataset corresponding to the medical data in the third dataset as the target dataset, and the corresponding ground truth values ​​are labeled by doctors or annotation personnel. Furthermore, the target collection unit 115 stores the storage location of the medical data in the target dataset and the image information in the database DB.

[0113] Specifically, the target collection unit 115 calculates the similarity between the reasoning results of the medical data in the third dataset and the reasoning results of the corresponding medical data in the first dataset, scores the reasoning results of the medical data in the third dataset based on the similarity, and collects the medical data of the second dataset corresponding to the medical data in the third dataset as the target dataset based on the score.

[0114] The reasoning unit 111 uses an artificial intelligence model to reason about the medical data in the third dataset and sends the reasoning results of the medical data in the third dataset to the target collection unit 115.

[0115] Next, the target collection unit 115 calculates the similarity between the inference result of the received third dataset of medical data and the inference result of the corresponding first dataset of medical data. Here, "the corresponding first dataset of medical data" refers to the medical data of the first dataset that serves as the basis for generating the third dataset of medical data.

[0116] Specifically, the target collection unit 115 uses the inference results of the medical data in the corresponding first dataset as the ground truth (GT), and calculates the Dice value of the inference results of the medical data in the third dataset as the similarity. The target collection unit 115 determines whether the calculated Dice value is within a pre-set collectable threshold range. The collectable threshold range can be defined according to the model task category and accuracy, for example, [0, 0.90].

[0117] If the calculated Dice value is within the collectable threshold range, the target collection unit 115 collects the medical data of the second dataset corresponding to the medical data of the third dataset as the target dataset. This is because a calculated Dice value within the collectable threshold range indicates that the inference result of the medical data of the third dataset containing the image features of the medical data of the second dataset has a large difference from the true value and low similarity. In this case, it can be considered that the artificial intelligence model cannot reason well about the medical data of the third dataset containing the image features of the medical data of the second dataset. Therefore, the medical data of the second dataset corresponding to the medical data of the third dataset is the unlearned or insufficiently learned medical data required for the training or testing of the artificial intelligence model, and the target collection unit 115 collects such medical data as the target dataset for the artificial intelligence model to train or test.

[0118] Furthermore, if the calculated Dice value is outside the collection threshold, it indicates that the inference result of the medical data in the third dataset has a small difference from the true value and a high similarity. In this case, it can be considered that the AI ​​model can reason well about the medical data in the third dataset that contains the image features of the medical data in the second dataset; that is, the AI ​​model already has sufficient ability to reason about the medical data in the third dataset that contains the image features of the medical data in the second dataset. Therefore, it is not necessary to collect medical data containing such image features as a target dataset for training or testing the AI ​​model.

[0119] In addition, the target collection unit 115 outputs the medical data of the collected target dataset to the training and testing unit 117, which will be described later.

[0120] The following is for reference Figure 7 The classification performed by the classification section 116 is explained. Figure 7 This is a schematic diagram showing the classification performed by the classification section 116 in the first embodiment.

[0121] The classification unit 116 is used to classify medical data. The classification unit 116 classifies the medical data in the target dataset collected by the target collection unit 115 using a clustering method. Figure 7In the example shown, the classification unit 116 classifies the medical data in the target dataset into category "0", category "1", category "2", and category "3" based on the description of the medical report corresponding to the medical data in the target dataset ("normal image", "calcified image", "mechanical valve", "other", etc.).

[0122] The training and testing unit 117 uses medical data to train or test the artificial intelligence model. Specifically, the training and testing unit 117 trains or tests the artificial intelligence model based on the received medical data of the target dataset, enabling the artificial intelligence model to learn from the medical data of the target dataset. At this time, the ground truth of the training or testing is either the inference result modified by the user or the ground truth obtained subsequently by the user or annotators. Since the medical data of the target dataset contains image features of the medical data of the second dataset that the artificial intelligence model cannot reason well about, training or testing the artificial intelligence model based on the medical data of the target dataset enables the artificial intelligence model to effectively learn new features.

[0123] Therefore, the data collection device 1 of the present invention can efficiently collect effective medical data for training or testing artificial intelligence models.

[0124] This is because the medical data collected by the first collection unit 112, the second collection unit 113, the third collection unit 114, and the target collection unit 115 for training or testing the artificial intelligence model is medical data that is required for the training process of the artificial intelligence model but has not been learned or has been insufficiently learned and is therefore unable to perform reasoning well. Thus, it is possible to efficiently collect effective medical data for training or testing the artificial intelligence model.

[0125] Furthermore, through the extraction unit 1131 and the detection unit 1132 of the second collection unit 113, only a portion of the medical data sequence is collected as a second dataset. Therefore, the amount of invalid data in the target dataset can be reduced, improving the collection efficiency of the target dataset. Consequently, it is possible to collect effective medical data for training or testing artificial intelligence models more efficiently.

[0126] Furthermore, the medical data in the target dataset is classified using the classification unit 116. Therefore, the ability to select appropriate medical data during the training of the artificial intelligence model can improve the efficiency of training or testing.

[0127] (Data collection method process)

[0128] The following is for reference Figure 8 The flow of the data collection method of the first embodiment will be described. Figure 8 This is a flowchart illustrating the data collection method of the first embodiment.

[0129] In step S11, the receiving unit 110 receives the original medical data stored in the medical data storage device 3. Then, the process proceeds to step S12.

[0130] In step S12, the inference unit 111 uses an artificial intelligence model to infer the medical data received by the receiving unit 110 and obtains the inference result. The inference unit 111 sends the inference result to the first collection unit 112, the third collection unit 114, and the target collection unit 115. Then, the process proceeds to step S13.

[0131] In step S13, the user evaluates the reasoning result of the reasoning unit 111 through the operation terminal 2 and sends the evaluation result to the first collection unit 112. Then, the process proceeds to step S14.

[0132] In step S14, the first collection unit 112 judges the user's evaluation of the reasoning result. If the evaluation result is "acceptable,"... Figure 8 If the evaluation result is "unacceptable", proceed to step S15. If the evaluation result is "unacceptable", the first collection unit 112 will display the evaluation result as "unacceptable". Figure 8 The medical data marked "No" in step S14 is sent to the second collection unit 113, proceeding to steps S16 and S18.

[0133] In step S15, the first collection unit 112 collects medical data whose evaluation results are “acceptable” as the first dataset.

[0134] In step S16, it is determined whether the user has modified the reasoning result that was deemed "unacceptable" through the operation terminal 2. If the user modifies the reasoning result... Figure 8 If the user does not modify the reasoning result (as indicated by "Yes" in step S16), proceed to step S17. Figure 8 If the condition in step S16 is "No", proceed to step S18. Then, the first collection unit 112 sends the medical data for which the inference result evaluation is "unacceptable" and the inference result has not been modified by the user to the second dataset 113. Additionally, the operation terminal 2 sends the medical data for which the inference result evaluation is "unacceptable" and the inference result has been modified by the user to the target dataset 115. The operation terminal 2 also sends the modified inference result and a message indicating that the user has made modifications to it to the target collection unit 115.

[0135] In step S17, the target collection unit 115 collects the user-modified reasoning results and corresponding medical data into a target dataset.

[0136] In step S18, the second collection unit 113 receives medical data sent by the first collection unit 112 that does not belong to the first dataset. Then, the process proceeds to step S19.

[0137] In step S19, the second collection unit 113 determines the matching degree between the received medical data that does not belong to the first dataset and the artificial intelligence model. If the matching degree is determined to be low... Figure 8 If the condition in step S19 is "No", proceed to step S20, where a high matching degree is determined ( Figure 8 If "yes" is selected in step S19, proceed to step S21.

[0138] In step S20, the second collection unit 113 does not collect medical data with low matching degree with the artificial intelligence model as the second dataset.

[0139] In step S21, the second collection unit 113 collects medical data that is determined to have a high degree of matching with the artificial intelligence model as a second dataset. Then, the process proceeds to step S22.

[0140] In step S22, the third collection unit 114 extracts image features of the target regions from the medical data in the first dataset and the medical data in the second dataset, respectively. Then, the process proceeds to step S23.

[0141] In step S23, the third collection unit 114 uses the image features of the target regions from the extracted medical data of the first dataset and the medical data of the second dataset to generate medical data for the third dataset. Then, the process proceeds to step S24.

[0142] In step S24, the inference unit 111 uses an artificial intelligence model to infer the medical data in the third dataset generated by the third collection unit 114 and obtain the inference result. Then, the process proceeds to step S25.

[0143] In step S25, the target collection unit 115 calculates the similarity between the reasoning result of the received third dataset's medical data and the reasoning result of the corresponding first dataset's medical data. Furthermore, the target collection unit 115 determines whether the similarity is within a pre-set collectable threshold range. If the calculated similarity is not within the collectable threshold range, the process proceeds to step S26. If the calculated similarity is within the collectable threshold range, the process proceeds to step S27.

[0144] In step S26, the target collection unit 115 does not collect the medical data of the second dataset corresponding to the medical data of the third dataset as the target dataset.

[0145] In step S27, the target collection unit 115 collects the medical data of the second dataset corresponding to the medical data of the third dataset whose similarity is within the collectable threshold range as the target dataset. Then, it proceeds to step S28.

[0146] In step S28, the classification unit 116 classifies the medical data of the collected target dataset. Then, the process proceeds to step S29.

[0147] In step S29, the training and testing unit 117 uses the classified medical data of the target dataset to train or test the artificial intelligence model. Alternatively, after collecting the target dataset through the target dataset 115 in step S27, the process can directly proceed to step S29 without going through step S28, where the classification unit 116 classifies the medical data of the target dataset. Then, the process ends.

[0148] Furthermore, step S19 specifically includes steps S191 to S197, which will be described later. Hereinafter, refer to... Figure 9 Steps S191 to S197 are explained in detail. Figure 9 This is a flowchart illustrating the process of determining the matching degree in the data collection method of the first embodiment.

[0149] In step S191, the extraction unit 1131 of the second collection unit 113 extracts the lesion keywords from the medical report stored in the medical data storage device 3. Then, the process proceeds to step S192.

[0150] In step S192, the extraction unit 1131 compares the training data features of the artificial intelligence model with the extracted lesion keywords. Furthermore, the extraction unit 1131 determines whether the extracted lesion keywords contain training data features missing from the artificial intelligence model. If it is determined that the lesion keywords do not contain training data features missing from the artificial intelligence model... Figure 9 If the condition is "No" in step S192, proceed to step S193. If it is determined that the lesion keyword contains training data features missing from the artificial intelligence model (…), then… Figure 9 If "yes" is selected in step S192, proceed to step S194.

[0151] In step S193, the second collection unit 113 does not collect medical data as a second dataset.

[0152] In step S194, the extraction unit 1131 extracts key images from the medical report and extracts sequences from the medical data corresponding to the medical report. Then, the process proceeds to step S195A.

[0153] In step S195A, the detection unit 1132 detects feature points in the key image and sequence extracted by the extraction unit 1131. Then, the process proceeds to step S196A.

[0154] In step S196A, the second collection unit 113 determines the consistency between the feature points of the key image and the feature points of the sequence. If the consistency between the feature points of the key image and the feature points of the sequence is high (…), then… Figure 9 If "Yes" is selected in step S196A, proceed to step S21. If the feature points of the image are determined to have low consistency with the feature points of the sequence (…), then… Figure 9 If "No" is selected in step S196A, proceed to step S197.

[0155] Alternatively, after step S194, step S195B can also be performed.

[0156] In step S195B, the detection unit 1132 detects keywords in the region of interest in the key image extracted by the extraction unit 1131, and detects feature points in the sequence extracted by the extraction unit 1131. Then, the process proceeds to step S196B.

[0157] In step S196B, the second collection unit 113 determines the consistency between the keywords in the region of interest of the key image and the feature points of the sequence. If the consistency between the keywords in the region of interest of the key image and the feature points of the sequence is high (…), then… Figure 9 If "Yes" is selected in step S196B, proceed to step S21. If the keyword in the region of interest of the image is determined to be a key image and the feature points of the sequence have low consistency (…), then… Figure 9 If "No" is selected in step S196B, proceed to step S197.

[0158] In step S197, the second collection unit 113 does not collect the sequence as a second dataset.

[0159] In step S21, the second collection unit 113 collects sequences with high consistency as a second dataset. Then, the process proceeds to the aforementioned step S22. Thus, steps S191 to S197, which determine the matching degree, end.

[0160] (Second Implementation)

[0161] The following is for reference Figure 10 , 11 The image processing apparatus 1A of the second embodiment will be described. Figure 10 This is a block diagram illustrating a configuration example of the data collection device 1A according to the second embodiment. Figure 11 This is a schematic diagram showing the division of blocks in the second embodiment.

[0162] In the description of the second embodiment, an example is given of an artificial intelligence model performing a segmentation task on a medical image containing bones. Such a segmentation task involves a large segmentation area, and the segmentation of the entire area or local areas may be suboptimal due to lesions or other factors. The medical data collected by the data collection device 1A in the second embodiment is used to train or test the artificial intelligence model, enabling the model to perform the bone segmentation task.

[0163] When training an AI model to perform bone segmentation tasks, if the training data contains a high proportion of images of normal bones but lacks image data containing lesions such as osteoporosis and bone hyperplasia, the trained AI model will be unable to segment bones in images containing these lesions effectively. To improve the robustness and generalization ability of such AI models, it is necessary to collect medical data containing lesions such as osteoporosis and bone hyperplasia as training data.

[0164] In the following description, only the parts that differ from the first embodiment will be described, and the description of the parts that are the same will be omitted. The main difference between the second embodiment and the first embodiment is that the processing circuit 11A of the data collection device 1A further includes a division section 118.

[0165] like Figure 11 As shown by the white line, the partitioning unit 118 divides the medical data of the first dataset and the medical data of the second dataset into multiple blocks. In the second embodiment, the medical data is, for example, three-dimensional image data containing bones; therefore, the multiple blocks divided by the partitioning unit 118 are three-dimensional blocks.

[0166] Next, the third collection unit 114 extracts local image features from each block of medical data in the first dataset and each block of medical data in the second dataset. Local image features include, for example, grayscale distribution features, contrast features, texture features, frequency distribution, and intensity features in each block of medical data.

[0167] Next, the third collection unit 114 calculates the difference between the local image features of each block of the medical data in the first dataset and the local image features of each block of the medical data in the second dataset. For example, the third collection unit 114 calculates the difference in grayscale distribution characteristics between the two datasets (e.g.,...). Figure 11 (as shown in the solid-line box in the image).

[0168] Additionally, the segmentation unit 118 can also identify one or more blocks with significant differences as local target regions. The segmentation unit 118 can also combine feature points detected in key images of medical reports corresponding to medical data to determine local target regions.

[0169] Specifically, the detection unit 1132 detects feature points in the medical data of the second dataset. The third collection unit 114 matches the feature points of key images in the medical report with the feature points of the medical data in the second dataset to locate local images in the medical data of the second dataset.

[0170] Additionally, the detection unit 1132 detects feature points in the medical data of the first dataset. The third collection unit 114 matches the feature points of key images in the medical report with the feature points of the medical data in the first dataset to locate local images in the medical data of the first dataset.

[0171] The third collection unit 114 calculates the difference in local image features between local images in the first dataset of medical data after localization and local images in the second dataset of medical data.

[0172] Next, the third collection unit 114 acquires local image features of the medical data from the second dataset, which has significant differences. Using the local image features of the medical data from the second dataset, based on the following mathematical formula 1, the third collection unit 114 applies the local image features of the medical data from the second dataset to the medical data from the first dataset through grayscale transformation and other methods to process the medical data from the first dataset and generate the medical data from the third dataset.

[0173] (Mathematical Formula 1)

[0174]

[0175] Here, the third collection unit 114 in the second embodiment uses local image features of medical data from a second dataset with large differences to generate medical data for a third dataset, which is similar to the first embodiment, so the description is omitted.

[0176] Next, the target collection unit 115 calculates the similarity between the overall inference result and / or block inference result of the medical data in the third dataset based on the artificial intelligence model and the inference result of the corresponding medical data in the first dataset. "Overall inference result" refers to the inference result obtained by using the artificial intelligence model to infer the entirety of the medical data in the third dataset. "Block inference result" refers to the inference result obtained by using the artificial intelligence model to infer each block of the medical data in the third dataset.

[0177] Next, the target collection unit 115 scores the reasoning results of the medical data in the third dataset based on similarity, and collects the medical data in the second dataset corresponding to the medical data in the third dataset as the target dataset based on the score.

[0178] Here, the processing performed by the target collection unit 115 in the second embodiment is similar to that in the first embodiment, so the description is omitted. The difference is that the target collection unit 115 calculates the overall reasoning result and / or block reasoning result of the medical data in the third dataset, and scores the overall reasoning result and / or block reasoning result respectively.

[0179] Here, the target collection unit 115 can collect the medical data from the third dataset as the target dataset if the overall reasoning result score (Dice value) is within the collection threshold range. In this case, it can be assumed that the artificial intelligence model cannot reason well about the overall structure of the medical data in the third dataset. Therefore, the overall structure of the medical data in the second dataset corresponding to the medical data in the third dataset is required for the training or testing of the artificial intelligence model.

[0180] The target collection unit 115 can also collect the medical data from the third dataset as the target dataset if the overall inference result is not within the collection threshold range, but the score (Dice value) of the block inference result is within the collection threshold range. In this case, it can be assumed that the artificial intelligence model can infer the overall structure of the medical data in the third dataset well, but cannot infer the local structure of the medical data in the third dataset well. Therefore, the local structure of the medical data in the second dataset corresponding to the medical data in the third dataset is required for the training or testing of the artificial intelligence model.

[0181] The target collection unit 115 collects such medical data into a target dataset for use in training or testing artificial intelligence models.

[0182] Therefore, the medical data in the target dataset contains overall and / or local structures that artificial intelligence models cannot reason about well. Such medical data is what is needed for training or testing artificial intelligence models. Thus, even when the medical data contains overall structure, it is possible to efficiently collect medical data effective for training or testing artificial intelligence models.

[0183] Furthermore, in the above embodiments, the processing circuit is not limited to being implemented by a single processor; it can also be constructed by combining multiple independent processors, with each processor executing a program to implement each processing function. Additionally, the processing functions of the processing circuit can be appropriately distributed or integrated into one or more processing circuits. Furthermore, the processing functions of the processing circuit can also be implemented through a combination of hardware (such as circuits) and software. While an example has been described where the program corresponding to each processing function is stored in a single storage circuit, the implementation is not limited to this. For example, it is also possible to configure the program corresponding to each processing function to be distributed and stored in multiple storage circuits, with the processing circuit reading and executing each program from each storage circuit.

[0184] Furthermore, while the above embodiments illustrate examples where each functional part of this specification is implemented by processing circuitry, the embodiments are not limited to these examples. For instance, each functional part of this specification may be implemented not only by the processing circuitry described in the embodiments, but also by hardware alone, software alone, or a combination of hardware and software to achieve the same function.

[0185] Furthermore, the term "processor" used in the above description of the embodiments may, for example, mean CPU (Central Processing Unit), GPU (Graphics Processing Unit), or Application Specific Integrated Circuit (ASIC), or programmable logic device (e.g., Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device).

[0186] Circuits such as Complex Programmable Logic Device (CPLD) and Field Programmable Gate Array (FPGA) can be used. Alternatively, instead of storing the computer program in a memory circuit, the circuit can be configured to directly load the computer program into the processor. In this case, the processor reads and executes the program loaded into the circuit to perform its function. Furthermore, the processors in this embodiment are not limited to being configured as a single circuit; multiple independent circuits can be combined to form a single processor and perform its function.

[0187] Furthermore, the "computer program product" used in the above description of the embodiments includes a computer program executed by a processor, which is provided pre-loaded into ROM (Read Only Memory) or storage circuitry, etc. Additionally, the computer program can also be provided as a file that can be installed on or executed on computer-readable non-volatile storage media such as CD (Compact Disk)-ROM, FD (Flexible Disk), CD-R (Recordable), DVD (Digital Versatile Disk). Furthermore, the computer program can also be stored on a computer connected to a network such as the Internet and provided or distributed via network download. For example, the computer program is composed of modules including the aforementioned processing functions. As actual hardware, the CPU reads and executes the computer program from a storage medium such as ROM, thereby loading each module onto the main storage device and generating it on the main storage device.

[0188] Furthermore, in the above embodiments, the constituent elements of each device illustrated are functional conceptual elements and do not necessarily need to be physically configured as shown in the figures. That is, the specific manner in which the devices are distributed or concentrated is not limited to that illustrated, and can be configured in any unit, functionally or physically, in accordance with various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can also be implemented by a CPU and the program parsed and executed by the CPU, or as hardware implementation based on wiring logic.

[0189] Furthermore, among the processes described in the above embodiments, all or part of the processes described as automatically performed can be performed manually, or all or part of the processes described as manually performed can be performed automatically using known methods. In addition, the information regarding the processing procedures, control procedures, specific names, various data, or parameters shown in the above documents or drawings can be arbitrarily changed, except where specifically stated.

[0190] Furthermore, the various types of data processed in this specification are typically digital data.

[0191] Several embodiments have been described above, but these embodiments are given as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other ways, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope or spirit of the invention, and are included in the scope of the invention as described in the claims and its equivalents.

Claims

1. A data collection device for collecting medical data for training or testing an artificial intelligence model, characterized in that, Possessing: a first collection section that collects medical data as a first data set according to an evaluation result of a user on an inference result of the medical data based on the artificial intelligence model; a second collection section that collects medical data as a second data set according to a matching degree of the medical data with the artificial intelligence model among medical data not belonging to the first data set; a third collection section that respectively extracts an image feature of a target region of medical data of the first data set and the second data set to generate medical data of a third data set; and a target collection section that collects medical data of a second data set corresponding to the medical data of the third data set as a target data set according to the inference result of the medical data of the third data set.

2. The data collection apparatus according to claim 1, wherein the third collection section calculates a difference between the image feature of the medical data of the first data set and the image feature of the medical data of the second data set, acquires the image feature of the medical data of the second data set having a large difference, and processes the medical data of the first data set using the image feature of the medical data of the second data set having a large difference to generate the medical data of the third data set.

3. The data collection apparatus according to claim 2, wherein the third collection section processes the medical data of the first data set using a histogram distribution feature of the medical data of the second data set in a histogram specification manner.

4. The data collection apparatus according to claim 3, wherein the image feature is at least any one of a histogram distribution feature, a shape feature, and a texture feature of the target region of the medical data.

5. The data collection apparatus according to claim 1, wherein the data collection apparatus further possesses an extraction section, the extraction section extracts a lesion keyword in a medical report corresponding to the medical data not belonging to the first data set, compares a training data feature of the artificial intelligence model with the lesion keyword, and determines whether the lesion keyword includes the training data feature missing from the artificial intelligence model, and the second collection section collects the medical data corresponding to the medical report from which the lesion keyword is extracted as a second data set when the lesion keyword includes the training data feature missing from the artificial intelligence model.

6. The data collection apparatus according to claim 5, wherein the data collection apparatus further possesses a detection section, the extraction section extracts a key image in the medical report and extracts a sequence in the medical data, the detection section detects a feature point in the key image and the sequence, respectively, and the second collection section collects the sequence as a second data set according to a degree of coincidence of the feature point of the key image with the feature point of the sequence.

7. The data collection apparatus according to claim 5, wherein the data collection apparatus further possesses a detection section, ​ The extraction unit extracts a key image in the medical report and a sequence in the medical data, The detection unit detects a region of interest keyword in the key image and a feature point in the sequence, The second collection unit collects the sequence as a second data set according to a degree of coincidence between the region of interest keyword and the feature point of the sequence.

8. The data collection apparatus according to claim 1, wherein The target collection unit calculates a similarity between the inference result of the medical data of the third data set and the inference result of the medical data of the corresponding first data set, scores the inference result of the medical data of the third data set according to the similarity, and collects the medical data of the second data set corresponding to the medical data of the third data set as the target data set according to the score.

9. The data collection apparatus according to claim 8, wherein The inference result modified by the user and the corresponding medical data are collected as the target data set.

10. The data collection apparatus according to any one of claims 1 to 9, wherein The first data set is a reference data set including the medical data as a reference, the second data set is a to-be-collected data set including the medical data to be collected, and the third data set is a scoring data set including the medical data to be scored.

11. The data collection apparatus according to any one of claims 1 to 9, wherein The data collection apparatus further includes a classification unit, The classification unit classifies the medical data of the target data set by clustering.

12. The data collection apparatus according to any one of claims 1 to 9, wherein The data collection apparatus further includes: a reception unit that receives the medical data, an inference unit that infers the medical data using the artificial intelligence model, and a training and testing unit that trains or tests the artificial intelligence model based on the medical data of the target data set.

13. The data collection apparatus according to claim 1, wherein The data collection apparatus further includes a division unit, The division unit divides the medical data of the first data set and the medical data of the second data set into a plurality of blocks, respectively, The third collection unit extracts a local image feature of each of the blocks of the medical data of the first data set and the medical data of the second data set, respectively, calculates a difference between the local image feature of each of the blocks of the medical data of the first data set and the local image feature of each of the blocks of the medical data of the second data set, acquires the local image feature of the medical data of the second data set having a large difference, generates the medical data of the third data set by processing the medical data of the first data set using the local image feature of the medical data of the second data set having a large difference, and The target collection unit calculates a similarity between an overall inference result and / or a block inference result of the medical data of the third data set based on the artificial intelligence model and the inference result of the medical data of the corresponding first data set, scores the inference result of the medical data of the third data set according to the similarity, and collects the medical data of the second data set corresponding to the medical data of the third data set as the target data set according to the score.

14. A data collection method for collecting medical data for training or testing an artificial intelligence model, characterized in that, Comprising a first data set collection step of collecting the medical data as a first data set according to an evaluation result of a user for an inference result of the medical data based on the artificial intelligence model; a second data set collection step of collecting the medical data as a second data set according to a matching degree of the medical data with the artificial intelligence model among the medical data not belonging to the first data set; a third data set collection step of extracting image features of a target region of the medical data of the first data set and the second data set, respectively, to generate medical data of a third data set; and a target data set collection step of collecting the medical data of the second data set corresponding to the medical data of the third data set as a target data set according to the inference result of the medical data of the third data set. The computer program, when executed, includes the following steps:

15. A computer program product comprising a computer program for collecting medical data for training or testing an artificial intelligence model when executed by a processor. a first data set collection step of collecting the medical data as a first data set according to an evaluation result of a user for an inference result of the medical data based on the artificial intelligence model; a second data set collection step of collecting the medical data as a second data set according to a matching degree of the medical data with the artificial intelligence model among the medical data not belonging to the first data set; a third data set collection step of extracting image features of a target region of the medical data of the first data set and the second data set, respectively, to generate medical data of a third data set; and a target data set collection step of collecting the medical data of the second data set corresponding to the medical data of the third data set as a target data set according to the inference result of the medical data of the third data set. ​ ​

Images