A DICOM file management method and system based on a lake warehouse architecture

Abstract

This invention discloses a DICOM file management method and system based on a lake-warehouse integrated architecture. The method involves: acquiring a target DICOM file (obtained from a PACS server); parsing the target DICOM file to obtain parsing results; storing the parsing results in a data lake according to data type; acquiring data to be calculated from the data lake, processing it, and then pushing it to a data warehouse; wherein the data to be calculated includes the parsing results of the target DICOM file; using the data warehouse's computing engine to perform calculations on the data to obtain calculation results; and storing the calculation results in the data warehouse database for querying and / or access by upper-layer applications. This solves the problem of managing massive amounts of DICOM files in existing technologies.

Description

Technical Field

[0001] This invention relates primarily to the field of medical image data processing, and more particularly to a DICOM file management method and system based on a lakeware architecture. Background Technology

[0002] In recent years, the rapid development of internet technology has propelled the medical field into the information age, and even towards the intelligent age. However, after in-depth research, the applicant discovered that a crucial type of medical data is DICOM files based on the DICOM protocol. However, DICOM files are unstructured files. To utilize massive amounts of DICOM files and provide a data foundation for medical big data analysis and data mining, it is essential to overcome the management challenges of storing and querying these files. Traditional data warehouses, using two-dimensional relational tables for data management, are ill-suited for managing unstructured DICOM files.

[0003] However, using a data lake to store DICOM files cannot meet the needs of in-depth processing and data mining of DICOM file data. Therefore, how to solve the management problem of DICOM files and remove the stumbling blocks that restrict the in-depth mining and utilization of DICOM data in the medical field has become a technical problem that technicians in this field must solve. Summary of the Invention

[0004] In view of this, this application provides a DICOM file management method and system based on a lake warehouse integrated architecture. By parsing DICOM files and storing the parsing results in a data lake, the data to be computed in the data lake is processed and pushed to the data warehouse. The computing engine of the data warehouse is used to compute the data to be computed, and the computing results are stored in the data warehouse database for querying and / or calling by upper-layer applications, thereby solving the problem of managing massive DICOM files in the prior art.

[0005] According to one aspect of this application, a DICOM file management method based on a lakeware architecture is provided, comprising:

[0006] Acquire the target DICOM file; this target DICOM file is obtained from the PACS server;

[0007] Parse the target DICOM file to obtain the parsing results; store the parsing results into the data lake according to data type.

[0008] The data to be computed in the data lake is retrieved, processed, and then pushed to the data warehouse; the data to be computed includes the parsing results of the target DICOM file.

[0009] The data warehouse's computing engine is used to perform calculations on the data to be calculated, thereby obtaining the calculation results. The calculation results are then stored in the data warehouse database for querying and / or access by upper-layer applications. The calculation results include query results for the target DICOM file.

[0010] In one exemplary embodiment, the data lake includes at least structured data storage units and unstructured data storage units; the parsing result includes at least the tag data and image data of the target DICOM file;

[0011] The parsing results will be stored in the data lake according to data type, specifically including:

[0012] Extract the tag data from the target DICOM file and store the tag data in the structured data storage unit;

[0013] Extract the image data from the target DICOM file and store the image data in the unstructured data storage unit.

[0014] In one exemplary embodiment, the process of acquiring the data to be computed in the data lake, processing it, and then pushing it to the data warehouse specifically includes:

[0015] Retrieve the data to be computed in batches from the data lake;

[0016] The data to be calculated is extracted, transformed, and loaded to obtain the processed data to be calculated.

[0017] The processed data to be calculated is pushed to the data warehouse.

[0018] In one exemplary embodiment, the process of acquiring the data to be computed in the data lake, processing it, and then pushing it to the data warehouse further includes:

[0019] Determine whether the data to be calculated is tag data. If so, obtain the storage path of the image data corresponding to the tag data in the unstructured data storage unit; associate the storage path with the tag data and write it into the data warehouse.

[0020] In one exemplary embodiment, extracting the tag data of the target DICOM file and storing the tag data in the structured data storage unit specifically includes:

[0021] Extract the tag data from the target DICOM file;

[0022] Generate a globally unique identifier for the tag based on the tag data;

[0023] The tag data and its corresponding globally unique identifier are encapsulated into a message body and distributed to the lake warehouse engine of the data lake through a message middleware to be written into the structured data storage unit of the data lake.

[0024] In one exemplary embodiment, after storing the calculation results in the data warehouse database, the method further includes:

[0025] The Tag data of the target DICOM file and its corresponding globally unique identifier are encapsulated into a message body and then distributed to the index building program via a message middleware.

[0026] Load the distributed full-text search database;

[0027] The index builder constructs a query index of the tag data based on the tag data of the target DICOM file;

[0028] The query index is stored in the distributed full-text search database.

[0029] In one exemplary embodiment, the data lake further includes a metadata storage unit; following the target DICOM file, it also includes:

[0030] Extract the metadata from the DICOM file and abstract the metamodel;

[0031] Store the metadata and the metamodel in the metadata storage unit.

[0032] In one exemplary embodiment, the unstructured data storage unit employs a file system or object storage system to manage the image data of the target DICOM file.

[0033] According to another aspect of this application, a DICOM file management system based on a lakeware architecture is provided, comprising:

[0034] The data acquisition module is used to acquire the target DICOM file, which is obtained from the PACS server; parse the target DICOM file to obtain the parsing results; and store the parsing results into the data lake according to data type.

[0035] The data lake module is used to store the parsing results of the target DICOM file;

[0036] The data processing module is used to acquire the data to be calculated in the data lake, process it, and then push it to the data warehouse; the data to be calculated includes the parsing results of the target DICOM file;

[0037] The data warehouse module is used to perform calculations on the data to be calculated using the data warehouse's computing engine, obtain the calculation results, and store the calculation results in the data warehouse database for querying and / or access by upper-layer applications; wherein, the calculation results include query results for the target DICOM file.

[0038] In one exemplary embodiment, the data processing module is used to acquire data to be computed in the data lake, process it, and then push it to the data warehouse, specifically including:

[0039] Determine whether the data to be calculated is Tag data. If so, obtain the storage path of the image data corresponding to the Tag data in the unstructured storage unit; associate the storage path with the Tag data and store it in the data warehouse.

[0040] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0041] 1. By parsing DICOM files, the tag data is extracted and stored in structured storage units, while the image data is extracted and stored in unstructured storage units. This allows DICOM files to be decomposed into structured and unstructured data types, which are then managed separately according to their data types. Furthermore, when a tag file needs to be pushed to the data warehouse for computation, the storage address of the corresponding image file is read, and this storage address is associated with the tag file and stored in the data warehouse database. This allows upper-layer applications to query and / or access the data. Additionally, queries can be aggregated into a single composite query command for upper-layer applications to query and / or access, thus enabling the rational and effective management of massive amounts of complex DICOM file data.

[0042] 2. Use message middleware to distribute tag data to ensure data distribution efficiency and reliable data transmission. Attached Figure Description

[0043] Figure 1 This is a DICOM file header structure illustrated in an exemplary embodiment of this application.

[0044] Figure 2 This is a schematic diagram of a DICOM file management system architecture based on a lake warehouse integrated architecture, according to an exemplary embodiment of this application.

[0045] Figure 3 This is a flowchart illustrating a DICOM file management method based on a lakeware architecture according to an exemplary embodiment of this application.

[0046] Figure 4This is a flowchart illustrating a tag data storage method according to an exemplary embodiment of this application.

[0047] Figure 5 This is a schematic diagram of the structure of a high-efficiency DICOM file retrieval system for real-time complex queries implemented using a DICOM file management system based on a lake-warehouse integrated architecture, according to an exemplary embodiment of this application. Detailed Implementation

[0048] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0049] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0050] The "DICOM" (Digital Imaging and Communications in Medicine) referred to in the embodiments of this application is a standard that regulates the management, storage, printing, and information transmission of medical imaging. Files generated and managed based on the DICOM standard are called DICOM files. The DICOM file format consists of a header and image data. The header contains basic patient information such as patient ID and name, as well as examination and equipment information corresponding to the DICOM file. A DICOM file is a standard binary stream file; it essentially encodes the attribute information and corresponding values ​​of various objects to form the tag data of the DICOM file.

[0051] If you open a DICOM file using the Dicompyler software, you can see that the DICOM file's data header is as follows: Figure 1As shown, each tag is determined by a combination of two hexadecimal numbers in the format (Group, Element). For example, the tag (0010, 0010) represents Patient's Name, which corresponds to the patient's name in this DICOM file. When reading information from a DICOM file, the corresponding tag must first be identified. Therefore, tag data can be understood as the data dictionary of the DICOM file; that is, to obtain data from a DICOM file, it is necessary to interpret the tag data to determine the data attributes stored in that group of tags. Thus, tag data can be considered structured data.

[0052] Of course, the categories of tag data include patient tag, study tag, series tag, and image tag. Among them, the image tag is followed by the image data (i.e., image data) of the DICOM file, which is unstructured data. In other words, a DICOM file is a hybrid type of file that integrates structured and unstructured data; overall, it belongs to the category of unstructured files.

[0053] The data lake referred to in this application is a highly scalable data storage area that can store all types of data, including structured, semi-structured, and even unstructured data. It can store massive amounts of data of any type without requiring any data transformation during writing.

[0054] The data warehouse referred to in this application is a data system used for storage, analysis, and reporting. The data in the data warehouse is obtained by extracting and cleaning existing, dispersed database data, followed by systematic processing, summarization, and organization. The data in the data warehouse is primarily used for enterprise decision-making analysis, and the data operations involved are mainly data queries.

[0055] Lakewarehousing is a new paradigm that combines the advantages of data lakes and data warehouses. It achieves similar data structures and data management functions in a data warehouse while providing low-cost storage in a data lake. Specifically, it centrally stores data in a "data lake storage," which provides data storage and management functions for both structured and unstructured data, and offers data manipulation interfaces to the data warehouse computing engine.

[0056] As mentioned earlier, DICOM files are a hybrid type of file that integrates structured and unstructured data. For the healthcare industry, how to store and manage massive amounts of DICOM files to serve as the data foundation for medical decision analysis has become a pressing technical challenge. To address this, embodiments of this application provide a DICOM file management method and system based on a lake-warehouse integrated architecture. This method parses DICOM files, stores the parsing results in a data lake, processes the data to be computed in the data lake, and pushes it to a data warehouse. The data warehouse's computing engine then performs calculations on the data to be computed, and the results are stored in the data warehouse database for querying and / or access by upper-layer applications. This solves the problem of managing massive amounts of DICOM files in existing technologies. Furthermore, the lake-warehouse integrated architecture described in this application can be deployed in electronic devices, including but not limited to servers, server clusters, and other hardware or virtual environments.

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0058] Figure 2 This is a schematic diagram of the architecture of a DICOM file management system based on a lake-warehouse integrated architecture, which is an exemplary embodiment of this application. The DICOM file management system based on a lake-warehouse integrated architecture in this exemplary embodiment includes at least a data source module 110, a data acquisition module 120, a data lake module 130, a data processing module 140, a data warehouse module 150, a data service module 160, and a data application module 170. The data flow is as follows: the data acquisition module 120 acquires DICOM files from the data source module 110; after local processing, it stores them in the data lake module 130; the data processing module 140 obtains the data to be processed from the data lake module 130, processes it, and pushes it to the data warehouse module 150; the data warehouse module 150 performs calculations on the data to be processed, obtains the calculation results, and saves the calculation results in the data warehouse database; the data service module 160 constructs a query index for the data stored in the data warehouse database and provides a query interface to the data application module 170 for the data application module 170 to call.

[0059] Specifically, the data source module 110 is used to store DICOM files. Typically, the data source module 110 is located at the medical institution, for example, it can be a PACS server of a PACS system used to store and manage medical examination data of a medical institution.

[0060] Data acquisition module 120 is used to acquire target DICOM files, parse the target DICOM files to obtain the parsing results, and store the parsing results into the data lake according to data type.

[0061] Data Lake Module 130, or Data Lake, is used to store the parsing results of the target DICOM file, the metadata of the DICOM file, and the abstracted meta-model.

[0062] The data processing module 140 is used to acquire the data to be calculated in the data lake, process it, and then push it to the data warehouse. The data to be calculated includes the parsing result of the target DICOM file. Specifically, the module acquires the data to be calculated in the data lake, processes it, and then pushes it to the data warehouse, including: determining whether the data to be calculated is tag data; if so, acquiring the storage path of the image data corresponding to the tag data in the unstructured storage unit; associating the storage path with the tag data and storing it in the data warehouse; and constructing a query index based on the association result between the storage path and the tag data, so that upper-layer applications can query the image data and associated patient information.

[0063] The data warehouse module 150, i.e. the data warehouse, is used to perform calculations on the data to be calculated using the data warehouse's computing engine, obtain the calculation results, and store the calculation results in the data warehouse database for querying and / or calling by upper-layer applications; wherein, the calculation results include the query results for the target DICOM file.

[0064] The data service module 160 receives the tag data of the target DICOM file and encapsulates it with its corresponding globally unique identifier into a message body; loads the distributed full-text search database; the index building program of the data service module 150 decapsulates the message body and builds a query index for the tag data based on the tag data of the target DICOM file; and stores the query index in the distributed full-text search database. In addition, it provides query and / or call interfaces to the data application module 170.

[0065] The data application module 170 is used to query and / or call the DICOM file management system based on the lake warehouse integrated architecture through the query interface according to actual analysis needs.

[0066] Therefore, as Figure 3 As shown, the DICOM file management method of the DICOM file management system based on the lake warehouse integrated architecture includes:

[0067] Step 210: Acquire the target DICOM file; the target DICOM file is obtained from the PACS server.

[0068] Specifically, the DICOM file management system based on the lakeware architecture creates a medical image data collector on the data acquisition module 120. This collector is used to download DICOM files from the hospital's PACS server to the local server where the collector runs. PACS (picture archiving and communication system) is a system used in hospital radiology departments. It stores massive amounts of various medical images generated daily in the hospital (including images from MRI, CT, ultrasound, various X-ray machines, various infrared instruments, microscopes, etc.) digitally on the hospital's server via a DICOM interface.

[0069] The acquisition frequency of the data collector can be set to periodic batch acquisition or real-time acquisition, depending on the actual production needs. There is no limitation here.

[0070] Step 230: Parse the target DICOM file to obtain the parsing result of the target DICOM file; store the parsing result into data lake 130 according to data type.

[0071] Specifically, the data acquisition module 120 needs to load a dedicated DICOM file parser locally, such as a "DICOM Processor". The data acquisition module 120 uses the DICOM file parser to parse the downloaded DICOM file, thereby obtaining the parsing result of the target DICOM file. In one possible implementation, this parsing result includes at least tag data and image data. As mentioned earlier, tag data has structured characteristics and can therefore be considered structured data; image data is essentially image data and has unstructured characteristics, therefore it is considered unstructured data. The data acquisition module 120 stores the parsing results in the data lake 130 according to their data types.

[0072] In one possible implementation, before the data acquisition module 120 stores the parsed results into the data lake 130 according to data type, the data lake 130 is divided into storage spaces according to storage requirements, at least into structured data storage units and unstructured data storage units. Of course, in actual use, there may also be semi-structured data that needs to be stored, so further semi-structured data storage units can be created. Furthermore, in order to perform unified global management of the stored data, a metadata storage unit also needs to be created within the storage space of the data lake 130; this metadata storage unit is used to store the metadata and abstracted meta-model of the data stored in the data lake 130.

[0073] Therefore, the data acquisition module 120 extracts the tag data of the target DICOM file from the parsing results of the target DICOM file and stores the tag data in the structured data storage unit of the data lake 130; it also extracts the image data of the target DICOM file, stores the image data in the unstructured data storage unit, and records the storage path of the image data of the target DICOM file in the unstructured data storage unit.

[0074] As mentioned above, the image data in a DICOM file is essentially image data, and therefore unstructured data. To efficiently store and manage this image data, one possible implementation is to use an object storage system to manage the unstructured data storage units. Alternatively, another possible implementation is to use a file system to manage the unstructured data storage units.

[0075] The data acquisition module 120 extracts the metadata of the target DICOM file and abstracts the meta-model. It stores the metadata and meta-model of the target DICOM file in the metadata storage unit of the data lake 130 and generates corresponding security settings and access control policies.

[0076] Step 250: Obtain the data to be calculated from the data lake 130, process it, and push it to the data warehouse 150; wherein, the data to be calculated includes the parsing result of the target DICOM file;

[0077] Specifically, the data processing module 140 obtains the data to be calculated from the data lake 130; one possible implementation is that the data processing module 140 periodically obtains batches of data to be calculated from the data lake 130; the data to be calculated can be tag data, metadata, or other data; after processing the data to be calculated, the data processing module 140 pushes it to the data warehouse 150.

[0078] One possible implementation is that, before pushing the data to be computed to the data warehouse 150, the data processing module 140 extracts, transforms, and loads the data to be computed to obtain processed data; then, the processed data is pushed to the data warehouse 150. Since the data lake 130 stores data in its raw format, the data obtained by the data processing module 140 from the data lake 130 needs to be processed before it can be used by the computing engine of the data warehouse 150. The processing of the data to be computed here can include extraction, transformation, and loading, i.e., performing an ETL task. An ETL task consists of three stages: extracting data from different data sources, processing and formatting the data according to certain data processing rules, and finally outputting the data to the target data table in the data warehouse 150, thus completing the processing of the data to be computed.

[0079] One possible implementation is that after the data processing module 140 obtains the data to be calculated from the data lake 130, it first needs to determine whether the data to be calculated is tag data. If it is tag data, it queries the data lake 130 for the storage path of the image data corresponding to the tag data within the data lake 130. It then obtains the storage path of the image data corresponding to the tag data in the unstructured data storage unit of the data lake 130; and associates this storage path with the tag data, writing it into the data warehouse 150 to meet the complex query requirements of the data application module 170 for the image data of the target DICOM file.

[0080] Step 270: Utilize the computing engine of the data warehouse 150 to perform calculations on the data to be calculated to obtain the calculation results; store the calculation results in the data warehouse database for querying and / or calling by upper-layer applications; wherein, the calculation results include the query results for the target DICOM file.

[0081] Specifically, data warehouse 150 receives the data to be computed from data processing module 140140 after ETL processing. Since data warehouse 150 itself is an integrated data environment oriented towards analysis, it uses a computing engine to perform calculations on the data to be computed to obtain results. One possible implementation is that data warehouse 150 uses the computing engine to layer, process, and integrate the data to be computed, establishing consistent dimensions, constructing reusable detailed fact tables for analysis and statistics, and summarizing common granular indicators. For example, different DICOM files of the same patient can summarize common granular indicators including basic information about the same patient. Another possible implementation is to provide SQL-based interactive query and data analysis capabilities through engines such as Trino and Presto. Trino is a distributed SQL query engine designed to query large datasets distributed across one or more heterogeneous data sources. It achieves horizontal scaling by distributing processing tasks across servers throughout the cluster. Based on this architecture, the Trino query engine can process SQL queries on massive amounts of data in parallel on computing nodes within the cluster. Presto is also a distributed SQL query engine suitable for interactive analytical queries.

[0082] The calculation results are stored in the data warehouse database for querying and / or access by upper-layer applications;

[0083] Specifically, the detailed fact table after the computing engine has layered, processed and integrated the data to be computed, as well as the summarized public granularity indicators, will be stored in the data warehouse database. One possible implementation is that the data warehouse database is set in the storage space of the data warehouse 150; another possible implementation is that the data warehouse database is set in the data lake 130 of the lake warehouse integrated architecture system.

[0084] One possible implementation involves creating a query index for the calculation results after storing them in a data warehouse database, and then storing the query index in a distributed full-text search database. This includes the steps of: loading the distributed full-text search database; the index building program constructing a query index for the tag data based on the tag data of the target DICOM file; and storing the query index in the distributed full-text search database.

[0085] Specifically, the data service module pre-creates a distributed full-text search database and runs an index builder locally. Upon receiving tag data, the index builder creates a query index for the tag data and stores this index in the distributed full-text search database. The data service module provides query and / or call interfaces to the data application module 170 for querying and / or calling by upper-layer applications.

[0086] Data application module 170 provides data application services to users; the patient image retrieval application uses the patient's identity identifier as a keyword to retrieve information in the distributed full-text search database, providing authorized queryers with the patient's examination information. The image data of the target DICOM file is stored in the unstructured data storage unit of data lake 130. For example, the image retrieval application can access the retrieval database of the data service module carrying the identity ID.

[0087] So, tag data, as structured data in unstructured files, is one method of storage in a lakeware architecture, such as... Figure 4 As shown;

[0088] Step 410: The collector extracts the Tag data from the DICOM file;

[0089] Step 420: The collector reports the Tag data of the DICOM file to the structured data gateway;

[0090] Step 430: The structured data gateway generates a globally unique identifier (gid identifier) ​​for the DICOM file in the lake warehouse based on the tag data;

[0091] Step 440: Encapsulate the gid identifier and tag into the message body;

[0092] Step 450a: Write the Tag data to data lake 130 via message queue;

[0093] Step 450b: Distribute the Tag data to the index building program of the data service module via message middleware to build a query index for the Tag data. The index building program builds the query index for the Tag data based on the GID identifier of the Tag data in the message body. For example, if the structure of the GID identifier is set to "Hospital ID - In-hospital PACS system ID - Image series ID - File generation time Unix timestamp (milliseconds)", then the constructed index example is: "H0001-01-xxxx-1667898585000".

[0094] Step 460: Update the medical image data warehouse offline.

[0095] Specifically, the GID identifier and Tag are encapsulated in the message body; this is then distributed to the lake warehouse engine via a message middleware to write the Tag data to data lake 130; additionally, it is also distributed to the index building program of the data service module; for details, please refer to [link to relevant documentation]. Figure 4 ;

[0096] Step 451: Deliver the message body to the distributed messaging platform;

[0097] Step 452: The distributed message consumer cluster pulls the message body;

[0098] Step 453: The distributed message consumer cluster distributes the message body to the index building program of the lake warehouse engine and data service module;

[0099] Step 454: The lake warehouse engine decapsulates the message body, extracts the Tag data of the target DICOM file, and writes it into the structured data storage unit of the data lake 130 in real time.

[0100] Based on a lake-warehouse integrated architecture and DICOM file storage, a high-efficiency DICOM file retrieval system for real-time complex queries can be implemented. This retrieval system allows data application modules 170 to query and / or access data in real time. The architecture of this retrieval system is as follows: Figure 5 As shown;

[0101] The retrieval system can be built on the distributed full-text retrieval database of the data service module 160.

[0102] Specifically, based on the medical field's requirements for DICOM file data, single index tables based on single query conditions and composite index tables based on composite query conditions are constructed on the basis of a distributed full-text search database. One possible approach is to construct four single index tables: Image Examination Request Form Index 161, Image Examination Form Index 162, Image Examination Report Form Index 163, and Image Series Index 164. In addition, a composite image examination index is constructed, which adds an image data storage path query index 165 from the DICOM file to the four single index tables. This image data storage path query index 165 is a query index for the storage path of image data in the unstructured data storage unit of data lake 130, and the four single index tables and image data storage path query index 165 are linked through an index association identifier. One possible implementation is that this index association identifier is index_link_id. In summary, the DICOM file management method and system based on a lakeware architecture in the first embodiment of this application parses DICOM files, extracts the Tag data from the DICOM files and stores it in a structured storage unit, and extracts the image data from the DICOM files and stores it in an unstructured storage unit. When a Tag file needs to be pushed to the data warehouse 150 for computation, the storage address of the corresponding image file is read, and this storage address is associated with the Tag file and stored in the data warehouse database, so that upper-layer applications can query and / or call it. This decomposes DICOM files into two types: structured data and unstructured data, and manages them separately according to their data types. Furthermore, it can aggregate the data into a single composite query command during querying, allowing upper-layer applications to query and / or call it, thus enabling reasonable and effective management of massive amounts of complex DICOM file data. In addition, a message middleware is used to distribute Tag data, ensuring efficient data distribution and reliable data transmission.

[0103] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A DICOM file management method based on a lakeware architecture, characterized in that, include: Acquire the target DICOM file; The target DICOM file is obtained from the PACS server; The DICOM file is a hybrid file that integrates structured and unstructured data; The target DICOM file is parsed to obtain the parsing result of the target DICOM file; the parsing result includes at least the tag data and image data of the target DICOM file. The parsing results are stored in a data lake according to data type; wherein the data lake includes at least structured data storage units and unstructured data storage units; the tag data of the target DICOM file is extracted and stored in the structured data storage unit; the image data of the target DICOM file is extracted and stored in the unstructured data storage unit. The system acquires the data to be computed from the data lake, processes it, and then pushes it to the data warehouse. The data to be computed includes the parsing result of the target DICOM file. The system also determines whether the data to be computed is tag data. If so, it acquires the storage path of the image data corresponding to the tag data in the unstructured data storage unit and associates the storage path with the tag data, writing it into the data warehouse. The computing engine of the data warehouse is used to perform calculations on the data to be calculated to obtain calculation results; the calculation results are stored in the data warehouse database for querying and / or calling by upper-layer applications; wherein, the calculation results include query results for the target DICOM file.

2. The DICOM file management method based on a lake-warehouse integrated architecture according to claim 1, characterized in that: The process of acquiring the data to be computed in the data lake, processing it, and then pushing it to the data warehouse specifically includes: The data to be computed is obtained in batches from the data lake; The data to be calculated is extracted, transformed, and loaded to obtain the processed data to be calculated. The processed data to be calculated is pushed to the data warehouse.

3. The DICOM file management method based on a lake-warehouse integrated architecture according to claim 1, characterized in that: The step of extracting the tag data of the target DICOM file and storing the tag data in the structured data storage unit specifically includes: Extract the tag data from the target DICOM file; Generate a globally unique identifier for the tag data based on the tag data; The tag data and its corresponding globally unique identifier are encapsulated into a message body and distributed to the lake warehouse engine of the data lake through a message middleware, so as to be written into the structured data storage unit of the data lake.

4. The DICOM file management method based on a lake-warehouse integrated architecture according to claim 3, characterized in that: After storing the calculation results in the data warehouse database, the process also includes: The Tag data of the target DICOM file and its corresponding globally unique identifier are encapsulated into a message body and then distributed to the index building program via a message middleware. Load the distributed full-text search database; The index building program constructs a query index of the tag data based on the tag data of the target DICOM file; The query index is stored in the distributed full-text search database.

5. The DICOM file management method based on a lake warehouse integrated architecture according to claim 1, characterized in that: The data lake also includes a metadata storage unit; Following the acquisition of the target DICOM file, the following is also included: Extract the metadata of the DICOM file and abstract the metamodel; The metadata and the metamodel are stored in the metadata storage unit.

6. The DICOM file management method based on a lake warehouse integrated architecture according to claim 1, characterized in that: The unstructured data storage unit uses a file system or object storage system to manage the image data of the target DICOM file.

7. A DICOM file management system based on a lake-warehouse integrated architecture, characterized in that: include: The data acquisition module is used to acquire the target DICOM file, which is obtained from the PACS server; The DICOM file is a hybrid file integrating structured and unstructured data; the target DICOM file is parsed to obtain the parsing result; the parsing result includes at least the tag data and image data of the target DICOM file; The parsing results are stored in the data lake according to data type; wherein the data lake includes at least structured data storage units and unstructured data storage units; the tag data of the target DICOM file is extracted and stored in the structured data storage unit; the image data of the target DICOM file is extracted and stored in the unstructured data storage unit. The data lake module is used to store the parsing results of the target DICOM file; A data processing module is used to acquire data to be calculated in the data lake, process it, and push it to the data warehouse; wherein, the data to be calculated includes the parsing result of the target DICOM file; and to determine whether the data to be calculated is tag data, if so, to acquire the storage path of the image data corresponding to the tag data in the unstructured data storage unit; and to associate the storage path with the tag data and write it into the data warehouse; The data warehouse module is used to perform calculations on the data to be calculated using the data warehouse's computing engine to obtain calculation results; and to store the calculation results in the data warehouse database for querying and / or access by upper-layer applications; wherein, the calculation results include query results for the target DICOM file.

Images