Digital slice and case information association system and method, electronic equipment and medium

Abstract

The invention provides a digital slice and case information association system and method, an electronic device and a medium, and the system comprises a data obtaining module which is used for obtaining a digital slice of a patient and pathological case information; the feature extraction module is used for extracting slice feature information of the digital slices and case feature information of the case information based on a preset AI model and / or an OCR character recognition technology; the feature matching module is used for calculating a matching score between the slice feature information and the case feature information, and judging whether the digital slices and the case information are successfully matched or not based on the matching score; and the association module is used for associating the successfully matched digital slices with the case information. According to the method, the digital slices and the case information can be automatically associated, and subsequent consultation or other operations of the user are facilitated.

Description

Technical Field

[0001] The present invention relates to the field of medical information technology, and in particular, to a system, method, electronic device and medium for associating digital slices with case information. Background Art

[0002] With the rapid development of technology and medicine, doctors can diagnose diseases through remote consultations, thus saving time and improving efficiency. During remote consultations, information such as the patient's case, slices, and images needs to be uploaded to the consultation platform so that the doctors participating in the consultation can understand the patient's condition through the consultation platform. However, currently, after receiving digital slices and case information, the consultation platform cannot automatically associate them; at the same time, for some pathological diagnosis systems deployed in small and medium-sized hospitals, digital slices cannot be uploaded to the consultation platform. Instead, physical slices need to be sent by mail and then uploaded to the consultation platform by relevant staff. During this process, the slice information and case information of the patient are uploaded separately, and the staff cannot know the corresponding relationship between the slice information and the case information. Moreover, due to the large amount of data, it is unrealistic to perform the association manually. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a system, method, electronic device and medium for associating digital slices with case information, which can automatically associate digital slices and case information, facilitating subsequent consultations or other operations for users.

[0004] To achieve the above purpose, the technical solution adopted by the present invention is as follows: In the first aspect, the present invention provides a system for associating digital slices with case information, including: a data acquisition module for acquiring the digital slices of a patient and the case information of the pathology; a feature extraction module for extracting the slice feature information of the digital slices and the case feature information of the case information based on a preset AI model and / or OCR text recognition technology; a feature matching module for calculating the matching score between the slice feature information and the case feature information, and judging whether the digital slices and the case information match successfully based on the matching score; an association module for associating the successfully matched digital slices and case information.

[0005] Optionally, the above-mentioned association system further includes a storage module. The data acquisition module includes: a digital slice acquisition unit and a case information acquisition unit; the digital slice acquisition unit is used to process the digital slices according to a preset chunking strategy to obtain multiple digital slice chunks, generate an identifier for each digital slice chunk, and upload each digital slice chunk to the storage module in parallel, and verify the integrity of the digital slices based on the identifier; the case information acquisition unit is used to acquire the case information input by the user, or acquire the picture of the pathology report / request form, and recognize the case information in the picture.

[0006] Optionally, a storage module is used to store digital slice blocks in an edge-cloud hybrid storage manner. The storage module includes: a hot data layer for storing digital slice blocks accessed in a first preset time period to an edge node; a warm data layer for storing digital slice blocks in a second preset time period to a regional object storage; and a cold data layer for archiving historical data to a GLACIER storage.

[0007] Optionally, the feature extraction module includes: a first feature extraction unit and a second feature extraction unit. The first feature extraction unit is used to input a digital slice and case information into a preset AI model to obtain slice feature information of the digital slice and case feature information of the case information. The second feature extraction unit is used to obtain a label picture of the digital slice, extract slice feature information from the label picture through OCR character recognition technology, and extract case feature information from the case information through OCR character recognition technology.

[0008] Optionally, the feature matching module is specifically used for: obtaining a weight coefficient of pre-configured feature information, where the feature information at least includes: pathology number, patient number, hospital name, label barcode number, case creation time, pathological specimen name, and patient clinical diagnosis; the slice feature information and case feature information include one or more of the feature information; performing combined matching on the slice feature information and case feature information through multi-field fuzzy matching to obtain combined feature information; calculating a similarity score of the combined feature information, and performing weighted calculation on the similarity score based on the weight coefficient to obtain a matching score of the digital slice and case information; if the matching score exceeds a preset matching score threshold, determining that the digital slice and case information match successfully, and generating a confidence level of the digital slice and case information based on the matching score.

[0009] Optionally, the feature matching module is further used for: inputting the slice feature information and case feature information into an AI model to obtain a matching score of the digital slice and case information; if the matching score exceeds a preset matching score threshold, determining that the digital slice and case information match successfully, and generating a confidence level of the digital slice and case information based on the matching score.

[0010] Optionally, the association module is specifically used for: associating the identifiers of the patient corresponding to the successfully matched digital slice and case information and the identifier of the digital slice, and saving them to a database, and determining an association method, where the association method includes: automatic association and manual association. For automatically associated digital slices and case information, the corresponding matching score and confidence level are displayed on the user's operation page.

[0011] Second aspect, the present invention provides a method for associating digital slices with case information, which is applied to the digital slice and case information association system according to any one of the above first aspects, and includes: obtaining the digital slices of a patient and the case information of the pathology; extracting the slice feature information of the digital slices and the case feature information of the case information based on a preset AI model and / or OCR character recognition technology; calculating the matching score between the slice feature information and the case feature information, and determining whether the digital slices and the case information match successfully based on the matching score; associating the successfully matched digital slices and case information.

[0012] Third aspect, the present invention provides an electronic device, including a processor and a memory, where the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the method provided in the above second aspect.

[0013] Fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, it executes the steps of the method provided in the above second aspect.

[0014] The present invention brings the following beneficial effects: The above digital slice and case information association system, method, electronic device and medium provided by the present invention include: a data acquisition module for acquiring the digital slices of a patient and the case information of the pathology; a feature extraction module for extracting the slice feature information of the digital slices and the case feature information of the case information based on a preset AI model and / or OCR character recognition technology; a feature matching module for calculating the matching score between the slice feature information and the case feature information, and determining whether the digital slices and the case information match successfully based on the matching score; an association module for associating the successfully matched digital slices and case information. The above system can extract the slice feature information of the digital slices and the case feature information of the case information, then perform matching according to the matching score between the slice feature information and the case feature information, and associate the successfully matched digital slices and case information, so as to automatically associate the digital slices and the case information, which is convenient for users to conduct subsequent consultations or other operations; at the same time, it improves the efficiency and accuracy of the association between digital slices and case information.

[0015] Other features and advantages of the present invention will be described in the following specification, and part of them will become obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification, claims and drawings.

[0016] To make the above objectives, features and advantages of the present invention more obvious and understandable, the following specific embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. Brief Description of the Drawings

[0017] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a schematic structural diagram of a digital slice and case information association system provided by an embodiment of the present invention; Figure 2 It is a schematic structural diagram of another digital slice and case information association system provided by an embodiment of the present invention; Figure 3 It is a flowchart of a digital slice and case information association method provided by an embodiment of the present invention; Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed Description of the Embodiments

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions of the present invention with reference to the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0020] Currently, after receiving digital slices and case information, the consultation platform cannot automatically associate them; at the same time, for the pathological diagnosis systems deployed in some small and medium-sized hospitals, digital slices cannot be uploaded to the consultation platform. Instead, physical slices need to be sent by mail and then uploaded to the consultation platform by relevant staff. During this process, the slice information and case information of patients are uploaded separately, and the staff cannot know the corresponding relationship between the slice information and the case information. Moreover, due to the large amount of data, it is unrealistic to perform the association manually.

[0021] Based on this, a digital slice and case information association system, method, electronic device, and medium provided by an embodiment of the present invention can automatically associate digital slices and case information, facilitating subsequent consultation or other operations by users.

[0022] To facilitate the understanding of this embodiment, first, a digital slice and case information association system disclosed in an embodiment of the present invention will be introduced in detail. This system can be applied to consultation platforms, film viewing platforms, etc.

[0023] See Figure 1 The structural schematic diagram of a digital slice and case information association system shown in the figure schematically shows that this posture mainly includes the following parts: The data acquisition module 101 is used to acquire the digital slices of patients and the case information of pathology. In specific implementation, the digital slices can be acquired by scanning the barcodes of pathological slices, or directly acquired from the pathology system; the acquisition of the case information of pathology can be manually input by the user, or directly extracted by docking with the pathology system, or automatically recognized and filled in through relevant photographing OCR such as pathology requisitions / reports.

[0024] The feature extraction module 102 is used to extract the slice feature information of the digital slices and the case feature information of the case information based on a preset AI model and / or OCR character recognition technology. In specific implementation, the slice feature information of the digital slices and the case feature information of the case information can be extracted by using the preset AI model, or the slice feature information in the label pictures of the digital slices can be recognized through the OCR character recognition technology, and the case feature information of the case information can be recognized through the OCR character recognition technology.

[0025] The feature matching module 103 is used to calculate the matching score between the slice feature information and the case feature information, and determine whether the digital slices and the case information match successfully based on the matching score. In specific implementation, the cosine similarity algorithm can be used to calculate the matching score between the slice feature information and the case feature information, or an AI model can be used to calculate the matching score between the slice feature information and the case feature information, and then it is determined whether the digital slices and the case information match successfully according to the matching score. If the matching score is greater than the matching score threshold, it is determined that the digital slices and the case information match successfully.

[0026] The association module 104 is used to associate the successfully matched digital slices and case information.

[0027] The above digital slice and case information association system provided by the embodiments of the present invention can extract the slice feature information of the digital slices and the case feature information of the case information, then perform matching according to the matching score between the slice feature information and the case feature information, and associate the successfully matched digital slices and case information, so that the digital slices and the case information can be automatically associated, which is convenient for users to conduct consultations or other operations in the future; at the same time, the efficiency and accuracy of the association between the digital slices and the case information are improved.

[0028] In an implementation manner, see Figure 2 As shown, the above association system further includes a storage module 105. The data acquisition module 101 includes: a digital slice acquisition unit and a case information acquisition unit.

[0029] Among them, the digital slice acquisition unit is used to process digital slices according to a preset chunking strategy to obtain multiple digital slice chunks, generate identifiers for each digital slice chunk, and upload each digital slice chunk to the storage module in parallel, and verify the integrity of the digital slices based on the identifiers.

[0030] In specific implementation, considering that digital slices occupy a large amount of memory, in order to improve the upload efficiency, in the embodiments of the present invention, a chunking strategy can be adopted for uploading digital slices. When scanning a new slice, the digital slice is segmented to support resume from breakpoint. For example: according to the percentage strategy, a breakpoint is set every 5%. When there are problems such as network fluctuations or storage, it will not affect the already uploaded part.

[0031] Specifically, in this embodiment, an adaptive chunking algorithm is adopted to adjust the chunk size in real time according to the network bandwidth (for example, initially chunked by 5%, and switched to a fixed 5MB / chunk when there is network fluctuation). Before uploading, the digital slice (such as TIFF) is automatically converted to a format that supports chunk compression (JPEG 2000), and multiple digital slice chunks are uploaded simultaneously through multi-thread / coroutine technology, and the throughput is improved by using the multiplexing feature of the HTTP / 3 protocol.

[0032] In order to achieve resume from breakpoint, in the embodiments of the present invention, a unique identifier (GUID) can also be assigned to each digital slice chunk, and the chunk status of each digital slice chunk is recorded, such as: uploaded: [0-5%, 5-10%], to be resumed: 10%-100%]. When the upload fails, the exponential backoff algorithm can be used for failure retry, that is, wait for 1 second for the first failure and retry, and then extend to 5 seconds and 10 seconds in sequence to avoid the avalanche effect.

[0033] Furthermore, when receiving digital slice chunks, in the embodiments of the present invention, the integrity of the chunks can be verified in real time according to the identifiers of the digital slice chunks (such as: using CRC32 check). If an incorrect chunk is encountered, the client is triggered to automatically clean up the chunk and regenerate it.

[0034] In addition, in this embodiment, the file space can be pre-declared in the object storage (MinIO) before uploading to avoid affecting performance due to multiple IO operations, and the server-side merge interface is called after all chunks arrive, and the memory mapping (Memory-MappedFile) technology is used to achieve second-level recombination to avoid the disk IO bottleneck.

[0035] The case information acquisition unit is used to obtain the case information input by the user, or obtain the pictures of the pathology report / requisition form, and identify the case information in the pictures.

[0036] In specific implementation, pathological case information can be obtained through multiple channels, including but not limited to: manual entry by users, direct extraction by docking with the pathological system, automatic recognition and filling in of relevant photos such as pathological requisition forms / reports through OCR, and direct extraction by docking with the remote consultation platform.

[0037] Among them, for the automated recognition technology, in the embodiments of the present invention, an Azure Form Recognizer custom model can be used to recognize the key fields of the requisition form (such as clinical diagnosis, sampling site), and a Transformer-HMM hybrid model can also be used to parse the doctor's handwritten annotations, supporting cursive writing and medical abbreviations (such as "CA" → cancer).

[0038] In order to improve the accuracy of information recognition and facilitate subsequent processing, in this embodiment, a standardization engine can be used to process the recognized data, including but not limited to: converting free text into standard codes (such as ICD-O-3 morphological codes) based on the UMLS knowledge graph; automatically converting different unit systems using dimensional analysis algorithms (such as "5cm" → "50mm"); verifying the rationality of numerical values through a rule engine (such as "male patient, endometrial section" triggering manual review); and attaching quadruple metadata (source system, collection time, operator, verification code) to each data item.

[0039] Since the digital slices are uploaded in segmented form during upload, marginalized segmented storage can be adopted according to the sliced segmented upload method. Based on this, in this embodiment, a storage module is used to store the digital slice blocks in an edge-cloud hybrid storage manner; among them, the storage module includes: a hot data layer for storing the digital slice blocks accessed in the first preset time period to the edge node; a warm data layer for storing the digital slice blocks in the second preset time period to the regional-level object storage; and a cold data layer for archiving historical data to GLACIER storage.

[0040] In specific implementation, the edge-cloud hybrid storage architecture includes: a hot data layer that stores the digital slice blocks accessed in the first preset time period to the edge node. Specifically, the slice blocks accessed in the most recent 7 days are stored in the edge node (such as the local NAS of the hospital), with a latency <5ms; a warm data layer that stores the digital slice blocks in the second preset time period to the regional-level object storage. Specifically, the slices in the most recent 3 months are stored in the regional-level object storage (Ceph cluster), supporting cross-hospital synchronization; a cold data layer that archives historical data to GLACIER storage. Specifically, the historical data is archived to GLACIER storage, using erasure coding (EC 12+4).

[0041] Further, in this embodiment, a geographical sharding strategy can be adopted to store multiple chunks of each digital slice in different availability zones (AWS us-east-1a / 1b / 1c), and the CRUSH algorithm is used to ensure the alignment of the chunks with the computing node topology; meanwhile, the Redis cluster is used to store the chunk mapping table (including MD5, storage location, access count), and the Raft protocol is adopted to ensure strong consistency of the metadata, supporting 100,000 queries per second.

[0042] In one implementation manner, the above-mentioned feature extraction module 102 includes: a first feature extraction unit and a second feature extraction unit.

[0043] The first feature extraction unit is used to input the digital slice and the case information into a preset AI model to obtain the slice feature information of the digital slice and the case feature information of the case information.

[0044] In specific implementation, the first feature extraction unit is used to perform feature extraction through the AI model. The AI model is not limited to digital slices only, and can also integrate, analyze, and extract case information, so as to perform matching and association in combination with the information of digital slices (such as: when the medical history in the case information shows long-term smoking, then the disordered arrangement of cilia of ciliated cells should be visible in the digital slices of lung slices).

[0045] The steps for the AI model to extract the slice feature information are as follows: 1. Use OpenSlide to read multi-level resolutions; 2. Tissue area detection (removing blank areas); 3. Color normalization (solving staining differences); 4. Generate a chunking strategy; 5. Perform tissue segmentation according to the chunking strategy; 6. Calculate the stroma / epithelium ratio; 7. Measure the vascular density; 8. Nucleus segmentation; 9. Extract nuclear morphological features; 10. Analyze the nuclear arrangement directionality; 11. Locate protein expression; 12. Identify the membrane / cytoplasm / nucleus distribution pattern.

[0046] In specific implementation, first, preprocessing operations such as denoising, contrast enhancement, and color correction are performed on digital slices to improve image quality. Second, according to the requirements of pathological diagnosis, regions of interest (ROIs) are selected for feature extraction, and manual annotation tools or automatic segmentation algorithms can be used to select ROIs. After that, morphological features of cells, such as cell size, shape, texture, etc., can be calculated using image processing algorithms; texture features of the image, such as gray-level co-occurrence matrix (GLCM), local binary pattern (LBP), etc., which can describe the local texture information of the image; color features of the image, such as color histogram, color moment, etc., which can describe the color distribution information of the image; geometric features of the image, such as area, perimeter, circularity, etc., which can describe the geometric shape information of the image. Finally, statistical methods (such as t-test, ANOVA, etc.) are used to screen out features with significant differences, and dimensionality reduction methods (such as principal component analysis PCA, linear discriminant analysis LDA, etc.) are used to reduce the feature dimension and improve the generalization ability of the model.

[0047] When the AI model extracts case feature information, it can match the corresponding feature information from the pathological information according to the pre-set fields, such as: pathology number, patient number, hospital name, etc.

[0048] The second feature extraction unit is used to obtain the label picture of the digital slice, extract the slice feature information from the label picture through OCR text recognition technology, and extract the case feature information from the case information through OCR text recognition technology.

[0049] In specific implementation, the second feature extraction unit is used to extract the slice feature information and the case feature information through OCR text recognition technology. Specifically, when extracting the slice feature information, the label picture of the digital slice can be obtained first. The label picture includes information such as pathology number, patient number, hospital name, pathological specimen name, barcode, etc., and relevant information is extracted from it through OCR text recognition technology as the slice feature information. When extracting the case feature information, information such as pathology number, patient number, hospital name, label barcode number, case creation time, pathological specimen name, patient's clinical diagnosis, etc. are extracted from the case information through OCR text recognition technology as the case feature information.

[0050] In the embodiments of the present invention, the above-mentioned feature extraction module 102 may further include a third feature extraction module, which is used to extract feature information through a pre-trained feature extraction model, such as a convolutional neural network (CNN) or a recurrent neural network (RNN). In specific implementation, model training can be carried out by providing a large number of digital slice annotation sets, and model training is carried out in the case of supervised learning. Feature extraction is carried out in a manner configured according to configuration items to extract fixed-position or recognition-position features. Specifically, model training includes but is not limited to the following steps: 1. Data preparation: Collect a large number of matching case information and corresponding pathological slices, and annotate the ground truth (such as the final diagnosis result).

[0051] 2. Training steps: Step 1: Separate pre-training.

[0052] Slice model: Use a large number of unannotated slices to learn basic pathological features; Case model: Use medical texts to train semantic understanding.

[0053] Step 2: Joint fine-tuning.

[0054] Input paired data (case + slice), and set two loss functions, including: Diagnostic accuracy: The prediction result is successfully bound; Feature relevance: The slice features of similar cases are closer in the vector space.

[0055] For example: When the input case is "Male, 65 years old, smoking for 40 years, hemoptysis for 2 months"; the model will: ① Focus on the ciliated cell area in the lung slice; ② Automatically calculate indicators such as the proportion of enlarged cell nuclei; ③ Output the percentage of "relevance 85%".

[0056] Step 3: Feature extraction.

[0057] Pathological slice part: Input the digital slice into the deep learning model, and the model automatically identifies important features: such as cell shape, tissue arrangement pattern, special protein distribution, etc., and outputs the identified feature description.

[0058] Case information part: Use a simple neural network to process structured data (such as clinical diagnosis, gross findings), and finally merge them into a comprehensive case feature vector.

[0059] Step 4: Feature binding.

[0060] Let the model know which slice features correspond to which case features through contrastive learning. For example: When "smoking for 40 years" appears in the case, the model will strengthen the weight of the cilia disorder feature in the lung slice.

[0061] In one implementation, the above-mentioned feature matching module 103 is specifically configured to: obtain the weight coefficients of the pre-configured feature information; wherein, the feature information at least includes: pathology number, patient number, hospital name, label barcode number, case creation time, pathology specimen name, patient clinical diagnosis; the slice feature information and the case feature information include one or more of the feature information; perform combined matching on the slice feature information and the case feature information through multi-field fuzzy matching to obtain combined feature information; calculate the similarity score of the combined feature information, and perform weighted calculation on the similarity score based on the weight coefficients to obtain the matching score of the digital slice and the case information; if the matching score exceeds the preset matching score threshold, it is determined that the digital slice and the case information match successfully, and a confidence level of the digital slice and the case information is generated based on the matching score.

[0062] In specific implementation, the feature matching module 103 can perform combined matching according to the pre-set matching rules based on the feature information such as the pathology number, patient number, hospital name, label barcode number, etc. that have been extracted or recognized and read. The specific steps are as follows: First, extract metadata such as pathology number, patient number, hospital name, label barcode number, etc. from the digital slice, and transmit the data back to the server data center in real time through the HTTPS secure channel.

[0063] Second, the server dynamically loads the weight coefficients of the feature information including pathology number, patient number, hospital name, label barcode number, case creation time, etc. In this embodiment, it supports customizing the weight coefficients according to the pathology number / patient number.

[0064] Then, perform multi-field fuzzy matching based on the SQL Server 2022 + full-text search engine (such as: matching the pathology number in the feature information of the strange heat slice with the pathology number in the case feature information as a combined feature), and calculate the matching score (0 - 1000 points) using the cosine similarity algorithm; at the same time, this embodiment also supports dynamic weight superposition calculation (such as: case creation time coefficient × 0.8 - 1.2).

[0065] After that, obtain the preset matching score threshold (such as: 850 points). In this embodiment, it supports setting different thresholds according to the machine / disease type. If the matching score exceeds the matching score threshold, the matching is successful; if it is lower than the matching score threshold, the manual review process is triggered.

[0066] Finally, return the matching score and the confidence level (high (such as: greater than 950 points) / medium (such as: 900 - 950 points) / low (such as 850 - 900 points)) in real time, and generate the matching details and record them in the database (including the matching contribution degree of each field).

[0067] Further, the feature matching module 103 is further configured to: input the slice feature information and the case feature information into the AI model to obtain the matching score of the digital slice and the case information; if the matching score exceeds the preset matching score threshold, determine that the digital slice and the case information match successfully, and generate the confidence level of the digital slice and the case information based on the matching score.

[0068] In specific implementation, the matching score of the digital slice and the case information can also be calculated by using the AI model. If the matching score exceeds the preset matching score threshold, it is determined that the digital slice and the case information match successfully, and the confidence level of the digital slice and the case information is generated based on the matching score.

[0069] Illustrate with examples: The digital slice feature extraction by the AI model ---- The digital slice feature extraction results include: the cilia arrangement of ciliated cells is disordered, and the cell nucleus is enlarged; The digital slice OCR feature extraction ---- The digital slice feature extraction results include: Pathology number: 100297851; The digital slice OCR feature extraction ---- The digital slice feature extraction results include: Hospital area: South Hospital of a certain hospital; The case information feature extraction ---- The patient's clinical diagnosis: Lung cancer; The case information feature extraction ---- Pathology specimen name: Lung mass; The case information feature extraction ---- Pathology number: 100297851; The case information feature extraction ---- Hospital area: South Hospital of a certain hospital; When all information matches successfully, read the weight coefficients of each feature information, and calculate the matching score and confidence level: 950 points, confidence level: high; When only the AI feature extraction results match, read the weight coefficients of each feature, and calculate the matching score and confidence level: 850 points, confidence level: medium.

[0070] In one implementation manner, the above-mentioned association module 104 is specifically configured to: associate the identifiers of the patient corresponding to the successfully matched digital slice and case information and the identifier of the digital slice, and then save them to the database, and determine the association method; wherein, the association method includes: automatic association and manual association. For the automatically associated digital slice and case information, the corresponding matching score and confidence level are displayed on the user's operation page.

[0071] In specific implementation, after successful matching, the association relationship between the patient's identifier and the identifier of the digital slice is recorded in the database, and it is written whether it is automatically associated or manually associated. If it is automatically associated, the matching score and confidence level are displayed in the user's operation window, and the matching score and confidence level can be seen every time the user opens the digital slice. For the association operation with a low confidence level, manual review is required.

[0072] If an error occurs in the association with a high confidence level, it means that the weight coefficient of the rule algorithm is incorrect and needs to be adjusted. The user can perform a replacement association operation on the digital slice that has been successfully associated and wrongly associated in the interface.

[0073] Considering that the scanning and slicing quality will affect the overall effect of AI feature extraction, in the implementation of the present invention, after obtaining the digital slice and the label image of the digital slice, the quality of the digital slice and the label image can also be evaluated first, and the weight coefficient of the slice feature information is adjusted according to the evaluation result.

[0074] In the embodiment of the present invention, the quality of the digital slice and the label image can be evaluated from the following several dimensions: Clarity: Whether the image is clear and free of blur; Contrast: Whether the contrast of the image is appropriate and the details are visible; No noise: Whether the image contains noise, such as spots, stripes, etc.; Integrity: Whether the image is complete and without missing or damaged parts; Label accuracy: Whether the label accurately reflects the image content.

[0075] Specifically, the digital slice and the label image are preprocessed first, including but not limited to: Format unification, ensuring that all image formats are the same, such as PNG, JPEG, etc.; Size adjustment, adjusting the image size according to needs to ensure consistency; Color correction, performing color correction on the image to reduce the influence of device or environmental factors on the quality.

[0076] Then, quality assessment is automatically performed. For each quality dimension, relevant image features are extracted, including: (1) Sharpness feature: The edge detection algorithm (such as Sobel, Canny edge detection) can be used to evaluate the image sharpness and extract edge information, or the gradient distribution of the image can be calculated to analyze the sharpness; (2) Contrast feature: Calculate the contrast value of the image, such as using histogram analysis for contrast; (3) Noise feature: The noise detection algorithm can be used, such as residual analysis after median filtering for noise; (4) Integrity feature: The size and structure of the image file can be checked to confirm no damage; (5) Color accuracy feature: Extract the color histogram or color moment of the image; (6) Label accuracy feature: The machine learning model (such as a classifier) can be used to verify the accuracy of the label, or calculate the intersection over union ratio of the label and the actual situation, and evaluate the label accuracy according to the intersection over union ratio. In addition, the image can be observed manually to check the consistency between the label and the image content to check for problems that may be missed by the automatic evaluation.

[0077] Finally, weights are assigned to each feature, and the comprehensive quality score is calculated based on the features. The quality score of each image is recorded in the database or file. Among them, calculating the comprehensive quality score includes the following steps: (1) Standardization: Normalize each feature value to the same scale, such as between 0 and 1. (2) Weighted summation: Calculate the weighted summation according to the weights of each feature to obtain the comprehensive quality score. (3) Score normalization: Normalize the comprehensive quality score to a specific range, such as 0 to 100.

[0078] Furthermore, the weight coefficient of the slice feature information is adjusted according to the comprehensive quality score. Specifically, the corresponding relationship between the comprehensive quality score and the weight coefficient can be set, and the weight coefficient is adjusted according to the corresponding relationship.

[0079] Furthermore, unassigned digital slices are allowed to exist in the system, which may be teaching slices or orphan slices. After being marked as teaching slices or orphan slices in the association status, they will no longer participate in the association matching. Otherwise, all unmatched digital slices will automatically enter the loop matching and association program. There is an associated field in the digital slice table and the binding relationship table to judge whether this field is unmatched, so as to distinguish whether the digital slice is associated with a case.

[0080] In the embodiment of the present invention, for a digital slice with a lost slice label, the digital slice can be scanned into the system to extract the slice feature information, and then automatic matching is performed first, that is, to judge whether there is a matching case information in the database (that is, to match the orphan case information and the orphan slice). Specifically, it can be judged whether the case description in the case information conforms to the slice feature information for matching. If it conforms, the matching is successful and the association is made. Otherwise, it is marked as an orphan slice.

[0081] In the embodiments of the present invention, sensitive information is encrypted and decrypted during feature extraction and when transmitting interactive information between the client and the server. The main information such as patient names and ID numbers is masked in real time through a dynamic desensitization algorithm to ensure that even if intercepted during the transmission process, sensitive information will not be leaked. The system adopts dynamic key rotation + sharding encryption, taking into account both security and performance. Specifically, a hierarchical encryption strategy can be adopted, including: Field-level dynamic desensitization: The ID number (18 digits / ending with X) and mobile phone number (starting with 1 and 11 digits) are matched in real time through a regular expression engine and replaced with ***; NLP entity recognition: Entities such as addresses and names in the report text are recognized through the BERT-BiLSTM model and replaced with hash values (such as SHA3-512); ROI area encryption: Pixel-level AES-CTR encryption is performed on sensitive areas such as label information areas in digital slices; Quantum-secure hybrid encryption for the transmission channel: X25519 key exchange is used in the TLS 1.3 handshake phase, and Kyber-1024+ChaCha20-Poly1305 is used for the data channel; Deploy a national cryptography dual stack based on OpenSSL (supporting both SM2 / SM4 and RSA / AES simultaneously) and integrate the WebCrypto API on the client to implement browser-side encryption.

[0082] In the embodiments of the present invention, in order to further ensure the security of data transmission, the following measures can also be adopted for the remote consultation system to ensure the security of data during transmission: (1) Use an encryption protocol.

[0083] Specifically, the HTTPS (HTTP Secure) protocol can be used to encrypt the communication between the client and the server. The HTTPS protocol uses the SSL / TLS protocol at the transport layer to encrypt HTTP requests and responses to ensure that data is not eavesdropped on or tampered with during transmission; configure the SSL / TLS certificate to ensure that strong encryption algorithms are used for data encryption during transmission. Select a suitable encryption suite and avoid using known weak encryption algorithms.

[0084] (2) Data encryption.

[0085] Specifically, during data transmission, symmetric encryption (such as AES) or asymmetric encryption (such as RSA) algorithms are used to encrypt the data to ensure that the data is not eavesdropped on or tampered with during transmission. When storing data, sensitive data is encrypted to ensure that the data is not accessed without authorization during storage. The built-in encryption function of the database or a third-party encryption tool can be used.

[0086] (3) Authentication and authorization.

[0087] Specifically, use a strong password policy that requires users to set complex passwords and change them regularly. Multi-factor authentication (such as SMS verification codes, email verification codes, etc.) can be used to enhance the security of user authentication; implement role-based access control (RBAC) to ensure that only authorized users can access sensitive data and functions. Classify users and assign different access permissions to different categories of users.

[0088] (4) Data integrity verification.

[0089] Specifically, a hash algorithm (such as SHA-256) can be used to calculate the hash of the data to generate a data digest. During data transmission, the data digest is transmitted together with the data, and the receiving party verifies the integrity of the data by comparing the data digest. Digital signature technology can also be used to sign the data to ensure the integrity and authenticity of the data. Digital signatures are based on asymmetric encryption algorithms, and only the holder of the private key can generate a digital signature, while the holder of the public key can verify the validity of the digital signature.

[0090] (5) Firewall and intrusion detection system.

[0091] Specifically, configure firewall rules to restrict access to the server and only allow authorized IP addresses and ports to communicate. The firewall can effectively prevent unauthorized access and attacks. Deploy an intrusion detection system to monitor network traffic in real time, detect and prevent potential attack behaviors. The IDS can identify known attack patterns and issue alerts in a timely manner.

[0092] (6) Security auditing and logging.

[0093] Specifically, record the operation logs of all users, including login time, operation content, operation results, etc. Logging can help track and audit user behaviors, and discover and resolve security issues in a timely manner. At the same time, conduct regular security audits to check the security configuration and logging of the system, and discover and fix potential security vulnerabilities.

[0094] (7) Regular updates and patch management.

[0095] Specifically, regularly update software such as the operating system, database, and applications to ensure the use of the latest security patches. Software updates can fix known security vulnerabilities and improve the security of the system. A patch management process can also be established to install security patches in a timely manner to ensure the security of the system.

[0096] Through the above measures and technologies, the security of data transmission in the remote consultation system can be effectively ensured, protecting the privacy of patients and the security of data.

[0097] In the embodiments of the present invention, when performing feature matching, the following methods may be adopted, including but not limited to: (1) Feature matching based on deep learning.

[0098] Use a deep learning model (such as a convolutional neural network CNN) to extract the image features of digital slices, and combine natural language processing (NLP) technology to extract the key features in the case text information.

[0099] (2) Multimodal feature fusion.

[0100] Fuse the image features of digital slices with the case text features, for example, by means of feature concatenation or splicing, to construct a unified multimodal feature space. Multimodal feature fusion can comprehensively utilize different types of data and enhance the robustness and comprehensiveness of matching.

[0101] (3) Semantic matching and graph matching algorithms.

[0102] Use semantic matching technology to analyze the semantic relationship between digital slices and case information, or construct an association model through graph matching algorithms. Semantic matching can understand the deep connections behind the data, while graph matching algorithms are suitable for modeling complex relationships.

[0103] (4) Similarity measurement methods.

[0104] Adopt more advanced similarity measurement methods, such as cosine similarity, edit distance, or similarity measurement methods based on deep learning, which can more accurately evaluate the similarity between different data, thereby improving the matching efficiency. In the matching of pathological images and case information, using similarity measurement methods based on deep learning can significantly improve the accuracy of matching.

[0105] (5) Dynamic matching strategy.

[0106] Dynamically adjust the matching strategy according to real-time data changes, for example, combine information such as the patient's medical history and treatment progress to optimize the matching results. The dynamic matching strategy can adapt to the continuously changing medical data and ensure the timeliness and accuracy of the association results.

[0107] The above system provided by the embodiments of the present invention can automatically associate digital slices and case information, facilitating subsequent consultation or other operations by users; at the same time, it avoids the situation where misdiagnosis may be caused by incorrect association.

[0108] For the digital slice and case information association system provided in the foregoing embodiments, the embodiments of the present invention also provide a digital slice and case information association method. Refer to Figure 3 The flowchart of a digital slice and case information association method shown, which shows that the method mainly includes the following steps S301 to step S304: Step S301: Obtain the digital slices of the patient and the case information of the pathology.

[0109] Step S302: Extract the slice feature information of the digital slices and the case feature information of the case information based on a preset AI model and / or OCR character recognition technology.

[0110] Step S303: Calculate the matching score between the slice feature information and the case feature information, and determine whether the digital slices and the case information match successfully based on the matching score.

[0111] Step S304: Associate the successfully matched digital slices and case information.

[0112] The above method for associating digital slices and case information provided by the embodiments of the present invention can extract the slice feature information of digital slices and the case feature information of case information, then perform matching according to the matching score between the slice feature information and the case feature information, and associate the successfully matched digital slices and case information, so as to automatically associate digital slices and case information, facilitating subsequent consultation or other operations by users; at the same time, it improves the efficiency and accuracy of associating digital slices and case information.

[0113] It should be noted that for the method provided by the embodiments of the present invention, its implementation principle and the technical effects produced are the same as those of the foregoing system embodiments. For the sake of brief description, for the parts not mentioned in the method embodiments, reference may be made to the corresponding content in the foregoing system embodiments. The specific numerical values provided in the embodiments of the present invention are only exemplary and are not limited herein.

[0114] The embodiments of the present invention also provide an electronic device. Specifically, the electronic device includes a processor and a storage device; a computer program is stored on the storage device, and the computer program, when run by the processor, executes the method according to any one of the above embodiments.

[0115] Figure 4 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. The electronic device 100 includes: a processor 40, a memory 41, a bus 42, and a communication interface 43. The processor 40, the communication interface 43, and the memory 41 are connected through the bus 42; the processor 40 is used to execute an executable module stored in the memory 41, such as a computer program.

[0116] Among them, the memory 41 may include high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk memory. The communication connection between this system network element and at least one other network element is realized through at least one communication interface 43 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0117] The bus 42 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 4 only a bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0118] Among them, the memory 41 is used to store programs. After receiving an execution instruction, the processor 40 executes the program. The methods executed by the device defined by the flow process disclosed in any embodiment of the foregoing embodiments of the present invention can be applied to the processor 40 or implemented by the processor 40.

[0119] The processor 40 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 40 or by instructions in software form. The above-mentioned processor 40 can be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it can also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed and completed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, register, etc. This storage medium is located in the memory 41, and the processor 40 reads the information in the memory 41 and combines its hardware to complete the steps of the above method.

[0120] The computer program product of the readable storage medium provided by the embodiments of the present invention includes a computer-readable storage medium storing program codes. The instructions included in the program codes can be used to execute the methods described in the foregoing method embodiments. For specific implementation, reference can be made to the foregoing method embodiments, which will not be elaborated herein.

[0121] If the above functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, etc., which can store program codes.

[0122] Finally, it should be noted that the above embodiments are only specific implementation manners of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A system for associating digital slices with case information, characterized in that: include: A data acquisition module is used to obtain digital slices and pathological case information of patients; A feature extraction module, used to extract slice feature information of the digital slice and case feature information of the case information based on a preset AI model and / or OCR text recognition technology; wherein the feature extraction module includes: a first feature extraction unit and a second feature extraction unit; the first feature extraction unit is used to input the digital slice and the case information into a preset AI model to obtain the slice feature information of the digital slice and the case feature information of the case information; the second feature extraction unit is used to obtain a label image of the digital slice, and extract slice feature information from the label image through OCR text recognition technology, and extract case feature information from the case information through OCR text recognition technology; A feature matching module, used to calculate a matching score between the slice feature information and the case feature information, and determine whether the digital slice and the case information are matched successfully based on the matching score; The association module is used to associate the successfully matched digital slices with the case information.

2. The association system according to claim 1, characterized in that: The association system further includes a storage module, and the data acquisition module includes: a digital slice acquisition unit and a case information acquisition unit; The digital slice acquisition unit is used to process the digital slice according to a preset block strategy to obtain multiple digital slice blocks, generate an identifier for each of the digital slice blocks, upload each of the digital slice blocks to the storage module in parallel, and verify the integrity of the digital slice based on the identifier; The case information acquisition unit is used to acquire case information input by a user, or acquire a picture of a pathology report / application form, and identify the case information in the picture.

3. The association system according to claim 2, characterized in that: The storage module is used to store the digital slice blocks in an edge-cloud hybrid storage manner; wherein the storage module includes: a hot data layer, used to store the digital slice blocks accessed in a first preset time period to an edge node; a warm data layer, used to store the digital slice blocks in a second preset time period to a regional object storage; and a cold data layer, used to archive historical data to GLACIER storage.

4. The association system according to claim 1, characterized in that: The feature matching module is specifically used to: obtain the weight coefficient of pre-configured feature information; wherein the feature information at least includes: pathology number, patient number, hospital name, label barcode number, case creation time, pathology specimen name, patient clinical diagnosis; the slice feature information and the case feature information include one or more of the feature information; The slice feature information and the case feature information are combined and matched by multi-field fuzzy matching to obtain combined feature information; Calculating a similarity score of the combined feature information, and performing weighted calculation on the similarity score based on the weight coefficient to obtain a matching score between the digital slice and the case information; If the matching score exceeds a preset matching score threshold, it is determined that the digital slice and the case information are matched successfully, and a confidence level of the digital slice and the case information is generated based on the matching score.

5. The association system according to claim 4, characterized in that: The feature matching module is also used for: Inputting the slice feature information and the case feature information into the AI ​​model to obtain a matching score between the digital slice and the case information; If the matching score exceeds a preset matching score threshold, it is determined that the digital slice and the case information are matched successfully, and a confidence level of the digital slice and the case information is generated based on the matching score.

6. The association system according to claim 1, characterized in that: The association module is specifically used to: associate the patient identifier and the identifier of the digital slice corresponding to the successfully matched digital slice and the case information and save them in a database, and determine the association method; wherein the association method includes: automatic association and manual association, and for the automatically associated digital slice and the case information, the corresponding matching score and confidence level are displayed on the user's operation page.

7. A method for associating digital slices with case information, characterized in that: The system for associating digital slices with case information as claimed in any one of claims 1 to 6 comprises: Obtain patient digital slides and pathology case information; Extracting slice feature information of the digital slice and case feature information of the case information based on a preset AI model and / or OCR text recognition technology; Calculating a matching score between the slice feature information and the case feature information, and judging whether the digital slice and the case information are matched successfully based on the matching score; The successfully matched digital slices are associated with the case information.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the method according to claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 7 are performed.

Images