Self-inspection method and system for medical units based on data analysis

Abstract

The present application discloses a self-inspection method and system for medical units based on data analysis, belonging to the field of information planning technology. The method includes: receiving and responding to a self-inspection request, retrieving multimodal data of a target medical unit, the multimodal data including text data, image data, sound data, and time series data; performing data processing and data fusion on the text data, image data, sound data, and time series data to obtain medical data; inputting the medical data into a self-inspection detection model, analyzing and detecting the medical data through the self-inspection detection model, and obtaining detection results output by the self-inspection detection model; analyzing the detection results, generating an assessment result of the target medical unit and matching it with rectification suggestions, and obtaining a self-inspection report of the target medical unit. This method can fully utilize the potential information in the data to achieve a comprehensive, personalized, and accurate assessment of the medical unit.

Description

Technical Field

[0001] The present application belongs to the field of information planning technology, and in particular relates to a self-inspection method and system for medical units based on data analysis. Background Art

[0002] Traditional self-inspections at medical institutions typically rely on manual inspections and expert evaluations, which can be inefficient and subjective. With the rapid development of information technology, artificial intelligence, machine learning, and big data analytics have been widely used in the medical industry, making it possible to apply automated analysis technology to self-inspections at medical institutions.

[0003] However, the medical environment is highly complex and diverse, with significant differences between different hospitals, departments, patients, and treatment situations. The amount of data generated by medical institutions is huge and complex, and different data types have different characteristics and structures. Existing medical self-inspection methods rely on rules and fixed standards to evaluate the behavior of medical staff, but the quality of medical care does not only depend on the behavior of medical staff, but also involves multiple factors such as the patient's treatment effect, the hospital's resource allocation, and the condition of the equipment. The self-inspection report obtained may not be able to adapt to personalized needs, and may therefore miss potential problems or be inapplicable to certain specific scenarios, and cannot accurately reflect the defects of the medical unit.

[0004] There is an urgent need for a systematic integration and analysis method to fully utilize the potential information in the data and achieve a comprehensive, personalized and accurate evaluation of medical units. Summary of the Invention

[0005] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a self-inspection method and system for medical institutions based on data analysis, which can fully utilize the potential information in the data to achieve a comprehensive, personalized and accurate evaluation of the medical institution.

[0006] In a first aspect, the present application provides a self-inspection method for a medical unit based on data analysis, the method comprising:

[0007] Receive and respond to self-inspection requests, and retrieve multimodal data of the target medical unit, wherein the multimodal data includes text data, image data, sound data, and time series data;

[0008] Performing data processing and data fusion on the text data, the image data, the sound data, and the time series data to obtain medical data;

[0009] Inputting the medical data into a self-checking detection model, analyzing and detecting the medical data through the self-checking detection model, and obtaining a detection result output by the self-checking detection model, wherein the self-checking detection model is constructed based on the large model;

[0010] The test results are analyzed to generate an assessment result of the target medical unit and match it with rectification suggestions to obtain a self-inspection report of the target medical unit.

[0011] According to one embodiment of the present application, data processing and data fusion are performed on the text data, the image data, the sound data, and the time series data to obtain medical data, including:

[0012] Based on the data modality, respectively performing data cleaning and standardization processing on the text data, the image data, the sound data, and the time series data;

[0013] dividing the processed text data, the image data, the sound data, and the time series data into patient data and medical operation data;

[0014] Time-aligning and data-correlating the patient data and the medical operation data to obtain correlation information;

[0015] Feature vectors are extracted from the associated information and data fusion is performed according to the hierarchy to obtain the medical data.

[0016] According to one embodiment of the present application, the patient data and the medical operation data carry a patient ID, and the patient data and the medical operation data are time-aligned and data-associated to obtain association information, including:

[0017] Performing time stamp alignment and frequency synchronization on the patient data and the medical operation data;

[0018] Based on the patient ID, the synchronized patient data and medical operation data are mapped and associated to obtain associated information.

[0019] According to one embodiment of the present application, extracting feature vectors from the associated information and performing data fusion according to the hierarchy to obtain the medical data includes:

[0020] determining structured data and unstructured data in the associated information;

[0021] Extracting a first feature vector from the structured data, and extracting a second feature vector from the unstructured data based on a data modality of the unstructured data;

[0022] Performing a first-layer fusion of the drug features of the drug treatment and the patient information features in the first feature vector to obtain a first fused feature;

[0023] Performing a second-layer fusion of the first fused feature and the operation data feature in the second feature vector to obtain a second fused feature;

[0024] The second fusion feature and the multimodal feature in the second feature vector are fused in a third layer to obtain the medical data.

[0025] According to one embodiment of the present application, the self-check detection model includes an input layer, a feature extraction layer, a representation layer, a multimodal fusion layer, a hidden layer, a decision layer, and an output layer connected in sequence. The feature extraction layer is connected to the multimodal fusion layer. The self-check detection model is used to analyze and detect the medical data to obtain a detection result output by the self-check detection model, including:

[0026] encoding the medical data through the input layer;

[0027] Performing feature extraction on the medical data according to data modality by the feature extraction layer to obtain a first feature group;

[0028] Performing a nonlinear transformation on each first feature in the first feature group through the representation layer to generate a corresponding high-dimensional feature representation;

[0029] The first feature groups are concatenated through a multimodal fusion layer to obtain early fusion features, and the high-dimensional feature representations are fused into late fusion features. The early fusion features and the late fusion features are weighted by an attention mechanism to obtain a comprehensive feature representation.

[0030] Performing a nonlinear transformation on the comprehensive feature representation through the hidden layer to obtain a nonlinear feature representation;

[0031] Through the decision layer, pattern recognition is performed on the nonlinear feature representation, a task category is determined and executed, abnormal patterns in the medical data are obtained, and abnormality detection is performed on the nonlinear feature representation to obtain abnormal medical data;

[0032] Through the output layer, the abnormal pattern and abnormal medical data are mapped to the result space, and the detection results are obtained and output.

[0033] According to one embodiment of the present application, the first feature groups are concatenated through a multimodal fusion layer to obtain early fusion features, and the high-dimensional feature representations are fused into late fusion features, including:

[0034] Through a multi-level attention mechanism, the first feature of each data modality is divided according to the modality level and abstraction level to determine the attention weights of low-level features, mid-level features, and high-level features;

[0035] Performing weighted calculation on the low-level features, the mid-level features, and the high-level features according to the attention weights to obtain early fusion features;

[0036] Transferring information between each of the high-dimensional feature representations through a cross-modal graph neural network, and fusing the high-dimensional feature representations based on the interactive relationships between the high-dimensional feature representations;

[0037] The fusion of the high-dimensional feature representations is optimized according to the deep association of the high-dimensional feature representations through a multimodal transformation network to obtain late fusion features.

[0038] According to one embodiment of the present application, a variational autoencoder and a decoder are provided in the hidden layer, and a nonlinear transformation is performed on the comprehensive feature representation through the hidden layer to obtain a nonlinear feature representation, including:

[0039] Mapping the comprehensive feature representation to a latent space by the variational autoencoder, and reconstructing data of the comprehensive feature representation in the latent space by the decoder;

[0040] Determine outliers based on the reconstruction error and probability distribution of each data point during the data reconstruction process;

[0041] Perform time series analysis on the reconstructed data points to determine the data change trend;

[0042] The abnormal value and the change trend are represented as the nonlinear features.

[0043] According to one embodiment of the present application, the reconstruction error R is:

[0044]

[0045] Among them, x is the input data represented by comprehensive features; It is the reconstructed data reconstructed by the decoder from the samples in the latent space.

[0046] According to one embodiment of the present application, the test results include abnormal patterns and abnormal medical data with classification labels. The analysis of the test results to generate the assessment results of the target medical unit and matching rectification suggestions includes:

[0047] Determine the scoring weights of the abnormal pattern and the abnormal medical data according to the classification labels, score the target medical unit, and generate an assessment result;

[0048] Matching the abnormal pattern and abnormal medical data to corresponding patient IDs;

[0049] Generating an exclusive case for the patient data and medical operation data corresponding to the patient ID, and marking the abnormal pattern and related content of the abnormal medical data in the exclusive case;

[0050] Analyze the ongoing medical operation data of the exclusive case and match it with the rectification suggestions.

[0051] In a second aspect, the present application provides a self-inspection system for medical units based on data analysis, which is used to execute the self-inspection method for medical units based on data analysis as described in the first aspect.

[0052] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the self-inspection method for medical units based on data analysis as described in the first aspect above is implemented.

[0053] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the self-inspection method of a medical unit based on data analysis as described in the first aspect above.

[0054] In a fifth aspect, the present application provides a chip comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the self-inspection method of a medical unit based on data analysis as described in the first aspect.

[0055] In a sixth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the self-inspection method of a medical unit based on data analysis as described in the first aspect above.

[0056] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application.

[0057] The present application provides a self-inspection method and system for medical institutions based on data analysis, which has the following beneficial effects compared with the existing technology:

[0058] (1) Through the joint analysis of multimodal data, the potential information in the data can be fully utilized to achieve a comprehensive, personalized and accurate evaluation of medical units, thereby improving the problem detection capability. The self-inspection report obtained is comprehensive and accurate, ensuring that different types of data can be effectively integrated for comprehensive analysis. Medical units can achieve more efficient and accurate self-inspection management, which can not only improve medical quality and work efficiency, but also enhance the early warning and response capabilities of medical units to potential problems.

[0059] (2) By analyzing patient data and medical operation data, the self-inspection report obtained can provide more comprehensive decision-making support for medical institutions, improve the selection of treatment plans and patient health management, optimize the resource allocation of medical institutions, improve the quality of medical services, reduce operating costs, and help hospitals achieve precision medicine. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0061] Figure 1 1 is a flow chart of a self-examination method for a medical unit based on data analysis provided in an embodiment of the present application;

[0062] Figure 2 It is a structural diagram of the self-check detection model provided in the embodiment of the present application;

[0063] Figure 3 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0065] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0066] Below, in combination with the accompanying drawings, the self-inspection method of a medical unit based on data analysis, the self-inspection system of a medical unit based on data analysis, the electronic device and the readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0067] Among them, the self-inspection method of the medical unit based on data analysis can be applied to the terminal, and can be specifically executed by the hardware or software in the terminal.

[0068] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer having a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but a desktop computer having a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0069] In the following embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse, and a joystick.

[0070] The embodiment of the present application provides a self-inspection method for medical units based on data analysis. The execution subject of the self-inspection method for medical units based on data analysis can be an electronic device or a functional module or functional entity in the electronic device that can implement the self-inspection method for medical units based on data analysis. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras and wearable devices, etc. The self-inspection method for medical units based on data analysis provided in the embodiment of the present application is explained below using electronic devices as the execution subject as an example.

[0071] The self-inspection method of a medical unit based on data analysis is applied to a self-inspection system of a medical unit based on data analysis.

[0072] like Figure 1 As shown, the self-inspection method of the medical unit based on data analysis includes:

[0073] Step 110: Receive and respond to a self-inspection request, and retrieve multimodal data of the target medical unit, wherein the multimodal data includes text data, image data, sound data, and time series data;

[0074] Step 120: Processing and fusing the text data, the image data, the sound data, and the time series data to obtain medical data.

[0075] Step 130: Input the medical data into a self-checking detection model, analyze and detect the medical data through the self-checking detection model, and obtain a detection result output by the self-checking detection model, wherein the self-checking detection model is constructed based on the large model;

[0076] Step 140: Analyze the test results, generate an assessment result of the target medical unit and match it with rectification suggestions, and obtain a self-inspection report of the target medical unit.

[0077] To ensure the comprehensiveness and accuracy of data, text data, image data, sound data, and time series data are heterogeneous data from different sources. They can be retrieved from multiple systems such as electronic health records, medical management systems, and drug procurement systems using standardized interfaces such as HL7, FHIR, and DICOM, so as to comprehensively improve the accuracy and reliability of self-inspection and ensure that the data collection process complies with privacy protection and security regulations.

[0078] Among them, the self-inspection detection model is built based on the Billows large model and is trained with data from a large number of medical institutions. The Billows large model can be obtained by integrated learning such as multimodal neural network (MultimodalNeuralNetwork), Random Forest, XGBoost, or joint modeling based on Transformer models such as BERT and T5.

[0079] The self-check detection model has been pre-trained and can analyze historical medical data (labeled data) to identify potential medical problems such as diagnostic errors, non-standard treatment plans, abnormal vital signs, etc.

[0080] In actual implementation, the self-inspection system of a medical unit based on data analysis receives self-inspection requests through interaction with users. The self-inspection requests can be input in the form of voice dialogue or text question and answer. After extracting the keywords of the self-inspection request, the multimodal data related to the self-inspection request is retrieved.

[0081] In some embodiments, step 120, processing and fusing the text data, the image data, the sound data, and the time series data to obtain medical data, includes:

[0082] Based on the data modality, respectively performing data cleaning and standardization processing on the text data, the image data, the sound data, and the time series data;

[0083] dividing the processed text data, the image data, the sound data, and the time series data into patient data and medical operation data;

[0084] Time-aligning and data-correlating the patient data and the medical operation data to obtain correlation information;

[0085] Feature vectors are extracted from the associated information and data fusion is performed according to the hierarchy to obtain the medical data.

[0086] Among them, multimodal data includes text data, image data, sound data, time series data, etc. For example, medical images such as X-ray films, CT scan images, endoscopes, texts such as medical records, laboratory test reports, prescription forms, sounds such as phonocardiograms, lung sound diagrams, and remote diagnosis audio, and time series data such as electrocardiographs, sphygmomanometers, thermometers, pulse oximeters, etc.

[0087] In actual execution, the data modalities of each type of data are different, and the corresponding cleaning and standardization methods are different. It is necessary to determine the type of multimodal data, perform corresponding data cleaning on each type of data, remove useless null values, duplicate records, or error data, and then standardize the cleaned data.

[0088] For text data, remove noise and irrelevant information such as extra spaces, punctuation marks, HTML tags, etc., decompose the text data (such as medical records, doctor's diagnosis reports, etc.) into words or phrases, remove meaningless stop words such as "of", "is", "in", etc., normalize different forms of words, and convert the text content into a unified format through UTF-8 encoding and case conversion.

[0089] For image data, standardize images of different sizes to a unified size, remove noise or interference in the images through median filtering or Gaussian filtering, and adjust the pixel value range of the images to the interval [0, 1].

[0090] For sound data, use a filter to remove background noise, standardize the audio length by cropping or padding with zeros, extract features such as Mel-frequency cepstral coefficients (MFCC) from the original audio, and standardize the features of the audio through zero mean and unit variance.

[0091] For time series data, use methods such as interpolation, mean filling, and previous value filling to fill in the missing values in the time series data, use the three-standard-deviation or isolation forest method to detect and process outliers, and normalize the data from different sensors to the same scale through Min-Max scaling or Z-score standardization.

[0092] Unify text data, image data, sound data, time series data, etc. in terms of time format, data unit, and disease coding for more efficient data analysis.

[0093] In this step, by cleaning, preprocessing, and standardizing the collected multimodal data, ensure high data quality and complete information for subsequent analysis, delete duplicate data, and avoid errors in calculation and analysis.

[0094] The processed multimodal data is divided into patient data including: personal health information, clinical data, patient monitoring data, patient feedback data and patient behavior activity data, etc.; medical operation data includes: medical process records, medical equipment data, nursing and operating room operation data, medical team data and resource call data, etc.

[0095] Multimodal data involves data sources at different time scales. Patient data and medical operation data need to be time-aligned or synchronized based on a reference time point to ensure they are effectively linked across the same time dimension. Resource call data can include information on the hospital's medication and consumable usage, inventory management, and resource allocation. For example, information on medication usage, consumable inventory and usage frequency, and surgical instrument configurations can be found.

[0096] In this embodiment, by analyzing patient data and medical operation data, the self-examination report obtained can provide medical units with more comprehensive decision-making support, improve the selection of treatment plans and patient health management, optimize the resource allocation of medical units, improve the quality of medical services, reduce operating costs, and help hospitals achieve precision medicine.

[0097] In some embodiments, the patient data and the medical operation data carry a patient ID, and the patient data and the medical operation data are time-aligned and data-associated to obtain association information, including:

[0098] Performing time stamp alignment and frequency synchronization on the patient data and the medical operation data;

[0099] Based on the patient ID, the synchronized patient data and medical operation data are mapped and associated to obtain associated information.

[0100] Each patient and medical staff has a unique ID, and each multi-module data carries a timestamp and patient ID. It can associate and match the patient's basic information, medical operation information, laboratory test results and other data with the patient ID and timestamp, determine the association between patient data and medical operation data, and obtain associated information.

[0101] In actual implementation, the timestamp formats of the patient data and the medical operation data from different sources are unified.

[0102] For example, due to the diversity of information systems, some systems store time based on Coordinated Universal Time (UTC), while others use local time zones. Alternatively, the JSON format uses the ISO 8601 format to represent time, for example, 2024-11-20T12:34:56Z, while the CSV format may use date and time separators such as 2024 / 11 / 20 12:34:56 or 2024-11-20 12:34:56, using UTC as the base time and the ISO 8601 format as a unified format to convert the patient data and the medical operation data into different formats.

[0103] Among them, the ISO 8601 format can clearly express date, time, and time zone information, and has strong cross-platform compatibility.

[0104] Different data may be collected at different frequencies, and the data may need to be frequency synchronized based on the lowest sampling frequency.

[0105] For example, time series data is collected every second, while surgical records are updated every hour. Based on the frequency of surgical records, the extreme value or median of the time series data (sensor data) between each surgical record and the previous surgical record is matched with the surgical record to achieve frequency synchronization.

[0106] The patient's treatment records and surgical records need to be linked with other patient data (such as physiological data, medical history data, etc.) based on the patient ID and timestamp.

[0107] Based on the patient ID and timestamp, the synchronized patient data and medical operation data are mapped and associated, and the data from two different sources (patient data and medical operation data) are associated in the time dimension to form a unified and meaningful data structure, which can be used as the input of the self-check detection model for further analysis.

[0108] After the timestamp alignment and frequency synchronization, the patient data shown in Table 1 and the medical operation data shown in Table 2 need to be mapped and associated.

[0109] Table 1 Patient data

[0110] Patient ID Timestamp age gender body temperature blood pressure heart rate diagnosis 001 2024-11-19 08:00:00 45 male 37.0℃ 120 / 80 75 hypertension 001 2024-11-19 09:00:00 45 male 36.8 120 / 80 72 hypertension 002 2024-11-19 08:30:00 38 female 36.5 110 / 70 80 normal

[0111] Table 2 Medical operation data

[0112]

[0113] During the initial mapping process based on patient ID and timestamp, since both patient data and medical operation data have unique patient IDs, data with the same patient ID are clustered.

[0114] In the patient ID-based clustering results, for each data item, find the closest matching timestamp. Considering the different sampling frequencies of operation data and patient data, you can set a time window, for example, [-15, 15] minutes, to find a pair of records with consistent timestamps between the patient data and the medical operation data. If the timestamps are not completely consistent, select the record with the closest timestamp for association.

[0115] For example, the patient data of the patient with the patient ID 001 at the timestamp of 2024-11-19 08:00:00 and the medical operation data at the timestamp of 2024-11-19 08:15:00 are in the same time window and can be associated.

[0116] When the timestamps of patient data and medical operation data are very close, they can be matched directly (for example, within ±1 minute). If the timestamps have a large deviation, the patient data and medical operation data can be matched through a time window (for example, ±15 minutes).

[0117] If there are multiple medical operation records within a time period, the multiple operation records can be associated with the corresponding patient data in a many-to-one manner to form an association result with multiple operation records.

[0118] Patient data and medical operation data will be aligned by timestamp and patient ID, and the associated information shown in Table 3 will be obtained:

[0119] Table 3 Related information

[0120]

[0121] For example, the patient's vital signs such as blood pressure and pulse and the doctor's operation records such as drug injection and operation time should be matched within the same time window.

[0122] In this embodiment, patient data refers to various types of information related to the patient's health, reflecting the patient's health status and medical history. Medical operation data is used to characterize various operations, treatments and resource utilization in the medical process, involving specific measures and resource utilization taken by the hospital during the diagnosis and treatment process. Dividing multimodal data into patient data and medical operation data can analyze the hospital's diagnosis and treatment effects, resource utilization, service quality, etc. from different angles, providing support for improving patient care and hospital operations.

[0123] In some embodiments, extracting feature vectors from the associated information and performing data fusion according to the hierarchy to obtain the medical data includes:

[0124] determining structured data and unstructured data in the associated information;

[0125] Extracting a first feature vector from the structured data, and extracting a second feature vector from the unstructured data based on a data modality of the unstructured data;

[0126] Performing a first-layer fusion of the drug features of the drug treatment and the patient information features in the first feature vector to obtain a first fused feature;

[0127] Performing a second-layer fusion of the first fused feature and the operation data feature in the second feature vector to obtain a second fused feature;

[0128] The second fusion feature and the multimodal feature in the second feature vector are fused in a third layer to obtain the medical data.

[0129] Among them, the related information after time alignment and data association is extracted and hierarchically fused to construct medical data that can be used as input for self-inspection detection models.

[0130] It can be understood that the data format of structured data is unified, stored in the database, and is easy to query and process. It can include: basic information of the patient (such as age, gender), vital signs (such as body temperature, heart rate, blood oxygen saturation, etc.) and laboratory test results (such as blood sugar, blood lipids, etc.). Data that can provide the patient's basic physiological indicators (such as blood pressure, body temperature, blood sugar level, etc.) and their changing trends can be directly used as the first eigenvector.

[0131] Unstructured data has flexible and diverse formats and contains large amounts of free text, images, and audio. Advanced technologies (such as natural language processing (NLP) and image recognition) are needed to process and analyze it to describe complex clinical scenarios and patient conditions. This data can include imaging data, text data, audio data, and time series data. Time series data can include electrocardiograms and blood pressure trends.

[0132] For image data, image features can be extracted using deep learning models such as convolutional neural networks (CNN). For example, the image features can be deep features of medical images.

[0133] For text data, the embedding vector or sentiment features of the text can be extracted through the natural language processing (NLP) model as text features.

[0134] For sound data, features such as Mel-Frequency Cepstral Coefficients (MFCC) and spectrograms can be extracted as sound features.

[0135] For time series data, for dynamically changing vital signs or sensor data, the statistical characteristics of the time series (such as mean, standard deviation, maximum, minimum, trend, etc.) can be extracted as time series features.

[0136] The combination of image features, text features, sound features and time series features is used as the second feature vector.

[0137] It should be noted that time series characteristics can be used to evaluate the hospital operating status H(t), including:

[0138] H(t)=σ(W1X(t)+W2E(t)+b)

[0139] Among them, X(t) is the time series feature; E(t) is the static feature; W1 and W2 are learning weights; b is the bias term; σ is the activation function.

[0140] Among them, static characteristics include the number of hospital beds, number of operating rooms, and types and quantities of equipment;

[0141] In this embodiment, by introducing time series features and combining static data to perform more accurate dynamic evaluation to characterize the patient's recovery status, the effectiveness of the diagnosis and treatment process can be intuitively judged.

[0142] For example, drug features include converting drug type and treatment regimen into continuous vectors through embedding methods, as well as numerical values ​​such as dosage, course of treatment, and medication time as text data.

[0143] In actual implementation, the first layer of fusion is a low-level fusion of drug-related features, which combines the drug features of drug treatment and the patient information features into a fusion vector to obtain the first fusion feature;

[0144] Operation record features include examination records as well as symptom and diagnosis information. Examination records can be obtained by embedding representation or extracting laboratory test results and imaging examination data through feature engineering. Symptom and diagnosis information can be obtained by extracting relevant text features from patient medical records or doctor's diagnosis through NLP models (such as BERT) and converting them into vector representations.

[0145] In the second layer of fusion, the first fusion feature and the operation data feature in the second feature vector are concatenated or weighted averaged to form a second fusion feature.

[0146] In the third layer of fusion, the second fusion feature is weightedly merged with the image feature, text feature, sound feature and time series feature in the second feature vector to obtain a comprehensive feature vector as medical data.

[0147] In this embodiment, the processed patient data and medical operation data are fused to obtain a unified input feature vector as medical data, which is used as the input of the self-check detection model for further analysis, prediction or classification.

[0148] In some embodiments, as Figure 2 As shown, the self-checking detection model includes an input layer, a feature extraction layer, a representation layer, a multimodal fusion layer, a hidden layer, a decision layer, and an output layer connected in sequence. The feature extraction layer is connected to the multimodal fusion layer. The medical data is analyzed and detected by the self-checking detection model to obtain the detection results output by the self-checking detection model, including:

[0149] encoding the medical data through the input layer;

[0150] Performing feature extraction on the medical data according to data modality by the feature extraction layer to obtain a first feature group;

[0151] Performing a nonlinear transformation on each first feature in the first feature group through the representation layer to generate a corresponding high-dimensional feature representation;

[0152] The first feature groups are concatenated through a multimodal fusion layer to obtain early fusion features, and the high-dimensional feature representations are fused into late fusion features. The early fusion features and the late fusion features are weighted by an attention mechanism to obtain a comprehensive feature representation.

[0153] Performing a nonlinear transformation on the comprehensive feature representation through the hidden layer to obtain a nonlinear feature representation;

[0154] Through the decision layer, pattern recognition is performed on the nonlinear feature representation, a task category is determined and executed, abnormal patterns in the medical data are obtained, and abnormality detection is performed on the nonlinear feature representation to obtain abnormal medical data;

[0155] Through the output layer, the abnormal pattern and abnormal medical data are mapped to the result space, and the detection results are obtained and output.

[0156] During the data fusion phase, the features of each modality play different roles. Some modalities may provide diagnostic support, while others may provide prediction or risk assessment information. Data from different modalities can complement the shortcomings of a single modality and may also contain similar information. Due to the differences in content and format, features need to be extracted separately to capture potential correlations and maximize the data characteristics of each modality.

[0157] For example, imaging data may reveal the location and morphology of the lesion, while the clinical history provided by text data may explain the cause of the lesion. The patient's body temperature changes can be reflected by time series data or inferred from the patient's medical record description.

[0158] Therefore, although the medical data is fused, each modality still requires separate feature extraction.

[0159] Abnormal patterns refer to operations or behaviors that deviate from standard procedures during medical processes, potentially indicating medical errors, improper operations, or missing procedures. By identifying abnormal patterns through models, medical staff can be alerted.

[0160] Abnormal medical data refers to data that deviates from the normal range, such as abnormally high medication dosage, equipment operating time exceeding the maintenance cycle, or a medical staff member working too long hours.

[0161] In actual implementation, the input layer encodes and processes the input medical data, preprocesses and normalizes the text data, image data, sound data and time series data in the medical data, and ensures that all types of data can adapt to the requirements of the self-inspection detection model.

[0162] The feature extraction layer extracts features from different modalities of medical data and combines these first features into a first feature group. For example, semantic information is extracted from text data such as disease symptoms, patient history, and medication information as first features. Low-level visual features such as edges, texture, and morphology are extracted from image data as first features. Frequency, pitch, and timbre are extracted from sound data as first features. And physiological changes, trends, and temporal characteristics of patients are extracted from time series data as first features. This allows each data type to be converted into a more representative first feature.

[0163] The representation layer converts the initially extracted first feature group into a unified, high-dimensional feature representation that is easy to integrate through nonlinear transformation, which helps to retain the key information of each modality data and improve the expressive ability of the features.

[0164] In some embodiments, the first feature groups are concatenated through a multimodal fusion layer to obtain early fusion features, and the high-dimensional feature representations are fused into late fusion features, including:

[0165] Through a multi-level attention mechanism, the first feature of each data modality is divided according to the modality level and abstraction level to determine the attention weights of low-level features, mid-level features, and high-level features;

[0166] Performing weighted calculation on the low-level features, the mid-level features, and the high-level features according to the attention weights to obtain early fusion features;

[0167] Transferring information between each of the high-dimensional feature representations through a cross-modal graph neural network, and fusing the high-dimensional feature representations based on the interactive relationships between the high-dimensional feature representations;

[0168] The fusion of the high-dimensional feature representations is optimized according to the deep association of the high-dimensional feature representations through a multimodal transformation network to obtain late fusion features.

[0169] The multimodal fusion layer has a built-in multimodal attention network, which can fuse features from different modalities into a comprehensive representation and capture the complementary information between different modalities, thereby improving the accuracy of diagnosis and prediction.

[0170] For example, the deep associations represented by high-dimensional features can be synergistic relationships, causal relationships, cross-modal alignment relationships, and temporal dependencies.

[0171] In the collaborative relationship, the collaboration between images and text can provide more accurate understanding in the image description generation task by providing visual information through images and semantic information through text; in the causal relationship, a shadow in a chest X-ray may be the cause or result of an event in the sound or video; in the cross-modal alignment relationship, the information of the text modality can be mapped to the relevant part of the image modality, and by aligning the objects in the image with the words in the text, the self-checking detection model can understand the semantic association between the image and the text; in the temporal dependency relationship, the association between time series data is manifested as a dependency that changes over time.

[0172] It can be understood that the hierarchy of the first feature can include low-level features, mid-level features and high-level features, among which the low-level features mainly include local and detailed features, such as local textures in images, word meanings at the vocabulary level, etc.; mid-level features are represented as semantic abstractions, such as partial areas in images, grammatical structures of sentences, etc.; high-level features include global abstract information, such as the overall structure of images, long-range dependencies of texts, etc.

[0173] In actual implementation, features at different levels should have different weights and attention. For the first feature, it can be divided into multiple levels, and different levels represent different modes and abstract levels of the first feature.

[0174] When dividing the layers between modalities, the first features of different modalities may correspond to different layers. For example, image edges and textures, as well as text words and phrases, can be classified as low-level features, while the global information (overall layout) of the image and the sentence structure or semantic information of the text can be considered higher-level features.

[0175] When dividing the abstraction level, for the first feature of a single modality, different levels can be obtained through layer-by-layer feature extraction. For example, in a convolutional neural network (CNN), low-level features are local edges or textures, while high-level features are more abstract shapes or objects.

[0176] Low-level features contain a large amount of detailed information and are highly localized, making significant contributions to tasks such as fine-grained detection and local pattern recognition. Low-level attention mechanisms can assign different attention weights to low-level features, highlighting the local information that is most critical to the current task.

[0177] Mid-level features involve a certain degree of semantic abstraction and may not be as globally informative as high-level features. Mid-level attention mechanisms can help strike a balance between local and global attention. For mid-level features, attention weights can be adjusted based on contextual relevance to enhance feature expressiveness.

[0178] High-level features capture global information and are therefore crucial for overall task judgment. High-level attention mechanisms can help the model focus on the most important global information by adjusting attention weights, while suppressing irrelevant information and improving the model's global understanding.

[0179] The attention weights of each level are used to perform weighted feature fusion on the low-level features, the middle-level features and the high-level features to obtain early fusion features.

[0180] The Cross-Modality Graph Neural Network (GMGNN) treats the high-dimensional feature representation of each modality as a node in a graph, while the relationships between modalities are represented by edges. Graph convolutional layers transmit information between nodes, and combined with the interactions between modalities, high-dimensional feature representations are fused to produce a richer, more comprehensive feature representation. GMGNN can automatically integrate information from different modalities spatially while maintaining the independence and complementarity of each modality.

[0181] The graph convolution operation is:

[0182]

[0183] Among them, H l is the node feature matrix of the lth layer; is the normalized adjacency matrix (reflecting the relationship between nodes); W l is the weight matrix of the lth layer; σ is the activation function.

[0184] Through the multimodal transformation network, the deep correlation between modalities is learned and the fusion strategy between modalities is automatically adjusted. The feature fusion process of GMGNN for high-dimensional feature representation is adaptively optimized, splicing operations are performed and the relationship weights between modalities are learned to achieve more efficient feature fusion and obtain late fusion features.

[0185] In some cases, medical data of one modality may be more important to the final decision, while in other cases, medical data of another modality may be more valuable. By introducing a multimodal transformation network based on a modality adaptation strategy, we can avoid the information redundancy caused by simple splicing and improve the expressive power of the model.

[0186] In this embodiment, information from different modalities can be effectively integrated, thereby improving the model's ability to handle complex tasks. These relationships are not simply a simple splicing of information between modalities, but rather a deep understanding and integration of the complex connections between modalities, enhancing the model's representational power and predictive accuracy.

[0187] A multimodal attention network based on the attention mechanism is used to assign weights to early fusion features and late fusion features. The low-dimensional early fusion features and high-dimensional late fusion features are preliminarily weighted fused at the shallow level. The preliminarily fused features are further merged in the deep high-dimensional space to obtain a comprehensive feature representation of medical data. This can gradually capture the relationship between low-dimensional and high-dimensional features, which helps the model handle more complex fusion patterns.

[0188] In some embodiments, the decision layer performs pattern recognition on the nonlinear feature representation, determines the task category and executes it, obtains abnormal patterns in the medical data, and performs abnormality detection on the nonlinear feature representation to obtain abnormal medical data, including:

[0189] In some embodiments, the hidden layer is provided with a variational autoencoder and a decoder, and a nonlinear transformation is performed on the comprehensive feature representation through the hidden layer to obtain a nonlinear feature representation, including:

[0190] Mapping the comprehensive feature representation to a latent space by the variational autoencoder, and reconstructing data of the comprehensive feature representation in the latent space by the decoder;

[0191] Determine outliers based on the reconstruction error and probability distribution of each data point during the data reconstruction process;

[0192] Perform time series analysis on the reconstructed data points to determine the data change trend;

[0193] The nonlinear feature representation is obtained according to the abnormal value and the change trend.

[0194] In practice, the Variational Auto Encoder (VAE) and decoder reconstruct the data by introducing probabilistic modeling and provide a probability distribution for each data point. During anomaly detection, if the reconstruction error of the data is large or its probability distribution differs significantly from normal data, it can be considered an outlier.

[0195] VAE can capture anomalies in data and reveal potential trends and changes in data through modeling of latent space, and is suitable for modeling and predicting long-term trends.

[0196] The input data is mapped to the latent space through the encoder and the data is reconstructed through the decoder.

[0197] In some embodiments, the reconstruction error R is:

[0198]

[0199] Among them, x is the input data represented by comprehensive features; It is the reconstructed data reconstructed by the decoder from the samples in the latent space.

[0200] The reconstruction error R can be calculated by the mean squared error (MSE).

[0201] The reconstructed data is the output generated from the latent space by the back-propagation learning process of the encoder-decoder framework. The difference between the input data and the reconstructed data is measured by calculating the square of the Euclidean distance between the two. The larger the reconstruction error, the worse the reconstruction quality of the sample, and vice versa.

[0202] If the reconstruction error of some data points is significantly higher than that of other data points, these data points can be regarded as outliers. The data distribution in the latent space can also be used to determine the outliers through the tail of the probability distribution.

[0203] In the latent space, clustering algorithms (such as K-means) or time series analysis methods are used to further analyze the representation of the latent space to find the potential trends of the data. By observing the changing trends of the latent space, the long-term changes and periodic fluctuations of the data can be analyzed.

[0204] The comprehensive feature representation can capture the nonlinear structure and complex relationships of the data by mapping it to the latent space. Outliers and data change trends adjust and optimize the representation of the latent space. The resulting nonlinear feature representation can capture the complex and implicit patterns in medical data and improve data understanding and modeling accuracy.

[0205] In this embodiment, the comprehensive features are nonlinearly transformed through the hidden layer to capture the main structure and regularity of the data, thereby improving the expressive power of the comprehensive feature representation and the abstract level of medical data, and improving the expressive power of the self-check detection model.

[0206] Next, the outliers and data change trends in the nonlinear feature representation are input into the decision layer. The decision layer uses the pattern recognition algorithm to perform pattern classification on the nonlinear feature representation, determine the classification task and regression task and execute them, judge the rationality and abnormality of the medical data based on the comprehensive feature representation, obtain the abnormal pattern in the medical data, and perform abnormality detection on the nonlinear feature representation to obtain abnormal medical data.

[0207] In classification tasks, anomalies in the data can be classified and classification labels can be generated.

[0208] In regression tasks, certain continuous results can be predicted, such as evaluating whether a certain indicator is within the normal range, or predicting the effect of treatment.

[0209] Through the output layer, the abnormal pattern and abnormal medical data are mapped to the result space to obtain the detection results and output them. For example, for the anomaly detection task, the result space may be a "normal" or "abnormal" label; for the disease classification task, the result space may be different disease categories.

[0210] Detection results can include abnormal patterns, abnormal medical data, and classification labels. Abnormal patterns indicate that operations or behaviors in the medical process deviate from standard procedures, potentially indicating medical errors, improper operations, or missing procedures. Models can identify abnormal patterns and provide warnings to medical personnel. Abnormal medical data indicates data inconsistencies, numerical anomalies, and other phenomena. Classification labels identify abnormal patterns and types of abnormal medical data.

[0211] Among them, the task of anomaly detection is to identify the parts of medical data that are inconsistent with the normal pattern. Through the output of the self-check detection model, abnormal patterns such as medical operation errors and abnormal patient status, as well as abnormal medical data such as non-standard medical records and incorrect diagnosis and treatment operations can be obtained, and the identified abnormal patterns and abnormal medical data can be marked with classification labels.

[0212] In some embodiments, the test results include abnormal patterns and abnormal medical data with classification labels, and analyzing the test results to generate assessment results for the target medical unit and matching rectification suggestions includes:

[0213] Determine the scoring weights of the abnormal pattern and the abnormal medical data according to the classification labels, score the target medical unit, and generate an assessment result;

[0214] Matching the abnormal pattern and abnormal medical data to corresponding patient IDs;

[0215] Generating an exclusive case for the patient data and medical operation data corresponding to the patient ID, and marking the abnormal pattern and related content of the abnormal medical data in the exclusive case;

[0216] Analyze the ongoing medical operation data of the exclusive case and match it with the rectification suggestions.

[0217] The test results can be shown in Table 4:

[0218] Table 4 Test results

[0219]

[0220]

[0221] The assessment results may include assessment scores and problem overviews. The problem overviews are used to provide a detailed description of the problems found, their severity, frequency of occurrence, and the risks they may cause. For example, the scoring items are divided according to classification labels, the scoring weights are matched according to the severity of the consequences, and the patient data and medical operation data that cause deductions in the scoring items.

[0222] For example, the self-inspection process of medical institutions can be assessed by giving them scores based on the frequency, type, and severity of errors, which can help measure the quality of self-inspection at medical institutions and identify potential weak links.

[0223] Corrective suggestions are solutions to problem types and may include process optimization, training suggestions, and system suggestions.

[0224] Using database queries and automated scripts, the detected abnormal patterns and abnormal medical data are associated based on the patient ID, the patient data and medical operation data of the patient are extracted, and the content related to the abnormal patterns and abnormal data is automatically annotated. For example, excessive use of drugs is marked in the medication usage record, and deviations in the operation steps are marked in the surgical record. An "Abnormal Analysis" section is automatically added to the case report, listing the detected abnormal patterns and their corresponding scores and impacts. Using natural language generation (NLG) technology, a unique case for the patient is generated in a templated format.

[0225] By introducing a standardized medical terminology library (such as UMLS), the terminology used in the exclusive medical record is standardized. The standardization process is defined as:

[0226] T std =f(T raw , C UMLS )

[0227] Among them, T raw are medical terms in the original text; C UMLSis the standard terminology database in UMLS; f(T raw , C UMLS ) is the mapping function, which transforms the original term T raw Mapping to standard terminology C UMLS .

[0228] Through standardization, we ensure that the medical terms in the self-examination reports are accurate and consistent, and improve the personalization and readability of the self-examination reports.

[0229] For example, "Patient ID is

[0001] , who underwent [surgery] on [date]. Testing revealed [abnormal pattern] for this patient, specifically [abnormal medical data]. The abnormal pattern score is [score]. [Correction suggestions] are recommended."

[0230] By combining patient data with a rules engine and machine learning algorithms, the system conducts in-depth analysis of medical procedure data within individual cases, assesses the rationale for the current treatment plan, and provides relevant corrective recommendations in a structured format for the relevant medical staff. For example, if a surgical procedure is not standardized, the system can provide standardized surgical procedures or recommend surgical quality training.

[0231] Based on the analysis results, the system automatically generates correction suggestions. For example, patient 0001's medication regimen may have an overuse problem, and it is recommended to adjust the drug dosage and re-evaluate the condition.

[0232] Through natural language generation (NLG), the assessment results and rectification suggestions are generated into a self-inspection report according to the report template. The rectification suggestions can be pushed to the doctor's workstation and mobile application through the self-inspection report.

[0233] In addition, based on abnormal patterns and abnormal medical data, we can analyze the operational links in the medical staff's operating procedures that are prone to errors or unreasonable operations, as well as medical resources with excessive or underutilized rates, and generate corresponding rectification suggestions.

[0234] Among them, based on historical medical data, seasonal changes, epidemic trends, etc., the future hospital patient visits, bed demand, doctor scheduling, etc. can be predicted to provide a basis for the allocation of medical resources; by analyzing patient waiting time and medical procedures, the hospital's processes can be optimized and patient waiting time can be reduced; by analyzing the usage of different departments and equipment in the hospital, resources can be reasonably allocated and dispatched to avoid waste of resources.

[0235] For example, for abnormal patterns of operational violations, training courses are matched and recommendations are given, including optimizing operational procedures and setting up supervision. For data quality issues, recommendations are given to strengthening data collection standards and adding automated inspection mechanisms. For abnormal resource usage, such as too many vacant beds in some departments and insufficient beds in some departments, a corresponding bed allocation plan is generated based on the number of inpatients.

[0236] Identified abnormal patterns may include various medical operations that do not meet standards, errors in patient information recording, mismatches between imaging data and clinical symptoms, etc. These patterns may point to potential problems with medical procedures, operators, or equipment.

[0237] Among abnormal medical data, imaging data may be unclear due to equipment failure, text data may contain grammatical errors or lose key information, and time series data may have inconsistencies or missing problems. Corrective suggestions can be generated to improve data collection, processing procedures or equipment.

[0238] The process of matching corrective action recommendations needs to be integrated with the medical unit's standard procedures, operating manuals, historical data, and best practices. For example, if abnormal data stems from a device issue, a recommendation might be made to check the device status and perform calibration. If the issue stems from insufficient operator training, recommendations might include strengthening professional skills training for medical staff in this area, improving data input standards, enhancing the integration of medical information systems, and improving automated monitoring capabilities.

[0239] In this embodiment, through auxiliary decision-making based on data analysis, the intelligence of medical decision-making is improved, and optimization suggestions for diagnosis and treatment plans are provided to medical personnel. Real-time adjustments and optimizations are made based on self-inspection results, which promotes the intelligent transformation of medical institutions and provides patients with safer and more refined medical services.

[0240] According to the self-inspection method for medical units based on data analysis provided in the embodiment of the present application, through the joint analysis of multimodal data, it is possible to fully utilize the potential information in the data, realize a comprehensive, personalized and accurate evaluation of the medical unit, and improve the problem detection capability. The obtained self-inspection report is comprehensive and accurate, ensuring that different types of data can be effectively integrated for comprehensive analysis. Medical units can achieve more efficient and accurate self-inspection management, which can not only improve medical quality and work efficiency, but also enhance the medical unit's early warning and response capabilities to potential problems.

[0241] The present application also provides an embodiment for optimizing the self-check detection model by setting up an adaptive learning and feedback mechanism, which can continuously make personalized adjustments and optimizations to the self-check detection model based on analysis and feedback of the detection results as the internal data of the medical unit changes and technology advances.

[0242] For example, in a federated learning environment, each medical unit shares data through self-checking and detection model updates instead of directly exchanging data. The self-checking and detection model update of each medical unit is defined as:

[0243]

[0244] in, is the model parameter of the kth medical unit at the tth iteration; is the local loss function of the kth medical unit; η is the learning rate, is the gradient of the model parameters with respect to the loss value.

[0245] Each medical unit updates the global model by averaging the updates to ensure shared learning among the system.

[0246] In the update of the self-checking detection model, the following objective function is introduced for incremental learning:

[0247]

[0248] Among them, L i (θ,D i ) is the number of the i-th data set D i The loss function on θ prev is the model parameter of the last updated self-check detection model; λ is the regularization coefficient, which prevents the self-check detection model from experiencing drastic fluctuations during incremental learning.

[0249] This allows for smooth model updates as new clinical data is generated, ensuring continuous improvement.

[0250] The present application also provides a self-inspection system for a medical unit based on data analysis, which is used to execute the self-inspection method for a medical unit based on data analysis as described in any of the above embodiments.

[0251] According to the self-inspection system for medical units based on data analysis provided in the embodiment of the present application, through the joint analysis of multimodal data, it is possible to fully utilize the potential information in the data, realize a comprehensive, personalized and accurate evaluation of the medical unit, and improve the problem detection capability. The obtained self-inspection report is comprehensive and accurate, ensuring that different types of data can be effectively integrated for comprehensive analysis. Medical units can achieve more efficient and accurate self-inspection management, which can not only improve medical quality and work efficiency, but also enhance the medical unit's early warning and response capabilities to potential problems.

[0252] The self-check system can also predict the peak visit period of the emergency department based on historical data, rationally allocate resources such as doctors and beds, and optimize the emergency department schedule; optimize the use of the operating room according to the type of surgery, the doctor's professional skills, the equipment condition of the operating room, etc.; by analyzing the patient's treatment process data, analyze the effects of different treatment pathways, find the best treatment process, optimize the clinical pathway, and improve treatment effects and resource utilization; identify differences between different medical staff in the treatment of the same disease, and analyze the reasons for these differences, such as medical technology level, choice of treatment plan, etc.

[0253] The self-check system provides a multi-task learning framework for the self-check detection model, which can optimize multiple tasks at the same time. The objective function of multi-task learning is defined as:

[0254]

[0255] Among them, L i (θ) is the loss function of the i-th task (such as disease prediction or treatment recommendation); α i is the weight of the task; is the previous model parameter on task i; λ is a regularization term to avoid overfitting.

[0256] By jointly optimizing multiple tasks, the system can simultaneously generate decision support information such as disease prediction and treatment recommendations.

[0257] The self-inspection system also supports question and answer. In multi-round conversations, it uses context models to understand the context and sentiment analysis models to analyze the sentiment of the conversation.

[0258] The probability model P(response|history) for dialogue understanding is defined as:

[0259]

[0260] Among them, P(history, response) is the joint probability of the dialogue history and the current response; P(history) is the probability of the dialogue history.

[0261] This model helps the system understand the current user's intention based on historical conversations and generate more accurate responses.

[0262] Improve the real-time responsiveness and accuracy of voice assistants and question-answering systems.

[0263] Define the sentiment analysis model as:

[0264] E sentiment =Sigmoid(W sentiment ·X voice +b)

[0265] Among them, X voise is the feature vector extracted from speech; W sentiment is the weight of the sentiment analysis model; b is the bias term.

[0266] In the process of routine data detection by the self-checking system, outliers can be detected by the threshold-based anomaly detection model. The judgment function of anomaly detection is:

[0267] A t =1(‖x t -μ‖>ασ)

[0268] Among them, x t is the data at time t; μ is the mean of historical data; σ is the standard deviation of the data; α is the set threshold factor; 1(·) is the indicator function, which returns 1 when the data exceeds the threshold, indicating an abnormality.

[0269] When the data is detected to be outside the normal range, the self-checking system will trigger an early warning and notify relevant medical staff.

[0270] The self-inspection system for medical units based on data analysis provided in the embodiment of the present application can implement the various processes implemented in the self-inspection method embodiment for medical units based on data analysis in the above-mentioned embodiment. To avoid repetition, they will not be described here.

[0271] In some embodiments, as Figure 3 As shown, an embodiment of the present application also provides an electronic device 300, including a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the program is executed by the processor 301, each process of the embodiment of the self-inspection method of a medical unit based on data analysis is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0272] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0273] In the description of this application, "first feature" and "second feature" may include one or more such features.

[0274] In the description of this application, “plurality” means two or more.

[0275] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

[0276] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0277] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and purpose of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A self-inspection method for medical institutions based on data analysis, characterized in that: include: Receive and respond to self-inspection requests, and retrieve multimodal data of the target medical unit, wherein the multimodal data includes text data, image data, sound data, and time series data; Performing data processing and data fusion on the text data, the image data, the sound data, and the time series data to obtain medical data; Inputting the medical data into a self-checking detection model, analyzing and detecting the medical data through the self-checking detection model, and obtaining a detection result output by the self-checking detection model, wherein the self-checking detection model is constructed based on the large model; Analyze the test results, generate assessment results of the target medical unit and match them with rectification suggestions, and obtain a self-inspection report of the target medical unit; The self-checking detection model includes an input layer, a feature extraction layer, a representation layer, a multimodal fusion layer, a hidden layer, a decision layer, and an output layer connected in sequence. The feature extraction layer is connected to the multimodal fusion layer. The medical data is analyzed and detected by the self-checking detection model to obtain the detection results output by the self-checking detection model, including: encoding the medical data through the input layer; Performing feature extraction on the medical data according to data modality by the feature extraction layer to obtain a first feature group; Performing a nonlinear transformation on each first feature in the first feature group through the representation layer to generate a corresponding high-dimensional feature representation; The first feature groups are concatenated through a multimodal fusion layer to obtain early fusion features, and the high-dimensional feature representations are fused into late fusion features. The early fusion features and the late fusion features are weighted by an attention mechanism to obtain a comprehensive feature representation. Performing a nonlinear transformation on the comprehensive feature representation through the hidden layer to obtain a nonlinear feature representation; Through the decision layer, pattern recognition is performed on the nonlinear feature representation, a task category is determined and executed, abnormal patterns in the medical data are obtained, and abnormality detection is performed on the nonlinear feature representation to obtain abnormal medical data; Through the output layer, the abnormal pattern and abnormal medical data are mapped to the result space, and the detection results are obtained and output; The first feature groups are concatenated through a multimodal fusion layer to obtain early fusion features, and the high-dimensional feature representations are fused into late fusion features, including: Through a multi-level attention mechanism, the first feature of each data modality is divided according to the modality level and abstraction level to determine the attention weights of low-level features, mid-level features, and high-level features; Performing weighted calculation on the low-level features, the mid-level features, and the high-level features according to the attention weights to obtain early fusion features; Transferring information between each of the high-dimensional feature representations through a cross-modal graph neural network, and fusing the high-dimensional feature representations based on the interactive relationships between the high-dimensional feature representations; Optimizing the fusion of the high-dimensional feature representations according to the deep correlation of the high-dimensional feature representations through a multimodal transformation network to obtain late fusion features; The hidden layer is provided with a variational autoencoder and a decoder, and the comprehensive feature representation is subjected to a nonlinear transformation through the hidden layer to obtain a nonlinear feature representation, including: Mapping the comprehensive feature representation to a latent space by the variational autoencoder, and reconstructing data of the comprehensive feature representation in the latent space by the decoder; Determine outliers based on the reconstruction error and probability distribution of each data point during the data reconstruction process; Perform time series analysis on the reconstructed data points to determine the data change trend; The abnormal value and the change trend are represented as the nonlinear features.

2. The self-examination method of a medical unit based on data analysis according to claim 1, characterized in that: Processing and fusing the text data, the image data, the sound data, and the time series data to obtain medical data includes: Based on the data modality, respectively performing data cleaning and standardization processing on the text data, the image data, the sound data, and the time series data; dividing the processed text data, the image data, the sound data, and the time series data into patient data and medical operation data; Time-aligning and data-correlating the patient data and the medical operation data to obtain correlation information; Feature vectors are extracted from the associated information and data fusion is performed according to the hierarchy to obtain the medical data.

3. The self-examination method of a medical unit based on data analysis according to claim 2, characterized in that: The patient data and the medical operation data carry a patient ID, and the patient data and the medical operation data are time-aligned and data-associated to obtain association information, including: Performing time stamp alignment and frequency synchronization on the patient data and the medical operation data; Based on the patient ID, the synchronized patient data and medical operation data are mapped and associated to obtain associated information.

4. The self-examination method of a medical unit based on data analysis according to claim 2, characterized in that: Extracting feature vectors from the associated information and performing data fusion according to the hierarchy to obtain the medical data includes: determining structured data and unstructured data in the associated information; Extracting a first feature vector from the structured data, and extracting a second feature vector from the unstructured data based on a data modality of the unstructured data; Performing a first-layer fusion of the drug features of the drug treatment and the patient information features in the first feature vector to obtain a first fused feature; Performing a second-layer fusion of the first fused feature and the operation data feature in the second feature vector to obtain a second fused feature; The second fusion feature and the multimodal feature in the second feature vector are fused in a third layer to obtain the medical data.

5. The self-examination method of a medical unit based on data analysis according to claim 1, characterized in that: The reconstruction error R is: Among them, x is the input data represented by comprehensive features; It is the reconstructed data reconstructed by the decoder from the samples in the latent space.

6. The self-examination method of a medical unit based on data analysis according to claim 1, characterized in that: The detection results include abnormal patterns and abnormal medical data with classification labels. The analysis of the detection results to generate the assessment results of the target medical unit and matching rectification suggestions includes: Determine the scoring weights of the abnormal pattern and the abnormal medical data according to the classification labels, score the target medical unit, and generate an assessment result; Matching the abnormal pattern and abnormal medical data to corresponding patient IDs; Generating an exclusive case for the patient data and medical operation data corresponding to the patient ID, and marking the abnormal pattern and related content of the abnormal medical data in the exclusive case; Analyze the ongoing medical operation data of the exclusive case and match it with the rectification suggestions.

7. A self-inspection system for medical institutions based on data analysis, characterized in that: A self-examination method for a medical unit based on data analysis is used to perform the self-examination method according to any one of claims 1 to 6.

Images