Medical information processing method, system and device based on video data and storage medium

Abstract

The invention discloses a medical information processing method, system and device based on video data and a storage medium. The quality and efficiency of medical services can be improved. The method comprises the following steps: acquiring medical processing data; preprocessing the medical processing data to obtain preprocessed data; constructing a medical knowledge graph, wherein the medical knowledge graph is used for representing the relationship among the disease, the symptom, the examination index and the medicine; constructing a medical model based on the medical knowledge graph, and performing training and cross validation on the medical model to obtain a target medical model; constructing and generating a health digital identity model according to the preprocessed data and the target medical model; acquiring real-time video data; processing the real-time video data to extract identity information and skeleton key point information of a user in the real-time video data; and inputting the identity information and the skeleton key point information of the user into the health digital identity model to obtain medical health information of the user.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a medical information processing method, system, device and storage medium based on video data. Background Art

[0002] The rise of digital medicine has brought revolutionary changes to the medical industry. Through the collection, storage, analysis and application of medical data, medical institutions can provide more accurate and efficient prevention, diagnosis and treatment services. This data-driven medical model makes medical services more personalized and scientific.

[0003] At the same time, the popularization of home camera technology has also brought great convenience to home security and life convenience. Functions such as high-definition image quality, remote monitoring, and intelligent recognition have made home cameras a powerful assistant for home security.

[0004] At present, by combining digital medicine with home camera technology, it is possible to monitor the health status of patients in real time through home cameras, analyze them in combination with medical data, and provide doctors with more accurate diagnosis basis. For example, for patients who need long-term home rehabilitation, home cameras can monitor their vital signs, activities, etc. in real time, provide doctors with timely feedback, and use home cameras for remote medical consultation, which can also improve the accessibility and efficiency of medical services. Patients can make video calls with doctors through home cameras to consult about their condition and obtain treatment plans.

[0005] Although the combination of digital medical care and home cameras can help understand the patient's physical condition to a certain extent, the current technology that combines health care data with cameras cannot automatically predict a person's health status, and thus cannot provide people with more personalized and accurate medical and health services. The quality and efficiency of medical services need to be improved. Summary of the invention

[0006] In order to solve the above technical problems, the present application provides a medical information processing method, system, device and storage medium based on video data.

[0007] The technical solution provided in this application is described below: The first aspect of the present application provides a medical information processing method based on video data, characterized in that the method comprises: Access to medical treatment data; Preprocessing the medical treatment data to obtain preprocessing data; Constructing a medical knowledge graph, wherein the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators and drugs; Building a medical model based on the medical knowledge graph, and training and cross-validating the medical model to obtain a target medical model; Constructing and generating a healthy digital identity model based on the preprocessed data and the target medical model; Get real-time video data; Processing the real-time video data to extract the user's identity information and skeleton key point information in the real-time video data; The user's identity information and skeletal key point information are input into the healthy digital identity model to obtain the user's medical health information, where the medical health information includes medical advice, health recommendations or health forecasts.

[0008] Optionally, obtain medical treatment data, including: Determine where medical data is stored; The medical data is extracted from the storage location, and the medical data is desensitized by generalization, randomization, rearrangement, substitution, shielding or replacement to obtain the medical treatment data.

[0009] Optionally, a medical knowledge graph is constructed, wherein the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators, and the relationship between diseases and drugs, including: Access to medical information resources, including clinical medical data, scientific publications, reference standard libraries, and openly shared medical knowledge bases; Extracting structured medical resource information and unstructured medical resource information from the clinical medical data, the scientific publications, the reference standard library, and the open and shared medical knowledge base; A medical knowledge graph is constructed based on the structured medical resource information and the unstructured medical resource information, and the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators and drugs.

[0010] Optionally, building a medical model based on the medical knowledge graph, and training and cross-validating the medical model to obtain a target medical model includes: Build a medical model based on a semi-supervised machine learning model; Acquire a medical data training collection according to the medical knowledge graph, wherein the medical data training collection includes labeled medical data and unlabeled medical data; Training the medical model according to the medical data training collection to obtain an initial medical model; Using a ten-fold cross validation method to validate the initial medical model; When it is determined that the verification result of the initial medical model meets the preset requirements, the target medical model is obtained.

[0011] Optionally, the real-time video data is processed to extract the user's identity information and skeleton key point information in the real-time video data, including: Identifying the position and movement posture of the user in the real-time video data; Determine the user's facial features according to the user's location, and obtain the user's identity information through the facial features; The human skeleton of the user is tracked and identified according to the posture of the user to obtain the body movement angle of the user, wherein the body movement angle includes the angles of neck bending, trunk bending, trunk thigh angle and knee joint angle.

[0012] Optionally, preprocessing the medical treatment data to obtain preprocessed data includes: Processing the medical treatment data by performing duplicate value and missing value processing, format and coding consistency check, logic verification and control mapping processing to obtain processing data; The processed data are standardized to obtain pre-processed data.

[0013] Optionally, obtain real-time video data, including: Get real-time video data through high-definition cameras.

[0014] The second aspect of the present application provides a medical information processing system based on video data, characterized in that the system comprises: A first acquisition unit, used for acquiring medical treatment data; A second acquisition unit, used for preprocessing the medical treatment data to obtain preprocessing data; A first construction unit is used to construct a medical knowledge graph, wherein the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators and drugs; A third acquisition unit constructs a medical model based on the medical knowledge graph, and trains and cross-validates the medical model to obtain a target medical model; A generating unit, configured to generate a healthy digital identity model according to the preprocessed data and the target medical model; A fourth acquisition unit, used to acquire real-time video data; An extraction unit, used for processing the real-time video data to extract the user's identity information and skeleton key point information in the real-time video data; The fifth acquisition unit is used to input the user's identity information and skeleton key point information into the healthy digital identity model to obtain the user's medical health information, where the medical health information includes medical advice, health recommendations or health forecasts.

[0015] The third aspect of the present application provides a talent data matching device based on big data, the device comprising: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method as described in the first aspect and any one of the first aspects.

[0016] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the method described in the first aspect and any one of the first aspects is performed.

[0017] It can be seen from the above technical solutions that this application has the following advantages: 1. By constructing a medical knowledge graph, this method integrates the complex relationship between diseases, symptoms, examination indicators and drugs, providing a solid theoretical basis for medical decision-making. Combined with medical models for training and cross-validation, it can ensure the high accuracy and generalization ability of the model, thereby improving the efficiency and accuracy of medical diagnosis.

[0018] 2. The health digital identity model built using preprocessed data and target medical models can provide personalized medical advice and health recommendations based on each user's situation. This personalized service helps improve the user's health management level and prevent the occurrence or deterioration of diseases.

[0019] 3. Through real-time video data processing, this method can extract the user's identity information and bone key point information, and then analyze the user's health status in combination with the healthy digital identity model. This real-time monitoring mechanism can promptly detect the user's health abnormalities, provide timely warnings and interventions, and reduce health risks.

[0020] 4. This method does not rely on traditional face-to-face diagnosis and treatment, but uses video data and intelligent algorithms to process and analyze remote medical information, which helps to break geographical restrictions, enable high-quality medical resources to be more widely covered, and promote the intelligent transformation of the medical industry.

[0021] 5. By processing medical data through automated and intelligent means, this method can reduce the workload of medical staff and improve their work efficiency. At the same time, accurate medical models and health digital identity models can assist medical staff in making more accurate decisions, thereby optimizing the allocation and utilization of medical resources.

[0022] 6. This method can automatically predict an individual's health status, thereby providing people with more personalized and accurate medical and health services, and helping to improve the quality and efficiency of medical services. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 A schematic diagram of an embodiment of a medical information processing method based on video data of the present application; Figure 2 This is a schematic diagram of another embodiment of the medical information processing method based on video data of the present application; Figure 3 This is a schematic diagram of another embodiment of the medical information processing method based on video data of the present application; Figure 4 This is a schematic diagram of another embodiment of the medical information processing method based on video data of the present application; Figure 5 This is a schematic diagram of another embodiment of the medical information processing method based on video data of the present application; Figure 6 This is a schematic diagram of another embodiment of the medical information processing method based on video data of the present application; Figure 7 A schematic diagram of an embodiment of a medical information processing system based on video data of the present application; Figure 8 This is a schematic diagram of an embodiment of a medical information processing device based on video data of the present application. DETAILED DESCRIPTION

[0026] It should be noted that the present application provides a medical information processing method based on video data, which can be applied to a terminal, a system, or a server. For example, the terminal can be a smart phone or a computer, a tablet computer, a smart TV, a smart watch, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of explanation, the present application uses the terminal as an example for the execution subject.

[0027] See also Figure 1 The present application first provides an embodiment of a medical information processing method based on video data, the embodiment comprising: 101. Obtain medical treatment data; 102. Preprocessing the medical treatment data to obtain preprocessing data; 103. Construct a medical knowledge graph, wherein the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators and drugs; 104. Construct a medical model based on the medical knowledge graph, and train and cross-validate the medical model to obtain a target medical model; 105. Constructing and generating a healthy digital identity model according to the preprocessed data and the target medical model; 106. Obtain real-time video data; 107. Processing the real-time video data to extract the user's identity information and skeleton key point information in the real-time video data; 108. Input the user's identity information and skeletal key point information into the healthy digital identity model to obtain the user's medical health information, where the medical health information includes medical advice, health recommendations, or health forecasts.

[0028] In an embodiment of the present application, medical treatment data is first acquired, and then the medical treatment data is preprocessed to acquire preprocessed data, and then a medical knowledge graph is further constructed, wherein the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators and drugs, and then a medical model is constructed based on the medical knowledge graph, and the medical model is trained and cross-validated to acquire a target medical model. After obtaining the target medical model, a healthy digital identity model is further constructed based on the preprocessed data and the target medical model, real-time video data is acquired, and the real-time video data is processed to extract the user's identity information and skeletal key point information in the real-time video data, and then the user's identity information and skeletal key point information are input into the healthy digital identity model to acquire the user's medical health information, which includes medical advice, health recommendations or health forecasts.

[0029] In step 101, medical treatment data is obtained, wherein the medical treatment data comes from multiple channels, such as hospital electronic medical record systems, medical research databases, public health data platforms, etc., which are not specifically limited here. The obtained data includes the user's disease diagnosis information, symptom descriptions, laboratory test results, and drug treatment records. After the user's medical treatment data is obtained, step 102 is executed.

[0030] In step 102, after obtaining the user's medical treatment data, the medical treatment data is preprocessed. Specifically, the data preprocessing is to improve the data quality so that it is suitable for subsequent modeling and analysis. The preprocessing includes data cleaning, data conversion and data normalization. Specifically, during the processing, in order to ensure data security, it is first necessary to adopt verification technology, encryption technology, electronic signatures and timestamps to ensure the security of data during collection, transmission and storage. Secondly, backup and recovery, residual information protection, storage media management and other measures should be implemented to ensure data storage security. Data users and controllers should be subject to approval authorization, identity authentication, access control and auditing, and an emergency response mechanism should be established to effectively respond to security threats. When disclosing, sharing and opening data, personal sensitive information should be processed and de-identified.

[0031] In step 103, a medical knowledge graph is constructed. Specifically, the medical knowledge graph is used to describe the relationship between medical entities such as diseases, symptoms, examination indicators and drugs. Constructing the knowledge graph involves identifying medical entities from text, determining the relationship between entities and organizing the entities and relationships in the form of a graph. By creating a medical knowledge graph, background knowledge and reasoning capabilities can be provided for the subsequent medical model, helping the medical model to better understand medical data and make accurate predictions.

[0032] In step 104, a medical model is constructed based on the medical knowledge graph, and the medical model is trained and cross-validated. Specifically, a medical model is first constructed based on the medical knowledge graph, and the medical model can be constructed based on a semi-supervised machine learning model. Then, the medical model is trained using the medical treatment data in the medical knowledge graph so that it can learn the complex relationship between diseases, symptoms, examination indicators and drugs. In order to evaluate the performance and generalization ability of the model, cross-validation is usually required, that is, the data set is divided into multiple parts, one part of which is used as a test set in turn, and the rest is used as a training set for model training and evaluation. After the medical model is trained and verified, the target medical model is obtained.

[0033] In step 105, a healthy digital identity model is constructed based on the preprocessed data and the target medical model. Specifically, after obtaining the preprocessed data and the target medical model, a healthy digital identity model is further constructed. The healthy digital identity model is a personal health record constructed based on the user's medical data and medical model. The healthy digital identity model integrates information such as the user's medical history, current health status, and possible health risks to provide the user with a comprehensive health portrait. Constructing the healthy digital identity model requires integrating the personal information in the preprocessed data, the prediction results output by the medical model, and possible other health-related data, such as living habits, genetic information, etc. After generating the healthy digital identity model, the video data is further combined with the healthy digital identity model to execute step 106.

[0034] In step 106, real-time video data is obtained, wherein the real-time video data is user video information obtained in real time through a high-definition camera or other video acquisition equipment.

[0035] In step 107, the real-time video data is processed to extract the user's identity information and skeleton key point information in the real-time video data. Based on these video data and the OpenPose algorithm, the user's identity information and skeleton key point information can be extracted to analyze the user's body posture, movement, etc. Among them, the identity information is used to confirm the user's identity and ensure that the subsequent medical and health information is for the correct user, and the skeleton key point information is used to analyze the user's body posture and movement to provide a basis for the generation of medical and health information.

[0036] In step 108, the user's identity information and skeletal key point information are input into the healthy digital identity model to obtain the user's medical health information. Specifically, the extracted user identity information and skeletal key point information are input into the healthy digital identity model, and the medical health information for the user is generated in combination with the user's medical history and current health status. The generated information includes medical advice, health recommendations or health forecasts. Therefore, through the combination of real-time video data and the healthy digital identity model, real-time monitoring and personalized management of the user's health status can be achieved. Therefore, the method of the present application provides people with more personalized and accurate medical and health services by automatically predicting the health status of individuals, which helps to improve the quality and efficiency of medical services.

[0037] For code examples of the processing process of the medical information processing method based on video data, please refer to the following: import numpy as np import pandas as pd from sklearn.model_selection import train_test_split from sklearn.ensemble import RandomForestClassifier import cv2 import mediapipe as mp # 1. Get medical treatment data def get_medical_data(): return pd.read_csv('medical_data.csv') # 2. Preprocessing of medical treatment data def preprocess_data(data): # Assume we only need to fill missing values ​​and standardize data = data.fillna(data.mean()) return data # 3. Build a medical knowledge graph def build_medical_knowledge_graph(): return { 'disease': {'symptom': [...], 'test': [...], 'medication': [...]}, ... } # 4. Build a medical model based on the medical knowledge graph, and train and cross-validate def build_and_train_model(data, knowledge_graph): X = data.drop('target', axis=1) y = data['target'] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2) model = RandomForestClassifier() model.fit(X_train, y_train) # Cross validation and evaluation return model # 5. Build a model to generate healthy digital identity def build_health_digital_identity_model(model): def predict(identity_info, skeleton_info): # Here we assume that identity_info and skeleton_info are sufficient for prediction # More complex processing may be required in actual applications features = extract_features(identity_info, skeleton_info) return model.predict([features]) return predict # 6. Get real-time video data def get_real_time_video_data(): cap = cv2.VideoCapture(0) Return cap # 7. Process real-time video data to extract identity information and skeleton key point information def process_video_data(cap): mp_pose = mp.solutions.pose pose = mp_pose.Pose() ret, frame = cap.read() if not ret: return None, None results = pose.process(frame) # Extract identity information and skeleton key point information identity_info = {'id': 'user123'} skeleton_info = results.pose_landmarks return identity_info, skeleton_info # Main process if __name__ == "__main__": # Acquiring and preprocessing medical data data = get_medical_data() preprocessed_data = preprocess_data(data) # Build knowledge graph and medical model knowledge_graph = build_medical_knowledge_graph() model = build_and_train_model(preprocessed_data, knowledge_graph) # Building a healthy digital identity model health_digital_identity_fn = build_health_digital_identity_model(model) # Get and process real-time video data cap = get_real_time_video_data() while True: identity_info, skeleton_info = process_video_data(cap) if identity_info and skeleton_info: # Get medical and health information health_info = health_digital_identity_fn(identity_info, skeleton_info) print("Medical health information:", health_info) # Display video in real time (optional) cv2.imshow('Real-Time Video', frame) if cv2.waitKey(1)&0xFF == ord('q'): break cap.release() cv2.destroyAllWindows() The code example involves a medical information processing method for video data, specifically, obtaining medical processing data, preprocessing the medical processing data to obtain preprocessed data, constructing a medical knowledge graph, the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators and drugs, constructing a medical model based on the medical knowledge graph, and training and cross-validating the medical model to obtain a target medical model; constructing a healthy digital identity model based on the preprocessed data and the target medical model, obtaining real-time video data, processing the real-time video data to extract the user's identity information and skeletal key point information in the real-time video data, and inputting the user's identity information and skeletal key point information into the healthy digital identity model to obtain the user's medical health information.

[0038] By constructing a medical knowledge graph, this method integrates the complex relationship between diseases, symptoms, examination indicators and drugs, providing a solid theoretical basis for medical decision-making. Combined with medical models for training and cross-validation, it can ensure the high accuracy and generalization ability of the model, thereby improving the efficiency and accuracy of medical diagnosis. It can also automatically predict an individual's health status, thereby providing people with more personalized and accurate medical and health services, which helps to improve the quality and efficiency of medical services.

[0039] Please refer to Figure 2 According to some embodiments of the present invention, obtaining medical treatment data in step 101 may specifically include, but is not limited to, the following: 201. Determine the storage location of medical data; 202. Extract the medical data from the storage location, and desensitize the medical data by generalizing, randomizing, rearranging, replacing, shielding or substituting to obtain the medical treatment data.

[0040] In the embodiment of the present application, when determining the storage location of medical data, it is necessary to further extract the medical data in the storage location, wherein the medical data contains highly sensitive personal information, such as patient identity, health status, etc. In order to protect the privacy of patients, the medical data must be desensitized. Desensitization refers to the use of a series of technical means to include only specific data and remove or reduce the identifiability of the patient. The following are several common desensitization methods: Generalization: Replace specific data values ​​with more general or abstract values. For example, replace the specific age of the patient with an age range (such as "20-30 years old"), replace specific dates with months or quarters, etc.

[0041] Randomization: Randomly perturb the original data so that it cannot be directly associated with a specific individual. For example, randomly encoding patient names or making small random adjustments to numerical data.

[0042] Shuffle: Shuffle the order of records in a data set so that originally adjacent records are no longer adjacent, thereby reducing the risk of identifying individuals through the order of data arrangement.

[0043] Permutation: Replacing certain parts of the original data with other similar but non-personally identifiable data. For example, replacing a patient's real name and address with simulated data.

[0044] Masking: Directly delete or hide certain sensitive parts of the data. For example, masking personal information such as the patient’s ID number or phone number in the medical record.

[0045] Substitution: Replace sensitive values ​​in the original data with alternative values ​​that have the same or similar statistical properties. This approach can reduce personal identification while maintaining the overall distribution characteristics of the data.

[0046] Therefore, when performing desensitization, it is necessary to select an appropriate desensitization method or a combination of multiple methods according to the specific characteristics and uses of the data, and no specific limitation is made here. After desensitization, the privacy of patients can be effectively protected.

[0047] Please refer to Figure 3 According to some embodiments of the present invention, in step 103, a medical knowledge graph is constructed, and the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators, and the relationship between diseases and drugs, which may specifically include, but are not limited to, the following: 301. Obtain medical information resources, including clinical medical data, scientific publications, reference standard libraries, and open and shared medical knowledge bases; 302. Extracting structured medical resource information and unstructured medical resource information from the clinical medical data, the scientific publications, the reference standard library, and the open and shared medical knowledge base; 303. Construct a medical knowledge graph based on the structured medical resource information and the unstructured medical resource information, wherein the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators and drugs.

[0048] In the embodiment of the present application, medical information resources are obtained, and the medical information resources include clinical medical data, scientific publications, reference standard libraries, and open and shared medical knowledge bases, wherein: Clinical medical data: usually comes from the hospital's electronic medical record system, patient database, clinical trial results, etc. These data contain a large amount of actual case information, such as patient symptoms, diagnosis results, treatment plans, drug use records, etc., and are an important basis for building a medical knowledge graph.

[0049] Scientific publications: including medical journal articles, conference papers, research reports, etc. These publications usually contain the latest medical research results, clinical case analysis, disease mechanism discussions, etc., and are cutting-edge and authoritative sources of medical knowledge.

[0050] Reference standard libraries: such as medical terminology dictionaries, disease classification standards (such as ICD codes), drug coding systems (such as ATC codes), etc. These standard libraries provide a basis for the normalization and standardization of medical information, ensuring the accuracy and consistency of information in the medical knowledge graph.

[0051] Open and shared medical knowledge bases: such as the medical section of Wikipedia, PubMed, Cochrane Library and other public medical knowledge bases. These resources provide extensive medical knowledge and data with high credibility and reliability.

[0052] Furthermore, after obtaining the medical information resources, the collected medical information resources should be deeply analyzed and processed to extract useful medical knowledge. Specifically, structured medical resource information and unstructured medical resource information should be extracted from the medical information resources, including: Structured medical resource information extraction: Information that exists in a structured form, such as symptom lists, drug use records, laboratory test results, etc. in electronic medical records, can be directly extracted through data parsing and conversion technology. This information is usually stored in the form of tables, databases, or XML to facilitate subsequent processing and analysis.

[0053] Unstructured medical resource information extraction: For unstructured information in the form of text, such as scientific publications and medical reports, natural language processing (NLP) technology is needed to extract key information. NLP technology can identify the names of diseases and drugs, the relationship between diseases and symptoms, and the treatment process in the text, and convert this information into a structured form.

[0054] After obtaining structured medical resource information and unstructured medical resource information, a medical knowledge graph is constructed based on the structured medical resource information and unstructured medical resource information. The medical knowledge graph is a graphical data structure used to represent diseases, symptoms, drugs and their relationships. The construction process is as follows: First, it is necessary to identify the medical entities in the text and classify them into different categories such as diseases, symptoms, and drugs. Next, it is necessary to determine the relationships between entities. For example, what symptoms a certain disease may cause, what drugs can be used to treat the disease, etc. Finally, the identified entities and extracted relationships are represented graphically to form a medical knowledge graph. The nodes in the graph represent entities, and the edges represent the relationships between entities.

[0055] By constructing a medical knowledge graph, medical knowledge scattered across multiple sources can be integrated together to form a unified and structured knowledge system, which facilitates subsequent information query and extraction.

[0056] Please refer to Figure 4 According to some embodiments of the present invention, building a medical model based on the medical knowledge graph in step 104, and training and cross-validating the medical model to obtain a target medical model may specifically include, but not limited to, the following: 401. Building a medical model based on a semi-supervised machine learning model; 402. Acquire a medical data training collection according to the medical knowledge graph, wherein the medical data training collection includes labeled medical data and unlabeled medical data; 403. Training the medical model according to the medical data training collection to obtain an initial medical model; 404. Validating the initial medical model using a ten-fold cross validation method; 405. When it is determined that the verification result of the initial medical model meets the preset requirements, the target medical model is obtained.

[0057] In the embodiments of the present application, semi-supervised learning is a learning method that combines the characteristics of supervised learning and unsupervised learning. It can use a small amount of labeled data and a large amount of unlabeled data to improve the performance of the model. In the medical field, since labeling medical data requires professional medical knowledge and a lot of time, labeled medical data is often scarce. Therefore, the use of a semi-supervised learning model can use these limited labeled data and rich unlabeled data to build a more accurate and robust medical model.

[0058] Furthermore, a medical data training collection is obtained based on the medical knowledge graph. Specifically, the medical knowledge graph is a structured knowledge base containing rich medical entities and relationships, which can provide a medical data training collection for the training of medical models. First, it is necessary to extract medical entities and relationships related to medical model training from the medical knowledge graph, such as diseases, symptoms, examination indicators, drugs, etc. Then, based on these entities and relationships, the corresponding labeled and unlabeled medical data are screened from the medical database to form a medical data training collection.

[0059] Among them, labeled medical data refers to data that has been marked with categories or labels, such as confirmed disease cases, known treatment effects, etc., and unlabeled medical data refers to data that has no marked categories or labels, such as patients' examination index data, undiagnosed disease symptoms, etc., which are not specifically limited here.

[0060] Finally, the acquired medical data training collection is used to train the medical model. First, the labeled medical data is used as the training set, and the medical model is initially trained using the supervised learning method. Then, the unlabeled medical data and semi-supervised learning strategy are used to further optimize and adjust the medical model. In this process, the medical model will continuously learn and adjust parameters to better fit the distribution and rules of the medical data.

[0061] Through continuous iteration and optimization, we can eventually obtain an initial medical model, which has the initial ability to process medical data, but still needs further verification and adjustment to become the final target medical model.

[0062] After obtaining the initial medical model, the ten-fold cross-validation method is used to verify the initial medical model. Specifically, the medical data training collection is divided into ten equal parts, nine of which are used as training sets and the remaining one as a test set in turn to train and test the model. In this way, each data point will be used as a test set once, thereby obtaining the performance indicators of the ten models.

[0063] By calculating the mean and standard deviation of these ten performance indicators, the stability and generalization ability of the initial medical model can be evaluated. If the performance indicators of the model meet the preset requirements, it means that the performance of the model is good, and a target medical model can be output. If the performance indicators of the model do not meet the preset requirements, it is necessary to readjust the model parameters or select other semi-supervised learning algorithms for training.

[0064] Figure 5 According to some embodiments of the present invention, the processing of the real-time video data in step 107 to extract the user's identity information and skeleton key point information in the real-time video data may specifically include, but is not limited to, the following: 501. Identify the position and action posture of the user in the real-time video data; 502. Determine the facial features of the user according to the position of the user, and obtain the identity information of the user through the facial features; 503. Track and identify the human skeleton of the user according to the posture and movement of the user to obtain the body movement angle of the user, where the body movement angle includes the angles of neck bending, trunk bending, trunk thigh angle and knee joint angle.

[0065] In the embodiment of the present application, the user's position and action posture in the real-time video data are first identified, aiming to determine the user's specific position in the video and their action posture. Specifically, the user's position can be identified in the real-time video data through computer vision technology, such as target detection algorithms. These algorithms analyze the pixel information in the video frame and find the area that matches the human body features, thereby determining the coordinate position of the user in the video.

[0066] After determining the user's position, posture estimation algorithms such as OpenPose or DeepLabCut are further used to identify the user's posture. These algorithms analyze the relative positions of various parts of the user's body, such as the head, shoulders, elbows, knees, etc., as well as the connection relationships between them, to construct a skeleton model of the user and estimate the user's posture, such as standing, walking, sitting, etc.

[0067] Furthermore, the user's facial features are determined based on the user's location, and the user's identity information is obtained through the facial features. Specifically, the extracted facial features are compared with a pre-stored facial feature database, and the user's identity is identified by calculating the similarity. If the similarity exceeds a set threshold, it is considered that the match is successful, thereby obtaining the user's identity information, such as name, age, gender, etc.

[0068] Furthermore, based on the identification of the user's movement posture, the human skeleton is tracked and identified, and the user's body movement angle is calculated. The neck bending angle is obtained by measuring the angle between the neck bone and the trunk bone, the trunk thigh angle is obtained by measuring the angle between the trunk bone and the thigh bone, and the knee joint angle is obtained by measuring the angle between the thigh bone and the calf bone and the angle between the calf bone and the foot. These angle information can be used to analyze the user's body posture and movement conditions, and provide data support for subsequent medical and health analysis.

[0069] Please refer to Figure 6 According to some embodiments of the present invention, preprocessing the medical treatment data in step 102 to obtain preprocessed data may specifically include, but is not limited to, the following: 601. Process the medical treatment data by performing duplicate value and missing value processing, format and coding consistency check, logic verification, and control mapping processing to obtain processing data; 602. Standardize the processed data to obtain pre-processed data.

[0070] In an embodiment of the present application, the medical treatment data is subjected to duplicate value and missing value processing, format and coding consistency check, logic verification and control mapping processing to obtain processed data. After the above processing, since the medical treatment data may come from different data sources or systems, the format and coding of the data may be inconsistent. In order to ensure the consistency and comparability of the data, it is necessary to check and unify the format and coding of the data. After standardization processing, preprocessed data can be obtained, and the obtained preprocessed data provides a more reliable and effective input for subsequent modeling and analysis.

[0071] See also Figure 7 In a second aspect, the present application provides a medical information processing system based on video data, the system comprising: A first acquisition unit 701 is used to acquire medical treatment data; A second acquisition unit 702 is used to pre-process the medical treatment data to obtain pre-processed data; The first construction unit 703 is used to construct a medical knowledge graph, wherein the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators and drugs; A third acquisition unit 704 constructs a medical model based on the medical knowledge graph, and trains and cross-validates the medical model to obtain a target medical model; A generating unit 705 is used to generate a healthy digital identity model according to the preprocessed data and the target medical model; The fourth acquisition unit 706 is used to acquire real-time video data; An extraction unit 707 is used to process the real-time video data to extract the user's identity information and skeleton key point information in the real-time video data; The fifth acquisition unit 708 is used to input the user's identity information and skeleton key point information into the healthy digital identity model to obtain the user's medical health information, where the medical health information includes medical advice, health recommendations or health forecasts.

[0072] See also Figure 8 The present application also provides a medical information processing device based on video data, comprising: Processor 801, memory 802, input and output unit 803, bus 804; The processor 801 is connected to the memory 802, the input and output unit 803 and the bus 804; The memory 802 stores a program, and the processor 801 calls the program to execute any of the above methods.

[0073] The present application also relates to a computer-readable storage medium on which a program is stored, wherein when the program is run on a computer, the computer is caused to execute any of the above methods.

[0074] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0075] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0076] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0077] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0078] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A medical information processing method based on video data, characterized in that: The method comprises: Access to medical treatment data; Preprocessing the medical treatment data to obtain preprocessing data; Constructing a medical knowledge graph, wherein the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators and drugs; Building a medical model based on the medical knowledge graph, and training and cross-validating the medical model to obtain a target medical model; Constructing and generating a healthy digital identity model based on the preprocessed data and the target medical model; Get real-time video data; Processing the real-time video data to extract the user's identity information and skeleton key point information in the real-time video data; The user's identity information and skeletal key point information are input into the healthy digital identity model to obtain the user's medical health information, where the medical health information includes medical advice, health recommendations or health forecasts.

2. The medical information processing method based on video data according to claim 1, characterized in that: Access to medical treatment data, including: Determine where medical data is stored; The medical data is extracted from the storage location, and the medical data is desensitized by generalization, randomization, rearrangement, substitution, shielding or replacement to obtain the medical treatment data.

3. The medical information processing method based on video data according to claim 1, characterized in that: Construct a medical knowledge graph, which is used to represent the relationship between diseases, symptoms, examination indicators, and the relationship between diseases and drugs, including: Access to medical information resources, including clinical medical data, scientific publications, reference standard libraries, and openly shared medical knowledge bases; Extracting structured medical resource information and unstructured medical resource information from the clinical medical data, the scientific publications, the reference standard library, and the open and shared medical knowledge base; A medical knowledge graph is constructed based on the structured medical resource information and the unstructured medical resource information, and the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators and drugs.

4. The medical information processing method based on video data according to claim 1, characterized in that: A medical model is constructed based on the medical knowledge graph, and the medical model is trained and cross-validated to obtain a target medical model, including: Build a medical model based on a semi-supervised machine learning model; Acquire a medical data training collection according to the medical knowledge graph, wherein the medical data training collection includes labeled medical data and unlabeled medical data; Training the medical model according to the medical data training collection to obtain an initial medical model; Using a ten-fold cross validation method to validate the initial medical model; When it is determined that the verification result of the initial medical model meets the preset requirements, the target medical model is obtained.

5. The method for processing medical information based on video data according to claim 1, characterized in that: Processing the real-time video data to extract the user's identity information and skeleton key point information in the real-time video data includes: Identifying the position and movement posture of the user in the real-time video data; Determine the user's facial features according to the user's location, and obtain the user's identity information through the facial features; The human skeleton of the user is tracked and identified according to the posture of the user to obtain the body movement angle of the user, wherein the body movement angle includes the angles of neck bending, trunk bending, trunk thigh angle and knee joint angle.

6. The method for processing medical information based on video data according to claim 1, characterized in that: Preprocessing the medical treatment data to obtain preprocessed data includes: Processing the medical treatment data by performing duplicate value and missing value processing, format and coding consistency check, logic verification and control mapping processing to obtain processing data; The processed data are standardized to obtain pre-processed data.

7. The method for processing medical information based on video data according to claim 1, characterized in that: Get real-time video data, including: Get real-time video data through high-definition cameras.

8. A medical information processing system based on video data, characterized in that: The system comprises: A first acquisition unit, used for acquiring medical treatment data; A second acquisition unit, used for preprocessing the medical treatment data to obtain preprocessing data; A first construction unit is used to construct a medical knowledge graph, wherein the medical knowledge graph is used to represent the relationship between diseases, symptoms, examination indicators and drugs; A third acquisition unit constructs a medical model based on the medical knowledge graph, and trains and cross-validates the medical model to obtain a target medical model; A generating unit, configured to generate a healthy digital identity model according to the preprocessed data and the target medical model; A fourth acquisition unit, used to acquire real-time video data; An extraction unit, used for processing the real-time video data to extract the user's identity information and skeleton key point information in the real-time video data; The fifth acquisition unit is used to input the user's identity information and skeleton key point information into the healthy digital identity model to obtain the user's medical health information, where the medical health information includes medical advice, health recommendations or health forecasts.

9. A medical information processing device based on video data, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the method according to any one of claims 1 to 7.

Images