Data processing method and device and electronic equipment

Abstract

The invention provides a data processing method and device and electronic equipment, and the method comprises the steps: obtaining to-be-processed data which comprises user data with different data dimensions and time information; the method comprises the steps of performing structured extraction operation on user data to obtain feature data under a preset feature dimension, performing standardized mapping operation on the feature data to obtain standard feature data, and performing data combination operation on the standard feature data according to time information of the standard feature data to obtain feature combination data in different time periods. Compared with a manual feature extraction mode, when the data processing mode is adopted for feature extraction, the data feature extraction efficiency can be improved, and then the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and more specifically, to a data processing method, device and electronic equipment. Background Art

[0002] In recent years, with the rapid growth and widespread use of electronic health records, the medical field has made significant progress in clinical decision support research such as disease risk assessment, patient outcome assessment, and disease subtype analysis. Currently, it is possible to determine similar relationships between patients by analyzing user data such as patient examinations, drugs, and surgeries, and then build analysis models based on user data of similar patients for subsequent analysis operations.

[0003] When building an analysis model, it is necessary to perform data feature extraction operations on the patient's clinical data in advance. When extracting features, manual extraction is currently used. This method has a slow processing speed, low efficiency, and poor user experience. Summary of the invention

[0004] In view of this, the present invention provides a data processing method, device and electronic device to solve the problem of low efficiency in data feature extraction.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A data processing method, comprising:

[0007] Acquire data to be processed; the data to be processed includes user data of different data dimensions and having time information;

[0008] Performing a structured extraction operation on the user data to obtain feature data under a preset feature dimension;

[0009] Performing a standardized mapping operation on the feature data to obtain standard feature data;

[0010] According to the time information of the standard feature data, a data combination operation is performed on the standard feature data to obtain feature combination data of different time periods.

[0011] Optionally, a structured extraction operation is performed on the user data to obtain feature data under a preset feature dimension, including:

[0012] Performing a data preprocessing operation on the user data to obtain intermediate data; the data preprocessing operation includes at least one of data cleaning, word segmentation, and stop word removal;

[0013] The natural language processing large model is called to perform a structured extraction operation on the intermediate data to obtain feature data under a preset feature dimension.

[0014] Optionally, performing a standardized mapping operation on the feature data to obtain standard feature data includes:

[0015] Obtaining candidate words of the characteristic words in the characteristic data, and calculating the similarity between the characteristic words and the candidate words;

[0016] Screening out candidate words with the highest similarity to the characteristic words and using them as target candidate words;

[0017] The characteristic words are replaced with the target candidate words to obtain standard characteristic data.

[0018] Optionally, according to the time information of the standard feature data, a data combination operation is performed on the standard feature data to obtain feature combination data of different time periods, including:

[0019] Get the preset time period;

[0020] Based on the time information of the standard feature data, the standard feature data in each preset time period are combined to obtain feature combination data of different preset time periods.

[0021] Optionally, after performing a data combination operation on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods, the method further includes:

[0022] Determine a word vector of the feature combination data of each of the preset time periods.

[0023] Optionally, determining the word vector of the feature combination data of each preset time period includes:

[0024] Get word vector conversion reference information;

[0025] Acquire standard feature data in the feature combination data of the preset time period;

[0026] Referring to the word vector conversion reference information, determining the word vector of the word in the standard feature data;

[0027] According to the order of the words in the standard feature data, concatenating the word vectors of the words to obtain the word vectors of the standard feature data;

[0028] Based on the word vector of the standard feature data, the word vector of the feature combination data is calculated.

[0029] Optionally, calculating the word vector of the feature combination data based on the word vector of the standard feature data includes:

[0030] The weighted average sum of the word vectors of the standard feature data is calculated and used as the word vector of the feature combination data.

[0031] Optionally, after determining the word vector of the feature combination data of each preset time period, the method further includes:

[0032] The word vectors of the feature combination data of each preset time period are combined in chronological order to obtain a user feature matrix.

[0033] A data processing device, comprising:

[0034] A data acquisition module is used to acquire data to be processed; the data to be processed includes user data of different data dimensions and with time information;

[0035] A structured extraction module, used to perform a structured extraction operation on the user data to obtain feature data under a preset feature dimension;

[0036] A standardized mapping module, used for performing a standardized mapping operation on the feature data to obtain standard feature data;

[0037] The data combination module is used to perform a data combination operation on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods.

[0038] Optionally, it also includes:

[0039] A word vector determination module, used to determine the word vector of the feature combination data of each preset time period;

[0040] The word vector determination module includes:

[0041] Information acquisition submodule, used to obtain word vector conversion reference information;

[0042] A data acquisition submodule, used to acquire standard feature data in the feature combination data of the preset time period;

[0043] A word vector determination submodule, used to determine the word vector of the word in the standard feature data with reference to the word vector conversion reference information;

[0044] A concatenation operation submodule, used for concatenating the word vectors of the words according to the order of the words in the standard feature data to obtain the word vectors of the standard feature data;

[0045] The word vector calculation submodule is used to calculate the word vector of the feature combination data based on the word vector of the standard feature data.

[0046] Optionally, it also includes:

[0047] The matrix generation module is used to combine the word vectors of the feature combination data of each preset time period in chronological order to obtain a user feature matrix.

[0048] An electronic device comprising: a memory and a processor;

[0049] Wherein, the memory is used to store programs;

[0050] The processor calls the program and is used to execute the above data processing method.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention provides a data processing method, device and electronic device. In the present invention, data to be processed is obtained, and the data to be processed includes user data of different data dimensions and time information. A structured extraction operation is performed on the user data to obtain feature data under a preset feature dimension, a standardized mapping operation is performed on the feature data to obtain standard feature data, and a data combination operation is performed on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods. When the above-mentioned data processing method in the present invention is used for feature extraction, compared with the manual feature extraction method, the efficiency of data feature extraction can be improved, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0054] Figure 1 A method flow chart of a data processing method provided by an embodiment of the present invention;

[0055] Figure 2 A flow chart of a method for obtaining standard feature data provided by an embodiment of the present invention;

[0056] Figure 3 A schematic diagram of a scenario of feature combination data provided by an embodiment of the present invention;

[0057] Figure 4 A flow chart of a method for determining a word vector of feature combination data provided by an embodiment of the present invention;

[0058] Figure 5 A method flow chart of another data processing method provided by an embodiment of the present invention;

[0059] Figure 6 A schematic diagram of the structure of a data processing device provided by an embodiment of the present invention;

[0060] Figure 7 A schematic diagram of the structure of another data processing device provided by an embodiment of the present invention;

[0061] Figure 8 A schematic diagram of the structure of yet another data processing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] In recent years, with the rapid growth and widespread application of electronic health records, the medical field has made significant progress in clinical decision support research such as disease risk assessment, user outcome assessment, and disease subtype analysis.

[0064] By analyzing users, such as clinical data such as examinations, medications, and surgeries of patients during hospitalization, similar relationships between users can be determined. Similar users usually have similar treatment paths and outcomes. By analyzing a large amount of user data, characteristics and risk factors associated with specific diseases can be discovered, and analytical models, such as scoring models, can be constructed. These models can then be used to assess users' potential disease risks and take appropriate intervention measures to prevent or treat the disease early. In addition, by comparing and analyzing users with similar disease characteristics and treatment histories, user outcomes, such as survival rates and recovery status, can be evaluated, which provides appropriate support for developing personalized treatment plans and evaluating the progress of rehabilitation.

[0065] When building the above analysis model or evaluating user outcomes, accurately obtaining each user's user feature data is the basis for similar user disease analysis. User data usually contains multiple observation indicators and time points, such as blood sugar or blood pressure measurement data, which has the characteristics of dynamic changes and time series association, which increases the complexity and computational complexity of feature extraction.

[0066] At present, when extracting features, the traditional manual feature extraction method is used to manually label, annotate or classify user data to achieve feature extraction. The manual feature extraction method is suitable for processing text data with small quantities and high quality requirements. However, for large-scale data with many labeling rules, a lot of manpower and time are required for labeling, which limits the efficiency of data processing. In addition, different labelers may have differences in labeling rules and standards, resulting in inconsistent labeling results. The use of manual extraction methods is time-consuming and labor-intensive and cannot provide consistent and objective labeling results, resulting in low efficiency and accuracy of feature extraction.

[0067] In order to improve the efficiency and accuracy of feature extraction, deep learning or machine learning training models can be used, and the models can be used for feature extraction. Although compared with manual labeling methods, they can quickly discover the relationships and patterns between data and improve the efficiency and accuracy of data analysis when processing large-scale data, machine learning or deep learning has stricter requirements on data input. When processing time series data, time series information is not considered and will be ignored, resulting in the inability to provide the model with data time series information when using the analysis model later.

[0068] If the time series information of the data is increased, the number of network layers needs to be increased to learn deeper features, but this will require a large amount of training data and computing resources. In addition, the models trained by deep learning and machine learning are end-to-end black box models, and it is difficult to explain their internal decision-making process and feature selection, and their explainability is relatively poor.

[0069] In practical applications, due to the large variability among users, the data is diverse and heterogeneous. Therefore, it is necessary to use appropriate feature extraction techniques to extract features from user data. This ensures that the extracted results are interpretable and can be associated with time series features, so that researchers can understand the relationship between data features and user conditions.

[0070] Therefore, in the embodiment of the present invention, the program corresponding to the data processing method is used to automatically perform the feature extraction operation, which can improve the processing efficiency and accuracy compared to the manual feature extraction method. In addition, in the embodiment of the present invention, the processed data to be processed has time information, and the feature combination data finally obtained by feature extraction will also refer to the time information, so the final extracted data has time sequence. In addition, in the embodiment of the present invention, the black box design is not adopted, there are corresponding implementation steps, and the interpretability is good. In addition, the present invention does not require a training model, so it does not require a large amount of training data and computing resources.

[0071] Specifically, the present invention provides a data processing method, device and electronic device. In the present invention, data to be processed is obtained, and the data to be processed includes user data of different data dimensions and time information. A structured extraction operation is performed on the user data to obtain feature data under a preset feature dimension, a standardized mapping operation is performed on the feature data to obtain standard feature data, and a data combination operation is performed on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods. When the above-mentioned data processing method in the present invention is used for feature extraction, compared with the manual feature extraction method, the efficiency of data feature extraction can be improved, thereby improving the user experience.

[0072] Based on the above content, an embodiment of the present invention provides a data processing method, and a program corresponding to the data processing method can be configured in an electronic device, such as a processor, a controller, or a server, so that the device can execute the data processing method.

[0073] Reference Figure 1 , the data processing method may include:

[0074] S11. Obtain data to be processed.

[0075] In detail, the users in this embodiment are generally patients with chronic diseases, and the patients need to go to the hospital for check-ups frequently, so there will be a large amount of patient data, which is called data to be processed in this embodiment.

[0076] The data to be processed includes user data of different data dimensions and time information. For example, the data dimensions may include but are not limited to user basic information, test data, real-world cases, adverse reaction conclusions and levels.

[0077] Among them, user basic information includes but is not limited to gender, age, etc., test data includes but is not limited to blood biochemical data, blood pressure, blood lipids, etc., real-world cases include symptoms, medication conditions, prescriptions and other information, and adverse reaction conclusions and levels specifically refer to adverse reactions to user medications or adverse reactions after surgery.

[0078] The user data corresponding to the above data dimensions have corresponding time information. For example, when blood pressure is tested, there will be corresponding blood pressure test time and blood pressure test value. The blood pressure test time is the time information of this blood pressure test. For another example, if the user takes medication, the user's medication time is the time information corresponding to the medication.

[0079] The data to be processed in this embodiment can be obtained from the database of one hospital or from the databases of multiple hospitals. If the data is obtained from the databases of multiple hospitals, the data to be processed of the same user needs to be integrated, that is, one user corresponds to one data to be processed.

[0080] S12: Perform a structured extraction operation on the user data to obtain feature data under a preset feature dimension.

[0081] In actual applications, feature data under a preset feature dimension will be extracted from the above-mentioned data to be processed, and the preset feature dimension may be, for example, diagnostic information feature, surgical feature, medication feature, etc. The type and quantity of the specific preset feature dimensions may be configured according to the actual scenario.

[0082] After obtaining the above-mentioned data to be processed, it is necessary to perform feature extraction operations on the user data therein. During feature extraction, in order to ensure the standardization and reliability of the processed data, data preprocessing operations can be performed on the user data to obtain intermediate data. Among them, the data preprocessing operations include at least one of data cleaning, word segmentation, and removal of stop words. Data cleaning is to clean out impurities and incorrect data, word segmentation is to avoid the problem of low data processing efficiency caused by processing longer data, and stop words are removed to avoid the impact of stop words on data processing and improve efficiency. In addition, data preprocessing operations such as data proofreading and data filling can also be performed to ensure the accuracy and quantity of the data. In practical applications, each data preprocessing operation can be performed, such as data cleaning, word segmentation, stop word removal, data proofreading, and data filling operations are performed successively to ensure the reliability, accuracy, and standardization of the data to be processed.

[0083] After data preprocessing, feature extraction can be performed on the intermediate data obtained by the data preprocessing. In this embodiment, a large natural language processing model based on local fine-tuning is used for feature extraction.

[0084] The natural language processing large model is used for feature extraction because the large model has been pre-trained. In this embodiment, a large number of samples are no longer needed for model training operations. Instead, the natural language processing large model can be directly called for feature extraction. Compared with the method of retraining the model, the model training time is reduced.

[0085] The natural language processing large model in this embodiment can be a deep learning model, such as a long short-term memory network (LSTM), a convolutional neural network (CNN), and a self-attention mechanism (Self-Attention), etc. These large models can perform feature representation learning on the user's data to be processed. By combining the clinical patient symptom terminology specifications and the structure and expression of common symptoms, the feature patterns and potential associations of the data to be processed can be automatically learned to achieve feature extraction.

[0086] In practical applications, large natural language processing models can be based on deep learning architectures. This type of model uses the self-attention mechanism during training to capture statistical laws and semantic relationships in the language by predicting the next word or ignoring some words. It has strong semantic understanding capabilities and can perform complex processing on large-scale data to achieve natural language processing tasks.

[0087] In addition, the natural language processing model can also be a machine learning model, and machine learning can be divided into supervised learning, semi-supervised learning, and unsupervised learning. In an embodiment of the present invention, the natural language processing model can use a supervised learning method, and the natural language processing model can use, for example, a support vector machine (SVM), a random forest (RandomForest), etc. to achieve feature extraction. After the features are extracted, the supervised learning method can also be used to train the analysis model, build a patient similarity network, and achieve subsequent analysis and processing.

[0088] This embodiment is based on the semantic understanding ability of the local fine-tuned natural language processing large model, and applies this ability to the feature extraction method, which can be independent of manual annotation and feature engineering, and can efficiently process a large amount of user data, reducing human resource costs and time consumption. Through machine learning and deep learning technology, it can automatically mine and extract key information in user data, improving the efficiency and accuracy of data analysis.

[0089] When using a large natural language processing model, the large natural language processing model is directly called to perform structured extraction operations on the intermediate data to obtain feature data under the preset feature dimension.

[0090] The structured extraction in this embodiment is to extract structured data to facilitate subsequent data analysis and processing.

[0091] When using a large natural language processing model, you need to customize the prompt word input, that is, input prompt information for the large natural language processing model in advance. The prompt information can be:

[0092] {"role":"system","content":"Assistant is a very professional doctor"}, which means to inform the natural language processing model of the application fields and application scenarios of the data processed, such as the application scenario of processing patient data in this embodiment.

[0093] Then the prompt information can be:

[0094] {"role":"user","content":"Extract anatomical parts, subject words, descriptive words, and occurrence status from the following text"}, which tells the natural language processing model what content needs to be extracted. It should be noted that the prompt information in this embodiment is only for illustration. In actual applications, the feature data under the above-mentioned preset feature dimensions should be extracted from the above-mentioned user data.

[0095] For example, feature data of diagnostic information features can be extracted from real-world cases. When using the natural language processing big model for feature extraction, the data to be processed by the same user can be input. The natural language processing big model selects the target data required for feature extraction from the data to be processed for each preset feature dimension, and then extracts the target data according to the feature extraction requirements to obtain the feature data of the preset feature dimension.

[0096] In addition, for user data of different time information of the user, the user data of each time information can be extracted separately.

[0097] If the user had an examination on 1.1 and 1.10 respectively and has corresponding user data, feature extraction is performed on the user data on 1.1 and 1.10 respectively, and the final extraction result may be feature data with two features, namely, diagnostic information feature and surgical feature, on 1.1, and feature data with three features, namely, diagnostic information feature, surgical feature, and medication feature, on 1.10. In other words, the number of features under each time information may be different, and at each time, it may not be possible to extract feature data under all preset feature dimensions.

[0098] It should be noted that in addition to using the above-mentioned natural language processing large model for feature extraction, if there are a large number of samples, model training can be performed. At this time, the natural language processing large model can be trained again, or the deep learning model or machine learning model can be retrained, and the trained model can be used for feature extraction.

[0099] S13. Perform a standardized mapping operation on the feature data to obtain standard feature data.

[0100] After extracting the feature data under the preset feature dimension, the feature words in the feature data are not standardized and may be common words in a certain hospital, but other hospitals may not agree with this word. Therefore, in order to unify the data when building the analysis model later, it is necessary to perform standardized mapping operations on the feature data to obtain standard feature data.

[0101] In practical applications, refer to Figure 2 , step S13 may include:

[0102] S21. Obtain candidate words of the feature words in the feature data, and calculate the similarity between the feature words and the candidate words.

[0103] Specifically, when performing standardized mapping operations, the large model can still be used. At this time, based on the standard mapping method of the large model, a most similar candidate word is selected from the candidate words. The candidate words in this embodiment are recognized standard words.

[0104] When using a large model, you also need to enter prompt information for the large model. The prompt information can be as follows:

[0105] “You are an experienced clinician, especially in the field of chronic diseases. Please use all your expertise to complete this task. Now you need to help me complete the vocabulary similarity search task. When you receive a disease input word and 10 disease candidate words, you must find the word that is most similar to the input word from the candidate words without any explanation.”

[0106] By inputting the above prompt information, the big model has the ability to screen candidate words. When screening candidate words, the big model first obtains candidate words of characteristic words in the characteristic data. These candidate words can be manually input or obtained by the big model by searching the disease type from the word library.

[0107] For example, when the characteristic data is the characteristic data under the diagnostic information characteristic, the characteristic data may be a paragraph of text, and the paragraph of text may be as follows:

[0108] Based on xxxx's condition, it was diagnosed as decidualitis.

[0109] In this embodiment, "based on" and "condition" are non-medical professional terms and do not need to be standardized. Medical professional terms generally need to be standardized, that is, "decidualitis" needs to be standardized, so "decidualitis" is used as the characteristic word in this embodiment.

[0110] In practical applications, semantic analysis can be performed on feature data to extract feature words that need to be standardized.

[0111] After determining the characteristic words and candidate words, the similarity between the characteristic words and the candidate words can be calculated by a conventional similarity calculation method, such as Euclidean distance, cosine distance, etc. The similarity between the characteristic words and the candidate words can be calculated by the similarity calculation method.

[0112] S22. Filter out the candidate word with the highest similarity to the feature word and use it as the target candidate word.

[0113] Still taking "decidual inflammation" as an example, the corresponding candidate words are:

[0114] "Endometritis", "premature rupture of membranes", "herpesvirus iritis", "viral meningitis", "acute pericarditis", "acute synovitis", "palatal mucosal inflammation", "hymenolepidium synovitis", "herpesvirus keratoconjunctivitis", "chronic buccal submucous inflammation".

[0115] After calculating the similarity between "decidual inflammation" and each candidate word, the highest similarity is screened out, and the candidate word corresponding to the highest similarity is determined, such as "endometritis", then "endometritis" is the target candidate word in this embodiment.

[0116] For example, the input feature word of the large model is {word}, ​​and the target candidate word is {words_stand}.

[0117] S23. Replace the feature words with target candidate words to obtain standard feature data.

[0118] Specifically, if "decidualitis" in the above feature data is replaced with "endometritis", the feature data is updated as follows:

[0119] Based on xxxx's condition, it was diagnosed as endometritis.

[0120] "Based on xxxx's condition, it is confirmed to be endometritis" is the standard characteristic data.

[0121] In this embodiment, the characteristic words are standardized to ensure the standardization and uniformity of the characteristic data.

[0122] S14. Perform a data combination operation on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods.

[0123] In practical applications, through the above steps S11-S13, standardized standard feature data of the same user under different preset feature dimensions can be obtained, and each standard feature data has corresponding time information, which is the time information of the corresponding feature data.

[0124] For example, suppose user A has feature data of two features, namely diagnosis information features and surgery features, at 1.1, feature data of three features, namely diagnosis information features, surgery features, and medication characteristics at 1.10, feature data of two features, namely surgery features and medication characteristics at 1.12, and feature data of three features, namely diagnosis information features, surgery features, and medication characteristics at 1.23.

[0125] In order to avoid the problem of low efficiency in subsequent data analysis due to the large amount of standard feature data, in this embodiment, the standard feature data of similar time information can be combined. For example, if the user has undergone examinations on two adjacent days and has corresponding standard feature data, the standard feature data of these two days can be combined.

[0126] In practical applications, in order to reduce the amount of subsequent processing, a preset time period can be set in advance, such as three days, one week, two weeks, or one month, etc. It can be set specifically according to the number of user inspections. Then, based on the time information of the standard feature data, the standard feature data within each preset time period is combined to obtain feature combination data for different preset time periods.

[0127] Specific reference Figure 3 , the standard feature data can be sorted in sequence based on the time information, and then the sorted standard feature data can be combined. When combining, the data within the same preset time period can be combined together.

[0128] Figure 3 In the example, the horizontal axis represents the time dimension, and the vertical axis represents the characteristics of different users, such as diagnostic information characteristics, medication information characteristics, and surgical characteristics. In this embodiment, diagnostic information characteristics, medication information characteristics, and surgical characteristics are taken as examples. Figure 3 As shown, the user's standard feature data is arranged and combined based on the time series.

[0129] For example, the standard feature data in t1 are combined together, the standard feature data in t2 are combined together, the standard feature data in t3 are combined together, and the standard feature data in t4 are combined together.

[0130] It should be noted that the combination in this embodiment is simply to put the data together, and no data processing operation is performed.

[0131] Taking the standard feature data in t1 as an example, within the same preset time period, multiple standard feature data are allowed under the same preset feature dimension. For example, within t1, there are three standard feature data for the medication status feature.

[0132] Likewise, within the same preset time period, it is allowed that a certain preset feature dimension does not have standard feature data, for example, within a certain preset time period, there is no standard feature data corresponding to a surgical feature.

[0133] Reference Figure 3 t1, we can see that in t1, the user has two checks (corresponding to Figure 1Each time there is corresponding standard feature data. Since they are all located in t1, the data of these two inspections can be combined together as the feature combination data of t1. t2, t3, and t4 are similar.

[0134] In this embodiment, data to be processed is obtained, and the data to be processed includes user data of different data dimensions and with time information. A structured extraction operation is performed on the user data to obtain feature data under a preset feature dimension, a standardized mapping operation is performed on the feature data to obtain standard feature data, and a data combination operation is performed on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods. When the above-mentioned data processing method in the present invention is used for feature extraction, compared with the manual feature extraction method, the efficiency of data feature extraction can be improved, thereby improving the user experience. In addition, in the present invention, compared with the manual feature extraction method, the accuracy of feature extraction can also be improved.

[0135] In addition, the present invention does not require a training model, which can save computing resources and does not require a large number of data samples.

[0136] In addition, the steps of the present invention are logically clear and interpretable compared to the black box model.

[0137] In addition, the present invention uses a pre-trained natural language processing large model to perform structured extraction of user data and unifies and standardizes data representation methods, making it convenient to compare and integrate user data from different sources, eliminating the problem of data heterogeneity.

[0138] After obtaining the feature combination data of different time periods through the above steps, since the feature combination data containing time series is large in amount and high in dimension, converting the data into word embedding representation can help reduce the dimension of the data, reduce the complexity of the feature space, improve computing efficiency and model training speed, and facilitate the application of data to machine learning and deep learning models. At the same time, it can also better represent the diagnosis and treatment conditions of different time stages, thereby better capturing the user's status changes at different time points.

[0139] After step S14, the word vector of the feature combination data of each preset time period can also be determined, that is, the feature combination data in different time stages are embedded and represented by word embedding.

[0140] Among them, word embedding is a technology that maps words to low-dimensional continuous vector space, which is conducive to reducing the complexity of feature space, improving computational efficiency and model training speed. Common word embedding methods include methods based on co-occurrence matrix, neural network, and pre-trained model. The method based on co-occurrence matrix takes the context of word appearance as the feature of the word, constructs the co-occurrence matrix, and then performs singular value decomposition (Svd) on the matrix to obtain the vector representation of the word. Use neural networks to encode words, such as Word2Vec and GloVe. Word2Vec uses Skip-gram and Cbow models to learn the vector representation of words by training neural networks. GloVe uses neural networks for optimization based on the co-occurrence matrix. Using pre-trained language models such as Bert and Gpt, words are mapped to vector space to achieve vector conversion. This method can obtain more accurate embedding representations, but requires a lot of computing resources and time. Different embedding methods have different advantages and disadvantages, and it is necessary to consider specific application scenarios and needs to choose the appropriate method.

[0141] Specifically, refer to Figure 4 , determining the word vector of the feature combination data of each preset time period may include:

[0142] S31. Obtain word vector conversion reference information.

[0143] In this embodiment, a vocabulary library can be pre-constructed, and the vocabulary library includes word vector conversion reference information. The word vector conversion reference information has a correspondence between words and word vectors, and the correspondence can be expressed in a table form.

[0144] The word vector conversion reference information can be pre-built, and a word list can be constructed for all words that may appear in all users and hospitals. The words in the word list are unique and non-repetitive. The word list serves as the basis for word embedding.

[0145] Then, a word vector determination method is used, such as using the Word2vec model to generate a word vector for each word in the vocabulary table, and a corresponding relationship between the word and the corresponding word vector is established to obtain word vector conversion reference information.

[0146] S32, obtaining standard feature data from the feature combination data of a preset time period.

[0147] Specifically, since the feature combination data may include multiple standard feature data, when performing word embedding, the standard feature data needs to be embedded. Therefore, in this embodiment, the processing object of word embedding is the words in the standard feature data.

[0148] S33. Refer to the word vector conversion reference information to determine the word vector of the word in the standard feature data.

[0149] For example, if a standard feature data is composed of word 1, word 2, word 3, word 4 and word 5, then word 1, word 2, word 3, word 4 and word 5 are obtained respectively.

[0150] For each word, query the corresponding relationship in the word vector conversion reference information to obtain the word vector corresponding to the word.

[0151] S34. Concatenate the word vectors of the words according to the order of the words in the standard feature data to obtain the word vectors of the standard feature data.

[0152] Specifically, since the standard feature data is composed of word 1, word 2, word 3, word 4 and word 5, the standard feature data is a long text composed of multiple words. When the word vectors corresponding to word 1, word 2, word 3, word 4 and word 5 are known, it is necessary to perform a splicing operation on the word vectors of multiple words. When splicing, the words can be spliced ​​sequentially according to the order of the words in the standard feature words to obtain the word vectors of the standard feature data.

[0153] In addition, in order to extract better features during splicing, reinforcement learning methods can be introduced to perform splicing operations. Reinforcement learning methods establish interactions between the agent and the environment, enabling it to take different actions based on data at different time points to achieve the best splicing effect. The introduction of reinforcement learning can also provide support for the optimization and decision-making of subsequent personalized treatment plans.

[0154] S35. Calculate the word vector of the feature combination data based on the word vector of the standard feature data.

[0155] In practical applications, since the feature combination data includes at least one standard feature data, after the word vector of the standard feature data is determined, the word vector of the feature combination data can be determined.

[0156] In this embodiment, in order to represent the complete time series data of a single user, it is necessary to combine the word embedding representations of all standard feature data of the user within the same preset time period. This embodiment adopts the weighted average sum method to combine, that is, the weighted average sum of the word vectors of the standard feature data is calculated and used as the word vector of the feature combination data. The specific formula is as follows:

[0157]

[0158] Among them, E(D) is the word vector of the feature combination data, E(ω n ) is the word vector of each standard feature data, and n is the number of standard feature data within a preset time period.

[0159] Reference Figure 3 In t1, there are six standard feature data under the preset feature dimension. Then, the word vector of each standard feature data is retrieved, and then the weighted average sum is calculated to obtain the word vector of the feature combination data in t1. The word vector is a vector of size 256*256, which is a two-dimensional matrix. t2, t3, and t4 are similar.

[0160] In order to further reduce the amount of data and obtain the word vectors of the same user in all preset time periods, the word vectors of the feature combination data of each preset time period can be combined in chronological order to obtain a user feature matrix. For example, the word vectors corresponding to t1, t2, t3, and t4 are sequentially combined to obtain a three-dimensional matrix. The three-dimensional matrix is ​​the user feature matrix in this embodiment, which is a three-dimensional matrix formed by adding the preset time period dimension on the basis of the above-mentioned two-dimensional matrix.

[0161] After the above processing, the time-series diagnosis and treatment information data of all patients can be obtained, based on which researchers and medical staff can continue to conduct follow-up research and treatment analysis.

[0162] For example, the three-dimensional matrix described above can be used as a direct input for subsequent analysis models, such as developing models to evaluate user outcomes, developing personalized treatment plans, and evaluating rehabilitation progress.

[0163] For example, based on the above three-dimensional matrix, a patient similarity network relationship map can be constructed. Based on the map, the potential risks of the current patients can be inferred to reduce the patients' disease risks and provide more health management solutions.

[0164] In this embodiment, word embedding technology is used to map the user's standard feature data into a two-dimensional matrix of size (256*256), and then a time dimension is added to the two-dimensional matrix according to the time series to obtain a three-dimensional matrix. The three-dimensional matrix can be directly used as the input of the model, providing the model with data with time series information, so that the user's time series information can be considered in subsequent model processing, thereby improving the accuracy of model analysis.

[0165] In addition, this embodiment uses word embedding technology to convert the characteristic data of chronic disease patients into low-dimensional continuous vector representation, thereby achieving effective extraction and representation of patient characteristics. Through word embedding, the similarities and differences between patients can be captured, providing strong support for subsequent data analysis and evaluation.

[0166] In addition, the present invention converts the user's feature combination data of different time periods into word embedding form, which greatly reduces the high dimensionality of the data and the complexity of the feature space. It not only improves the computing efficiency and model training speed, but also can better represent the diagnosis and treatment conditions of different time stages in this way, thereby better capturing the patient's status changes at different time points.

[0167] On the basis of the above, Figure 5 An overall implementation scheme of this embodiment is provided, which specifically includes:

[0168] S41, obtaining data to be processed; the data to be processed includes user data of different data dimensions and having time information.

[0169] S42: Perform a structured extraction operation on the user data to obtain feature data under a preset feature dimension.

[0170] S43. Perform a standardized mapping operation on the feature data to obtain standard feature data.

[0171] S44. Perform a data combination operation on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods.

[0172] S45, determining the word vector of the feature combination data of each preset time period.

[0173] S46, combining the word vectors of the feature combination data of each preset time period in chronological order to obtain a user feature matrix.

[0174] It should be noted that the specific implementation process of each step in this embodiment, please refer to the corresponding description in the above embodiment. In addition, steps S41-S46 in this embodiment can be encapsulated into a data processing model, that is, steps S41-S46 are executed through the model. In addition, the output results of the model can also be input into an analysis model, such as a time series model, so that the time dependency and trend of the data can be better captured through the time series model, providing more accurate scoring and analysis results.

[0175] In this embodiment, we can use big models and natural language processing technology to automatically label and identify user data using word embedding models, thereby reducing the burden of manual labeling and improving data processing efficiency.

[0176] This embodiment can solve the problem of processing various high-dimensional diagnostic data of patients in multiple time series in current clinical applications, so that medical personnel and scientific researchers can obtain standardized patient diagnosis and treatment data, determine similar relationships between patients, evaluate the potential risks of specific diseases, and develop targeted treatment plans and programs.

[0177] Optionally, based on the above data processing method embodiment, another embodiment of the present invention provides a data processing device, referring to Figure 6 , which may include:

[0178] The data acquisition module 11 is used to acquire the data to be processed; the data to be processed includes user data of different data dimensions and with time information;

[0179] The structured extraction module 12 is used to perform a structured extraction operation on the user data to obtain feature data under a preset feature dimension;

[0180] A standardization mapping module 13 is used to perform a standardization mapping operation on the feature data to obtain standard feature data;

[0181] The data combination module 14 is used to perform a data combination operation on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods.

[0182] Further, the structured extraction module 12 is specifically used for:

[0183] Performing data preprocessing operations on user data to obtain intermediate data; the data preprocessing operations include at least one of data cleaning, word segmentation, and stop word removal;

[0184] Call the natural language processing large model to perform structured extraction operations on the intermediate data to obtain feature data under the preset feature dimension.

[0185] Further, the standardized mapping module 13 is specifically used for:

[0186] The candidate words of the characteristic words in the characteristic data are obtained, and the similarity between the characteristic words and the candidate words is calculated, and the candidate words with the highest similarity to the characteristic words are screened out and used as the target candidate words, and the characteristic words are replaced with the target candidate words to obtain the standard characteristic data.

[0187] Further, the data combination module 14 is specifically used for:

[0188] A preset time period is obtained, and based on the time information of the standard feature data, the standard feature data within each preset time period is combined to obtain feature combination data of different preset time periods.

[0189] Further, refer to Figure 7 , also includes a word vector determination module 15, the word vector determination module 15 is used to determine the word vector of the feature combination data of each preset time period.

[0190] Furthermore, the word vector determination module includes:

[0191] An information acquisition submodule 151 is used to acquire word vector conversion reference information;

[0192] The data acquisition submodule 152 is used to acquire the standard feature data in the feature combination data of the preset time period;

[0193] A word vector determination submodule 153, used to determine the word vector of the word in the standard feature data with reference to the word vector conversion reference information;

[0194] A concatenation operation submodule 154 is used to concatenate the word vectors of the words according to the order of the words in the standard feature data to obtain the word vectors of the standard feature data;

[0195] The word vector calculation submodule 155 is used to calculate the word vector of the feature combination data based on the word vector of the standard feature data.

[0196] Furthermore, the word vector calculation submodule 155 is specifically used for:

[0197] Calculate the weighted average sum of the word vectors of the standard feature data and use it as the word vector of the feature combination data.

[0198] Further, refer to Figure 8 , also includes:

[0199] The matrix generation module 16 is used to combine the word vectors of the feature combination data of each preset time period in chronological order to obtain a user feature matrix.

[0200] In this embodiment, data to be processed is obtained, and the data to be processed includes user data of different data dimensions and time information. A structured extraction operation is performed on the user data to obtain feature data under a preset feature dimension, a standardized mapping operation is performed on the feature data to obtain standard feature data, and a data combination operation is performed on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods. When the above-mentioned data processing method in the present invention is used for feature extraction, compared with the manual feature extraction method, the efficiency of data feature extraction can be improved, thereby improving the user experience.

[0201] It should be noted that, for the working process of each module in this embodiment, please refer to the corresponding description in the above embodiment, which will not be repeated here.

[0202] Optionally, based on the above data processing method and device embodiments, another embodiment of the present invention provides an electronic device, including: a memory and a processor;

[0203] Wherein, the memory is used to store programs;

[0204] The processor calls the program and is used to execute the above data processing method.

[0205] In this embodiment, data to be processed is obtained, and the data to be processed includes user data of different data dimensions and time information. A structured extraction operation is performed on the user data to obtain feature data under a preset feature dimension, a standardized mapping operation is performed on the feature data to obtain standard feature data, and a data combination operation is performed on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods. When the above-mentioned data processing method in the present invention is used for feature extraction, compared with the manual feature extraction method, the efficiency of data feature extraction can be improved, thereby improving the user experience.

[0206] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method, characterized in that: include: Get the data to be processed; The data to be processed includes user data of different data dimensions and having time information; Performing a structured extraction operation on the user data to obtain feature data under a preset feature dimension; Performing a standardized mapping operation on the feature data to obtain standard feature data; According to the time information of the standard feature data, a data combination operation is performed on the standard feature data to obtain feature combination data of different time periods.

2. The data processing method according to claim 1, characterized in that: Performing a structured extraction operation on the user data to obtain feature data under a preset feature dimension, including: Performing a data preprocessing operation on the user data to obtain intermediate data; the data preprocessing operation includes at least one of data cleaning, word segmentation, and stop word removal; The natural language processing large model is called to perform a structured extraction operation on the intermediate data to obtain feature data under a preset feature dimension.

3. The data processing method according to claim 1, characterized in that: Performing a standardized mapping operation on the feature data to obtain standard feature data includes: Obtaining candidate words of the characteristic words in the characteristic data, and calculating the similarity between the characteristic words and the candidate words; Screening out candidate words with the highest similarity to the characteristic words and using them as target candidate words; The characteristic words are replaced with the target candidate words to obtain standard characteristic data.

4. The data processing method according to claim 1, characterized in that: According to the time information of the standard feature data, a data combination operation is performed on the standard feature data to obtain feature combination data of different time periods, including: Get the preset time period; Based on the time information of the standard feature data, the standard feature data in each preset time period are combined to obtain feature combination data of different preset time periods.

5. The data processing method according to claim 1, characterized in that: After performing a data combination operation on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods, the method further includes: Determine a word vector of the feature combination data of each of the preset time periods.

6. The data processing method according to claim 5, characterized in that: Determining the word vector of the feature combination data of each preset time period includes: Get word vector conversion reference information; Acquire standard feature data in the feature combination data of the preset time period; Referring to the word vector conversion reference information, determining the word vector of the word in the standard feature data; According to the order of the words in the standard feature data, concatenating the word vectors of the words to obtain the word vectors of the standard feature data; Based on the word vector of the standard feature data, the word vector of the feature combination data is calculated.

7. The data processing method according to claim 6, characterized in that: Calculating the word vector of the feature combination data based on the word vector of the standard feature data includes: The weighted average sum of the word vectors of the standard feature data is calculated and used as the word vector of the feature combination data.

8. The data processing method according to claim 5, characterized in that: After determining the word vector of the feature combination data of each preset time period, the method further includes: The word vectors of the feature combination data of each preset time period are combined in chronological order to obtain a user feature matrix.

9. A data processing device, characterized in that: include: A data acquisition module, used to acquire data to be processed; The data to be processed includes user data of different data dimensions and having time information; A structured extraction module, used to perform a structured extraction operation on the user data to obtain feature data under a preset feature dimension; A standardized mapping module, used for performing a standardized mapping operation on the feature data to obtain standard feature data; The data combination module is used to perform a data combination operation on the standard feature data according to the time information of the standard feature data to obtain feature combination data of different time periods.

10. The data processing device according to claim 9, characterized in that: Also includes: A word vector determination module, used to determine the word vector of the feature combination data of each preset time period; The word vector determination module includes: Information acquisition submodule, used to obtain word vector conversion reference information; A data acquisition submodule, used to acquire standard feature data in the feature combination data of the preset time period; A word vector determination submodule, used to determine the word vector of the word in the standard feature data with reference to the word vector conversion reference information; A concatenation operation submodule, used for concatenating the word vectors of the words according to the order of the words in the standard feature data to obtain the word vectors of the standard feature data; The word vector calculation submodule is used to calculate the word vector of the feature combination data based on the word vector of the standard feature data.

11. The data processing device according to claim 10, characterized in that: Also includes: The matrix generation module is used to combine the word vectors of the feature combination data of each preset time period in chronological order to obtain a user feature matrix.

12. An electronic device, characterized in that: include: Memory and processor; Wherein, the memory is used to store programs; The processor calls the program and is used to execute the data processing method according to any one of claims 1 to 8.

Images