Information feature extraction method, device, readable storage medium and electronic device

Abstract

The present disclosure relates to an information feature extraction method, device, readable storage medium and electronic device, the method comprising: obtaining user visit record information, constructing structured data according to the user visit record information, constructing a feature representation of the user visit record information based on the structured data, the feature representation comprising spatial features, temporal features and spatiotemporal features of the user visit record information, extracting the spatiotemporal dependency features of the user visit record information according to the spatial features, temporal features and spatiotemporal features, the spatiotemporal dependency features being the semantic relationship and time series features between target events in multiple visit records of the user. While incorporating prior knowledge related to the user visit record information, the features of the user visit record information are captured in all directions from the three dimensions of time, space and spatiotemporal, learning a more robust and accurate feature representation of the user visit record information, and improving the prediction effect of the disease prediction task.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to an information feature extraction method, device, readable storage medium and electronic device. Background Art

[0002] Medical information prediction refers to the task of using computer technology to predict the diseases that a patient may suffer from in the future based on his or her historical medical records. Medical information prediction can assist doctors in improving work efficiency and reducing misdiagnosis rates, and has great application value in the medical field. It is also an important research field in medical informationization. With the accumulation of a large amount of medical data and the development of data mining technology, research on personalized medicine and precision medicine is also constantly progressing. However, due to the complex, sparse, and high-dimensional characteristics of medical data, traditional data mining algorithms cannot achieve satisfactory prediction results. In recent years, deep learning technology has achieved remarkable results in the medical field, especially in the field of medical information prediction, and has produced many excellent methods and achieved good results. The core of these methods revolves around how to accurately and effectively extract the features of patient medical record information. Therefore, feature extraction of medical record information has become the core issue of disease prediction.

[0003] Therefore, there is an urgent need for a method that can accurately extract the features of medical record information. Summary of the invention

[0004] The purpose of the present disclosure is to provide an information feature extraction method, device, readable storage medium and electronic device to solve the problem of inaccurate disease prediction caused by inaccurate feature extraction of existing medical record information.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an information feature extraction method, comprising:

[0006] Optionally, user visit record information is obtained, and structured data is constructed based on the user visit record information, wherein the user visit record information includes multiple visit records of the user; a feature representation of the user visit record information is constructed based on the structured data, and the feature representation includes spatial features, temporal features and spatiotemporal features of the user visit record information; the spatial features represent the semantic relationship between target events in the same visit record, the temporal features are the time series features between the same target event in different visit records, and the spatiotemporal features represent the semantic relationship between target events in different visit records; based on the spatial features, the temporal features and the spatiotemporal features, the spatiotemporal dependency features of the user visit record information are extracted; the spatiotemporal dependency features represent the semantic relationship and time series features between target events in multiple visit records of the user.

[0007] Optionally, the step of constructing a feature representation of the user visit record information based on the structured data includes: constructing a feature matrix of the user visit record information according to the structured data; the feature matrix is ​​a matrix of original data representing the user visit record information; constructing a spatial adjacency matrix of the user visit record information according to the structured data; the spatial adjacency matrix is ​​a matrix representing the spatial features of the user visit record information; extracting the time features in the user visit record information; constructing a spatiotemporal adjacency matrix of the user visit record information according to the structured data; the spatiotemporal adjacency matrix is ​​a matrix representing the spatiotemporal features of the user visit record information.

[0008] Optionally, the step of extracting the spatiotemporal dependency features of the user visit record information based on the spatial features, the temporal features and the spatiotemporal features includes: calculating the spatial tensor of the user visit record information based on the feature matrix and the spatial adjacency matrix; based on the temporal features, performing a temporal convolution operation on the spatial tensor in the time dimension to obtain a temporal tensor; performing a spatiotemporal convolution operation on the time tensor and the spatiotemporal adjacency matrix to obtain the spatiotemporal dependency features.

[0009] Optionally, the step of calculating the spatial tensor of the user visit record information based on the feature matrix and the spatial adjacency matrix includes: expanding the feature matrix through a convolutional neural network to obtain an extended feature matrix; performing a graph convolution operation on the extended feature matrix and the spatial adjacency matrix to obtain the spatial tensor.

[0010] Optionally, the calculation formula of the space tensor includes:

[0011]

[0012] in, D is the degree matrix, GA is the spatial adjacency matrix, and I is the identity matrix of the feature matrix; represents the graph convolution operation; x′ is the extended feature matrix, x′=∑x·W cnn +b, x is the feature matrix, x∈R |V|×(T) , |V| is the number of target events, T is the number of visit records, b is the bias vector of the convolutional neural network, W cnn is the weight matrix of the convolutional neural network, W cnn ∈R d×d , d is the feature dimension of the extended feature matrix; W gcn is the weight matrix of the graph convolution.

[0013] Optionally, the calculation formula of the time tensor includes:

[0014]

[0015] Wherein, |V| is the number of target events, d is the feature dimension of the extended feature matrix, and W tcn is the weight matrix of the temporal convolution, F spatial(*,-1,*) is the spatial tensor, T is the number of access records, b tcn is the bias vector of the temporal convolution.

[0016] Optionally, the calculation formula of the spatiotemporal dependency feature includes:

[0017]

[0018] in, GTA is the spatiotemporal adjacency matrix, D is the degree matrix, I is the identity matrix of the feature matrix, Indicates graph convolution operation; F temporal(|V|,T-1,d) is the time tensor, |V| is the number of target events, T is the number of visit records, and d is the feature dimension of the extended feature matrix; W stcn is the weight matrix of the spatiotemporal convolution.

[0019] Optionally, after the step of extracting the spatiotemporal dependency features of the user visit record information according to the spatial features, the temporal features and the spatiotemporal features, the step includes:

[0020] Extracting the spatiotemporal dependency features through a fully connected neural network to generate an extraction vector;

[0021] The loss calculation is performed on the extracted vector and the label vector to obtain a loss function; the label vector is obtained based on the last visit record in the user visit record information;

[0022] The parameters of the feature extraction model are updated according to the loss function; the feature extraction model is used to extract the spatiotemporal dependency features, and predict the next visit record of the user according to the spatiotemporal dependency features.

[0023] Optionally, the label vector is y p ∈{0,1} |V| , |V| is the number of the target events, and the calculation formula of the extraction vector includes:

[0024] The calculation formula of the loss function includes:

[0025]

[0026] Where i is the number of target events, F stis the spatiotemporal dependency feature, FCN(F st ) represents the features extracted from the spatiotemporal dependent features by the fully connected neural network, y i is the label vector corresponding to the i-th target event in the last visit record, is FCN(F stcn ) is the extraction vector corresponding to the i-th target event.

[0027] According to a second aspect of an embodiment of the present disclosure, there is provided an information feature extraction device, comprising: an acquisition module, for acquiring user visit record information, and constructing structured data according to the user visit record information, wherein the user visit record information includes multiple visit records of the user; a processing module, for constructing a feature representation of the user visit record information based on the structured data, wherein the feature representation includes spatial features, temporal features and spatiotemporal features of the user visit record information; the spatial features represent the semantic relationship between target events in the same visit record, the temporal features are the time series features between the same target event in the different visit records, and the spatiotemporal features represent the semantic relationship between target events in different visit records; an extraction module, for extracting the spatiotemporal dependency features of the user visit record information according to the spatial features, the temporal features and the spatiotemporal features; the spatiotemporal dependency features represent the semantic relationship and time series features between target events in the multiple visit records of the user.

[0028] According to a third aspect of an embodiment of the present disclosure, there is provided a non-temporary computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the aforementioned information feature extraction method are implemented.

[0029] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the steps of the aforementioned information feature extraction method.

[0030] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: the present disclosure can obtain user visit record information, construct structured data based on the user visit record information, and construct a feature representation of the user visit record information based on the structured data. The feature representation includes the spatial features, time features, and spatiotemporal features of the user visit record information. Based on the spatial features, time features, and spatiotemporal features, the spatiotemporal dependency features of the user visit record information are extracted. The spatiotemporal dependency features are the semantic relationships and time series features between target events in the user's multiple visit records. While incorporating prior knowledge related to the user visit record information, the features of the user visit record information are captured in all directions from the three dimensions of time, space, and spatiotemporal, and a more robust and accurate feature representation of the user visit record information is learned to improve the prediction effect of the disease prediction task.

[0031] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0033] Figure 1 It is a schematic diagram of the structure of a computer system shown in an exemplary embodiment of the present disclosure.

[0034] Figure 2 It is a flow chart of an information feature extraction method shown in an exemplary embodiment of the present disclosure.

[0035] Figure 3 The figure is a schematic diagram of structured data of medical record information shown in an exemplary embodiment of the present disclosure.

[0036] Figure 4 It is a flow chart of another information feature extraction method shown in an exemplary embodiment of the present disclosure.

[0037] Figure 5 is a flowchart of sub-steps of step S102 shown in an exemplary embodiment of the present disclosure.

[0038] Figure 6 is a flowchart of sub-steps of step S103 shown in an exemplary embodiment of the present disclosure.

[0039] Figure 7 It is a schematic diagram of an exemplary embodiment of the present disclosure showing that a spatial tensor is obtained by performing a graph convolution operation on an extended feature matrix and a spatial adjacency matrix.

[0040] Figure 8It is a schematic diagram of an exemplary embodiment of the present disclosure showing that a time tensor is obtained by performing a time convolution operation on a spatial tensor in the time dimension.

[0041] Fig. 9 It is a schematic diagram of an exemplary embodiment of the present disclosure showing how to perform a spatiotemporal convolution operation on a time tensor and a spatiotemporal adjacency matrix to obtain spatiotemporal dependency features.

[0042] Fig.10 It is a block diagram of an information feature extraction device shown in an exemplary embodiment of the present disclosure.

[0043] Fig.11 It is a block diagram of an electronic device shown in an exemplary embodiment of the present disclosure.

[0044] Fig.12 is a block diagram of another electronic device according to an exemplary embodiment of the present disclosure.

[0045] Description of Reference Numerals

[0046] 120-terminal; 140-server; 20-information feature extraction device; 201-acquisition module; 203-processing module; 205-extraction module; 400-electronic device; 401-processor; 402-memory; 403-multimedia component; 404-input / output (I / O) interface; 405-communication component; 500-electronic device; 522-processor; 532-memory; 526-power supply component; 550-communication component; 558-input / output (I / O) interface. DETAILED DESCRIPTION

[0047] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0048] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are only examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0049] At present, in the field of medical information prediction, the feature extraction methods of medical record information are mainly divided into three categories:

[0050] 1. Time series neural network model: Multi-hot encoding is used to model the user's medical treatment characteristics, and the time series features are captured and extracted through the recurrent neural network (RNN) cluster series correlation model.

[0051] 2. Attention mechanism model. Incorporating medical prior knowledge, the attention mechanism is used to model the features of user medical record information and perform feature extraction. The attention mechanism model introduces medical ontology to encode the structure and relationship of medical concepts, narrowing the search space, but lacks modeling of the overall structure of user medical record information.

[0052] 3. Graph convolution model. Graph convolution technology is used to capture the spatiotemporal characteristics of user medical record information in combination with prior knowledge for feature extraction. The time series model cannot capture the fine-grained development pattern of the user's condition and is easily affected by the noise data of the user's electronic medical record. The graph convolution model is currently the most cutting-edge feature extraction model in academia. It models the user's medical record information as a graph structure and captures user features from both spatial and temporal dimensions. However, it ignores the correlation between the user's medical record information.

[0053] In order to solve the above problems, the present invention introduces a spatiotemporal convolutional network and proposes a new medical record information feature extraction model. While incorporating medical prior knowledge, it captures the user's medical record information features from three dimensions: time, space, and spatiotemporal. It obtains the medical record information features in a more comprehensive and fine-grained manner, learns more robust and accurate user feature representations, and improves the prediction effect of disease prediction tasks.

[0054] Figure 1 A schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure is shown. The computer system includes a terminal 120 and a server 140 .

[0055] The terminal 120 and the server 140 are connected to each other via a wired or wireless network.

[0056] The terminal 120 may include at least one of a smart phone, a laptop computer, a desktop computer, a tablet computer, a smart speaker, and a smart robot.

[0057] The terminal 120 includes a display; the display can be used to display the extraction results of the feature extraction model.

[0058] The terminal 120 includes a first memory and a first processor. A first program is stored in the first memory; the first program is called and executed by the first processor to implement the information feature extraction method provided in the present disclosure. The first memory may include but is not limited to the following: Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), and Electric Erasable Programmable Read-Only Memory (EEPROM).

[0059] The first processor may be composed of one or more integrated circuit chips. Optionally, the first processor may be a general-purpose processor, such as a central processing unit (CPU) or a network processor (NP). Optionally, the first processor may implement the information feature extraction method provided by the present disclosure by calling a feature extraction model. Exemplarily, the trained feature extraction model in the terminal may be obtained by training the terminal; or, may be obtained by training the server, and the terminal obtains it from the server.

[0060] The server 140 includes a second memory and a second processor. A second program is stored in the second memory, and the second program is called by the second processor to implement the information feature extraction method provided by the present disclosure. Exemplarily, a feature extraction model is stored in the second memory, and the feature extraction model is called by the second processor to implement the information feature extraction method. Optionally, the second memory may include but is not limited to the following: RAM, ROM, PROM, EPROM, EEPROM. Optionally, the second processor may be a general-purpose processor, such as a CPU or NP.

[0061] The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this disclosure.

[0062] Exemplarily, the information feature extraction method provided by the present disclosure can be used for feature extraction of knowledge graphs in multiple fields such as medical treatment, shopping, recommendation, etc. For example, in the medical field, it can be used to predict the patient's next medical record information based on the patient's previous medical records.

[0063] See also Figure 2 , Figure 2 The flowchart of an information feature extraction method shown in the exemplary embodiment of the present disclosure is shown in FIG. The method is executed by a computer device, for example, Figure 1 The computer system shown is executed by a terminal or a server. Figure 2 The information feature extraction method shown includes the following steps:

[0064] In step S101, user visit record information is obtained, and structured data is constructed according to the user visit record information.

[0065] It should be noted that, in order to explain the present disclosure more clearly, the application of the present disclosure in the medical field is taken as an example, and the user visit record information is taken as the user medical record information for explanation.

[0066] The user's visit record information includes the user's multiple visit records, that is, the user's medical record information includes the user's multiple medical records. In addition, the user's medical record information also includes the medical event (such as disease, symptoms, medication, etc.) nodes contained in each medical record, the specific time and specific content of each medical event, as well as the semantic relationship (or association relationship) between medical events in the same medical record, the time relationship between the same medical event in different medical records, and the semantic relationship between medical events in different medical records.

[0067] According to the user's medical record information, Figure 3 The structured data shown in the figure shows a user's medical record. Figure 3 There are three faces in it, which means the user has three medical records, where V represents medical events. Figure 3 Each visit record includes four medical events. Arrow a represents the semantic relationship between medical events in the same visit record; arrow b represents the temporal relationship between the same medical event in different visit records; arrow c represents the semantic relationship between medical events in different visit records.

[0068] In step S102, a feature representation of user visit record information is constructed based on structured data.

[0069] Exemplarily, the feature representation includes the spatial features, temporal features and spatiotemporal features of the user's visit record information, that is, the spatial features, temporal features and spatiotemporal features of the user's medical treatment record information.

[0070] Please continue reading Figure 3 The line connected by arrow a models the spatial semantic relationship between medical events, which can also be called the spatial characteristics of medical record information. The spatial characteristics include hierarchical relationships, synonymous relationships between concepts, and semantic relationships that reflect medical business rules customized based on medical experience; the line connected by arrow b models the time series relationship between the same medical events in different medical records, which can also be called the time series characteristics of medical record information. The time series characteristics reflect the impact of time on medical events; the line connected by arrow c models the semantic relationship between medical events in the joint dimension of space and time, which can also be called the spatiotemporal characteristics of medical record information. The spatiotemporal characteristics reflect the impact of time and space on medical events.

[0071] In step S103, the spatiotemporal dependency features of the user visit record information are extracted based on the spatial features, the temporal features and the spatiotemporal features.

[0072] It should be noted that the spatiotemporal dependency feature represents the semantic relationship and time series characteristics between target events in the user's medical record information.

[0073] Based on the spatial features, temporal features and spatiotemporal features obtained above, a fully connected neural network is used to extract the spatiotemporal dependency features of the user's medical record information.

[0074] See also Figure 4 , Figure 4 FIG. 1 is a flowchart of another information feature extraction method shown in an exemplary embodiment of the present disclosure. The method is executed by a computer device, for example, Figure 1 The computer system shown is executed by a terminal or a server. Figure 4 The information feature extraction method shown includes the following steps:

[0075] In step S101, user visit record information is obtained, and structured data is constructed according to the user visit record information.

[0076] It should be noted that Figure 4 Steps S101-S103 of the information feature extraction method shown in FIG. Figure 2 The implementation method of the information feature extraction method shown is consistent with the content, Figure 4 For reference not mentioned in Figure 2 The description will not be repeated here. Figure 4 The information feature extraction method shown includes the following steps:

[0077] In step S102, a feature representation of user visit record information is constructed based on structured data.

[0078] It should be noted that step S102 may also include sub-steps S1021, S1022, S1023 and S1024. The specific method of constructing the characteristic representation of the user visit record information based on the structured data will be described in detail in the sub-steps of step S102. Figure 5 , Figure 5 is a flowchart of sub-steps of step S102 shown in an exemplary embodiment of the present disclosure.

[0079] In sub-step S1021, a feature matrix of user visit record information is constructed based on the structured data.

[0080] The feature matrix is ​​a matrix representing the original data of the user's visit record information, that is, a matrix of the original data of the user's medical information record.

[0081] For example, the user's medical information record is used for illustration, and the user's structured data is defined as G = (V, E), where V is the set of medical event nodes, V = {v1, v2, v3, ..., v |v|}, |V| is the total number of medical event nodes. Each medical event node V is encoded as a one-hot encoding vector (one-hot vector), which can be expressed as v * ∈(0,1) |V| . E is the set of edges, which can be expressed as E = {e1, e2, e3, …, e |H|}, |H| is the total number of edges, please continue to refer to Figure 3 , where V nodes represent medical events, or medical event nodes, and the lines connecting arrows a, b, and c represent edges. Therefore, the feature matrix of users with T medical records can be expressed as P p ∈R |V|×|T-1| , a medical record is defined as a vector

[0082] In sub-step S1022, a spatial adjacency matrix of user visit record information is constructed according to the structured data.

[0083] The spatial adjacency matrix is ​​a matrix that characterizes the spatial characteristics of the user's visit record information, that is, a matrix of the spatial characteristics of the user's medical information record.

[0084] As defined above for structured data, the spatial adjacency matrix can be expressed as GA∈R |V|×|V| .

[0085] In sub-step S1023, the time feature in the user visit record information is extracted.

[0086] Exemplarily, time features in the user's medical consultation information records are extracted, including the time of each medical consultation record and the time of each medical event in each medical consultation record.

[0087] In sub-step S1024, a spatiotemporal adjacency matrix of user visit record information is constructed based on the structured data.

[0088] The spatiotemporal adjacency matrix is ​​a matrix that represents the spatiotemporal characteristics of the user's visit record information, that is, the matrix of the spatiotemporal characteristics of the user's medical information record.

[0089] As defined above for structured data, the spatiotemporal adjacency matrix can be expressed as GTA∈R |V|×|V| .

[0090] It should be noted that there is no order relationship between the execution of sub-steps S1021 - S1024 and they can also be executed simultaneously.

[0091] In step S103, the spatiotemporal dependency features of the user visit record information are extracted based on the spatial features, the temporal features and the spatiotemporal features.

[0092] It should be noted that step S103 may also include sub-steps S1031, S1032 and S1033. The specific method of extracting the spatiotemporal dependency features of the user visit record information will be described in detail in the sub-steps of step S103. Figure 6 , Figure 6 is a flowchart of sub-steps of step S103 shown in an exemplary embodiment of the present disclosure.

[0093] In sub-step S1031, the spatial tensor of the user visit record information is calculated based on the feature matrix and the spatial adjacency matrix.

[0094] For example, the user's medical record information is used for illustration. The feature matrix x of the user's medical record information is input, x∈R |V|×(T-1), the extended feature matrix x′ is obtained by expanding the feature matrix through a convolutional neural network, and the shape of x′ is (|V|, d), where |V| is the total number of medical event nodes and d is the feature dimension of x′. In one embodiment, the convolutional neural network can be a two-dimensional convolutional neural network (2D CNN), and the convolution kernel of the 2D CNN is |1×1| and the step size is (1, 1).

[0095] Perform graph convolution on the expanded feature matrix and the spatial adjacency matrix to obtain a spatial tensor. Figure 7 , Figure 7 This is a schematic diagram of an exemplary embodiment of the present disclosure showing a spatial tensor obtained by performing a graph convolution operation on an extended feature matrix and a spatial adjacency matrix. The spatial tensor is a three-dimensional tensor with a shape of (|V|, d, T-1).

[0096] The calculation formula of the spatial tensor includes:

[0097]

[0098] in, D is the degree matrix, GA is the above-mentioned spatial adjacency matrix, and I is the identity matrix of the above-mentioned feature matrix; Indicates graph convolution operation; x′ is the extended feature matrix, x′=∑x·W cnn +b, x is the feature matrix, x∈R |V|×(T-1) , |V| is the number of medical events, T is the number of user medical records, b is the bias vector of the above convolutional neural network, W cnn is the weight matrix of the convolutional neural network, W cnn ∈R d×d , d is the feature dimension of the extended feature matrix; W gcn is the weight matrix of graph convolution.

[0099] In sub-step S1032, based on the time feature, a time convolution operation is performed on the spatial tensor in the time dimension to obtain a time tensor.

[0100] In the above spatial tensor calculation process, although the dimension of the feature matrix of the user's medical record information is expanded, the neighbor nodes (such as Figure 3 As shown in the figure, the medical event nodes in the same medical record can be called neighbor nodes), but they have no effect on the time dimension. Therefore, in this step, based on the aforementioned time feature, the spatial tensor is convolved in the time dimension to obtain a three-dimensional tensor with a shape of (|V|, T-1, d), which is the time tensor. Figure 8 , Figure 8 A schematic diagram of performing a temporal convolution operation on a spatial tensor in the time dimension to obtain a temporal tensor according to an exemplary embodiment of the present disclosure.

[0101] The calculation formula of the time tensor includes:

[0102]

[0103] Where |V| is the number of the above medical events, d is the feature dimension of the extended feature matrix, and W tcn is the weight matrix of the temporal convolution, F spatial(*,T-1,*) is the input spatial tensor, T is the number of user visits, b tcn is the bias vector for temporal convolution.

[0104] In sub-step S1033, a spatiotemporal convolution operation is performed on the time tensor and the spatiotemporal adjacency matrix to obtain spatiotemporal dependency features.

[0105] In sub-steps S1031-S1032, the spatial and temporal features of the user's medical record information are extracted, but the data dependency between time and space is not considered, that is, the lack of Figure 3 The feature extraction of the relationship described by the arrow c in the figure is performed by performing a spatiotemporal convolution operation on the time tensor and the spatiotemporal adjacency matrix to obtain the spatiotemporal dependency features. Fig. 9 , Fig. 9 A schematic diagram of an exemplary embodiment of the present disclosure showing a spatiotemporal convolution operation of a time tensor and a spatiotemporal adjacency matrix to obtain spatiotemporal dependency features.

[0106] The calculation formulas for spatiotemporal dependency features include:

[0107]

[0108] in, GTA is the aforementioned spatiotemporal adjacency matrix, D is the degree matrix, I is the identity matrix of the feature matrix, Indicates graph convolution operation; F temporal(|V|,T-1,d) is the time tensor, |V| is the number of medical events, T is the number of user medical records, and d is the feature dimension of the extended feature matrix; W stcn is the weight matrix of spatiotemporal convolution.

[0109] After obtaining the spatiotemporal dependency feature, the shape of the spatiotemporal dependency feature can also be adjusted, that is, the dimension of the spatiotemporal dependency feature can be adjusted and the values ​​of different dimensions of the spatiotemporal dependency feature can be replaced. For example, (1, 3, 2) can be adjusted to (2, 1, 3) so that the shape of the spatiotemporal dependency feature conforms to the original business representation.

[0110] In step S104, feature extraction is performed on the spatiotemporal dependency features through a fully connected neural network to generate an extraction vector.

[0111] The calculation formula of the extraction vector includes: F st For the aforementioned spatiotemporal dependency features, FCN (F st ) represents the features extracted from the spatiotemporal dependent features through the fully connected neural network, sigmoid(FCN(F st )) indicates that FCN(F st ) as input to the sigmoid activation function.

[0112] In step S105, the loss of the extracted vector and the label vector is calculated to obtain a loss function.

[0113] Exemplarily, the label vector is obtained based on the last visit record in the user's visit record information, that is, based on the last visit record in the user's medical record information.

[0114] Taking the last visit record as the predicted label, it can be expressed as Will Encoded as a multi-hot vector, it can be represented as a vector y p ∈{0,1} |V| .

[0115] In step S106, the parameters of the feature extraction model are updated according to the loss function.

[0116] The calculation formula of the loss function includes:

[0117]

[0118] Where i is the number of medical events, F st is the spatiotemporal dependency feature, FCN (F st ) represents the features extracted from the spatiotemporal dependent features through the fully connected neural network, y i is the label vector corresponding to the i-th medical event in the last visit record, is FCN(F stcn ) corresponds to the extracted vector of the i-th medical event.

[0119] The parameters of the feature extraction model are updated according to the above loss function. The feature extraction model is used to extract the spatiotemporal dependency features in the user's medical record information, and predict the next possible medical visit of the user based on the spatiotemporal dependency features.

[0120] In summary, the information feature extraction method provided by the present disclosure includes obtaining user visit record information, constructing structured data based on the user visit record information, and constructing a feature representation of the user visit record information based on the structured data, wherein the feature representation includes spatial features, temporal features, and spatiotemporal features of the user visit record information, and extracting the spatiotemporal dependency features of the user visit record information based on the spatial features, temporal features, and spatiotemporal features, wherein the spatiotemporal features are the semantic relationships between target events in different visit records, and the spatiotemporal dependency features are the semantic relationships and time series features between target events in multiple visit records of the user. While incorporating prior knowledge related to the user visit record information, the features of the user visit record information are captured in all directions from the three dimensions of time, space, and spatiotemporal, and a more robust and accurate feature representation of the user visit record information is learned to improve the prediction effect of the disease prediction task.

[0121] Fig.10 is a block diagram of an information feature extraction device shown in an exemplary embodiment of the present disclosure. Fig.10 The device 20 includes an acquisition module 201, a processing module 203 and an extraction module 205.

[0122] The acquisition module 201 is used to acquire user visit record information and construct structured data according to the user visit record information, wherein the user visit record information includes multiple visit records of the user;

[0123] The processing module 203 is used to construct a feature representation of the user visit record information based on the structured data, and the feature representation includes the spatial feature, time feature and spatiotemporal feature of the user visit record information; the spatial feature represents the semantic relationship between target events in the same visit record, the time feature is the time series feature between the same target event in different visit records, and the spatiotemporal feature represents the semantic relationship between target events in different visit records;

[0124] The extraction module 205 is used to extract the spatiotemporal dependency features of the user's visit record information based on the spatial features, the temporal features and the spatiotemporal features; the spatiotemporal dependency features represent the semantic relationship and time series features between target events in the user's multiple visit records.

[0125] Optionally, the acquisition module 203 is further used to construct a feature matrix of the user visit record information according to the structured data; the feature matrix is ​​a matrix representing the original data of the user visit record information;

[0126] Constructing a spatial adjacency matrix of the user visit record information according to the structured data; the spatial adjacency matrix is ​​a matrix that characterizes the spatial features of the user visit record information;

[0127] Extracting time features from the user visit record information;

[0128] A spatiotemporal adjacency matrix of the user visit record information is constructed according to the structured data; the spatiotemporal adjacency matrix is ​​a matrix that characterizes the spatiotemporal features of the user visit record information.

[0129] Optionally, the extraction module 205 is further used to calculate the spatial tensor of the user visit record information based on the feature matrix and the spatial adjacency matrix;

[0130] Based on the time feature, performing a time convolution operation on the spatial tensor in the time dimension to obtain a time tensor;

[0131] The time tensor and the space-time adjacency matrix are subjected to a space-time convolution operation to obtain the space-time dependency feature.

[0132] Optionally, the extraction module 205 is further configured to expand the feature matrix through a convolutional neural network to obtain an expanded feature matrix;

[0133] The extended feature matrix and the spatial adjacency matrix are subjected to a graph convolution operation to obtain the spatial tensor.

[0134] Optionally, the calculation formula of the space tensor includes:

[0135]

[0136] in, D is the degree matrix, GA is the spatial adjacency matrix, and I is the identity matrix of the feature matrix; represents the graph convolution operation; x′ is the extended feature matrix, x′=∑x·W cnn +b, x is the feature matrix, x∈R |V|×(T-1) , |V| is the number of target events, T is the number of visit records, b is the bias vector of the convolutional neural network, W cnn is the weight matrix of the convolutional neural network, W cnn ∈R d×d , d is the feature dimension of the extended feature matrix; W gcn is the weight matrix of the graph convolution.

[0137] Optionally, the calculation formula of the time tensor includes:

[0138]

[0139] Wherein, |V| is the number of target events, d is the feature dimension of the extended feature matrix, and W tcn is the weight matrix of the temporal convolution, Fspatial(*,T-1,*) is the spatial tensor, T is the number of access records, b tcn is the bias vector of the temporal convolution.

[0140] Optionally, the calculation formula of the spatiotemporal dependency feature includes:

[0141]

[0142] in, GTA is the spatiotemporal adjacency matrix, D is the degree matrix, I is the identity matrix of the feature matrix, Indicates graph convolution operation; F temporal(|V|,T-1,d) is the time tensor, |V| is the number of target events, T is the number of visit records, and d is the feature dimension of the extended feature matrix; W stcn is the weight matrix of the spatiotemporal convolution.

[0143] Optionally, the extraction module 205 is further used to extract the spatiotemporal dependency features through a fully connected neural network to generate an extraction vector;

[0144] The loss calculation is performed on the extracted vector and the label vector to obtain a loss function; the label vector is obtained based on the last visit record in the user visit record information;

[0145] The parameters of the feature extraction model are updated according to the loss function; the feature extraction model is used to extract the spatiotemporal dependency features, and predict the next visit record of the user according to the spatiotemporal dependency features.

[0146] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0147] Fig.11 FIG. 4 is a block diagram of an electronic device 400 according to an exemplary embodiment. Fig.11 As shown, the electronic device 400 may include: a processor 401 and a memory 402. The electronic device 400 may also include one or more of a multimedia component 403, an input / output (I / O) interface 404, and a communication component 405.

[0148] The processor 401 is used to control the overall operation of the electronic device 400 to complete all or part of the steps in the above-mentioned information feature extraction method. The memory 402 is used to store various types of data to support the operation of the electronic device 400, and these data may include, for example, instructions for any application or method used to operate on the electronic device 400, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 402 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 403 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 402 or sent through the communication component 405. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 404 provides an interface between the processor 401 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 405 is used for wired or wireless communication between the electronic device 400 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 405 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0149] In an exemplary embodiment, the electronic device 400 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned information feature extraction method.

[0150] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned information feature extraction method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 402 including program instructions, and the above-mentioned program instructions can be executed by the processor 401 of the electronic device 400 to complete the above-mentioned information feature extraction method.

[0151] Fig.12 is a block diagram of another electronic device 500 according to an exemplary embodiment. For example, the electronic device 500 may be provided as a server. Fig.12 The electronic device 500 includes a processor 522, which may be one or more, and a memory 532 for storing a computer program executable by the processor 522. The computer program stored in the memory 532 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 522 may be configured to execute the computer program to perform the above-mentioned information feature extraction method.

[0152] In addition, the electronic device 500 may further include a power supply component 526 and a communication component 550. The power supply component 526 may be configured to perform power management of the electronic device 500, and the communication component 550 may be configured to implement communication of the electronic device 500, for example, wired or wireless communication. In addition, the electronic device 500 may further include an input / output (I / O) interface 558. The electronic device 500 may operate based on an operating system stored in the memory 532, such as Windows Server 2008. TM , Mac OSX TM , Unix TM , Linux TM etc.

[0153] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned information feature extraction method are implemented. For example, the non-transitory computer-readable storage medium can be the above-mentioned memory 532 including program instructions, and the above-mentioned program instructions can be executed by the processor 522 of the electronic device 500 to complete the above-mentioned information feature extraction method.

[0154] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned information feature extraction method when executed by the programmable device.

[0155] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0156] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0157] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for extracting information features, characterized in that: include: Acquire user visit record information, and construct structured data according to the user visit record information, wherein the user visit record information includes multiple visit records of the user; Constructing a characteristic representation of the user visit record information based on the structured data, the characteristic representation includes the spatial feature, time feature and spatiotemporal feature of the user visit record information; the spatial feature represents the semantic relationship between target events in the same visit record, the time feature is the time series feature between the same target event in different visit records, and the spatiotemporal feature represents the semantic relationship between target events in different visit records; the step of constructing the characteristic representation of the user visit record information based on the structured data includes: constructing a characteristic matrix of the user visit record information according to the structured data; the characteristic matrix is ​​a matrix representing the original data of the user visit record information; constructing a spatial adjacency matrix of the user visit record information according to the structured data; the spatial adjacency matrix is ​​a matrix representing the spatial features of the user visit record information; Extracting the spatiotemporal dependency features of the user's visit record information according to the spatial features, the temporal features and the spatiotemporal features; the spatiotemporal dependency features represent the semantic relationship and time series features between target events in multiple visit records of the user; The step of extracting the spatiotemporal dependency features of the user visit record information according to the spatial features, the time features and the spatiotemporal features comprises: Calculating the spatial tensor of the user visit record information based on the feature matrix and the spatial adjacency matrix; The step of calculating the spatial tensor of the user visit record information based on the feature matrix and the spatial adjacency matrix includes: Expanding the feature matrix through a convolutional neural network to obtain an expanded feature matrix; Performing a graph convolution operation on the extended feature matrix and the spatial adjacency matrix to obtain the spatial tensor; The calculation formula of the space tensor includes: in, D is the degree matrix, GA is the spatial adjacency matrix, and I is the identity matrix of the feature matrix; represents the graph convolution operation; x′ is the extended feature matrix, x′=∑x·W cnn +b, x is the feature matrix, x∈R |V|×(T-1) , |V| is the number of target events, T is the number of visit records, b is the bias vector of the convolutional neural network, W cnn is the weight matrix of the convolutional neural network, W cnn ∈R d×d , d is the feature dimension of the extended feature matrix; W gcn is the weight matrix of the graph convolution.

2. The method according to claim 1, characterized in that The step of constructing a characteristic representation of the user visit record information based on the structured data comprises: Extracting time features from the user visit record information; A spatiotemporal adjacency matrix of the user visit record information is constructed according to the structured data; the spatiotemporal adjacency matrix is ​​a matrix that characterizes the spatiotemporal features of the user visit record information.

3. The method according to claim 2, characterized in that The step of extracting the spatiotemporal dependency features of the user visit record information according to the spatial features, the temporal features and the spatiotemporal features comprises: Based on the time feature, performing a time convolution operation on the spatial tensor in the time dimension to obtain a time tensor; The time tensor and the space-time adjacency matrix are subjected to a space-time convolution operation to obtain the space-time dependency feature.

4. The method according to claim 3, characterized in that The calculation formula of the time tensor includes: Wherein, |V| is the number of target events, d is the feature dimension of the extended feature matrix, and W tcn is the weight matrix of the temporal convolution, F spatial(*,T-1,*) is the spatial tensor, T is the number of access records, b tcn is the bias vector of the temporal convolution.

5. The method according to claim 3, characterized in that: The calculation formula of the spatiotemporal dependence feature includes: in, GTA is the spatiotemporal adjacency matrix, D is the degree matrix, I is the identity matrix of the feature matrix, Indicates graph convolution operation; F temporal(|V|,T-1,d) is the time tensor, |V| is the number of target events, T is the number of visit records, and d is the feature dimension of the extended feature matrix; W stcn is the weight matrix of the spatiotemporal convolution.

6. The method according to claim 1, characterized in that After the step of extracting the spatiotemporal dependency features of the user visit record information according to the spatial features, the temporal features and the spatiotemporal features, the following steps are included: Extracting the spatiotemporal dependency features through a fully connected neural network to generate an extraction vector; The loss calculation is performed on the extracted vector and the label vector to obtain a loss function; the label vector is obtained based on the last visit record in the user visit record information; The parameters of the feature extraction model are updated according to the loss function; the feature extraction model is used to extract the spatiotemporal dependency features, and predict the next visit record of the user according to the spatiotemporal dependency features.

7. The method according to claim 6, characterized in that The label vector is y p ∈{0,1} |V| , |V| is the number of the target events, and the calculation formula of the extraction vector includes: The calculation formula of the loss function includes: Where i is the i-th target event, F st is the spatiotemporal dependency feature, FCN(F st ) represents the features extracted from the spatiotemporal dependent features by the fully connected neural network, y i is the label vector corresponding to the i-th target event in the last visit record, is FCN(F stcn ) is the extraction vector corresponding to the i-th target event.

8. An information feature extraction device, characterized in that: include: An acquisition module, used to acquire user visit record information and construct structured data according to the user visit record information, wherein the user visit record information includes multiple visit records of the user; A processing module, for constructing a feature representation of the user visit record information based on the structured data, the feature representation includes the spatial features, time features and spatiotemporal features of the user visit record information; the spatial features represent the semantic relationship between target events in the same visit record, the time features are the time series features between the same target event in different visit records, and the spatiotemporal features represent the semantic relationship between target events in different visit records; the step of constructing the feature representation of the user visit record information based on the structured data includes: constructing a feature matrix of the user visit record information according to the structured data; the feature matrix is ​​a matrix representing the original data of the user visit record information; constructing a spatial adjacency matrix of the user visit record information according to the structured data; the spatial adjacency matrix is ​​a matrix representing the spatial features of the user visit record information; An extraction module, used to extract the spatiotemporal dependency features of the user's visit record information according to the spatial features, the temporal features and the spatiotemporal features; the spatiotemporal dependency features represent the semantic relationship and time series features between target events in multiple visit records of the user; The step of extracting the spatiotemporal dependency features of the user visit record information according to the spatial features, the temporal features and the spatiotemporal features comprises: Calculating the spatial tensor of the user visit record information based on the feature matrix and the spatial adjacency matrix; The step of calculating the spatial tensor of the user visit record information based on the feature matrix and the spatial adjacency matrix includes: Expanding the feature matrix through a convolutional neural network to obtain an expanded feature matrix; Performing a graph convolution operation on the extended feature matrix and the spatial adjacency matrix to obtain the spatial tensor; The calculation formula of the space tensor includes: in, D is the degree matrix, GA is the spatial adjacency matrix, and I is the identity matrix of the feature matrix; represents the graph convolution operation; x′ is the extended feature matrix, x′=∑x·W cnn +b, x is the feature matrix, x∈R |V|×(T-1) , |V| is the number of target events, T is the number of visit records, b is the bias vector of the convolutional neural network, W cnn is the weight matrix of the convolutional neural network, W cnn ∈R d×d , d is the feature dimension of the extended feature matrix; W gcn is the weight matrix of the graph convolution.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

Images