Data processing method and device, storage medium and electronic equipment

Abstract

The present disclosure provides a data processing method and device, a storage medium and an electronic device; and relates to the technical field of data processing. The method comprises: acquiring current full data of a target service, and constructing a plurality of connected subgraphs according to the current full data, wherein the current full data at least comprises current incremental data; identifying a target field in the current incremental data based on a data record table corresponding to the current incremental data, to determine a data mode of the current incremental data; determining a first connected subgraph corresponding to the current incremental data, and updating the first connected subgraph according to the data mode of the current incremental data, to obtain a second connected subgraph. The present disclosure can reduce the consumption of computing resources and improve the data processing efficiency by calculating the connected graph based on the incremental data.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and in particular, to a data processing method, a data processing apparatus, a computer readable storage medium, and an electronic device. BACKGROUND

[0002] As an important form of relationship data representation, a connected graph is widely used in some business data analysis scenarios. For example, a connected graph set corresponding to a business can be calculated, and further mining can be performed on the connected graph set to analyze the business data.

[0003] With the update of business data, the connected graphs in the connected graph set corresponding to the business also need to be updated. Currently, when calculating the connected graphs using the full data of the related business, the changed data needs to be added to the historical full data after each data change, and then the whole calculation is performed based on the latest full data. The connected graph calculation of the historical full data has been performed at the system cold start, and the historical full data is repeatedly calculated in the calculation process every day, which causes waste of computing resources, and the whole calculation process takes too long, resulting in low data processing efficiency.

[0004] It should be noted that the information disclosed in the above background section is only used to enhance the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The purpose of the present disclosure is to provide a data processing method, a data processing apparatus, a computer readable storage medium, and an electronic device, thereby at least partially overcoming the problem of large data volume in calculating connected graphs, waste of excessive computing resources, and long calculation process time resulting in low data processing efficiency caused by related technologies.

[0006] According to a first aspect of the present disclosure, a data processing method is provided, comprising:

[0007] obtaining current full data of a target business, and constructing a plurality of connected subgraphs according to the current full data, wherein the current full data at least includes current incremental data;

[0008] identifying a target field in the current incremental data based on a data record table corresponding to the current incremental data, to determine a data pattern of the current incremental data;

[0009] determining a first connected subgraph corresponding to the current incremental data, and updating the first connected subgraph according to the data pattern of the current incremental data to obtain a second connected subgraph.

[0010] In an example embodiment of the present disclosure, the data mode of the current incremental data at least includes any one of the following:

[0011] The current incremental data is first new data, the first new data does not affect the relationship between the connected subgraphs, and the first new data includes first new entity data and first new relationship data.

[0012] The current incremental data is change data, the change data does not affect the relationship between the connected subgraphs, and the change data includes change entity data and change relationship data.

[0013] The current incremental data is second new relationship data, and the second new relationship data causes a relationship between entity data in the connected subgraphs.

[0014] The current incremental data is second new entity data, and the second new entity data has a relationship with the entity data in the connected subgraphs.

[0015] In an example embodiment of the present disclosure, the current full data further includes historical full data; after obtaining the current full data of the target business, the method further includes:

[0016] Constructing a full data record table according to entity data and relationship data in the historical full data, the full data record table including a full entity table and a full relationship table;

[0017] Constructing an incremental data record table according to entity data and relationship data in the current incremental data, the incremental data record table including an incremental entity table and an incremental relationship table;

[0018] Constructing a mode recognition result table according to a first field in the full data record table and a second field in the incremental data record table, the mode recognition result table including a mode recognition entity table and a mode recognition relationship table.

[0019] In an example embodiment of the present disclosure, the target field is a connected subgraph identifier field; and the identifying the target field in the current incremental data based on the data record table corresponding to the current incremental data to determine the data mode of the current incremental data includes:

[0020] Obtaining a unique identifier of the current incremental data, and determining whether the connected subgraph identifier field in the mode recognition result table where the unique identifier is located is empty;

[0021] If the connected subgraph identifier field is empty, it is determined that the current incremental data is first new data.

[0022] In an example embodiment of the present disclosure, the target field is a connected subgraph identification field, a historical attribute field and a current attribute field; and the identifying the target field in the current incremental data based on the data record table corresponding to the current incremental data to determine the data mode of the current incremental data comprises:

[0023] obtaining the unique identifier of the current incremental data, and judging whether the connected subgraph identification field in the mode recognition result table where the unique identifier is located is empty, and whether the current attribute field in the mode recognition result table is same as the historical attribute field;

[0024] if the connected subgraph identification field is not empty and the current attribute field is not same as the historical attribute field, it is determined that the current incremental data is change data.

[0025] In an example embodiment of the present disclosure, the identifying the target field in the current incremental data based on the data record table corresponding to the current incremental data to determine the data mode of the current incremental data comprises:

[0026] querying the first newly added relationship data in the mode recognition relationship table, and determining the target entity data connected with the first newly added relationship data;

[0027] judging whether the connected subgraph identification field corresponding to the target entity data in the full-amount entity table is empty;

[0028] if the connected subgraph identification field is not empty, it is determined that the current incremental data is second newly added relationship data.

[0029] In an example embodiment of the present disclosure, the identifying the target field in the current incremental data based on the data record table corresponding to the current incremental data to determine the data mode of the current incremental data comprises:

[0030] querying the first newly added entity data in the mode recognition entity table;

[0031] querying the target relationship data connected with the first newly added entity data in the mode recognition relationship table, the target relationship data connecting the first newly added entity data and historical entity data;

[0032] judging whether the connected subgraph identification field corresponding to the historical entity data in the full-amount entity table is empty;

[0033] if the connected subgraph identification field is not empty, it is determined that the current incremental data is second newly added entity data.

[0034] In an example embodiment of the present disclosure, the current incremental data is first new data; the first connected subgraph is updated according to the data mode of the current incremental data to obtain a second connected subgraph, including:

[0035] The first new entity data in the mode recognition entity table and the first new relationship data in the mode recognition relationship table are obtained.

[0036] The second connected graph is constructed according to the first new entity data and the first new relationship data.

[0037] In an example embodiment of the present disclosure, the current incremental data is change data; the first connected subgraph is updated according to the data mode of the current incremental data to obtain a second connected subgraph, including:

[0038] The historical data in the first connected subgraph is replaced by the change data to obtain the second connected subgraph.

[0039] In an example embodiment of the present disclosure, the current incremental data is second new relationship data; the first connected subgraph is updated according to the data mode of the current incremental data to obtain a second connected subgraph, including:

[0040] A plurality of first connected subgraphs connected with the second new relationship data are determined.

[0041] The plurality of first connected subgraphs are merged to obtain the second connected subgraph.

[0042] In an example embodiment of the present disclosure, the current incremental data is second new entity data; the first connected subgraph is updated according to the data mode of the current incremental data to obtain a second connected subgraph, including:

[0043] A target connected subgraph is constructed according to the first new entity data and target relationship data connected with the first new entity data, the target relationship data connecting the first new entity data and historical entity data.

[0044] A first connected subgraph corresponding to the historical entity data is determined.

[0045] The target connected subgraph and the first connected subgraph are merged to obtain the second connected subgraph.

[0046] In an example embodiment of the present disclosure, after obtaining the second connected subgraph, the method further includes:

[0047] Equity penetration is performed on the second connected subgraph to determine equity data of each target entity in the second connected subgraph.

[0048] According to a second aspect of the present disclosure, a data processing apparatus is provided, comprising:

[0049] a connected subgraph construction module configured to obtain current full data of a target business, and construct a plurality of connected subgraphs according to the current full data, the current full data comprising historical full data and current incremental data;

[0050] a data pattern determination module configured to identify a target field in the current incremental data based on a data record table corresponding to the current incremental data, to determine a data pattern of the current incremental data;

[0051] a connected subgraph update module configured to determine a first connected subgraph corresponding to the current incremental data, and update the first connected subgraph according to the data pattern of the current incremental data, to obtain a second connected subgraph.

[0052] According to a third aspect of the present disclosure, a data processing apparatus is provided, comprising:

[0053] a connected subgraph obtaining module configured to obtain the second connected subgraph as described in the data processing method;

[0054] a stock right data determination module configured to perform stock right penetration on the second connected subgraph, to determine stock right data of each target entity in the second connected subgraph.

[0055] According to a fourth aspect of the present disclosure, a computer readable storage medium is provided, having a computer program stored thereon, the computer program being executed by a processor to implement the method of any of the above.

[0056] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the method of any of the above by executing the executable instructions.

[0057] The exemplary embodiments of the present disclosure can have the following partial or all beneficial effects:

[0058] In the data processing method provided by the exemplary embodiments of the present disclosure, the current full data of the target business is obtained, and a plurality of connected subgraphs are constructed according to the current full data, the current full data comprising historical full data and current incremental data; a target field in the current incremental data is identified based on a data record table corresponding to the current incremental data, to determine a data pattern of the current incremental data; a first connected subgraph corresponding to the current incremental data is determined, and the first connected subgraph is updated according to the data pattern of the current incremental data, to obtain a second connected subgraph. The present disclosure calculates the connected graph based on the incremental data, which can reduce the consumption of computing resources and improve the data processing efficiency.

[0059] It should be understood that the general description and detailed description below are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0060] The drawings hereof are incorporated into the specification and constitute a part of the specification, illustrating embodiments consistent with the present disclosure, and together with the specification, serve to explain the principles of the present disclosure. It is apparent that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0061] Figure 1 A system architecture schematic diagram of a data processing method and device to which embodiments of the present disclosure can be applied is shown;

[0062] Figure 2 A flowchart of a data processing method in embodiments of the present disclosure is schematically shown;

[0063] Figure 3 A connected subgraph schematic diagram in embodiments of the present disclosure is schematically shown;

[0064] Figure 4 An update schematic diagram of a connected subgraph in embodiments of the present disclosure is schematically shown;

[0065] Figure 5 A connected subgraph schematic diagram in an incremental data mode in embodiments of the present disclosure is schematically shown;

[0066] Figure 6 Another connected subgraph schematic diagram in an incremental data mode in embodiments of the present disclosure is schematically shown;

[0067] Figure 7 A block diagram of a data processing device in embodiments of the present disclosure is schematically shown;

[0068] Figure 8 A block diagram of a stock right penetration device in embodiments of the present disclosure is schematically shown;

[0069] Figure 9 A structural schematic diagram of an electronic device suitable for implementing embodiments of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0070] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0071] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0072] Figure 1 A schematic diagram of a system architecture for an exemplary application environment in which an embodiment of the data processing method and apparatus of the present disclosure can be applied is shown.

[0073] like Figure 1 As shown, system architecture 100 may include one or more of terminal devices 101, 102, and 103, a network 104, and a server 105. Terminal devices 101, 102, and 103 may be various electronic devices, including but not limited to desktop computers, laptops, smartphones, and tablets. Server 105 may be a single server, a server cluster consisting of multiple servers, a virtualization platform, or a cloud computing service center. Network 104 is used as the medium to provide communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. It should be understood that... Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0074] The data processing method provided by the example embodiments of the present disclosure can be executed by the server 105, and accordingly, the data processing apparatus can be arranged in the server 105. For example, after the server 105 receives the current full volume data input by the terminal device 101, by executing the data processing method, the connected subgraph of the historical full volume data and the second connected subgraph corresponding to the incremental data can be sent to the terminal device 101 for display to the relevant user. However, it is easy for those skilled in the art to understand that the data processing method provided by the example embodiments of the present disclosure can also be executed by one or more of the terminal devices 101, 102, and 103, and accordingly, the data processing apparatus can also be arranged in the terminal devices 101, 102, and 103. For example, by executing the data processing method by the terminal device 101, the connected subgraph of the current full volume data can be directly calculated, and the calculated connected subgraph of the historical full volume data and the second connected subgraph corresponding to the incremental data can be directly displayed on the display screen of the terminal device 101 for display to the relevant user, which is not specially limited by the present disclosure.

[0075] The technical solutions of the embodiments of the present disclosure are described in detail as follows:

[0076] In the example embodiments of the present disclosure, the business scenario of calculating equity penetration in the financial field can be taken as an example for illustration. Equity penetration refers to the analysis of the equity structure of an enterprise, which can penetrate the shareholder company upwards and the subsidiary company downwards, so as to obtain the partner information of the shareholder company or the subsidiary company. The equity penetration is based on the calculation of the connected graph of business data. In graph theory, the connected graph is based on the concept of connectivity. In an undirected graph G, the undirected graph can be divided into several isolated subgraphs by using the connected graph algorithm,

[0077] At present, when the connected graph is calculated by using the full volume data of the related business, the changed data needs to be added to the historical full volume data after the data is changed each time, and then the whole calculation is performed based on the latest full volume data. For example, in the business scenario of calculating equity penetration in a certain project, the project has about 1.3 billion pieces of historical full volume data, including entity data and relationship data. After the historical full volume data is stored in the graph database, about 43 million connected subgraphs can be generated by using the connected graph algorithm. Then, the equity penetration algorithm is executed on the 43 million connected subgraphs respectively, and 43 million equity graphs are generated, which are the output results required in the business scenario.

[0078] However, subsequent equity penetration calculations are all based on 1.3 billion historical full data, plus the incremental data of each day, and the connected graph algorithm and equity penetration algorithm are executed in turn. The entire calculation process needs to be calculated every day, which consumes about 13 hours of time. Especially, the connected graph calculation of the historical full data has been executed at the system cold start. The historical full data is repeatedly calculated in the calculation process of each day, which wastes computing resources and makes the entire calculation process time-consuming.

[0079] Based on one or more of the above problems, the example embodiment provides a data processing method, which calculates the connected graph based on incremental data, can reduce the amount of data calculation, so that the historical full data that has been calculated before does not need to participate in the subsequent calculation, and only the incremental data needs to be calculated each time, thereby achieving the purpose of saving computing resources and shortening the calculation time. Referring to Figure 2 As shown in the figure, the data processing method can include steps S210 to S230:

[0080] Step S210. Obtain the current full data of the target business, and construct a plurality of connected subgraphs according to the current full data, wherein the current full data at least includes current incremental data;

[0081] Step S220. Based on the data record table corresponding to the current incremental data, identify the target field in the current incremental data to determine the data mode of the current incremental data;

[0082] Step S230. Determine the first connected subgraph corresponding to the current incremental data, and update the first connected subgraph according to the data mode of the current incremental data to obtain the second connected subgraph.

[0083] In the data processing method provided by the example embodiment of the present disclosure, by obtaining the current full data of the target business, and constructing a plurality of connected subgraphs according to the current full data, the current full data includes historical full data and current incremental data; based on the data record table corresponding to the current incremental data, the target field in the current incremental data is identified to determine the data mode of the current incremental data; the first connected subgraph corresponding to the current incremental data is determined, and the first connected subgraph is updated according to the data mode of the current incremental data to obtain the second connected subgraph. The present disclosure calculates the connected graph based on incremental data, which can reduce the consumption of computing resources and improve the data processing efficiency.

[0084] Next, the above steps of the example embodiment will be described in more detail.

[0085] In step S210, the current full data of the target business is obtained, and a plurality of connected subgraphs are constructed according to the current full data, wherein the current full data at least includes current incremental data.

[0086] In the business scenario of calculating equity penetration in the financial field in the example implementation of the present disclosure, the target business can be a business of analyzing equity data of an enterprise. Current full data of the target business can be obtained according to a preset update period, where the update period can be set according to actual needs, for example, the update period can be 1 day, 3 days or a week, and the present disclosure does not make a specific limitation thereon. The current full data can include historical full data and current incremental data. For example, when the update period is 1 day, the current full data is the full data obtained on the day, and the full data on the day includes the historical full data and the newly added data on the day. For example, in this business scenario, the newly added data on a daily basis is about 1 million.

[0087] After obtaining the current full data of the target business, a full data record table can be constructed according to the entity data and the relationship data in the historical full data. The full data record table can include a full entity table and a full relationship table. As shown in Table 1, the table structure of the full entity table and each field in the table are shown, and the table records the unique identifier of the entity data, the label of the entity data, the attribute of the entity data, the connected subgraph identifier to which the entity data belongs, and the attribute encryption value of the entity data.

[0088] Table 1

[0089] id label props componentId propsMd5

[0090] Wherein, id represents the unique identifier of the entity data, which can be the primary key of the entity data, or the encryption calculation of the primary key of the entity data, such as the Md5 value of the primary key, or the Md5 value of the label of the entity data or the Md5 value of the attribute of the entity data, and the present disclosure does not make a limitation thereon. Label represents the label of the entity data, such as the category of the entity data, such as company, legal person, natural person, etc. Props represents the attribute of the entity data, such as age, gender, company name, etc. The attribute format of the entity data is a json (java script object notation, data interaction format) string. ComponentId represents the result of connected graph calculation, that is, the connected subgraph identifier to which the entity data belongs, such as the connected subgraph ID. PropsMd5 represents the attribute encryption value of the entity data, such as the Md5 value of the attribute json string, which can be used to determine whether the attribute of the entity data has changed.

[0091] As shown in Table 2, the table structure of the full relationship table and each field in the table are shown, and the table records the unique identifier of the relationship data, the label of the relationship data, the identifier and label of the entity data connected by the relationship data, the attribute of the relationship data, the connected subgraph identifier to which the relationship data belongs, and the attribute encryption value of the relationship data.

[0092] Table 2

[0093] id label srcId dstId srcLabel dstLabel props componentId propsMd5

[0094] Wherein, id represents the unique identification of the relationship data, the id can be the relationship data primary key, such as order number, transaction serial number, etc. It can also be the encryption calculation of the relationship data primary key, such as the Md5 value of the primary key, and it can also be the Md5 value of the relationship data label, the Md5 value of the relationship data attribute, etc. The present disclosure does not limit this. Label represents the label of the relationship data, such as the category of the relationship data. SrcId represents the source entity ID of the two entity data connected by the relationship data, dstId represents the destination entity ID of the two entity data connected by the relationship data, srcLabel represents the label of the source entity, and dstLabel represents the label of the destination entity. Wherein, the source entity is the entity used as the source in the relationship data, and the destination entity is the entity used as the target in the relationship data. Props represents the attribute of the relationship data, such as the transfer amount, transaction time, etc. The attribute format of the relationship data is a json string. ComponentId represents the result of the connected graph calculation, that is, the connected subgraph identification to which the relationship data belongs, such as the connected subgraph ID. PropsMd5 represents the attribute encryption value of the relationship data, such as the Md5 value of the attribute json string, which can be used to determine whether the attribute of the relationship data has changed.

[0095] Similarly, the current incremental data can be stored separately, and an incremental data record table can be constructed according to the entity data and the relationship data in the current incremental data. The incremental data record table can include an incremental entity table and an incremental relationship table. As shown in Table 3, the table structure of the incremental entity table and each field in the table are shown, and the unique identification of the entity data in the current incremental data, the label of the entity data, the attribute of the entity data, the attribute encryption value of the entity data, the time of recording the entity data, etc. Data are recorded in the table.

[0096] Table 3

[0097] id label props propsMd5 date

[0098] Wherein, date represents the time of recording the entity data, and the meanings of the other fields are the same as those of the corresponding fields in Table 1, which will not be repeated here.

[0099] As shown in Table 4, the table structure of the incremental relationship table and each field in the table are shown, and the unique identification of the relationship data in the current incremental data, the label of the relationship data, the identification and label of the entity data connected by the relationship data, the attribute of the relationship data, the attribute encryption value of the relationship data, the time of recording the entity data, etc. Data are recorded in the table.

[0100] Table 4

[0101] id label srcId dstId srcLabel dstLabel props propsMd5 date

[0102] wherein date represents the time when the entity data is recorded, and the meanings of the other fields are the same as those of the corresponding fields in Table 2, which are not described herein again.

[0103] After obtaining the full-amount data record table and the incremental data record table, a pattern recognition result table can be constructed according to the first field in the full-amount data record table and the second field in the incremental data record table. The pattern recognition result table can include a pattern recognition entity table and a pattern recognition relationship table.

[0104] For example, the pattern recognition entity table can be constructed according to the first field in the full-amount entity table and the second field in the incremental entity table. The first field in the full-amount entity table includes two fields of propsMd5 and componentId, and the second field in the incremental entity table includes four fields of id, label, props and propsMd5. When the date field in the incremental entity table is the current time and the id in the full-amount entity table is the same as the id in the incremental entity table, the pattern recognition entity table can be constructed by the propsMd5 and componentId fields in the full-amount entity table and the id, label, props and propsMd5 fields in the incremental entity table. In the pattern recognition entity table, the propsMd5 field in the full-amount entity table is replaced by a histPropsMd5 field. In addition, the pattern recognition entity table further includes an incrPattern field, which represents the data pattern of the incremental data.

[0105] As shown in Table 5, the table structure of the pattern recognition entity table and the fields in the table are shown, and the table records the unique identification (id) of the entity data, the label of the entity data (label), the attribute of the entity data (props), the attribute encryption value of the entity data (propsMd5), the connected subgraph identification to which the entity data belongs (componentId), the historical attribute encryption value of the entity data (histPropsMd5), the data pattern of the entity data (incrPattern), and the like.

[0106] Table 5

[0107] id label props propsMd5 componentId histPropsMd5 incrPattern

[0108] Similarly, a pattern recognition relation table can be constructed based on the first field of the full relation table and the second field of the incremental relation table. The first field of the full relation table includes two fields: `propsMd5` and `componentId`. The second field of the incremental relation table includes six fields: `id`, `label`, `srcId`, `dstId`, `props`, and `propsMd5`. When the `date` field in the incremental relation table is the current time and the `id` in both the full and incremental relation tables is the same, a pattern recognition relation table can be constructed using the `propsMd5` and `componentId` fields from the full relation table and the `id`, `label`, `srcId`, `dstId`, `props`, and `propsMd5` fields from the incremental relation table. In the pattern recognition relation table, the `propsMd5` field from the full relation table is replaced with the `histPropsMd5` field. Additionally, the pattern recognition relation table includes an `incrPattern` field, representing the data pattern of the incremental data.

[0109] Table 6 shows the table structure of the pattern recognition relation table and the various fields in the table. The table records the unique identifier (id) of the relation data, the label of the relation data, the source entity ID (srcId) and the destination entity ID (dstId) of the two entity data connected by the relation data, the attributes of the relation data (props), the encrypted value of the attributes of the relation data (propsMd5), the identifier of the connected subgraph to which the relation data belongs (componentId), the encrypted value of the historical attributes of the relation data (histPropsMd5), and the data pattern of the relation data (incrPattern), etc.

[0110] Table 6

[0111] id label srcId dstId props propsMd5 componentId histPropsMd5 incrPattern

[0112] After obtaining the current full data for the target business, a connectivity graph calculation can be performed on the current full data to obtain multiple connected subgraphs. (Reference) Figure 3 As shown, multiple connected subgraphs (sub-) are schematically given. Figure 1 ,son Figure 2 This is a schematic diagram of a subgraph (n, ..., n), where each connected subgraph consists of entity data and relational data, and any two entity data are connected through relational data. Then, each entity data and relational data, including fields such as the connected subgraph ID to which each entity data and relational data belongs, can be recorded in the full data record table and the incremental data record table.

[0113] The data mode of the current incremental data is determined by calculating a connected graph of the current full data, and recording entity data and relationship data in each connected graph in a table structure, so as to identify the data mode of the entity data and the relationship data in the incremental data, and to realize flexible processing of the incremental data of different data modes, and to improve the data processing efficiency.

[0114] In step S220, the target field in the current incremental data is identified based on the data record table corresponding to the current incremental data, so as to determine the data mode of the current incremental data.

[0115] In the example embodiment of the present disclosure, the data mode of the current incremental data can include at least any one of the following:

[0116] Mode one: the current incremental data is first new data, the first new data does not affect the relationship between the connected subgraphs, and the first new data can include first new entity data and first new relationship data. For example, the first new data is new data that does not exist in the historical full data, the first new entity data can be new entity data such as natural person, company, etc., and the first new relationship data can be new investment relationship data, etc.

[0117] Mode two: the current incremental data is change data, the change data does not affect the relationship between the connected subgraphs, and the change data can include changed entity data and changed relationship data. For example, the change data is data that exists in the historical full data and is changed at this time of data update, the changed entity data can be attribute change of entity such as natural person or company, and the changed relationship data can be increase, decrease or withdrawal of equity share, etc.

[0118] Reference Figure 4 As shown, a schematic diagram of the change of the connected subgraph with the addition of the current incremental data is schematically given. Figure 4 The connected subgraphs at three time points intercepted from the time axis of the change of the current incremental data are included, and it is assumed that the connected subgraph G t at t time point is constructed according to the base data of the historical full data. t At t+1 time point, due to the addition of three entity data g, h and i, and five relationship data 8, 9, 10, 11 and 12, the connected subgraph G t+1 is updated to the connected subgraph G t+1 At t+2 time point, due to the attribute change of the entity data a in the connected subgraph G t+1 to a', and the attribute change of the relationship data 1 to 1', which represents the case that the investment share of a to b is increased or decreased when the attribute of the entity data a is changed.

[0119] Pattern three: the current incremental data is second newly added relationship data, and the second newly added relationship data can generate a relationship between entity data in multiple connected subgraphs. For reference Figure 5 As shown in the figure, subgraph M and subgraph N are originally two mutually isolated connected subgraphs, and when the second newly added relationship data is added, a relationship can be generated between the entity data in subgraph M and subgraph N.

[0120] Pattern four: the current incremental data is second newly added entity data, and the second newly added entity data generates a relationship with the entity data in multiple connected subgraphs. For reference Figure 6 As shown in the figure, subgraph M and subgraph N are originally two mutually isolated connected subgraphs, and when the second newly added entity data X is added, the second newly added entity data X can generate a relationship M with the entity data in subgraph M, and the second newly added entity data X can also generate a relationship N with the entity data in subgraph N. It should be noted that the second newly added entity data can also be a new subgraph.

[0121] Example implementation one:

[0122] The target field can be a connected subgraph identifier field, and the data record table corresponding to the current incremental data is a pattern recognition result table, which can be a pattern recognition entity table or a pattern recognition relationship table. Illustratively, the unique identifier of the current incremental data can be obtained, and it is determined whether the connected subgraph identifier field in the pattern recognition result table where the unique identifier is located is empty; if the connected subgraph identifier field is empty, it is determined that the current incremental data is first newly added data.

[0123] Specifically, when the current incremental data is newly added entity data, the unique identifier of the entity data, such as the id of the entity data, can be obtained. The pattern recognition entity table to which the entity data belongs can be queried and obtained according to the id of the entity data, and it is determined whether the componentId field in the pattern recognition entity table is empty. If the componentId field is empty, it indicates that the connected subgraph identifier to which the entity data belongs is not recorded in the full-amount entity table, that is, the entity data has not been subjected to connected graph calculation in the historical calculation process, and it can be determined that the entity data is first newly added entity data. At this time, the incrPattern field in the pattern recognition entity table can be updated to "1.a", and 1.a indicates that the data pattern of the current incremental data is first newly added entity data.

[0124] When the current incremental data is newly added relationship data, an identifier of the relationship data can be obtained, such as an id of the relationship data. A pattern recognition relationship table to which the relationship data belongs can be queried and obtained according to the id of the relationship data, and it is determined whether a componentId field in the pattern recognition relationship table is empty. If the componentId field is empty, it indicates that a connected subgraph identifier to which the relationship data belongs is not recorded in the full-amount relationship table, that is, the relationship data has not been subjected to connected graph calculation in the historical calculation process, and it is determined that the relationship data is first newly added relationship data. At this time, an incrPattern field in the pattern recognition relationship table can be updated to "1.b", and 1.b indicates that a data mode of the current incremental data is first newly added relationship data.

[0125] Example implementation two:

[0126] The target field can be a connected subgraph identifier field, a historical attribute field, and a current attribute field, and a data record table corresponding to the current incremental data is a pattern recognition result table, which can be a pattern recognition entity table or a pattern recognition relationship table. For example, an identifier of the current incremental data can be obtained, and it is determined whether a connected subgraph identifier field in a pattern recognition result table in which the identifier is located is empty, and whether a current attribute field and a historical attribute field in the pattern recognition result table are the same. If the connected subgraph identifier field is not empty and the current attribute field and the historical attribute field are not the same, it is determined that the current incremental data is changed data.

[0127] Specifically, when the current incremental data is newly added entity data, an identifier of the entity data can be obtained, such as an id of the entity data. A pattern recognition entity table to which the entity data belongs can be queried and obtained according to the id of the entity data, and it is determined whether a componentId field in the pattern recognition entity table is empty, and whether a propsMd5 field and a histPropsMd5 field in the pattern recognition entity table are the same. If the componentId field is not empty, it indicates that a connected subgraph identifier to which the entity data belongs is recorded in the full-amount entity table, that is, the entity data has been subjected to connected graph calculation in the historical calculation process, that is, the entity data is not newly added data. At the same time, if the propsMd5 field and the histPropsMd5 field in the pattern recognition entity table are not the same, it indicates that the entity data has been changed, and it is determined that the entity data is changed entity data. At this time, an incrPattern field in the pattern recognition entity table can be updated to "2.a", and 2.a indicates that a data mode of the current incremental data is changed entity data.

[0128] When the current incremental data is a newly added relationship data, an identifier of the relationship data can be obtained, such as an id of the relationship data. A pattern recognition relationship table to which the relationship data belongs can be queried and obtained according to the id of the relationship data, and it is determined whether a componentId field in the pattern recognition relationship table is empty and whether a propsMd5 field and a histPropsMd5 field in the pattern recognition relationship table are the same. If the componentId field is not empty, it is indicated that a connected subgraph identifier of the relationship data is recorded in the full entity table, that is, the relationship data has been calculated in the historical calculation process, that is, the relationship data is not new data. At the same time, if the propsMd5 field and the histPropsMd5 field in the pattern recognition relationship table are different, it is indicated that the relationship data has changed, and it can be determined that the relationship data is changed relationship data. At this time, the incrPattern field in the pattern recognition relationship table can be updated to "2.b", and 2.b indicates that the data mode of the current incremental data is changed relationship data.

[0129] Example implementation three:

[0130] The first newly added relationship data in the pattern recognition relationship table can be queried, and target entity data connected with the first newly added relationship data can be determined, and it is determined whether a connected subgraph identifier field corresponding to the target entity data in the full entity table is empty. If the connected subgraph identifier field is not empty, it is determined that the current incremental data is second newly added relationship data.

[0131] Specifically, the first newly added relationship data with the incrPattern field being "1.b" in the pattern recognition relationship table can be obtained, and identifiers of target entity data connected with the first newly added relationship data are srcId and dstId respectively. Further, it is determined whether a componentId field in the full entity table where the srcId is located is empty, and whether a componentId field in the full entity table where the dstId is located is empty. If both componentId fields are not empty, it is indicated that the entity data connected with the relationship data is not new data, and it is determined that the current incremental data is second newly added relationship data.

[0132] In which it has been determined that the current incremental data is first newly added relationship data, based on which, if the entity data connected with the relationship data is historical data, it is indicated that the relationship between the entity data in the originally isolated connected subgraph is generated after the current incremental data is added, and the corresponding relationship data is the current incremental data, and therefore, it is determined that the current incremental data is second newly added relationship data. At this time, the incrPattern field in the pattern recognition relationship table can be updated to "3", and 3 indicates that the data mode of the current incremental data is second newly added relationship data.

[0133] In this example, if it is determined that the componentId field in the full entity table where the srcId is located and the componentId field in the full entity table where the dstId is located are both not empty, the two componentId fields can be marked to facilitate subsequent processing of the connected subgraph corresponding to the two componentId fields.

[0134] Example Implementation Four:

[0135] The first newly added entity data in the pattern recognition entity table can be queried, and target relationship data connected with the first newly added entity data in the pattern recognition relationship table can be queried, the target relationship data connecting the first newly added entity data and the corresponding historical entity data. Further, it can be determined whether the connected subgraph identifier field corresponding to the historical entity data in the full entity table is empty. If the connected subgraph identifier field is not empty, it is determined that the current incremental data is the second newly added entity data.

[0136] Specifically, the first newly added entity data with the incrPattern field of "1.a" in the pattern recognition entity table can be obtained, and the target relationship data with the incrPattern field of "1.b" and connected with the first newly added entity data in the pattern recognition relationship table can be obtained, the target relationship data can connect the first newly added entity data and the corresponding historical entity data. For example, the identifiers of the two entity data connected by the target relationship data are srcId and dstId, respectively, wherein the srcId can be the first newly added entity data, and the dstId can be the corresponding historical entity data, or the dstId can be the first newly added entity data, and the srcId can be the corresponding historical entity data.

[0137] When the dstId is the corresponding historical entity data, it can be determined whether the componentId field in the full entity table where the dstId is located is empty. If the componentId field is not empty, it indicates that the entity data represented by the dstId is historical data, which means that after the current incremental data (i.e., the first newly added entity data srcId) is added, a relationship is generated between the current incremental data and the entity data in the historical connected subgraph, i.e., a relationship is generated between the current incremental data and the entity data represented by the dstId. Therefore, it can be determined that the current incremental data is the second newly added entity data.

[0138] Similarly, when the srcId is the corresponding historical entity data, it can be determined whether the componentId field in the full entity table where the srcId is located is empty. If the componentId field is not empty, it indicates that the entity data represented by the srcId is historical data, and it indicates that the current incremental data and the entity data in the historical connected subgraph have a relationship after the current incremental data (i.e., the first newly added entity data dstId) is added, that is, the current incremental data and the entity data represented by the srcId have a relationship, and therefore, it can be determined that the current incremental data is the second newly added entity data. At this time, the incrPattern field in the pattern recognition entity table can be updated to "4", and 4 indicates that the data pattern of the current incremental data is the second newly added entity data.

[0139] In this example, when the dstId is the corresponding historical entity data, if it is determined that the componentId field in the full entity table where the dstId is located is not empty, the componentId field can be marked to facilitate subsequent processing of the connected subgraph corresponding to the componentId field. Alternatively, when the srcId is the corresponding historical entity data, if it is determined that the componentId field in the full entity table where the srcId is located is not empty, the componentId field can be marked to facilitate subsequent processing of the connected subgraph corresponding to the componentId field.

[0140] The present disclosure identifies the data pattern of the current incremental data to facilitate subsequent different connected graph calculations for current incremental data of different data patterns, thereby achieving flexible processing of incremental data of different data patterns and improving data processing efficiency.

[0141] In step S230, a first connected subgraph corresponding to the current incremental data is determined, and the first connected subgraph is updated according to the data pattern of the current incremental data to obtain a second connected subgraph.

[0142] The incremental data record table can be queried according to the unique identifier of the current incremental data to obtain the connected subgraph identifier field corresponding to the unique identifier, and the first connected subgraph corresponding to the current incremental data is determined according to the connected subgraph identifier field.

[0143] When it is determined that the current incremental data is the first newly added data, the first newly added entity data in the pattern recognition entity table and the first newly added relationship data in the pattern recognition relationship table can be obtained, and a second connected graph is constructed according to the first newly added entity data and the first newly added relationship data. For example, the entity data with incrPattern field of 1.a in the pattern recognition entity table and the relationship data with incrPattern field of 1.b in the pattern recognition relationship table can be obtained, a connected graph algorithm is performed on all the obtained entity data and relationship data to obtain a second connected graph, and the componentId field in the full data record table in which the entity data and the relationship data in the second connected graph are located is updated to the componentId of the second connected graph. In addition, the componentId of the second connected subgraph corresponding to the first newly added data can be recorded in the set S1.

[0144] When it is determined that the current incremental data is the change data, the historical data in the first connected subgraph can be replaced with the change data to obtain a second connected subgraph. Then, the entity data with incrPattern field of 2.a in the pattern recognition entity table and the relationship data with incrPattern field of 2.b in the pattern recognition relationship table can be obtained, the entity data in the full entity table is replaced with the entity data in the pattern recognition entity table according to the unique identifier of the entity data, and the relationship data in the full relationship table is replaced with the relationship data in the pattern recognition relationship table according to the unique identifier of the relationship data. In addition, the componentId of the second connected subgraph corresponding to the change data can be recorded in the set S2.

[0145] When it is determined that the current incremental data is the second newly added relationship data, a plurality of first connected subgraphs connected with the second newly added relationship data can be determined, and the plurality of first connected subgraphs are merged to obtain a second connected subgraph. For example, the identifiers of the target entity data connected with the current incremental data are srcId and dstId respectively, the componentIds corresponding to srcId and dstId respectively can be determined, and the plurality of first connected subgraphs are merged to obtain a second connected subgraph according to the componentIds. After the plurality of first connected subgraphs are merged to obtain a second connected subgraph, the smallest componentId in the plurality of first connected subgraphs can be selected as the identifier of the second connected subgraph. At the same time, the componentId in the full relationship table in which the current incremental data is located can be updated to the componentId of the second connected subgraph, the componentId in the full entity table in which the target entity data connected with the current incremental data is located can be updated to the componentId of the second connected subgraph, and the componentId in the pattern recognition relationship table in which the current incremental data is located can be updated to the componentId of the second connected subgraph.

[0146] When it is determined that the current incremental data is second newly added entity data, a target connected subgraph can be constructed according to the first newly added entity data and target relationship data connected with the first newly added entity data, wherein the target relationship data connects the first newly added entity data and the historical entity data. A first connected subgraph corresponding to the historical entity data can be determined, and the target connected subgraph and the first connected subgraph are merged to obtain a second connected subgraph.

[0147] For example, the first newly added entity data with the incrPattern field of "1.a" in the pattern recognition entity table can be obtained, and the target relationship data with the incrPattern field of "1.b" in the pattern recognition relationship table and connected with the first newly added entity data can be obtained, and the connected graph algorithm is performed on the first newly added entity data and the target relationship data to obtain a target connected subgraph. When the first newly added entity data and the historical entity data connected by the target relationship data have identifiers srcId and dstId, respectively, the componentId corresponding to the historical entity data can be determined, and the first connected subgraph corresponding to the historical entity data can be determined according to the componentId. After the first connected subgraph corresponding to the historical entity data and the target connected subgraph are merged to obtain a second connected subgraph, the smallest componentId in the first connected subgraph and the target connected subgraph can be selected as the identifier of the second connected subgraph. At the same time, the componentId in the full entity table where the current incremental data is located can be updated to the componentId of the second connected subgraph, the componentId in the full entity table where the target relationship data connected with the current incremental data is located can be updated to the componentId of the second connected subgraph, and the componentId in the pattern recognition entity table where the current incremental data is located can be updated to the componentId of the second connected subgraph.

[0148] In addition, when the current incremental data is second newly added relationship / entity data, the componentId of the second connected subgraph corresponding to the second newly added relationship / entity data can be recorded in the set S3. It should be noted that after the second connected subgraph corresponding to the current incremental data is obtained, the full data record table can also be updated.

[0149] The present disclosure can reduce the consumption of computing resources, improve data processing efficiency, and flexibly process incremental data of different data patterns, thereby further improving data processing efficiency.

[0150] After obtaining the second connected subgraph corresponding to the current incremental data according to steps S210 to S230, equity penetration can be performed on the second connected subgraph to determine the equity data of each target entity in the second connected subgraph. It can be understood that if the current incremental data is a data mode, equity penetration can be directly performed on the second connected subgraph corresponding to the current incremental data. If the current incremental data contains incremental data of multiple data modes, for example, if the componentId of the second connected subgraph corresponding to the current incremental data is a set S = {S1, S2, S3}, the entity data and relationship data corresponding to the componentId recorded in the set S can be obtained from the full data record table, and equity penetration calculation can be performed according to the obtained entity data and relationship data.

[0151] For example, the equity relationship in the connected subgraph corresponding to each componentId can be calculated according to the grouping of componentId. The holding ratio data of each target entity in the connected subgraph can be obtained from the props field in the full relationship table, and iteration can be performed in the connected subgraph according to the holding ratio data of each entity to calculate the equity data of each target entity. For example, the target entity is an investor A, the holding ratio of A in company A is 100%, company A holds 50% equity of company B, the holding ratio of A in company B is 30%, company B holds 40% equity of company C, and the holding ratio of A in company C is 20%. Through calculation, it can be obtained that the total holding ratio of A in company C is 20% + 30% * 40% + 100% * 50% * 40% = 52%, and it can be seen that A is the actual controlling shareholder of company C.

[0152] In the business scenario shown in the embodiments of the present disclosure, it is assumed that the connectivity graph calculation on the historical full data can obtain about 43 million connected subgraphs, and the daily incremental data is about 1 million data, wherein the entity data and the relationship data in the incremental data: 1) both exist in the 43 million connected subgraphs; 2) both do not exist in the 43 million connected subgraphs; 3) part exists in the 43 million connected subgraphs, and part does not exist in the 43 million connected subgraphs; 4) part exists in the 43 million connected subgraphs, and part constitutes connectivity subgraph merging; 5) part exists in the 43 million connected subgraphs, and part does not exist in the 43 million connected subgraphs, and part constitutes connectivity subgraph merging. It can be seen that the calculation amount in case 1) is the largest. Moreover, if the entity data in the 1 million data are all independent nodes, 1 million connected subgraphs can be constructed, and subsequent only needs to perform equity penetration on the 1 million connected subgraphs. Compared with performing equity penetration on the 43 million connected subgraphs, the calculation amount is greatly reduced, a large amount of calculation resources is saved, the data processing efficiency is improved, and finally the optimization of the connectivity graph calculation based on incremental data in the equity penetration application is realized.

[0153] It should be noted that the data processing method of the present disclosure can be applied to various scenarios involving graph mining algorithms in knowledge graph related businesses, and the present disclosure only takes the business scenario of calculating equity penetration in the financial field as an example for description.

[0154] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. In addition or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be divided into multiple steps, etc.

[0155] Further, in the example embodiment, a data processing apparatus is also provided. The apparatus can be applied to a terminal device or a server. Referring to Figure 7 The data processing apparatus 700 can include a connected subgraph construction module 710, a data pattern determination module 720, and a connected subgraph update module 730, wherein:

[0156] The connected subgraph construction module 710 is configured to obtain current full data of a target business, and construct a plurality of connected subgraphs according to the current full data, wherein the current full data at least includes current incremental data;

[0157] The data pattern determination module 720 is configured to identify a target field in the current incremental data based on a data record table corresponding to the current incremental data, to determine a data pattern of the current incremental data;

[0158] The connected subgraph updating module 730 is configured to determine a first connected subgraph corresponding to the current incremental data, and update the first connected subgraph according to a data mode of the current incremental data to obtain a second connected subgraph.

[0159] In an optional implementation, the data mode of the current incremental data in the data processing apparatus 700 at least includes any one of the following:

[0160] The current incremental data is first new data, the first new data does not affect the relationship between the connected subgraphs, and the first new data includes first new entity data and first new relationship data.

[0161] The current incremental data is change data, the change data does not affect the relationship between the connected subgraphs, and the change data includes change entity data and change relationship data.

[0162] The current incremental data is second new relationship data, and the second new relationship data causes the entity data in the connected subgraph to have a relationship.

[0163] The current incremental data is second new entity data, and the second new entity data has a relationship with the entity data in the connected subgraph.

[0164] In an optional implementation, the current full data further includes historical full data; and the data processing apparatus 700 further includes:

[0165] The full data record table construction module is configured to construct a full data record table according to entity data and relationship data in the historical full data, and the full data record table includes a full entity table and a full relationship table.

[0166] The incremental data record table construction module is configured to construct an incremental data record table according to entity data and relationship data in the current incremental data, and the incremental data record table includes an incremental entity table and an incremental relationship table.

[0167] The mode recognition result table construction module is configured to construct a mode recognition result table according to a first field in the full data record table and a second field in the incremental data record table, and the mode recognition result table includes a mode recognition entity table and a mode recognition relationship table.

[0168] In an optional implementation, the target field is a connected subgraph identifier field; and the data mode determination module 720 includes:

[0169] The first target field judging module is configured to obtain the unique identifier of the current incremental data, and judge whether the connected subgraph identifier field in the pattern recognition result table where the unique identifier is located is empty.

[0170] The first data pattern determining module is configured to determine the current incremental data as first new data if the connected subgraph identifier field is empty.

[0171] In an optional embodiment, the target fields are the connected subgraph identifier field, the historical attribute field and the current attribute field; and the data pattern determining module 720 comprises:

[0172] The second target field judging module is configured to obtain the unique identifier of the current incremental data, and judge whether the connected subgraph identifier field in the pattern recognition result table where the unique identifier is located is empty, and whether the current attribute field and the historical attribute field in the pattern recognition result table are the same;

[0173] The second data pattern determining module is configured to determine the current incremental data as changed data if the connected subgraph identifier field is not empty and the current attribute field and the historical attribute field are not the same.

[0174] In an optional embodiment, the data pattern determining module 720 comprises:

[0175] The target entity data determining module is configured to query the first new relationship data in the pattern recognition relationship table, and determine the target entity data connected with the first new relationship data;

[0176] The third target field judging module is configured to judge whether the connected subgraph identifier field corresponding to the target entity data in the full-amount entity table is empty.

[0177] The third data pattern determining module is configured to determine the current incremental data as second new relationship data if the connected subgraph identifier field is not empty.

[0178] In an optional embodiment, the data pattern determining module 720 comprises:

[0179] The new entity data determining module is configured to query the first new entity data in the pattern recognition entity table;

[0180] The target relationship data determining module is configured to query the target relationship data connected with the first new entity data in the pattern recognition relationship table, the target relationship data connecting the first new entity data and the historical entity data.

[0181] The fourth target field judging module is configured to judge whether the connected subgraph identifier field corresponding to the historical entity data in the full-amount entity table is empty.

[0182] The fourth data mode determining module is configured to determine that the current incremental data is second newly-added entity data if the connected subgraph identification field is not empty.

[0183] In an optional implementation, the current incremental data is first newly-added data; and the connected subgraph updating module 730 includes:

[0184] The newly-added data obtaining module is configured to obtain first newly-added entity data in a pattern recognition entity table and first newly-added relationship data in a pattern recognition relationship table.

[0185] The second connected graph constructing module is configured to construct the second connected graph according to the first newly-added entity data and the first newly-added relationship data.

[0186] In an optional implementation, the current incremental data is change data; and the connected subgraph updating module 730 is configured to replace historical data in the first connected subgraph with the change data to obtain the second connected subgraph.

[0187] In an optional implementation, the current incremental data is second newly-added relationship data; and the connected subgraph updating module 730 includes:

[0188] The first connected subgraph determining module is configured to determine a plurality of first connected subgraphs connected with the second newly-added relationship data.

[0189] The second connected subgraph generating module is configured to merge the plurality of first connected subgraphs to obtain the second connected subgraph.

[0190] In an optional implementation, the current incremental data is second newly-added entity data; and the connected subgraph updating module 730 includes:

[0191] The target connected subgraph constructing module is configured to construct a target connected subgraph according to first newly-added entity data and target relationship data connected with the first newly-added entity data, the target relationship data connecting the first newly-added entity data and historical entity data.

[0192] The first connected subgraph determining module is configured to determine a first connected subgraph corresponding to the historical entity data.

[0193] The second connected subgraph generating module is configured to merge the target connected subgraph and the first connected subgraph to obtain the second connected subgraph.

[0194] In an optional implementation, the data processing apparatus 700 further includes:

[0195] The equity penetration module is configured to perform equity penetration on the second connected subgraph to determine equity data of each target entity in the second connected subgraph.

[0196] The specific details of each module in the data processing apparatus are described in detail in the corresponding data processing method, and thus will not be described here.

[0197] Further, in the example embodiment, an equity penetration apparatus is also provided. The apparatus can be applied to a terminal device or a server. Referring to Figure 8 As shown in the figure, the equity penetration apparatus 800 can include a connected subgraph obtaining module 810 and an equity penetration module 820, wherein:

[0198] The connected subgraph obtaining module is configured to obtain the second connected subgraph in the data processing apparatus 700.

[0199] The equity data determining module is configured to perform equity penetration on the second connected subgraph to determine equity data of each target entity in the second connected subgraph.

[0200] Each module in the apparatus can be a general processor, including a central processing unit, a network processing unit, etc. It can also be a digital signal processor, an application specific integrated circuit, a field programmable gate array, or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each module can also be implemented in the form of software, firmware, etc. Each processor in the apparatus can be an independent processor, or can be integrated together.

[0201] The example embodiments of the present disclosure also provide a computer readable storage medium having a program product stored thereon, which can implement the method described in the specification. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing an electronic device to perform the steps described in the “example method” section of the specification according to various example embodiments of the present disclosure when the program product is run on the electronic device. The program product can take the form of a portable compact disc read-only memory (CD-ROM) and include program codes, and can be run on an electronic device, such as a personal computer. However, the program product of the present disclosure is not limited to this, and in this document, the readable storage medium can be any tangible medium containing or storing a program, which can be used or combined with an instruction execution system, device or apparatus.

[0202] A program product can take any combination of one or more computer-readable media. The computer-readable media can be a computer-readable storage medium or a computer-readable signal medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0203] The computer-readable signal medium can include a computer-readable storage medium that is propagated as a carrier wave in a baseband or propagated as part of a propagated data signal in a carrier, such as a propagated signal. The propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transport programming for use by or in connection with an instruction execution system, apparatus, or device.

[0204] The program code embodied on the computer-readable media can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical, RF, etc., or any suitable combination of the above.

[0205] Program code implemented as routine(s) 920 can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider. The program code can also be written in, or deployed using, a cloud computing model.

[0206] An example embodiment of the present disclosure also provides an electronic device capable of implementing the above-described method. The electronic device 900 according to this example embodiment of the present disclosure will be described below with reference to FIG. 9. Figure 9 Figure 9 The electronic device 900 shown is merely an example and should not impose any limitation on the function and scope of use of the embodiments of the present disclosure. The electronic device 900 shown is merely an example and should not impose any limitation on the function and scope of use of the embodiments of the present disclosure.

[0207] As Figure 9 shown in FIG. 9, the electronic device 900 can be a form of a general computing device. Components of the electronic device 900 can include, but are not limited to, at least one processing unit 910, at least one memory unit 920, a bus 930 that connects different system components (including the memory unit 920 and the processing unit 910), and a display unit 940.

[0208] The memory unit 920 stores program codes that can be executed by the processing unit 910, such that the processing unit 910 performs steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Methods" section of the specification. For example, the processing unit 910 can perform the method steps in the above "Exemplary Methods" section. Figure 2

[0209] The memory unit 920 can include a readable medium in the form of a volatile memory unit, such as a random access memory (RAM) 921 and / or a cache memory unit 922, and can further include a read-only memory (ROM) 923.

[0210] The memory unit 920 can further include program / utility 924 having a set of at least one program modules 925, including but not limited to, an operating system, one or more application programs, other program modules, and program data, and can include an implementation of a network environment, each or a combination of these examples.

[0211] The bus 930 can be representative of one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0212] The electronic device 900 can also communicate with one or more external devices 1000 such as a keyboard or pointing device, a Bluetooth device, etc., as well as with one or more devices that enable a user to interact with the electronic device 900 and / or any devices (e.g., a router, a modem, etc.) that enables the electronic device 900 to communicate with one or more other computing devices. Such communication can occur via an input / output (I / O) interface 950. Still yet, the electronic device 900 can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the Internet) through a network adapter 960. As Figure 9 illustrated in FIG. 9, the network adapter 960 communicates with the other components of the electronic device 900 through the bus 930. It should be appreciated that although the network adapter 960 is illustrated as a separate component, the network adapter 960 can be incorporated as part of the processing unit 910 or the bus 930. Figure 9 ​Other hardware and / or software modules not shown can be used in conjunction with the electronic device 900. Such hardware and / or software modules include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0213] From the above description of the embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the methods according to the example embodiments of the present disclosure.

[0214] In addition, the above-described flowcharts are only schematic representations of the processes included in the methods according to the example embodiments of the present disclosure, and are not intended to limit the purpose of the processes. It is easy to understand that the processes shown in the above-described flowcharts do not indicate or limit the time sequence of the processes. In addition, it is also easy to understand that the processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0215] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. Indeed, according to the embodiments of the present disclosure, the features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functionalities of one module or unit described above can be further divided into embodied by multiple modules or units.

[0216] It should be understood that the present disclosure is not limited to the precise structures described above and illustrated in the drawings and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method of equity penetration, characterized by, The method comprises: obtaining current full data of a target service, the target service being a stock data analysis service of an enterprise, and constructing a plurality of connected subgraphs according to the current full data, the current full data at least comprising current incremental data; wherein the current full data further comprises historical full data; after obtaining the current full data of the target service, the method further comprises: constructing a full data record table according to entity data and relationship data in the historical full data, the full data record table comprising a full entity table and a full relationship table; constructing an incremental data record table according to entity data and relationship data in the current incremental data, the incremental data record table comprising an incremental entity table and an incremental relationship table; and constructing a pattern recognition result table according to a first field in the full data record table and a second field in the incremental data record table, the pattern recognition result table comprising a pattern recognition entity table and a pattern recognition relationship table; wherein the full entity table comprises labels of entity data, the labels being companies or legal persons or natural persons, and the types of entity data being ages or genders or company names; the full relationship table comprises unique identifiers representing relationship data, which are primary keys of relationship data and comprise order numbers or transaction serial numbers; based on a data record table corresponding to the current incremental data, identifying a target field in the current incremental data to determine a data pattern of the current incremental data; wherein the target field is a connected subgraph identification field; the determination of the data pattern of the current incremental data comprises: obtaining a unique identifier of the current incremental data, and judging whether the connected subgraph identification field in the pattern recognition result table where the unique identifier is located is empty; if the connected subgraph identification field is empty, it is determined that the current incremental data is first new data; determining a first connected subgraph corresponding to the current incremental data, and updating the first connected subgraph according to the data pattern of the current incremental data to obtain a second connected subgraph.

2. The equity penetration method of claim 1, wherein, The data pattern of the current incremental data at least comprises any one of the following: the current incremental data is first new data, the first new data does not affect the relationship between the connected subgraphs, and the first new data comprises first new entity data and first new relationship data; the current incremental data is change data, the change data does not affect the relationship between the connected subgraphs, and the change data comprises change entity data and change relationship data; the current incremental data is second new relationship data, the second new relationship data causes the entity data in the connected subgraph to have a relationship; the current incremental data is second new entity data, and the second new entity data has a relationship with the entity data in the connected subgraph.

3. The equity penetration method of claim 1, wherein, The target field is a connected subgraph identification field, a historical attribute field and a current attribute field; the identification of the target field in the current incremental data based on the data record table corresponding to the current incremental data to determine the data pattern of the current incremental data comprises: obtain a unique identifier of the current incremental data, and determine whether a connected subgraph identifier field in a pattern recognition result table where the unique identifier is located is empty, and whether a current attribute field in the pattern recognition result table and a historical attribute field are same; if the connected subgraph identifier field is not empty and the current attribute field and the historical attribute field are not same, it is determined that the current incremental data is change data.

4. The equity penetration method of claim 1 wherein, the determining, based on the data record table corresponding to the current incremental data, a target field in the current incremental data, to determine a data pattern of the current incremental data, comprises: querying first added relationship data in a pattern recognition relationship table, and determining target entity data connected with the first added relationship data; determining whether a connected subgraph identifier field corresponding to the target entity data in a full-amount entity table is empty; if the connected subgraph identifier field is not empty, it is determined that the current incremental data is second added relationship data.

5. The equity penetration method of claim 1 wherein, the determining, based on the data record table corresponding to the current incremental data, a target field in the current incremental data, to determine a data pattern of the current incremental data, comprises: querying first added entity data in a pattern recognition entity table; querying target relationship data connected with the first added entity data in a pattern recognition relationship table, the target relationship data connecting the first added entity data and historical entity data; determining whether a connected subgraph identifier field corresponding to the historical entity data in a full-amount entity table is empty; if the connected subgraph identifier field is not empty, it is determined that the current incremental data is second added entity data.

6. The method of claim 1-5, wherein, the current incremental data is first added data; updating the first connected subgraph according to the data pattern of the current incremental data to obtain a second connected subgraph, comprising: obtaining first added entity data in a pattern recognition entity table and first added relationship data in a pattern recognition relationship table; constructing the second connected graph according to the first added entity data and the first added relationship data.

7. The method of claim 1-5, wherein, the current incremental data is change data; updating the first connected subgraph according to the data pattern of the current incremental data to obtain a second connected subgraph, comprising: replacing historical data in the first connected subgraph with the change data to obtain the second connected subgraph.

8. The method of claim 1-5, wherein, the current incremental data is second added relationship data; updating the first connected subgraph according to the data pattern of the current incremental data to obtain a second connected subgraph, comprising: determining a plurality of first connected subgraphs connected with the second added relationship data; merging a plurality of the first connected subgraphs to obtain the second connected subgraph.

9. The method of claim 1-5, wherein, the current incremental data is second added entity data; updating the first connected subgraph according to the data pattern of the current incremental data to obtain a second connected subgraph, comprising: constructing a target connected subgraph according to first added entity data and target relationship data connected with the first added entity data, the target relationship data connecting the first added entity data and historical entity data; determining a first connected subgraph corresponding to the historical entity data; merge the target connected subgraph and the first connected subgraph to obtain the second connected subgraph.

10. A method of equity penetration, characterized by, The method comprises: obtaining the second connected subgraph according to any one of claims 1-9; performing equity penetration on the second connected subgraph to determine the equity data of each target entity in the second connected subgraph.

11. A stock penetrating device characterized by, The method comprises: A connected subgraph construction module is configured to obtain current full data of a target service, the target service being an equity data analysis service of an enterprise, and construct a plurality of connected subgraphs according to the current full data, the current full data at least including current incremental data; wherein the current full data further includes historical full data; after obtaining the current full data of the target service, the device further comprises: constructing a full data record table according to entity data and relationship data in the historical full data, the full data record table including a full entity table and a full relationship table; constructing an incremental data record table according to entity data and relationship data in the current incremental data, the incremental data record table including an incremental entity table and an incremental relationship table; and constructing a pattern recognition result table according to a first field in the full data record table and a second field in the incremental data record table, the pattern recognition result table including a pattern recognition entity table and a pattern recognition relationship table; wherein the full entity table includes labels of entity data, the labels being companies or legal persons or natural persons, and the types of entity data being ages or genders or company names; the full relationship table contains unique identifiers representing relationship data, which are primary keys of relationship data and include order numbers or transaction serial numbers; A data pattern determination module is configured to identify a target field in the current incremental data based on a data record table corresponding to the current incremental data, to determine a data pattern of the current incremental data; wherein the target field is a connected subgraph identification field; the determination of the data pattern of the current incremental data includes: obtaining a unique identifier of the current incremental data, and judging whether the connected subgraph identification field in the pattern recognition result table where the unique identifier is located is empty; if the connected subgraph identification field is empty, it is determined that the current incremental data is first new data; A connected subgraph update module is configured to determine a first connected subgraph corresponding to the current incremental data, and update the first connected subgraph according to the data pattern of the current incremental data, to obtain a second connected subgraph.

12. A stock penetrating device characterized by, The method comprises: A connected subgraph acquisition module is configured to obtain the second connected subgraph according to any one of claims 1-9; An equity data determination module is configured to perform equity penetration on the second connected subgraph to determine the equity data of each target entity in the second connected subgraph.

13. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the method of any one of claims 1-10.

14. An electronic device, comprising: The method comprises: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method of any one of claims 1-10 by executing the executable instructions. The computer program is executed by a processor to implement the method of any one of claims 1-10.

Images