Data information processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data information processing method and device. The method comprises the following steps: automatically generating an initial data relationship according to a basic data label; obtaining basic data according to the initial data relationship; constructing data nodes, and mining a new data relationship through a node similarity degree; when the first matrix A1 is a behavior matrix, M1 (a, b) = sqrt ((a1 (a)-a1 (b)) 2), M2 (a, b) = sqrt ((a2 (a)-a2 (b)) 2) and Mn (a, b) = sqrt (((an (a)-an (b)) 2), taking an array generated by M minimum value data, and defining the array as a new data node; when the second matrix A2 is a similar matrix, M1 (i, j) = a1 (a)-a1 (b), M2 (i, j) = a2 (a)-a2 (b) and Mn (i, j) = an (a)-an (b), taking an array generated by M maximum value data, and defining the array as a new data node; wherein the parameter M represents the logic distance between the data a and the data b, an (a) represents the value of the nth feature of the sample i, and an (b) represents the value of the nth feature of the sample j. According to the data information processing method and device provided by the invention, the new data relationship is mined through the node similarity degree, so that diversity and accuracy are provided for data extraction.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing, and in particular to a data information processing method and device. Background Art

[0002] At present, commonly used data information is to directly parse and store data or files, and then perform unified calculation processing. When querying data information, the data response will be very slow, which is very inconvenient for data analysis and data viewing.

[0003] The publication number is CN115098506A, and the name is “A storage method, device, equipment and storage medium for associated data”, which discloses a storage method for associated data, including: obtaining each data node and multiple node attributes corresponding to each data node in the target database, and determining the attribute identifiers corresponding to each node attribute for each data node according to a preset numerical range; constructing a node information table corresponding to each data node according to the multiple node attributes corresponding to each data node and the attribute identifiers corresponding to each node attribute; obtaining the attribute identifier set corresponding to each sample under different associated nodes according to the associated node attributes corresponding to each sample in the target database and each node information table; constructing a sample associated information table according to the attribute identifier set corresponding to each sample under different associated nodes and the sample identifier of each sample. However, the above method is too simple and cannot effectively solve the above problems.

[0004] Therefore, it is necessary to provide a data information processing method and device to solve the above problems. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a data information processing method and device, which mines new data relationships through node similarity, thereby providing diversity and accuracy for data extraction.

[0006] According to one aspect of the present invention, a data information processing method is provided, the method comprising:

[0007] Automatically generate initial data relationships based on basic data tags;

[0008] Obtaining basic data according to the initial data relationship;

[0009] Build data nodes and mine new data relationships through node similarity.

[0010] Preferably, the basic data tags are established according to business attributes and information attributes.

[0011] Preferably, mining new data relationships by node similarity includes:

[0012] Set a first data node and a second data node, where the first data node and the second data node are in a many-to-many relationship;

[0013] The first data node is used as the row M and the second data node is used as the column N to obtain a first matrix A1, wherein the first matrix A1 is an M*N matrix and the element value A of the first matrix A1 is ij is the number of times the i-th first data node is associated with the j-th second data node, and the first matrix A1 is a row distance matrix;

[0014] The distance values ​​between rows of the first matrix A are calculated, and the distance values ​​are normalized. The larger the distance value is, the smaller the value after normalization is, and the smaller the similarity is. The smaller the distance value is, the larger the value after normalization is, and the greater the similarity is.

[0015] Preferably, the first data nodes are used as rows M and columns M to obtain a second matrix A2, wherein the second matrix A2 is an M*M matrix, and the second matrix A2 is a similarity matrix.

[0016] Preferably, the number of all data nodes to the central data point is calculated, and the average value is calculated, and the data nodes exceeding the average value are removed.

[0017] Preferably, the ratio of the number of nodes to the total number of points in each single relationship is counted, the lift value is calculated, and the relationship with the lift value greater than 1 is removed.

[0018] Preferably, when the first matrix A1 is a behavior matrix,

[0019] M1(a,b)=sqrt((a1(a)-a1(b))^2)

[0020] M2(a,b)=sqrt((a2(a)-a2(b))^2)

[0021] Mn(a,b)=sqrt(((an(a)-an(b))^2)

[0022] Take the array generated by the M minimum value data and define it as a new data node;

[0023] When the second matrix A2 is a similarity matrix,

[0024] M1(i,j)=|a1(a)-a1(b)|

[0025] M2(i,j)=|a2(a)-a2(b)|

[0026] Mn(i,j)=|an(a)-an(b)|

[0027] Take the array generated by the M maximum value data and define it as a new data node;

[0028] Wherein, parameter M represents the logical distance between data a and data b, an(a) represents the value of the nth feature of sample i, and an(b) represents the value of the nth feature of sample j. According to another aspect of the present invention, a data information processing device is provided, the device comprising:

[0029] An initial data relationship generation module, which is used to automatically generate initial data relationships based on basic data tags;

[0030] A basic data acquisition module, which is used to obtain basic data according to the initial data relationship;

[0031] A new data relationship mining module is used to construct data nodes and mine new data relationships through node similarity.

[0032] The present application discloses a data information processing method and device, the method comprising: automatically generating an initial data relationship based on basic data tags; obtaining basic data based on the initial data relationship; constructing data nodes, mining new data relationships through node similarity, and mining new data relationships through node similarity, thereby providing diversity and accuracy for data extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present invention, not all embodiments. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0034] Figure 1 is a flow chart of a data information processing method according to an embodiment of the present invention;

[0035] Figure 2 is a module schematic diagram of a data information processing device according to an embodiment of the present invention;

[0036] Figure 3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

[0037] Figure 4 It is a data relationship image obtained by a data information processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. According to the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0040] In view of the problems existing in the prior art, an embodiment of the present invention provides a data information processing method and device, which mines new data relationships through node similarity, thereby providing diversity and accuracy for data extraction.

[0041] Figure 1 is a flow chart of a data information processing method according to an embodiment of the present invention. Figure 1 According to one aspect of the present invention, a data information processing method is provided, the method comprising:

[0042] Step S101: Automatically generate initial data relationships based on basic data tags;

[0043] Step S102: obtaining basic data according to the initial data relationship;

[0044] Step S103: construct data nodes and mine new data relationships through node similarity.

[0045] In a specific implementation, the basic data tags are established according to business attributes and information attributes.

[0046] In a specific implementation, mining new data relationships by node similarity includes:

[0047] Set a first data node and a second data node, where the first data node and the second data node are in a many-to-many relationship;

[0048] The first data node is used as the row M and the second data node is used as the column N to obtain a first matrix A1, wherein the first matrix A1 is an M*N matrix and the element value A of the first matrix A1 is ij is the number of times the i-th first data node is associated with the j-th second data node, and the first matrix A1 is a row distance matrix;

[0049] The distance values ​​between rows of the first matrix A are calculated, and the distance values ​​are normalized. The larger the distance value is, the smaller the value after normalization is, and the smaller the similarity is. The smaller the distance value is, the larger the value after normalization is, and the greater the similarity is.

[0050] Specifically, the first matrix A1 is a row spacing matrix as shown in the following table:

[0051]

[0052] In a specific implementation, the first data nodes are used as rows M and columns M to obtain a second matrix A2, the second matrix A2 is an M*M matrix, and the second matrix A2 is a similarity matrix.

[0053] Specifically, the second matrix A2 is a similarity matrix as shown in the following table:

[0054]

[0055] In a specific implementation, the number of all data nodes to the central data point is calculated, and the average value is calculated, and the data nodes exceeding the average value are removed.

[0056] In a specific implementation, the ratio of the number of nodes to the total number of points in each single relationship is counted, the lift value is calculated, and the relationship with the lift value greater than 1 is removed.

[0057] In a specific implementation, when the first matrix A1 is a behavior matrix,

[0058] M1(a,b)=sqrt((a1(a)-a1(b))^2)

[0059] M2(a,b)=sqrt((a2(a)-a2(b))^2)

[0060] Mn(a,b)=sqrt(((an(a)-an(b))^2)

[0061] Take the array generated by the M minimum value data and define it as a new data node;

[0062] When the second matrix A2 is a similarity matrix,

[0063] M1(i,j)=|a1(a)-a1(b)|

[0064] M2(i,j)=|a2(a)-a2(b)|

[0065] Mn(i,j)=|an(a)-an(b)|

[0066] Take the array generated by the M maximum value data and define it as a new data node;

[0067] Among them, the parameter M represents the logical distance between data a and data b, an(a) represents the value of the nth feature of sample i, and an(b) represents the value of the nth feature of sample j.

[0068] Figure 2 is a schematic diagram of a module of a data information processing device according to an embodiment of the present invention. Figure 2 According to another aspect of the present invention, a data information processing device is provided, the device comprising:

[0069] An initial data relationship generation module 21, which is used to automatically generate an initial data relationship according to basic data tags;

[0070] A basic data acquisition module 22, which is used to obtain basic data according to the initial data relationship;

[0071] The new data relationship mining module 23 is used to construct data nodes and mine new data relationships through node similarity.

[0072] Figure 3 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. Figure 3 As shown, an electronic device 300 provided in this embodiment includes: a processor 301 and a memory 302; wherein:

[0073] The memory 302 is used to store computer programs, and the memory may also be a flash memory.

[0074] The processor 301 is used to execute the execution instructions stored in the memory to implement each step in the above method. For details, please refer to the relevant description in the above method embodiment.

[0075] Optionally, the memory 302 may be independent or integrated with the processor 401 .

[0076] When the memory 302 is a device independent of the processor 301, the electronic device 300 may further include:

[0077] The bus 303 is used to connect the memory 302 and the processor 301 .

[0078] This embodiment further provides a readable storage medium, in which a computer program is stored. When at least one processor of an electronic device executes the computer program, the electronic device executes the methods provided in the above-mentioned various implementation modes.

[0079] This embodiment also provides a program product, which includes a computer program stored in a readable storage medium. At least one processor of the electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device implements the methods provided in the above various embodiments.

[0080] Figure 4 is a data relationship image obtained by a data information processing method according to an embodiment of the present invention. Figure 4 The data relationship image obtained by a data information processing method according to an embodiment of the present invention is a one-to-many relationship image with a closed loop.

[0081] In summary, a data information processing method and device disclosed in an embodiment of the present application automatically generates an initial data relationship based on basic data tags; obtains basic data based on the initial data relationship; constructs data nodes, and mines new data relationships through node similarity, thereby providing diversity and accuracy for data extraction.

[0082] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0083] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data information processing method, characterized in that: The method comprises: Automatically generate initial data relationships based on basic data tags; Obtaining basic data according to the initial data relationship; Build data nodes and mine new data relationships through node similarity.

2. The data information processing method according to claim 1, characterized in that: The basic data tags are established according to business attributes and information attributes.

3. The data information processing method according to claim 1, characterized in that: The mining of new data relationships by node similarity includes: Set a first data node and a second data node, where the first data node and the second data node are in a many-to-many relationship; The first data node is used as the row M and the second data node is used as the column N to obtain a first matrix A1, wherein the first matrix A1 is an M*N matrix and the element value A of the first matrix A1 is ij is the number of times the i-th first data node is associated with the j-th second data node, and the first matrix A1 is a row distance matrix; The distance values ​​between rows of the first matrix A are calculated, and the distance values ​​are normalized. The larger the distance value is, the smaller the value after normalization is, and the smaller the similarity is. The smaller the distance value is, the larger the value after normalization is, and the greater the similarity is.

4. The data information processing method according to claim 3, characterized in that: The first data nodes are used as rows M and columns M to obtain a second matrix A2, wherein the second matrix A2 is an M*M matrix, and the second matrix A2 is a similarity matrix.

5. The data information processing method according to claim 4, characterized in that: Calculate the number of data nodes to the central data point, calculate the average value, and remove the data nodes that exceed the average value.

6. The data information processing method according to claim 5, characterized in that: The ratio of the number of nodes to the total number of points in each single relationship is counted, the lift value is calculated, and the relationships with the lift value greater than 1 are removed.

7. The data information processing method according to claim 4, characterized in that: When the first matrix A1 is a behavior matrix, M1(a,b)=sqrt((a1(a)-a1(b))^2) M2(a,b)=sqrt((a2(a)-a2(b))^2) Mn(a,b)=sqrt(((an(a)-an(b))^2) Take the array generated by the M minimum value data and define it as a new data node; When the second matrix A2 is a similarity matrix, M1(i,j)=|a1(a)-a1(b)| M2(i,j)=|a2(a)-a2(b)| Mn(i,j)=|an(a)-an(b)| Take the array generated by the M maximum value data and define it as a new data node; Among them, the parameter M represents the logical distance between data a and data b, an(a) represents the value of the nth feature of sample i, and an(b) represents the value of the nth feature of sample j.

8. A data information processing device, characterized in that: The device comprises: An initial data relationship generation module, which is used to automatically generate initial data relationships based on basic data tags; A basic data acquisition module, which is used to obtain basic data according to the initial data relationship; A new data relationship mining module is used to construct data nodes and mine new data relationships through node similarity.

9. An electronic device, characterized in that: include: processor; as well as, A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images