Data information processing method, device, electronic device and storage medium

Abstract

The present invention discloses a data information processing method and device, which includes: automatically generating an initial data relationship based on basic data tags; obtaining basic data based on the initial data relationship; constructing data nodes, and mining new data relationships through node similarity; when the first matrix A1 is a behavior matrix, M1(a,b)=sqrt((a1(a)‑a1(b))^2), M2(a,b)=sqrt((a2(a)‑a2(b))^2), Mn(a,b)=sqrt(((an(a)‑an(b))^2), taking the array generated by the minimum value of M data and defining it as a new data node; when When the second matrix A2 is a similarity matrix, M1(i,j) = |a1(a)-a1(b)|, M2(i,j) = |a2(a)-a2(b)|, and Mn(i,j) = |an(a)-an(b)|. The array generated by taking the maximum value of M is defined as a new data node; where the parameter M represents the logical distance between data a and data b, an(a) represents the value of the nth feature of sample i, and an(b) represents the value of the nth feature of sample j. The data information processing method and device provided by the present invention mine new data relationships through node similarity, thereby providing diversity and accuracy for data extraction.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing, and in particular to a data information processing method and device. Background Art

[0002] Currently, commonly used data information is directly parsed and stored as data or files, and then processed in a unified manner. When querying data information, the data response will be very slow, which is very inconvenient for data analysis and data viewing.

[0003] Publication number CN115098506A, entitled "A method, apparatus, device and storage medium for storing associated data", discloses a method for storing associated data, including: obtaining each data node and multiple node attributes corresponding to each data node in a target database, and determining the attribute identifiers corresponding to each node attribute for each data node according to a preset numerical range; constructing a node information table corresponding to each data node based on the multiple node attributes corresponding to each data node and the attribute identifiers corresponding to each node attribute; obtaining the attribute identifier set corresponding to each sample under different associated nodes based on the associated node attributes corresponding to each sample in the target database and each node information table; constructing a sample associated information table based on the attribute identifier set corresponding to each sample under different associated nodes and the sample identifier of each sample. However, the above method is too simple and cannot effectively solve the above problems.

[0004] Therefore, it is necessary to provide a data information processing method and device to solve the above problems. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to propose a data information processing method and device, which mines new data relationships through node similarity, thereby providing diversity and accuracy for data extraction.

[0006] According to one aspect of the present invention, a data information processing method is provided, the method comprising:

[0007] Automatically generate initial data relationships based on basic data tags;

[0008] Obtaining basic data according to the initial data relationship;

[0009] Build data nodes and mine new data relationships through node similarity.

[0010] Preferably, the basic data tags are established according to business attributes and information attributes.

[0011] Preferably, mining new data relationships by node similarity includes:

[0012] Setting a first data node and a second data node, wherein the first data node and the second data node are in a many-to-many relationship;

[0013] The first data node is used as row M and the second data node is used as column N to obtain a first matrix A1. The first matrix A1 is an M*N matrix. The element value A of the first matrix A1 is ij is the number of times the i-th first data node is associated with the j-th second data node, and the first matrix A1 is a row spacing matrix;

[0014] Calculate the distance values ​​between rows of the first matrix A and perform normalization on the distance values. The larger the distance value, the smaller the normalized value, and the smaller the similarity. The smaller the distance value, the larger the normalized value, and the greater the similarity.

[0015] Preferably, the first data nodes are used as rows M and columns M to obtain a second matrix A2, the second matrix A2 is an M*M matrix, and the second matrix A2 is a similarity matrix.

[0016] Preferably, the number of nodes from all data points to the central data point is calculated, and the average value is calculated, and the data nodes exceeding the average value are removed.

[0017] Preferably, the ratio of the number of nodes to the total number of points in each single relationship is counted to calculate the lift value, and relationships with lift values ​​greater than 1 are removed.

[0018] Preferably, when the first matrix A1 is a behavior matrix,

[0019] M1(a,b)=sqrt((a1(a)-a1(b))^2)

[0020] M2(a,b)=sqrt((a2(a)-a2(b))^2)

[0021] Mn(a,b)=sqrt(((an(a)-an(b))^2)

[0022] Take the array generated by the M minimum value data and define it as a new data node;

[0023] When the second matrix A2 is a similar matrix,

[0024] M1(i,j)=|a1(a)-a1(b)|

[0025] M2(i,j)=|a2(a)-a2(b)|

[0026] Mn(i,j)=|an(a)-an(b)|

[0027] Take the array generated by the M maximum value data and define it as a new data node;

[0028] Wherein, parameter M represents the logical distance between data a and data b, an(a) represents the value of the nth feature of sample i, and an(b) represents the value of the nth feature of sample j. According to another aspect of the present invention, a data information processing device is provided, comprising:

[0029] An initial data relationship generation module, which is used to automatically generate initial data relationships based on basic data tags;

[0030] A basic data acquisition module, configured to acquire basic data according to the initial data relationship;

[0031] A new data relationship mining module is used to construct data nodes and mine new data relationships through node similarity.

[0032] The present application discloses a data information processing method and device, which includes: automatically generating initial data relationships based on basic data tags; obtaining basic data based on the initial data relationships; constructing data nodes, and mining new data relationships through node similarity, thereby providing diversity and accuracy for data extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention, not all embodiments. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0034] Figure 1 is a flow chart of a data information processing method according to an embodiment of the present invention;

[0035] Figure 2 is a module diagram of a data information processing device according to an embodiment of the present invention;

[0036] Figure 3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

[0037] Figure 4 It is a data relationship image obtained by a data information processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field without making creative efforts based on the embodiments of the present invention are within the scope of protection of the present invention.

[0039] The technical solution of the present invention is described in detail below with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0040] In view of the problems existing in the prior art, an embodiment of the present invention provides a data information processing method and apparatus, which mines new data relationships by analyzing the similarity of nodes, thereby providing diversity and accuracy for data extraction.

[0041] Figure 1 This is a flow chart of a data information processing method according to an embodiment of the present invention. Figure 1 According to one aspect of the present invention, a data information processing method is provided, the method comprising:

[0042] Step S101: Automatically generate initial data relationships based on basic data tags;

[0043] Step S102: obtaining basic data according to the initial data relationship;

[0044] Step S103: construct data nodes and mine new data relationships through node similarity.

[0045] In a specific implementation, the basic data tags are established according to business attributes and information attributes.

[0046] In a specific implementation, mining new data relationships based on node similarity includes:

[0047] Setting a first data node and a second data node, wherein the first data node and the second data node are in a many-to-many relationship;

[0048] The first data node is used as row M and the second data node is used as column N to obtain a first matrix A1. The first matrix A1 is an M*N matrix. The element value A of the first matrix A1 is ij is the number of times the i-th first data node is associated with the j-th second data node, and the first matrix A1 is a row spacing matrix;

[0049] Calculate the distance values ​​between rows of the first matrix A and perform normalization on the distance values. The larger the distance value, the smaller the normalized value, and the smaller the similarity. The smaller the distance value, the larger the normalized value, and the greater the similarity.

[0050] Specifically, the first matrix A1 is a row spacing matrix as shown in the following table:

[0051]

[0052] In a specific implementation, the first data node is used as a row M and a column M to obtain a second matrix A2, the second matrix A2 is an M*M matrix, and the second matrix A2 is a similarity matrix.

[0053] Specifically, the second matrix A2 is a similarity matrix as shown in the following table:

[0054]

[0055] In a specific implementation, the number of nodes from all data nodes to the central data point is calculated, and the average value is calculated, and the data nodes exceeding the average value are removed.

[0056] In a specific implementation, the ratio of the number of nodes to the total number of points in each single relationship is counted, the lift value is calculated, and the relationships with the lift value greater than 1 are removed.

[0057] In a specific implementation, when the first matrix A1 is a behavior matrix,

[0058] M1(a,b)=sqrt((a1(a)-a1(b))^2)

[0059] M2(a,b)=sqrt((a2(a)-a2(b))^2)

[0060] Mn(a,b)=sqrt(((an(a)-an(b))^2)

[0061] Take the array generated by the M minimum value data and define it as a new data node;

[0062] When the second matrix A2 is a similar matrix,

[0063] M1(i,j)=|a1(a)-a1(b)|

[0064] M2(i,j)=|a2(a)-a2(b)|

[0065] Mn(i,j)=|an(a)-an(b)|

[0066] Take the array generated by the M maximum value data and define it as a new data node;

[0067] Among them, the parameter M represents the logical distance between data a and data b, an(a) represents the value of the nth feature of sample i, and an(b) represents the value of the nth feature of sample j.

[0068] Figure 2 This is a module diagram of a data information processing device according to an embodiment of the present invention. Figure 2 According to another aspect of the present invention, a data information processing device is provided, the device comprising:

[0069] An initial data relationship generation module 21 is used to automatically generate initial data relationships based on basic data tags;

[0070] A basic data acquisition module 22, configured to acquire basic data according to the initial data relationship;

[0071] The new data relationship mining module 23 is used to construct data nodes and mine new data relationships through node similarity.

[0072] Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. Figure 3 As shown, an electronic device 300 provided in this embodiment includes: a processor 301 and a memory 302; wherein:

[0073] The memory 302 is used to store computer programs, and the memory may also be a flash memory.

[0074] The processor 301 is configured to execute the execution instructions stored in the memory to implement each step in the above method. For details, please refer to the relevant description in the above method embodiment.

[0075] Optionally, the memory 302 may be independent or integrated with the processor 401 .

[0076] When the memory 302 is a device independent of the processor 301, the electronic device 300 may further include:

[0077] The bus 303 is used to connect the memory 302 and the processor 301 .

[0078] This embodiment further provides a readable storage medium, in which a computer program is stored. When at least one processor of an electronic device executes the computer program, the electronic device executes the methods provided in the various aforementioned embodiments.

[0079] This embodiment further provides a program product, which includes a computer program stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor can execute the computer program to cause the electronic device to implement the methods provided in the various embodiments described above.

[0080] Figure 4 It is a data relationship image obtained by a data information processing method according to an embodiment of the present invention. Figure 4 The data relationship image obtained by a data information processing method according to an embodiment of the present invention is a one-to-many relationship image with a closed loop.

[0081] In summary, the data information processing method and device disclosed in the embodiments of the present application automatically generate initial data relationships based on basic data tags; obtain basic data based on the initial data relationships; construct data nodes, and mine new data relationships through node similarity, thereby providing diversity and accuracy for data extraction.

[0082] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present application are indicated by the claims.

[0083] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data information processing method, characterized in that: The method comprises: Automatically generate initial data relationships based on basic data tags; Obtaining basic data according to the initial data relationship; Construct data nodes and mine new data relationships through node similarity; set a first data node and a second data node, the first data node and the second data node are in a many-to-many relationship; take the first data node as the row M and the second data node as the column N to obtain the first matrix , the first matrix for Matrix, the first matrix The element value of is the number of times the i-th first data node is associated with the j-th second data node, the first matrix is a row distance matrix; calculating the distance value between rows of the first matrix A, and normalizing the distance value. The larger the distance value, the smaller the normalized value, and the smaller the similarity. The smaller the distance value, the larger the normalized value, and the greater the similarity. With the first data node as row M and column M, we get the second matrix , the second matrix for Matrix, the second matrix is a similarity matrix; Calculate the number of data nodes to the central data point, calculate the average value, and remove data nodes that exceed the average value; When the first matrix When is the behavior matrix, Take the array generated by the M minimum value data and define it as a new data node; When the second matrix When is a similarity matrix, Take the array generated by the M maximum value data and define it as a new data node; Among them, the parameter M represents the logical distance between data a and data b, an(a) represents the value of the nth feature of sample i, and an(b) represents the value of the nth feature of sample j.

2. The data information processing method according to claim 1, characterized in that: The basic data tags are established according to business attributes and information attributes.

3. The data information processing method according to claim 1, characterized in that: The ratio of the number of nodes in each single relationship to the total number of points is counted, and the lift value is calculated. Relationships with a lift value greater than 1 are removed.

4. A data information processing device, characterized in that: The device comprises: An initial data relationship generation module, which is used to automatically generate initial data relationships based on basic data tags; A basic data acquisition module, configured to acquire basic data according to the initial data relationship; A new data relationship mining module is used to construct data nodes and mine new data relationships based on node similarity; a first data node and a second data node are set, and the first data node and the second data node are in a many-to-many relationship; the first data node is used as the row M and the second data node is used as the column N to obtain the first matrix , the first matrix for Matrix, the first matrix The element value of is the number of times the i-th first data node is associated with the j-th second data node, the first matrix is a row distance matrix; calculating the distance value between rows of the first matrix A, and normalizing the distance value. The larger the distance value, the smaller the normalized value, and the smaller the similarity. The smaller the distance value, the larger the normalized value, and the greater the similarity. With the first data node as row M and column M, we get the second matrix , the second matrix for Matrix, the second matrix is a similarity matrix; Calculate the number of data nodes to the central data point, calculate the average value, and remove data nodes that exceed the average value; When the first matrix When is the behavior matrix, Take the array generated by the M minimum value data and define it as a new data node; When the second matrix When is a similarity matrix, Take the array generated by the M maximum value data and define it as a new data node; Among them, the parameter M represents the logical distance between data a and data b, an(a) represents the value of the nth feature of sample i, and an(b) represents the value of the nth feature of sample j.

5. An electronic device, characterized in that: include: processor; as well as, a memory for storing executable instructions of the processor; The processor is configured to perform the method according to any one of claims 1 to 3 by executing the executable instructions.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 3 when executed by a processor.

Images