Data resource map generation method and system based on big data mining technology

Abstract

The embodiment of the application discloses a data resource map generation method and system based on big data mining technology, comprising: obtaining a component B1 in a front-end page A1 in a development environment, a back-end data interface or data interaction calling information request C1 operated by the component B1, obtaining server log data based on the data interaction calling information request C1; obtaining the front-end page A1, the component B1, the data interaction calling information request C1, a data set, a data table, a data field and a data processing rule called by a back-end triggered task and a triggering mechanism under different request conditions; associating the relationship between the data set, the data table, the data field and the data processing rule, forming a first modularized data resource map and a second modularized data resource map, and finally forming a global data resource map M. The problem that an enterprise global data resource map cannot be formed in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to a data resource map generation system, method, electronic device, and storage medium based on big data mining technology. Background Technology

[0002] Data classification and grading is based on the natural attributes of data assets, combined with national and industry standards, to formulate suitable classification and grading standards for data, and to classify and grade customer data according to these standards to determine the security level of data assets.

[0003] Currently, many enterprises do not manage their own systems, especially cross-system data resources, in a systematic way, and are unable to form a complete data resource map for the enterprise. As a result, they spend a lot of money on data governance, data security and data traceability.

[0004] Therefore, there is an urgent need for a method that can generate a comprehensive data resource map for an enterprise based on big data mining technology. Summary of the Invention

[0005] The purpose of this invention is to provide a data resource map generation system, method, electronic device, and storage medium based on big data mining technology, in order to solve the problem that existing technologies cannot generate a complete data resource map for an enterprise.

[0006] To achieve the above objectives, embodiments of the present invention provide a method for generating data resource maps based on big data mining technology, wherein the method for generating data resource maps based on big data mining technology specifically includes:

[0007] The system retrieves component B1 from the front-end page A1 in the development environment, as well as the back-end data interface or data interaction call information request C1 that component B1 interacts with. Based on the data interaction call information request C1, the system retrieves server log data. The number of components B1 is one or more, and the number of data interaction call information requests C1 is one or more.

[0008] Obtain the dataset, data table, data fields, and data processing rules corresponding to the server log data under different request conditions;

[0009] The relationships between front-end page A1, component B1, data interaction call information request C1, dataset, data table, data field and data processing rules are linked to form the first modular data resource map;

[0010] Obtain the dataset, data table, data field, and data processing rules called by the backend triggering task and triggering mechanism, and associate the relationships between the frontend page A1, component B1, data interaction call information request C1, dataset, data table, data field, and data processing rules to form a second modular data resource map;

[0011] A global data resource map M is formed based on the first modular data resource map and the second modular map, wherein the number of the first modular data resource map and the second modular map is one or more.

[0012] Based on the above technical solution, the present invention can be further improved as follows:

[0013] Furthermore, the step of obtaining the dataset, data table, data field, and data processing rules corresponding to the server log data under different request conditions, and obtaining the dataset, data table, data field, and data processing rules called by the backend triggering task and triggering mechanism, includes:

[0014] Generate a unique A for front-end page A1, and generate a unique D for each data entry element component D1 in A;

[0015] Generate a unique B for each component B1, generate a unique C for each data interaction call information request C1, generate a unique T based on different request conditions of C, and generate a unique TT based on T.

[0016] Furthermore, the data resource map generation method based on big data mining technology also includes:

[0017] Establish relationships between A, D, B, C, T, TT, and M;

[0018] A unique identifier is generated for each relationship between A, D, B, C, T, TT, M and A, D, B, C, T, TT, M, and the identifier is stored.

[0019] Furthermore, the data resource map generation method based on big data mining technology also includes:

[0020] Determine whether the relationships between A, D, B, C, T, TT, M and A, D, B, C, T, TT, M have been stored. If so, do not store them.

[0021] Furthermore, the process of associating the relationships between datasets, data tables, data fields, and data processing rules to form a first modular data resource map includes:

[0022] Obtain the preceding data table FT, preceding dataset FA, preceding data field FT-FT, preceding data field FA-FT, data processing rule PR, following data table BT, following data field BT-FT, following dataset BA, and following data field BA-FT corresponding to TT;

[0023] Based on the decomposition of data processing rules PR, the preceding data table FT, preceding dataset FA, preceding data field FT-FT, and preceding data field FA-FT are associated with the following data tables BT, following data field BT-FT, following dataset BA, and following data field BA-FT through data processing logic, forming the first modular data resource map.

[0024] Furthermore, the step of associating the relationships between datasets, data tables, data fields, and data processing rules to form a first modular data resource map also includes:

[0025] If the preceding data table FT and the preceding dataset FA do not exist, the data source is determined to be the combination of data entry elements D in A.

[0026] Furthermore, the data resource map generation method based on big data mining technology also includes:

[0027] For each data table, establish a relationship between at least one upstream data resource and one downstream data resource;

[0028] Based on the relationship between upstream and downstream data resources corresponding to each data table, the data resources are integrated and linked to form a global data resource map M.

[0029] A data resource map generation system based on big data mining technology includes:

[0030] The acquisition module is used to acquire component B1 in the front-end page A1 in the development environment, the back-end data interface or data interaction call information request C1 that component B1 operates and connects to, and acquire server log data based on the data interaction call information request C1. The number of components B1 is one or more, and the number of data interaction call information requests C1 is one or more.

[0031] Obtain the dataset, data table, data fields, and data processing rules corresponding to the server log data under different request conditions;

[0032] The map generation module is used to associate the relationships between the front-end page A1, component B1, data interaction call information request C1, dataset, data table, data field and data processing rules to form the first modular data resource map;

[0033] Obtain the dataset, data table, data field, and data processing rules called by the backend triggering task and triggering mechanism, and associate the relationships between the frontend page A1, component B1, data interaction call information request C1, dataset, data table, data field, and data processing rules to form a second modular data resource map;

[0034] A global data resource map M is formed based on the first modular data resource map and the second modular map, wherein the number of the first modular data resource map and the second modular map is one or more.

[0035] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the steps of the method described herein.

[0036] A non-transitory computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the method when executed by a processor.

[0037] The embodiments of the present invention have the following advantages:

[0038] The data resource map generation method based on big data mining technology in this invention uses identifier resolution technology to connect with various server environments, collect information including but not limited to server logs, and form at least one upstream data resource and downstream data resource association relationship for each data table information, thereby forming a complete data resource map of the enterprise, which solves the problem that existing technologies cannot form a complete data resource map of the enterprise. Attached Figure Description

[0039] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0040] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0041] Figure 1 This is a flowchart of the data resource map generation method based on big data mining technology of the present invention;

[0042] Figure 2 This is the first architecture diagram of the data resource map generation system based on big data mining technology of the present invention;

[0043] Figure 3 This is the second architecture diagram of the data resource map generation system based on big data mining technology of the present invention;

[0044] Figure 4 This is a schematic diagram of the physical structure of the electronic device provided by the present invention.

[0045] The attached figures are labeled as follows:

[0046] Acquisition module 10, map generation module 20, storage module 30, association module 40, electronic device 50, processor 501, memory 502, bus 503. Detailed Implementation

[0047] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Figure 1 This is a flowchart illustrating an embodiment of the data resource map generation method based on big data mining technology of the present invention. Figure 1 As shown, an embodiment of the present invention provides a data resource map generation method based on big data mining technology, which includes the following steps:

[0049] S101, obtain component B1 in the front-end page A1 of the development environment, the back-end data interface or data interaction call information request C1 that component B1 operates and connects to, and obtain server log data based on data interaction call information request C1;

[0050] Specifically: There are one or more components B1, and one or more data interaction call information request C1.

[0051] S102, Obtain the dataset, data table, data fields, and data processing rules corresponding to server log data under different request conditions;

[0052] S103, associate the relationships between front-end page A1, component B1, data interaction call information request C1, dataset, data table, data field and data processing rules to form the first modular data resource map;

[0053] S104: Obtain the dataset, data table, data field, and data processing rules called by the backend triggering task and triggering mechanism; associate the relationships between the frontend page A1, component B1, data interaction call information request C1, dataset, data table, data field, and data processing rules to form a second modular data resource map.

[0054] Specifically, it involves collecting data from which fields of which data tables are used by the operational functions of each system's front-end, and which components on the front-end are used for display.

[0055] Collect data on which front-end components' operational functions are stored, and what rules are used to transform the front-end data to obtain the stored data, and in which field of the data table it is stored;

[0056] How does the data acquisition backend process the field data of the data table to obtain the field data of the new data table and dataset?

[0057] Generate a unique A for the front-end page A1, and generate a unique D (D1, D2, D3...) for each data entry element component D1 in A, such as front-end text boxes, radio buttons, and other front-end data acquisition elements that represent front-end business attributes and have front-end business attributes;

[0058] Generate a unique B for each component B1, generate a unique C for each data interaction call information request C1, generate a unique T based on different request conditions of C, and generate a unique TT based on T;

[0059] Establish relationships between A, D, B, C, T, TT, and M. ；

[0060] A unique identifier is generated for each of A, D, B, C, T, TT and for each association between A, D, B, C, T, TT, and the identifier is stored.

[0061] Determine whether the relationships between A, D, B, C, T, TT and A, D, B, C, T, TT, M have been stored. If so, do not store them.

[0062] Based on the backend program TT, the system decomposes the data tables / processing rules called by the backend program and writes data into a new data table based on the data table data and processing rules (a. add or modify operations), or decomposes the data tables / processing rules called by the backend program to obtain data for displaying the dataset (b. query operations), or decomposes the frontend data parameters / processing rules received by the backend program and writes them into a new data table (c. add operations). It uses pre-configured data processing syntax corresponding to different backend development languages ​​and data processing terms (e.g., addition, subtraction, multiplication, division, rounding to two decimal places, etc.); it obtains the preceding data table FT, preceding dataset FA, preceding data field FT-FT, preceding data field FA-FT, data processing rule PR, following data table BT, following data field BT-FT, following dataset BA, and following data field BA-FT corresponding to TT.

[0063] Based on the decomposition of data processing rules PR, the preceding data table FT, preceding dataset FA, preceding data field FT-FT, and preceding data field FA-FT are associated with the following data tables BT, following data field BT-FT, following dataset BA, and following data field BA-FT through data processing logic, forming the first modular data resource map.

[0064] If the preceding data table FT and the preceding dataset FA do not exist, the data source is determined to be the combination of data entry elements D in A.

[0065] For example:

[0066] a. Obtain server log data, analyze the current front-end operation business system module, page and operation function name, and mark this operation function name with a unique ID1. Based on the dataset called by this operating system, obtain the data table and data processing rules called by the dataset.

[0067] If it is a query operation, the component that retrieves data metrics and displays them on the front-end page will generate ID2 under different conditions, as well as the corresponding business system, system module, page, operation function name, called data table or dataset, data processing rules, data metrics, and the components displayed on the front-end page, for the unique ID1 mentioned above.

[0068] If it is a CRUD operation, the same analysis is performed on the current front-end operation business system module, page and operation function name, and a unique ID1 is marked for this operation function name. Then the business system, system module, page and component corresponding to the data processed by CRUD operations are obtained, as well as the data processing rules. If there are differences in the components corresponding to the data processed under different conditions in the server log data, then different conditions generate ID2 and the corresponding business system, system module, page and component, as well as the data processing rules.

[0069] For data processing based on backend task triggering mechanisms, a unique ID1 is generated for each task triggering or mechanism. Similarly, ID2 is generated based on different conditions, as well as the data table and data rules called under the corresponding conditions, and the data table or generated dataset is stored after processing.

[0070] b. A device based on identifier resolution technology generates corresponding identifier data from the acquired information and registers a unique identifier ID3, corresponding to the aforementioned storage-related information;

[0071] c. For each ID1 and ID2, only a unique ID3 is generated. Based on the monitoring of the log data of each server, the device identifies the above-mentioned new operation or task / mechanism information and condition information, and determines whether it has been stored. If it has been stored, then the retrieval in a and the storage in b are not performed.

[0072] S105, a global data resource map M is formed based on the first modular data resource map and the second modular map;

[0073] Specifically, there may be one or more first modular data resource maps and second modular maps.

[0074] For each data table, establish a relationship between at least one upstream data resource and one downstream data resource;

[0075] Based on the relationship between upstream and downstream data resources corresponding to each data table, the data resources are integrated and linked to form a global data resource map M.

[0076] The query operation retrieves data from the data source table based on different conditions and the data processing rules; the add, delete, and modify operation retrieves data from the front-end page components based on different conditions and the data source table retrieves data processing rules; and the task trigger or mechanism retrieves data from the data source table based on different conditions and the data processing rules. This generates a production data resource map of elements such as business systems, system modules, system operations or task triggers (mechanisms), data source tables, and data processing rules.

[0077] For example, the generation steps are as follows: The query operation retrieves the data source table and data processing rules. The result is displayed in the component, so the query operation is associated with the data source table and data processing rules. The result displayed on the front end is associated with the component. The data addition, deletion, and modification operations on the page are performed on the component display based on the rules and stored in the original data source table or a new data source table. So the addition, deletion, and modification operations are associated with the component, that is, with the data source table and data processing rules retrieved by the query operation, then with the data processing rules for displaying data in the component, and finally with the stored data source table or the new data source table. In this way, each step of the operation is associated with data retrieval, data processing rules, and data storage.

[0078] Similarly, based on task triggering or mechanisms, the task triggering or mechanisms, the data source table being called, and the data processing rules are associated with the data source table or new data source table; based on the above, a global production data resource map of the business is generated, and a data flow map is also generated;

[0079] Obtain a list of system modules, pages, and related operation functions for each system;

[0080] Obtain the classification and grading level of each data table and its corresponding data fields (including structured and unstructured data);

[0081] The relationship between each operation function item in the operation function list and each data table and data field is obtained through the production data resource map. Then, all data tables and data fields corresponding to each operation function item in the operation function list are obtained. Finally, the classification and grading levels of all data tables and data fields corresponding to each operation function item are obtained. The highest classification and grading level are used as the current classification and grading level of each operation function item.

[0082] By defining the access levels for each employee based on the operable data categories of each business system, the security of system data within the enterprise is intelligently controlled.

[0083] This data resource map generation method based on big data mining technology obtains component B1 from the front-end page A1 in the development environment, and the back-end data interface or data interaction call information request C1 that component B1 interacts with. Based on the data interaction call information request C1, server log data is obtained. The number of components B1 and data interaction call information requests C1 can be one or more. The method then obtains the datasets, data tables, data fields, and data processing rules corresponding to the server log data under different request conditions. The relationships between the datasets, data tables, data fields, and data processing rules are associated to form a first modular data resource map. Next, the method obtains the datasets, data tables, data fields, and data processing rules called by back-end triggering tasks and triggering mechanisms, and the relationships between these datasets, data tables, data fields, and data processing rules are associated to form a second modular data resource map. Finally, a global data resource map M is formed based on the first and second modular data resource maps. The number of the first and second modular data resource maps can be one or more. This data resource map generation method based on big data mining technology solves the problem in existing technologies that cannot form a comprehensive data resource map for an enterprise.

[0084] Figures 2-3 This is a flowchart of an embodiment of the data resource map generation system based on big data mining technology of the present invention; as follows: Figures 2-3 As shown in the figure, an embodiment of the present invention provides a data resource map generation system based on big data mining technology, which includes the following steps:

[0085] The acquisition module 10 is used to acquire component B1 in the front-end page A1 in the development environment, the back-end data interface or data interaction call information request C1 that component B1 operates and connects to, and acquire server log data based on the data interaction call information request C1. The number of components B1 is one or more, and the number of data interaction call information requests C1 is one or more.

[0086] Obtain the dataset, data table, data fields, and data processing rules corresponding to the server log data under different request conditions;

[0087] The acquisition module 10 is further configured to:

[0088] Generate a unique A for front-end page A1, and generate a unique D for each data entry element component D1 in A;

[0089] Generate a unique B for each component B1, generate a unique C for each data interaction call information request C1, generate a unique T based on different request conditions of C, and generate a unique TT based on T.

[0090] The map generation module 20 is used to associate the relationships between the front-end page A1, component B1, data interaction call information request C1, dataset, data table, data field and data processing rules to form the first modular data resource map;

[0091] Obtain the dataset, data table, data field, and data processing rules called by the backend triggering task and triggering mechanism, and associate the relationships between the frontend page A1, component B1, data interaction call information request C1, dataset, data table, data field, and data processing rules to form a second modular data resource map;

[0092] A global data resource map M is formed based on the first modular data resource map and the second modular map, wherein the number of the first modular data resource map and the second modular map is one or more.

[0093] The map generation module 20 is also used for:

[0094] Obtain the preceding data table FT, preceding dataset FA, preceding data field FT-FT, preceding data field FA-FT, data processing rule PR, following data table BT, following data field BT-FT, following dataset BA, and following data field BA-FT corresponding to TT;

[0095] Based on the decomposition of data processing rules PR, the preceding data table FT, preceding dataset FA, preceding data field FT-FT, and preceding data field FA-FT are associated with the following data tables BT, following data field BT-FT, following dataset BA, and following data field BA-FT through data processing logic, forming the first modular data resource map.

[0096] If the preceding data table FT and the preceding dataset FA do not exist, the data source is determined to be the combination of data entry elements D in A.

[0097] Storage module 30 is used to establish relationships between A, D, B, C, T, TT, and M. ；

[0098] A unique identifier is generated for each relationship between A, D, B, C, T, TT, M and A, D, B, C, T, TT, M, and the identifier is stored.

[0099] Determine whether the relationships between A, D, B, C, T, TT, M and A, D, B, C, T, TT, M have been stored. If so, do not store them.

[0100] The association module 40 is used to form an association relationship between at least one upstream data resource and one downstream data resource for each data table;

[0101] Based on the relationships between upstream and downstream data resources corresponding to each data table, integration and linking are performed to ultimately form a global data resource map M.

[0102] This invention discloses a data resource map generation system based on big data mining technology. The system acquires components B1 and backend data interfaces or data interaction call requests C1 from a frontend page A1 in the development environment via an acquisition module 10. Based on the data interaction call requests C1, it acquires server log data. The system includes one or more components B1 and one or more data interaction call requests C1. It then acquires the datasets, data tables, data fields, and data processing rules corresponding to the server log data under different request conditions. A map generation module 20 associates the relationships between the frontend page A1, components B1, data interaction call requests C1, datasets, data tables, data fields, and data processing rules to form a first modular data resource map. Next, it acquires the datasets, data tables, data fields, and data processing rules called by backend triggering tasks and triggering mechanisms, and associates the relationships between the frontend page A1, components B1, data interaction call requests C1, datasets, data tables, data fields, and data processing rules to form a second modular data resource map. Finally, it forms a global data resource map M based on the first and second modular maps. The system includes one or more of the first and second modular maps. This data resource map generation system, based on big data mining technology, solves the problem that existing technologies cannot generate a comprehensive data resource map for an enterprise.

[0103] Figure 4 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 4 As shown, the electronic device 50 includes: a processor 501, a memory 502, and a bus 503;

[0104] The processor 501 and the memory 502 communicate with each other via the bus 503.

[0105] The processor 501 is used to call program instructions in the memory 502 to execute the methods provided in the above-described method embodiments, such as: obtaining component B1 in the front-end page A1 of the development environment, and the back-end data interface or data interaction call information request C1 that component B1 operates and interfaces with; obtaining server log data based on the data interaction call information request C1, wherein the number of components B1 is one or more, and the number of data interaction call information requests C1 is one or more; obtaining the dataset, data table, data field, and data processing rules corresponding to the server log data under different request conditions; and connecting the front-end page A1, component B1, and data interaction call information request C1 to the back-end data interface or data interaction call information request C1. The relationships between C1, datasets, data tables, data fields, and data processing rules are established to form a first modular data resource map. The datasets, data tables, data fields, and data processing rules called by backend triggering tasks and mechanisms are obtained, and the relationships between the frontend page A1, component B1, data interaction call information request C1, datasets, data tables, data fields, and data processing rules are established to form a second modular data resource map. A global data resource map M is formed based on the first and second modular data resource maps, wherein there are one or more of the first and second modular data resource maps. 。

[0106] This embodiment provides a non-transitory computer-readable storage medium that stores computer instructions. These instructions cause a computer to execute the methods provided in the above-described method embodiments. For example, the instructions include: obtaining component B1 in a front-end page A1 in a development environment, and a back-end data interface or data interaction call information request C1 that component B1 interacts with; obtaining server log data based on the data interaction call information request C1; wherein the number of components B1 is one or more, and the number of data interaction call information requests C1 is one or more; obtaining the dataset, data table, data fields, and data processing rules corresponding to the server log data under different request conditions; and storing computer instructions in front-end page A1, component B1, and back-end data interface or data interaction call information request C1. B1, the data interaction call information request C1, the dataset, data table, data field, and data processing rules are associated to form a first modular data resource map; the dataset, data table, data field, and data processing rules called by the backend triggering task and triggering mechanism are obtained, and the relationships between the frontend page A1, component B1, data interaction call information request C1, dataset, data table, data field, and data processing rules are associated to form a second modular data resource map; a global data resource map M is formed based on the first modular data resource map and the second modular map, wherein the number of the first modular data resource map and the second modular map is one or more.

[0107] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various storage media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0108] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0109] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0110] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for generating data resource maps based on big data mining technology, characterized in that, The data resource map generation method based on big data mining technology specifically includes: The system retrieves component B1 from the front-end page A1 in the development environment, as well as the back-end data interface or data interaction call information request C1 that component B1 interacts with. Based on the data interaction call information request C1, the system retrieves server log data. The number of components B1 is one or more, and the number of data interaction call information requests C1 is one or more. Obtain the dataset, data table, data fields, and data processing rules corresponding to the server log data under different request conditions; The relationships between front-end page A1, component B1, data interaction call information request C1, dataset, data table, data field, and data processing rules are linked to form the first modular data resource map, including: Obtain the preceding data table FT, preceding dataset FA, preceding data field FT-FT, preceding data field FA-FT, data processing rule PR, following data table BT, following data field BT-FT, following dataset BA, and following data field BA-FT corresponding to TT; Based on the decomposition of data processing rules PR, the preceding data table FT, preceding dataset FA, preceding data field FT-FT, and preceding data field FA-FT are associated with the following data tables BT, following data field BT-FT, following dataset BA, and following data field BA-FT to form the first modular data resource map. Obtain the dataset, data table, data field, and data processing rules called by the backend triggering task and triggering mechanism, and associate the relationships between the frontend page A1, component B1, data interaction call information request C1, dataset, data table, data field, and data processing rules to form a second modular data resource map; A global data resource map M is formed based on the first modular data resource map and the second modular map, wherein the number of the first modular data resource map and the second modular map is one or more; For each data table, establish a relationship between at least one upstream data resource and one downstream data resource; Based on the relationship between upstream and downstream data resources corresponding to each data table, the data resources are integrated and linked to form a global data resource map M.

2. The data resource map generation method based on big data mining technology according to claim 1, characterized in that, The process of obtaining the datasets, data tables, data fields, and data processing rules corresponding to the server log data under different request conditions, and obtaining the datasets, data tables, data fields, and data processing rules called by the backend triggering task and triggering mechanism, includes: Generate a unique A for front-end page A1, and generate a unique D for each data entry element component D1 in A; Generate a unique B for each component B1, generate a unique C for each data interaction call information request C1, generate a unique T based on different request conditions of C, and generate a unique TT based on T.

3. The data resource map generation method based on big data mining technology according to claim 1, characterized in that, The data resource map generation method based on big data mining technology also includes: Establish relationships between A, D, B, C, T, TT, and M; A unique identifier is generated for each relationship between A, D, B, C, T, TT, M and A, D, B, C, T, TT, M, and the identifier is stored.

4. The data resource map generation method based on big data mining technology according to claim 3, characterized in that, The data resource map generation method based on big data mining technology also includes: Determine whether the relationships between A, D, B, C, T, TT, M and A, D, B, C, T, TT, M have been stored. If so, do not store them.

5. The data resource map generation method based on big data mining technology according to claim 4, characterized in that, The relationships between datasets, data tables, data fields, and data processing rules are linked to form the first modular data resource map, which also includes: If the preceding data table FT and the preceding dataset FA do not exist, the data source is determined to be the combination of data entry elements D in A.

6. A data resource map generation system based on big data mining technology, characterized in that, include: The acquisition module is used to acquire component B1 in the front-end page A1 in the development environment, the back-end data interface or data interaction call information request C1 that component B1 operates and connects to, and acquire server log data based on the data interaction call information request C1. The number of components B1 is one or more, and the number of data interaction call information requests C1 is one or more. Obtain the dataset, data table, data fields, and data processing rules corresponding to the server log data under different request conditions; The map generation module is used to associate the relationships between the front-end page A1, component B1, data interaction call information request C1, dataset, data table, data field and data processing rules to form the first modular data resource map; The map generation module is also used for: Obtain the preceding data table FT, preceding dataset FA, preceding data field FT-FT, preceding data field FA-FT, data processing rule PR, following data table BT, following data field BT-FT, following dataset BA, and following data field BA-FT corresponding to TT; Based on the decomposition of data processing rules PR, the preceding data table FT, preceding dataset FA, preceding data field FT-FT, and preceding data field FA-FT are associated with the following data tables BT, following data field BT-FT, following dataset BA, and following data field BA-FT to form the first modular data resource map. Obtain the dataset, data table, data field, and data processing rules called by the backend triggering task and triggering mechanism, and associate the relationships between the frontend page A1, component B1, data interaction call information request C1, dataset, data table, data field, and data processing rules to form a second modular data resource map; A global data resource map M is formed based on the first modular data resource map and the second modular map, wherein the number of the first modular data resource map and the second modular map is one or more; For each data table, establish a relationship between at least one upstream data resource and one downstream data resource; Based on the relationship between upstream and downstream data resources corresponding to each data table, the data resources are integrated and linked to form a global data resource map M.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images