Information processing method and device of big data component, equipment, medium and product

Abstract

The invention relates to an information processing method and device for a big data component, equipment, a medium and a product, and relates to the technical field of big data. The method comprises the following steps: reading a big data component record from a database, and storing the record into a dependency linked list; a plurality of big data component records are stored in the database; traversing the dependency chain table, filling the dependency chain table into a dictionary data structure, and storing a big data component name and a big data component name set having a direct association relationship with the big data component name; and traversing the dictionary data structure, filling the dictionary data structure into the total mapping structure, storing the big data component names and a big data component name set having an association relationship with the big data component names, and obtaining a dependency relationship processing result. According to the method, the database containing the corresponding relation between the big data component and the dependent component is designed, a user only needs to add a component record of the database for a newly-added component, and the upgrading difficulty is reduced. The user can freely select the big data component combination, and the flexibility of the big data component combination is improved.

Description

Technical Field

[0001] This application relates to the field of big data technology, and particularly to a method, apparatus, computer device, computer-readable storage medium, and computer program product for processing information of big data components. Background Art

[0002] Currently, most cloud computing providers implement big data component deployment solutions based on scenarios, such as data lakes, data services, real-time data streams, etc. In each scenario, the combination of big data components is fixed. This solution has many defects, specifically: (1) lack of flexibility, the fixed combination of big data components in each scenario may not meet user needs; (2) it is troublesome to put on the shelves for scenarios. There are many types of big data components, and the number of their combinations is even more huge. If the component combinations are fixed for each scenario, it means that different scenarios need to be put on the shelves frequently, which is time-consuming and laborious; (3) difficult to upgrade. If the version of a certain big data component needs to be upgraded, all scenarios containing this component need to be upgraded simultaneously, resulting in a very large upgrade workload. Summary of the Invention

[0003] Based on this, in view of the above technical problems, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for processing information of big data components, which can reduce the difficulty of upgrading, enhance the scalability of big data components, and improve the flexibility of big data component combinations.

[0004] In a first aspect, this application provides a method for processing information of big data components, including:

[0005] Read big data component records from a database and store them in a dependency linked list; there are multiple big data component records stored in the database, and each big data component record includes a big data component name and a dependent component name;

[0006] Traverse the dependency linked list and fill the traversed data into a dictionary data structure; the dictionary data structure is used to store the big data component name and a set of big data component names that have a direct association relationship with the big data component name; the direct association relationship is a direct dependency relationship or a direct being-dependent relationship;

[0007] Traverse the dictionary data structure and fill the traversed data into a total mapping structure to obtain a dependency relationship processing result; the total mapping structure is used to store the big data component name and a set of big data component names that have an association relationship with the big data component name; the association relationship is a dependency relationship or a being-dependent relationship; the dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the being-dependent relationship includes a direct being-dependent relationship and an indirect being-dependent relationship.

[0008] In one embodiment, the method further includes:

[0009] Receive component configuration instructions; the component configuration instructions are used to select or cancel a target big data component;

[0010] According to the component configuration instructions, query the corresponding total mapping structure and determine at least one dependent component related to the target big data component;

[0011] Perform corresponding configuration processing on the target big data component and at least one dependent component.

[0012] In one embodiment, traverse the dictionary data structure and fill the traversed data into the total mapping structure, including:

[0013] Establish a relationship list, an access list, and a processing queue;

[0014] Traverse the dictionary data structure and store the currently traversed big data component name in the access list and the processing queue;

[0015] Obtain the current element from the processing queue;

[0016] From the dictionary data structure, obtain the set of big data component names corresponding to the current element;

[0017] Traverse the obtained set of big data component names;

[0018] If the traversed associated component name is not in the access list, add the traversed associated component name to the relationship list, the access list, and the processing queue respectively;

[0019] When it is detected that the processing queue is empty, assign the current big data component name to the current key of the total mapping structure and assign the relationship list to the current value of the total mapping structure.

[0020] In one embodiment, after traversing the obtained set of big data component names, the method further includes:

[0021] If the traversed associated component name is in the access list, continue to traverse the obtained set of big data component names.

[0022] In one embodiment, traverse the dependency linked list and fill the traversed data into the dictionary data structure, including:

[0023] Traverse the dependency linked list and obtain the target name from the traversed linked list object;

[0024] Detect whether there is a target key in the dictionary data structure that matches the target name;

[0025] If it exists, store the attribute value of the linked list object in the value corresponding to the target key.

[0026] In one embodiment, the method further includes:

[0027] If it does not exist, a target object is created, and the attribute values of the linked list object are stored in the target object;

[0028] The target name of the linked list object is assigned to the current key of the dictionary data structure, and the target object is assigned to the current value of the dictionary data structure.

[0029] In a second aspect, the present application further provides an information processing device for a big data component, including:

[0030] A reading module, configured to read big data component records from a database and store them in a dependency linked list; there are multiple big data component records stored in the database, and each big data component record includes a big data component name and a dependency component name;

[0031] A first traversal module, configured to traverse the dependency linked list and fill the traversed data into a dictionary data structure; the dictionary data structure is used to store the big data component name and a set of big data component names that have a direct association relationship with the big data component name; the direct association relationship is a direct dependency relationship or a directly dependent relationship;

[0032] A second traversal module, configured to traverse the dictionary data structure and fill the traversed data into a total mapping structure to obtain a dependency relationship processing result; the total mapping structure is used to store the big data component name and a set of big data component names that have an association relationship with the big data component name; the association relationship is a dependency relationship or a dependent relationship; the dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the dependent relationship includes a directly dependent relationship and an indirect dependent relationship.

[0033] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect above are implemented.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0035] In a fifth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0036] The above information processing method, device, computer equipment, computer-readable storage medium and computer program product of big data components read big data component records from a database and store them in a dependency linked list; there are multiple big data component records stored in the database, and each big data component record includes a big data component name and a dependent component name; traverse the dependency linked list and fill the traversed data into a dictionary data structure; the dictionary data structure is used to store the big data component name and the set of big data component names that have a direct association relationship with the big data component name; the direct association relationship is a direct dependency relationship or a direct being-dependent relationship; traverse the dictionary data structure and fill the traversed data into a total mapping structure to obtain a dependency relationship processing result; the total mapping structure is used to store the big data component name and the set of big data component names that have an association relationship with the big data component name; the association relationship is a dependency relationship or a being-dependent relationship; the dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the being-dependent relationship includes a direct being-dependent relationship and an indirect being-dependent relationship. Through the above method, a database containing the one-to-one correspondence relationship between big data components and dependent components is designed, which is convenient for forward traversal or reverse traversal to construct a dictionary data structure, enabling the reuse of the logic of component dependency relationship operations and component being-dependent relationship operations, and greatly reducing the implementation complexity of the algorithm. For newly added components, users only need to add component records to the database without changing the existing component records and operation logic, reducing the upgrade difficulty and enhancing the scalability of big data components. It can automatically generate big data component combinations based on the database without being fixed according to scenarios, enabling users to freely select big data component combinations and improving the flexibility of big data component combinations. Description of the Drawings

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can be obtained based on these drawings.

[0038] Figure 1 It is a flowchart of the information processing method of big data components in one embodiment;

[0039] Figure 2 It is a flowchart of the information processing method of big data components in another embodiment;

[0040] Figure 3 It is a flowchart of the information processing method of big data components in yet another embodiment;

[0041] Figure 4 It is a structural block diagram of the information processing device of big data components in one embodiment;

[0042] Figure 5 It is the internal structure diagram of a computer device in an embodiment. Specific implementation manners

[0043] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0044] In an exemplary embodiment, as Figure 1 shown, an information processing method for big data components is provided. In this embodiment, an example is given where this method is applied to a computer device. It can be understood that this method can also be applied to a server, and can also be applied to a system including a computer device and a server, and is implemented through the interaction between the computer device and the server. The method includes:

[0045] Step 102: Read big data component records from the database and store them in a dependency linked list; there are multiple big data component records stored in the database, and each big data component record includes a big data component name and a dependent component name.

[0046] Among them, the database in this embodiment stores big data components and the directly dependent data of big data components, that is, the one-to-one correspondence between storage components and directly dependent components. The advantage of such a design is that the forward traversal and reverse traversal logics are the same. Optionally, the data table of the database adopts the data table structure shown in Table 1 below. The big data component records stored in the database include big data component names, dependent component names, creation time, and update time.

[0047] Table 1:

[0048]

[0049] Exemplarily, referring to Table 2, Table 2 is an example of the dependency relationship data representation of any big data component in the database.

[0050] Table 2:

[0051]

[0052] Among them, a new dependency linked list is created to save big data component records. Specifically, a new database table field mapping object componentDependency is created, and a new dependency linked list dependencyList: List <componentdependency>Object. Store all the records read from the database into the dependencyList data structure.

[0053] Step 104: Traverse the dependency linked list and fill the traversed data into the dictionary data structure. The dictionary data structure is used to store the big data component names and the set of big data component names that have a direct association relationship with the big data component names. The direct association relationship is a direct dependency relationship or a direct being-dependent relationship.

[0054] Among them, the direct dependency relationship means that the operation of the subject directly relies on the support of the object, and there is no intermediate medium between the two. The direct being-dependent relationship means that the function of the object is directly relied on by the subject. Optionally, traverse the dependency linked list through the first traversal strategy to generate the first dictionary data structure, which is used to store the big data component names and the set of big data component names that have a direct dependency relationship with the big data component names. Optionally, traverse the dependency linked list through the second traversal strategy to generate the second dictionary data structure, which is used to store the big data component names and the set of big data component names that have a direct being-dependent relationship with the big data component names. Among them, the first traversal strategy is, for example, to traverse based on the big data component names in the big data component records; the second traversal strategy is, for example, to traverse based on the dependent component names in the big data component records.

[0055] The dictionary data structure refers to the map data structure, which stores and operates data in the form of key-value pairs. Among them, in the dictionary data structure corresponding to the direct dependency relationship, the key stores the big data component names, and the value stores the set of big data component names that have a direct dependency relationship with the big data component names. In the dictionary data structure corresponding to the direct being-dependent relationship, the key stores the big data component names, and the value stores the set of big data component names that have a direct being-dependent relationship with the big data component names.

[0056] Step 106: Traverse the dictionary data structure and fill the traversed data into the total mapping structure to obtain the dependency relationship processing result. The total mapping structure is used to store the big data component names and the set of big data component names that have an association relationship with the big data component names. The association relationship is a dependency relationship or a being-dependent relationship. The dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the being-dependent relationship includes a direct being-dependent relationship and an indirect being-dependent relationship.

[0057] Among them, the total mapping structure refers to an object that maps keys to values, that is, the map data structure. For the dictionary data structure corresponding to the direct dependency relationship, by traversing this dictionary data structure, all big data components that the currently traversed big data component depends on from the first layer to the nth layer can be exhausted, where n is an integer greater than 1. For the dictionary data structure corresponding to the directly depended-on relationship, by traversing this dictionary data structure, all big data components that directly or indirectly depend on the currently traversed big data component can be determined, that is, all big data components that depend on the current big data component from the first layer to the nth layer are exhausted.

[0058] It can be understood that based on the total mapping structure corresponding to the dependency relationship, the set of all big data components that a big data component depends on can be managed, facilitating users to query and use combinations of big data components with dependency relationships. Similarly, based on the total mapping structure corresponding to the depended-on relationship, the set of all big data components that depend on a big data component can be managed, facilitating users to query and use combinations of big data components with depended-on relationships.

[0059] In the above information processing method of big data components, big data component records are read from the database and stored in a dependency linked list; there are multiple big data component records stored in the database, and each big data component record includes a big data component name and a dependent component name; the dependency linked list is traversed, and the data traversed is filled into a dictionary data structure; the dictionary data structure is used to store the big data component name and the set of big data component names that have a direct association relationship with the big data component name; the direct association relationship is a direct dependency relationship or a directly depended-on relationship; the dictionary data structure is traversed, and the data traversed is filled into the total mapping structure to obtain a dependency relationship processing result; the total mapping structure is used to store the big data component name and the set of big data component names that have an association relationship with the big data component name; the association relationship is a dependency relationship or a depended-on relationship; the dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the depended-on relationship includes a directly depended-on relationship and an indirect depended-on relationship. Through the above method, a database containing a one-to-one correspondence between big data components and dependent components is designed, facilitating forward or reverse traversal to construct a dictionary data structure, enabling the reuse of the logic of component dependency relationship operations and component depended-on relationship operations, greatly reducing the implementation complexity of the algorithm. For newly added components, users only need to add component records to the database without changing the existing component records and operation logic, reducing the upgrade difficulty and enhancing the scalability of big data components. It is possible to automatically generate combinations of big data components based on the database without being fixed according to scenarios, enabling users to freely select combinations of big data components and improving the flexibility of combinations of big data components.

[0060] In an exemplary embodiment, as Figure 2 shown, the method further includes:

[0061] Step 202, receive a component configuration instruction; the component configuration instruction is used to select or cancel a target big data component.

[0062] Among them, the computer device provides an input unit, and the user inputs a component configuration instruction to the computer device through the input unit. The user can select or cancel any big data component through the component configuration instruction, and the big data component targeted by the component configuration instruction is the target big data component.

[0063] Step 204, according to the component configuration instruction, query the corresponding total mapping structure, and determine at least one dependent component related to the target big data component.

[0064] Among them, if the component configuration instruction is used to select a target big data component, query the total mapping structure corresponding to the dependency relationship, and determine at least one dependent component that has a direct or indirect dependency relationship with the target big data component. If the component configuration instruction is used to cancel the target big data component, query the total mapping structure corresponding to the be-dependent relationship, and determine at least one dependent component that has a direct or indirect be-dependent relationship with the target big data component.

[0065] Step 206, perform corresponding configuration processing on the target big data component and at least one dependent component.

[0066] Among them, when the user selects any big data component, the computer device automatically completes other components on which the big data component depends according to the dependency relationship processing result. Or, when the user cancels any big data component, the computer device automatically cancels other components that depend on the big data component according to the dependency relationship processing result. It is possible to freely select a combination of big data components according to user needs, ensure that the combination of big data components can be successfully deployed, and enhance the stability and reliability of the system.

[0067] In an exemplary embodiment, step 106 includes: establishing a relationship list, an access list, and a processing queue; traversing the dictionary data structure, storing the currently traversed big data component name in the access list and the processing queue; obtaining the current element from the processing queue; obtaining the set of big data component names corresponding to the current element from the dictionary data structure; traversing the obtained set of big data component names; if the traversed associated component name is not in the access list, add the traversed associated component name to the relationship list, the access list, and the processing queue respectively; when it is detected that the processing queue is empty, assign the current big data component name to the current key of the total mapping structure, and assign the relationship list to the current value of the total mapping structure.

[0068] Among them, taking the dictionary data structure corresponding to the direct dependency relationship as an example for illustration, the dictionary data structure obtained in step 104 includes the set of direct dependency component names corresponding to each big data component. The big data components in these direct dependency name sets directly depend on some other big data components. Step 106 is to traverse all the big data components that the big data components depend on from the first layer to the nth layer.

[0069] New Map<String, Set <string>The total mapping structure `totalMapping` is used to store the big data component names and all other big data components that the big data component depends on from the first layer to the nth layer. Traverse the dictionary data structure `dependencyMap`, and the traversal logic is as follows:

[0070] 1. Create a relationship list `allDependencies`, an access list `visited`, and a processing queue `queue`; for each element traversed in `dependencyMap`, create a new `Set` <string>The allDependencies object, visited object, and Queue of the data structure <string>The queue object of the data structure. The relationship list allDependencies is used to store all big data components that the key value of the currently traversed dependencyMap depends on from the first layer to the nth layer; the access list visited is used to record the visited nodes to avoid repeated traversal and circular dependencies; the queue is used to store the unvisited nodes.

[0071] 2. Initialize the access list visited and the processing queue queue: Store the key value (big data component name) of the dependencyMap element to be traversed into queue and visited to complete the initialization of queue and visited.

[0072] 3. If queue is not empty, execute the tracing loop. The logic of each loop is as follows: (1) Pop an element from the head of the queue queue, and this element is the big data component name; (2) According to the obtained big data component name, take out the set of directly dependent component names corresponding to this big data component name from the dependencyMap. (3) If the obtained set of directly dependent component names is not empty, traverse this set of directly dependent component names. The traversal logic is: Check whether this element (the traversed associated component name) is in the access list visited. If it is not in the access list visited, add this element to the processing queue queue, add this element to the access list visited, and add this element to the relationship list allDependencies.

[0073] 4. After the tracing loop execution ends, assign the traversed big data component name (the key value of the dependencyMap element) to the key of totalMapping, and assign the relationship list allDependencies to the value of totalMapping.

[0074] 5. After traversing the dependencyMap, generate the final totalMapping, which is used to save the big data component names and all big data components that depend on them from the first layer to the nth layer.

[0075] For the dictionary data structure dependedMap corresponding to the direct dependency relationship, execute according to the above traversal logic, and all big data components that directly or indirectly depend on each big data component can be obtained.

[0076] In this embodiment, when traversing the dictionary data structure, a design of temporarily storing the traversed components is adopted, which can avoid repeated traversal of the traversed components and improve the execution efficiency of the algorithm.

[0077] In an exemplary embodiment, after traversing the obtained set of big data component names, the method further includes: if the traversed associated component name is in the access list, continue traversing the obtained set of big data component names.

[0078] Wherein, if the traversed associated component name is in the access list, continue traversing the next element in the set of big data component names.

[0079] In an exemplary embodiment, step 104 includes: traversing the dependency linked list, obtaining a target name from the traversed linked list object; detecting whether there is a target key in the dictionary data structure that matches the target name; if so, storing the attribute value of the linked list object into the value corresponding to the target key.

[0080] Among them, taking the dictionary data structure corresponding to the direct dependency relationship as an example for illustration, create a new dictionary data structure dependencyMap: Map<String, Set <string>> Objects of the type. The key of the dependencyMap is the big data component name, and the value is the set of the names of the directly dependent components corresponding to the big data component. Traverse each element componentDependency (i.e., the linked list object) in the dependency list dependencyList, and the logic is as follows: Determine whether the value of the name attribute of the current componentDependency exists in the key of the dependencyMap. If it exists, store the value of the dependency attribute of the componentDependency into the target object collectionDependency corresponding to the value of the dependencyMap in the dictionary data structure, and finally assign the target object collectionDependency to the value of the dependencyMap.

[0081] Similarly, create a dictionary data structure dependedMap corresponding to the directly depended relationship: Map<String,Set <string>>Type of object. The key of the dependedMap is the name of the big data component being depended on, and the value is the set of names of the big data components that depend on it. Traverse each element componentDependency (i.e., the linked list object) in the dependency list dependencyList, and the logic is as follows: Determine whether the value of the dependency attribute of the current componentDependency exists in the key of the dependedMap. If it exists, store the value of the name attribute of the componentDependency into the target object collectionDepended corresponding to the value of the dependedMap, and finally assign the target object collectionDepended to the value of the dependedMap.

[0082] In an exemplary embodiment, the method further includes: If not, create a new target object, store the attribute values of the linked list object into the target object; assign the target name of the linked list object to the current key of the dictionary data structure, and assign the target object to the current value of the dictionary data structure.

[0083] Among them, taking the dictionary data structure corresponding to the direct dependency relationship as an example, if the value of the name attribute of the current componentDependency does not exist in the key of the dependencyMap, then create a new Set <string>For the target object collectionDependency of type, store the dependency attribute value of componentDependency into the collectionDependency set. Finally, assign the name attribute value of componentDependency to the key of dependencyMap, and assign collectionDependency to the value of dependencyMap. The target object collectionDependency can be of the hashset type, which is used to store non-repeating data.

[0084] Finally, after traversing all elements of the dependency list dependencyList, a dictionary data structure is obtained. The dictionary data structure stores all big data component names and the set of big data component names that have a direct dependency relationship with each big data component name.

[0085] Similarly, taking the dictionary data structure corresponding to the directly dependent relationship as an example, if the dependency attribute value of the current componentDependency does not exist in the key of dependedMap, a new Set is created. <string>For the target object collectionDepended of the type, store the value of the name attribute of componentDependency into the collectionDepended set. Finally, assign the value of the dependency attribute of componentDependency to the key of dependedMap, and assign collectionDepended to the value of dependedMap. The target object collectionDepended can be of the hashset type, which is used to store non-duplicate data.

[0086] Finally, after traversing all elements of the dependency list dependencyList, a dictionary data structure is obtained, which stores all big data component names and the set of big data component names that have a direct dependency relationship with each big data component name.

[0087] In an alternative implementation, taking the processing of the dependency relationship of big data components as an example, the information processing method of big data components includes:

[0088] 1. Read all big data component dependency relationship data in the database and store it in the dependency list dependencyList.

[0089] 2. Create a dictionary data structure dependencyMap.

[0090] 3. Traverse the dependency list. Check if the value of the name attribute of the current list object exists in the key of the dictionary data structure. If it does not exist, create a target object, store the value of the dependency attribute of the current list object into the target object; assign the value of the name attribute of the current list to the key of the dictionary data structure; and assign the target object to the value of the dictionary data structure. If it exists, store the value of the dependency attribute of the current list object into the target object corresponding to the value of the dictionary data structure.

[0091] 4. Create a new total mapping structure totalMapping and traverse the dictionary data structure dependencyMap.

[0092] 5. Create a new relationship list, access list, and processing queue.

[0093] 6. Initialize the access list and the processing queue.

[0094] 7. Check if the processing queue is empty. If the processing queue is empty, store the key of the traversed dictionary data structure and the relationship list in the total mapping structure.

[0095] 8. If the processing queue is not empty, dequeue the head element of the processing queue, and retrieve the set of big data component names from the dictionary data structure according to the head element.

[0096] 9. Determine whether the retrieved set of big data component names is non-empty. If the retrieved set of big data component names is non-empty, traverse the retrieved set of big data component names.

[0097] 10. Check whether the element being traversed exists in the access list. If it does not exist, add the element to the relationship list, add the element to the access list, and add the element to the processing queue. If it exists, continue to traverse the retrieved set of big data component names.

[0098] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are displayed in sequence according to the indications of the arrows, these steps do not necessarily need to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily need to be executed at the same moment, but can be executed at different moments. The execution order of these steps or stages does not necessarily need to be sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0099] Based on the same inventive concept, an embodiment of the present application also provides an information processing device for a big data component for implementing the information processing method for a big data component involved above. The implementation solution provided by this device for solving problems is similar to the implementation solution recorded in the above method. Therefore, the specific limitations in one or more embodiments of the information processing device for a big data component provided below can refer to the limitations on the information processing method for a big data component in the above text, and will not be repeated here.

[0100] In an exemplary embodiment, as Figure 4 shown, an information processing device for a big data component is provided, including:

[0101] A reading module 402, configured to read big data component records from a database and store them in a dependency linked list; there are multiple big data component records stored in the database, and each big data component record includes a big data component name and a dependent component name.

[0102] The first traversal module 404 is configured to traverse the dependency linked list and fill the traversed data into a dictionary data structure; the dictionary data structure is used to store the big data component names and the set of big data component names that have a direct association relationship with the big data component names; the direct association relationship is a direct dependency relationship or a direct being-dependent relationship.

[0103] The second traversal module 406 is configured to traverse the dictionary data structure and fill the traversed data into the total mapping structure to obtain a dependency relationship processing result; the total mapping structure is used to store the big data component names and the set of big data component names that have an association relationship with the big data component names; the association relationship is a dependency relationship or a being-dependent relationship; the dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the being-dependent relationship includes a direct being-dependent relationship and an indirect being-dependent relationship.

[0104] In the information processing device for the above big data components, the big data component records are read from the database and stored in the dependency linked list; there are multiple big data component records stored in the database, and each big data component record includes a big data component name and a dependency component name; the dependency linked list is traversed, and the traversed data is filled into the dictionary data structure; the dictionary data structure is used to store the big data component names and the set of big data component names that have a direct association relationship with the big data component names; the direct association relationship is a direct dependency relationship or a direct being-dependent relationship; the dictionary data structure is traversed, and the traversed data is filled into the total mapping structure to obtain a dependency relationship processing result; the total mapping structure is used to store the big data component names and the set of big data component names that have an association relationship with the big data component names; the association relationship is a dependency relationship or a being-dependent relationship; the dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the being-dependent relationship includes a direct being-dependent relationship and an indirect being-dependent relationship. By the above method, a database containing the one-to-one correspondence between the big data components and the dependency components is designed, which is convenient for forward traversal or reverse traversal to construct the dictionary data structure, enabling the reuse of the logic for component dependency relationship calculation and component being-dependent relationship calculation, and greatly reducing the implementation complexity of the algorithm. For newly added components, the user only needs to add the component records in the database without changing the existing component records and operation logic, reducing the upgrade difficulty and enhancing the scalability of the big data components. It can automatically generate big data component combinations based on the database without being fixed according to the scenario, enabling the user to freely select big data component combinations and improving the flexibility of the big data component combinations.

[0105] In an exemplary embodiment, the information processing device of the big data component further includes an instruction processing module, which is configured to receive a component configuration instruction; the component configuration instruction is used to select or cancel a target big data component; according to the component configuration instruction, query the corresponding total mapping structure to determine at least one dependent component related to the target big data component; and perform corresponding configuration processing on the target big data component and the at least one dependent component.

[0106] In an exemplary embodiment, the second traversal module 406 is further configured to establish a relationship list, an access list, and a processing queue; traverse the dictionary data structure, and store the currently traversed big data component name in the access list and the processing queue; obtain the current element from the processing queue; obtain the set of big data component names corresponding to the current element from the dictionary data structure; traverse the obtained set of big data component names; if the traversed associated component name is not in the access list, add the traversed associated component name to the relationship list, the access list, and the processing queue respectively; when it is detected that the processing queue is empty, assign the current big data component name to the current key of the total mapping structure, and assign the relationship list to the current value of the total mapping structure.

[0107] In an exemplary embodiment, the second traversal module 406 is further configured to continue traversing the obtained set of big data component names if the traversed associated component name is in the access list.

[0108] In an exemplary embodiment, the first traversal module 404 is further configured to traverse the dependency linked list, and obtain the target name from the traversed linked list object; detect whether there is a target key in the dictionary data structure that matches the target name; if it exists, store the attribute value of the linked list object in the value corresponding to the target key.

[0109] In an exemplary embodiment, the first traversal module 404 is further configured to, if it does not exist, create a new target object, store the attribute value of the linked list object in the target object; assign the target name of the linked list object to the current key of the dictionary data structure, and assign the target object to the current value of the dictionary data structure.

[0110] Each module in the above information processing device of the big data component can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above respective modules.

[0111] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 5 As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store big data component records. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a method for processing information of big data components.

[0112] Those skilled in the art can understand that Figure 5 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component arrangement.

[0113] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented: reading big data component records from the database and storing them in a dependency linked list; there are multiple big data component records stored in the database, and each big data component record includes a big data component name and a dependent component name; traversing the dependency linked list and filling the traversed data into a dictionary data structure; the dictionary data structure is used to store big data component names and a set of big data component names that have a direct association relationship with the big data component names; the direct association relationship is a direct dependency relationship or a directly dependent relationship; traversing the dictionary data structure and filling the traversed data into a total mapping structure to obtain a dependency relationship processing result; the total mapping structure is used to store big data component names and a set of big data component names that have an association relationship with the big data component names; the association relationship is a dependency relationship or a dependent relationship; the dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the dependent relationship includes a directly dependent relationship and an indirect dependent relationship.

[0114] In one embodiment, when the processor executes the computer program, the following steps are further implemented: receiving a component configuration instruction; the component configuration instruction is used to select or cancel a target big data component; according to the component configuration instruction, querying the corresponding total mapping structure to determine at least one dependent component related to the target big data component; performing corresponding configuration processing on the target big data component and the at least one dependent component.

[0115] In one embodiment, when the processor executes the computer program, the following steps are further implemented: establishing a relationship list, an access list, and a processing queue; traversing the dictionary data structure and storing the currently traversed big data component name in the access list and the processing queue; obtaining the current element from the processing queue; obtaining, from the dictionary data structure, the set of big data component names corresponding to the current element; traversing the obtained set of big data component names; if the traversed associated component name is not in the access list, adding the traversed associated component name to the relationship list, the access list, and the processing queue respectively; when it is detected that the processing queue is empty, assigning the current big data component name to the current key of the total mapping structure and assigning the relationship list to the current value of the total mapping structure.

[0116] In one embodiment, when the processor executes the computer program, the following steps are further implemented: if the traversed associated component name is in the access list, continuing to traverse the obtained set of big data component names.

[0117] In one embodiment, when the processor executes the computer program, the following steps are further implemented: traversing the dependency linked list and obtaining the target name from the traversed linked list object; detecting whether there is a target key in the dictionary data structure that matches the target name; if it exists, storing the attribute value of the linked list object in the value corresponding to the target key.

[0118] In one embodiment, when the processor executes the computer program, the following steps are further implemented: if it does not exist, creating a new target object, storing the attribute value of the linked list object in the target object; assigning the target name of the linked list object to the current key of the dictionary data structure and assigning the target object to the current value of the dictionary data structure.

[0119] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: reading big data component records from a database and storing them in a dependency linked list; multiple big data component records are stored in the database, and each big data component record includes a big data component name and a dependent component name; traversing the dependency linked list and filling the traversed data into a dictionary data structure; the dictionary data structure is used to store the big data component name and a set of big data component names that have a direct association relationship with the big data component name; the direct association relationship is a direct dependency relationship or a directly dependent relationship; traversing the dictionary data structure and filling the traversed data into a total mapping structure to obtain a dependency relationship processing result; the total mapping structure is used to store the big data component name and a set of big data component names that have an association relationship with the big data component name; the association relationship is a dependency relationship or a dependent relationship; the dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the dependent relationship includes a directly dependent relationship and an indirect dependent relationship.

[0120] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: receiving a component configuration instruction; the component configuration instruction is used to select or cancel a target big data component; according to the component configuration instruction, querying the corresponding total mapping structure to determine at least one dependent component related to the target big data component; performing corresponding configuration processing on the target big data component and at least one dependent component.

[0121] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: establishing a relationship list, an access list, and a processing queue; traversing the dictionary data structure and storing the traversed current big data component name in the access list and the processing queue; obtaining the current element from the processing queue; obtaining, from the dictionary data structure, the set of big data component names corresponding to the current element; traversing the obtained set of big data component names; if the traversed associated component name is not in the access list, adding the traversed associated component name to the relationship list, the access list, and the processing queue respectively; when it is detected that the processing queue is empty, assigning the current big data component name to the current key of the total mapping structure and assigning the relationship list to the current value of the total mapping structure.

[0122] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: if the traversed associated component name is in the access list, continuing to traverse the obtained set of big data component names.

[0123] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: traverse the dependency linked list, and obtain the target name from the traversed linked list object; detect whether there is a target key in the dictionary data structure that matches the target name; if it exists, store the attribute value of the linked list object into the value corresponding to the target key.

[0124] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: if it does not exist, create a new target object, and store the attribute value of the linked list object into the target object; assign the target name of the linked list object to the current key of the dictionary data structure, and assign the target object to the current value of the dictionary data structure.

[0125] In one embodiment, a computer program product is provided, including a computer program, which when executed by a processor, implements the following steps: read big data component records from a database and store them in a dependency linked list; there are multiple big data component records stored in the database, and each big data component record includes a big data component name and a dependent component name; traverse the dependency linked list and fill the traversed data into a dictionary data structure; the dictionary data structure is used to store the big data component name and a set of big data component names that have a direct association relationship with the big data component name; the direct association relationship is a direct dependency relationship or a directly dependent relationship; traverse the dictionary data structure and fill the traversed data into a total mapping structure to obtain a dependency relationship processing result; the total mapping structure is used to store the big data component name and a set of big data component names that have an association relationship with the big data component name; the association relationship is a dependency relationship or a dependent relationship; the dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the dependent relationship includes a directly dependent relationship and an indirect dependent relationship.

[0126] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: receive a component configuration instruction; the component configuration instruction is used to select or cancel a target big data component; according to the component configuration instruction, query the corresponding total mapping structure to determine at least one dependent component related to the target big data component; perform corresponding configuration processing on the target big data component and at least one dependent component.

[0127] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: establishing a relationship list, an access list, and a processing queue; traversing the dictionary data structure, and storing the currently traversed big data component name in the access list and the processing queue; obtaining the current element from the processing queue; obtaining, from the dictionary data structure, the set of big data component names corresponding to the current element; traversing the obtained set of big data component names; if the traversed associated component name is not in the access list, adding the traversed associated component name to the relationship list, the access list, and the processing queue respectively; when it is detected that the processing queue is empty, assigning the current big data component name to the current key of the total mapping structure, and assigning the relationship list to the current value of the total mapping structure.

[0128] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: if the traversed associated component name is in the access list, continue traversing the obtained set of big data component names.

[0129] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: traversing the dependency linked list, and obtaining the target name from the traversed linked list object; detecting whether there is a target key in the dictionary data structure that matches the target name; if it exists, storing the attribute value of the linked list object in the value corresponding to the target key.

[0130] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: if it does not exist, creating a new target object, and storing the attribute value of the linked list object in the target object; assigning the target name of the linked list object to the current key of the dictionary data structure, and assigning the target object to the current value of the dictionary data structure.

[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0132] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, the memories, databases, or other media mentioned in the various embodiments provided in the present application can all include at least one of non-volatile memories and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the various embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the various embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0133] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in the present application.

[0134] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.< / string> < / string> < / string> < / string> < / string> < / string> < / string> < / componentdependency>

Claims

1. A method for processing information of a big data component, characterized in that: The method comprises: Reading a big data component record from a database and storing it in a dependency list; the database stores a plurality of big data component records, each of which includes a big data component name and a dependent component name; Traversing the dependency linked list, and filling the traversed data into a dictionary data structure; the dictionary data structure is used to store the big data component name, and a set of big data component names that have a direct association relationship with the big data component name; the direct association relationship is a direct dependency relationship or a direct dependency relationship; Traverse the dictionary data structure, fill the traversed data into the total mapping structure, and obtain the dependency processing result; the total mapping structure is used to store the big data component name, and a set of big data component names that have an association relationship with the big data component name; the association relationship is a dependency relationship or a dependent relationship; the dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the dependent relationship includes a direct dependent relationship and an indirect dependent relationship.

2. The method according to claim 1, characterized in that The method further comprises: Receive a component configuration instruction; the component configuration instruction is used to select or cancel a target big data component; According to the component configuration instruction, query the corresponding overall mapping structure to determine at least one dependent component related to the target big data component; The target big data component and the at least one dependent component are configured accordingly.

3. The method according to claim 1, characterized in that The traversing the dictionary data structure and filling the traversed data into the total mapping structure includes: Establish relationship lists, access lists, and processing queues; Traversing the dictionary data structure, and storing the traversed current big data component name into the access list and the processing queue; Get the current element from the processing queue; Obtaining a set of large data component names corresponding to the current element from the dictionary data structure; Traverse the acquired big data component name set; If the traversed associated component name is not in the access list, then the traversed associated component name is added to the relationship list, the access list and the processing queue respectively; When it is detected that the processing queue is empty, the current big data component name is assigned to the current key of the overall mapping structure, and the relationship list is assigned to the current value of the overall mapping structure.

4. The method according to claim 3, characterized in that: After traversing the acquired big data component name set, the method further includes: If the traversed associated component name is in the access list, continue to traverse the acquired large data component name set.

5. The method according to any one of claims 1 to 4, characterized in that: The traversing the dependency linked list and filling the traversed data into the dictionary data structure includes: Traversing the dependency linked list, and obtaining the target name from the traversed linked list object; detecting whether a target key matching the target name exists in the dictionary data structure; If it exists, the attribute value of the linked list object is stored in the value corresponding to the target key.

6. The method according to claim 5, characterized in that The method further comprises: If it does not exist, create a new target object and store the attribute value of the linked list object into the target object; The target name of the linked list object is assigned to the current key of the dictionary data structure, and the target object is assigned to the current value of the dictionary data structure.

7. An information processing device for a big data component, characterized in that: The device comprises: A reading module is used to read a big data component record from a database and store it in a dependency list; the database stores a plurality of big data component records, each of which includes a big data component name and a dependent component name; A first traversal module is used to traverse the dependency list and fill the traversed data into a dictionary data structure; the dictionary data structure is used to store the big data component name and a set of big data component names that have a direct association relationship with the big data component name; the direct association relationship is a direct dependency relationship or a direct dependency relationship; The second traversal module is used to traverse the dictionary data structure, fill the traversed data into the total mapping structure, and obtain the dependency processing result; the total mapping structure is used to store the big data component name, and the set of big data component names that have an association relationship with the big data component name; the association relationship is a dependency relationship or a dependent relationship; the dependency relationship includes a direct dependency relationship and an indirect dependency relationship, and the dependent relationship includes a direct dependent relationship and an indirect dependent relationship.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images