Data processing method and device

Abstract

The invention discloses a data processing method and device, and relates to the technical field of computers and data processing. A specific embodiment of the method comprises the following steps: in response to triggering of a task for changing hierarchies of a tree structure, obtaining a mapping relationship between a target change hierarchy and an adjacent hierarchy in the tree structure and analyzing the mapping relationship; the metadata table of the tree structure is maintained and updated through metadata reflecting the incidence relation between the target change hierarchy and the previous hierarchy, the metadata table is pre-configured for the tree structure and stored in a system, and the data table of the tree structure is updated based on a description rule between the metadata and data. And obtaining an updated data table corresponding to the changed tree structure. According to the embodiment, flexible maintenance and analysis of the organizations can be realized, various changes and application requirements are supported, and dynamic response and flexible adaptation to changes of the organizations are realized.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology and data processing, and in particular, to a method and apparatus for data processing. Background Art

[0002] For a tree structure with a hierarchical mapping relationship, such as an organizational structure, making hierarchical changes (such as adding a level, reducing a level, replacing a level, etc.) is a common way of organizational management. By actively updating the organizational structure, it helps to strengthen refined management, consolidate the internal driving force for enterprise development, and further build an efficient and collaborative management team. However, this is followed by an impact on the stability of the enterprise's existing systems and huge transformation costs (human resource inputs such as R & D, testing, and operation and maintenance).

[0003] The current technical solutions mainly fall into three categories: field reuse, reusing the existing organizational structure fields in the system and replacing their contents with new organizational structure values; adding fields, adding fields for all applications and modules in the system that involve organizational structure management; adding mappings, creating a new mapping relationship table to store the mapping relationship between the newly added organizational structure and the atomic organizational structure (the smallest management unit, a physical existence rather than a virtual organizational structure).

[0004] However, each of these three methods has its own disadvantages. For field reuse, the universality is poor; for adding fields, the cost is very high; for adding mappings, the performance is very poor. The common disadvantage of the three methods is that they cannot avoid repeated development, and each time the organizational structure changes, continuous investment in development, testing, and operation and maintenance resources is required. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for data processing, which can achieve flexible maintenance and parsing of a tree structure, support various changes and application requirements, realize dynamic response and flexible adaptation to hierarchical changes of the tree structure, improve the stability and maintainability of the entire system, and are also beneficial to maintaining the transparency and compatibility of the application, facilitating subsequent expansion and maintenance, so that the system can more easily adapt to future changes.

[0006] To achieve the above object, according to one aspect of the embodiments of the present invention, a method for data processing is provided for processing hierarchical changes of a tree structure, including:

[0007] In response to the triggering of a task to change the level of the tree structure, obtain and parse the mapping relationship between the target changed level and adjacent levels in the tree structure,

[0008] Based on the obtained results, maintain and update the metadata table of the tree structure with metadata reflecting the association relationship between the target change level and the upper level, where the metadata table is pre-configured for the tree structure and stored in the system, and

[0009] Based on the description rules between metadata and data, update the data table of the tree structure to obtain an updated data table corresponding to the changed tree structure.

[0010] According to the data processing method with the above composition, in response to the need to change the levels of the tree structure, the change information can be converted into the form of metadata for parsing to obtain information such as the hierarchical relationship of the tree structure. Each time a query, display, or other application of the tree structure is required, according to the parsed hierarchical relationship and other information, requests such as queries and displays are converted into operations on the data table, facilitating subsequent application expansion and adjustment. Through the tree structure level change dynamic adaptation device with the hierarchical relationship of the tree structure as metadata, dynamic response and flexible adaptation to the level change of the tree structure can be achieved, improving the stability and maintainability of the entire system.

[0011] Optionally, in the data processing method of the first aspect of this embodiment, it further includes: pre-transforming the ID of the tree structure to make it have a tree structure composite ID, where the tree structure composite ID includes prefix retrieval, the field length of the tree structure composite ID is at least the length from the highest level to the second lowest level of the tree structure, and the field of the tree structure composite ID is constructed from any field in the data table, and the updated data table includes the update of the tree structure composite ID.

[0012] According to the data processing method with the above composition, by designing a tree structure composite ID with prefix retrieval, a flexible and efficient retrieval method is provided, which can be widely applied to various types of database systems to achieve compatibility. Whether it is a relational or non-relational database, whether it is row storage or column storage, the characteristics of the index can be fully utilized to improve the query efficiency. Moreover, when making the first transformation, any tree structure field of the current data table can be optionally selected as the subsequent composite ID field under the condition that the field length is sufficient, and then the values of this field in the data table can be updated in full. Moreover, after the transformation, it has the ability of configuration and does not require re-development, which can greatly improve the maintainability and scalability of the system.

[0013] Optionally, in the data processing method of the first aspect of this embodiment, it further includes: pre - establishing a tree - structured domain model, where the tree - structured domain model includes business elements and technical elements. The business elements are the information of each node of the tree structure, including the hierarchical identifier and the actual value of each node. The technical elements are the positioning of each node in the tree structure, including the upper - level hierarchical identifier of the upper level of each node and the hierarchical value of the level where it is located.

[0014] According to the data processing method with the above composition, an organizational structure domain model involving business elements and technical elements is pre - configured. The business elements can reflect the specific tree structure, including the identifier and actual value of each node of the tree structure, through which business operators can understand the name, location, and other relevant detailed information of each node at each level. The technical elements can include the positioning of the nodes at each level of the tree structure in the tree structure, that is, the level where they are located and the upper level, and this information can help the system understand and process the hierarchical changes of the tree structure. Therefore, the tree - structured domain model with these design elements can not only help business operators better understand and manage the tree structure, but also improve the flexibility and scalability of the system, making the system more easily adaptable to future changes.

[0015] Optionally, in the data processing method of the first aspect of this embodiment, updating the data table of the tree structure based on the description rule between metadata and data to obtain an updated data table corresponding to the changed tree structure includes: updating the data table in the cache area and the database respectively, and the update in the cache area and the update in the database are asynchronous.

[0016] According to the data processing method with the above composition, by combining cache update and database update, the performance of the solution of the present invention can be improved. The cache has a small scale and can be updated quickly. Moreover, when reading data fails due to a dirty cache, the cache can be actively updated and then retried, and it is possible to distinguish whether the read failure is caused by a dirty cache. Specifically, during the process of the tree structure level change taking effect, the new level information can be first stored in the cache, and then the database can be updated. After the database update is completed, the information in the cache is updated to ensure the consistency of the data in the cache and the database. In this way, even when the data in the cache and the database is inconsistent, the effectiveness and correctness of the tree structure level change can be ensured. Because before the tree structure level change takes effect, all operations are performed in the cache, and the data synchronization between the cache and the database is performed after the database update is completed, which can avoid the situation of data inconsistency. In addition, to further improve the performance, an asynchronous cache update method can be considered. That is, when the database is updated, the cache can be updated asynchronously, which can avoid the blocking of the cache update on the database update, further improve the performance of the system, and ensure the effectiveness and correctness of the tree structure level change.

[0017] Optionally, in the data processing method of the first aspect of this embodiment, the update performed asynchronously includes: performing a full parse on the metadata table to generate a first cache and a second cache. The first cache has a key-value pair with the atomic organization ID as the key and the tree structure composite ID as the value. The second cache has two key-value pairs. One of the two key-value pairs has the level identifier as the key and the level value as the value, and the other of the two key-value pairs has the composite value of the level value and the actual value as the key and the prefix retrieval of the tree structure composite ID as the value, where the atomic organization is the lowest level of the organization tree structure;

[0018] Perform a full update of the data table according to the first cache, complete the initialization update of the information table, update the data table in the database to obtain an updated data table, and perform a full update of the data table according to the second cache for parameter conversion during application.

[0019] According to the data processing method with the above composition, the cache is designed to have a first cache and a second cache. When initializing and updating the data table, the system can use the first cache to quickly find the corresponding key-value pair in the first cache through the atomic organization ID, obtain the tree structure composite ID, and then use this tree structure composite ID to perform the update operation on the data table. Since the access speed of the cache is much faster than that of the database, this design can significantly improve the performance and efficiency of the system. Additionally, the first key-value pair in the second cache is the correspondence between the hierarchical identifier of the tree structure and the hierarchical value, which is used to represent the hierarchy corresponding to different hierarchical identifiers. When the system receives an input parameter, it can quickly find the hierarchy corresponding to this parameter through this key-value pair, and then perform corresponding operations. The second key-value pair in the second cache is the correspondence between the hierarchical value and the retrieval prefix of the tree structure composite ID of the organizational structure, which is used to represent the retrieval prefix of the tree structure composite ID corresponding to different organizational structure levels. When the system needs to query information about a specific node, it can quickly find the retrieval prefix of the tree structure composite ID corresponding to this node through this key-value pair, and then perform corresponding query operations. This design can ensure that when there are duplicate values in the tree structure ID under different hierarchical identifiers, the system can still correctly identify and process these duplicate values.

[0020] Optionally, in the data processing method of the first aspect of this embodiment, the asynchronous update includes: updating the data table of the tree structure in the database and updating the cache area, and the update of the cache area is configured to: perform a full parsing of the meta data table to generate a third cache, and the third cache uses key-value pairs with the actual value as the key and the prefix retrieval of the tree structure composite ID as the value to perform a full update of the data table.

[0021] According to the data processing method with the above composition, instead of the design of the two caches, the first cache and the second cache, a single third cache can be adopted. Thus, the storage space and management overhead of the cache are reduced, and the performance and efficiency of the system are further improved.

[0022] Optionally, in the data processing method of the first aspect of this embodiment, it further includes: in response to the input parameter of the tree structure field, replacing all the original fields according to the tree structure domain model, performing input parameter conversion in the second cache, obtaining the tree structure composite ID according to the tree structure information of the input parameter, and reading the updated data table in the database according to the obtained tree structure composite ID.

[0023] According to the method of data processing with the above composition, based on the tree - structure domain model, relevant attributes and data of the tree structure are encapsulated. When receiving the input parameter of the tree - structure field, the system generates a corresponding tree - structure domain model instance according to the business - element information, and parses and fills it. Then, the system uses the technical elements (such as the tree - structure synthesis ID) in the tree - structure domain model instance as the query condition to read the relevant update data table from the database and perform corresponding operations (such as updating the tree structure). By separating the business elements and technical elements, it is possible to better focus on different aspects of business logic and technical implementation, improve the readability and maintainability of the code. At the same time, it also makes the system more flexible and extensible, and can easily handle the demand changes in different scenarios.

[0024] The second aspect of the embodiments of the present invention provides a data - processing device, including:

[0025] An information - acquisition unit: in response to the triggering of a task to change the level of the tree structure, acquires and parses the mapping relationship between the target change level and the adjacent levels in the tree structure.

[0026] An information - processing unit: based on the acquired result, maintains and updates the metadata table of the tree structure with the metadata reflecting the association relationship between the target change level and the upper level, where the metadata table is pre - configured for the tree structure and stored in the system, and

[0027] An information - change unit: based on the description rule between metadata and data, updates the data table of the tree structure to obtain an updated data table corresponding to the changed tree structure.

[0028] The third aspect of the embodiments of the present invention provides an electronic device for data processing, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the method described in the first aspect.

[0029] The third aspect of the embodiments of the present invention provides a computer - readable medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method described in the first aspect.

[0030] The above - mentioned second to third aspects can achieve the same various technical effects as the first aspect.

[0031] Generally speaking, one embodiment of the above invention has the following advantages or beneficial effects: in response to the triggering of a task to change the level of the tree structure, obtain the mapping relationship between the target change level and the adjacent levels in the tree structure and parse it. Based on the obtained result, maintain and update the metadata table of the tree structure with metadata reflecting the association relationship between the target change level and the upper level, where the metadata table is pre-configured for the tree structure and stored in the system, and update the data table of the tree structure based on the description rule between metadata and data to obtain an updated data table corresponding to the changed tree structure. By means of this technology, technical problems such as high cost and poor performance are overcome, and thus the flexible maintenance and parsing of the tree structure can be achieved, supporting various changes and application requirements, realizing dynamic response and flexible adaptation to level changes, improving the stability and maintainability of the entire system, and being beneficial to maintaining the transparency and compatibility of the application, facilitating subsequent expansion and maintenance, and enabling the system to more easily adapt to future changes.

[0032] The further effects of the above non-conventional optional methods will be described below in combination with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings are used to better understand the present invention and do not constitute an improper limitation to the present invention. Among them:

[0034] Figure 1 is a schematic diagram of the main process of the data processing method according to an embodiment of the present invention;

[0035] Figure 2 is a schematic diagram of an example of the data processing method according to an embodiment of the present invention;

[0036] Figure 3 is a schematic diagram of the design of the organization structure composite ID according to an example of the data processing method according to an embodiment of the present invention;

[0037] Figure 4 is a schematic diagram of the design of the organization structure domain model object abstraction and metadata storage according to an example of the data processing method according to an embodiment of the present invention;

[0038] Figure 5 is a schematic diagram of the cache design according to an example of the data processing method according to an embodiment of the present invention;

[0039] Figure 6 is a schematic diagram of the dynamic adaptation process of the organization structure change according to an example of the data processing method according to an embodiment of the present invention;

[0040] Figure 7A schematic diagram of data reading, which is an example of a data processing method according to an embodiment of the present invention;

[0041] Figure 8 A schematic diagram of the main units of a data processing apparatus according to an embodiment of the present invention;

[0042] Figure 9 An exemplary system architecture diagram to which an embodiment of the present invention can be applied;

[0043] Figure 10 A schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. Detailed implementation manners

[0044] The following describes exemplary embodiments of the present invention with reference to the accompanying drawings. Various details of the embodiments of the present invention are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following.

[0045] Figure 1 A schematic diagram of the main process of a data processing method for processing hierarchical changes of a tree structure according to an embodiment of the present invention. In the embodiment, the tree structure is described by taking an organizational structure as an example. However, those skilled in the art can understand that the tree structure is not limited to the organizational structure, but is applicable to various scenarios of tree structures with hierarchical mapping relationships.

[0046] Organizational structure management usually divides an organization into different departments or units through the hierarchical relationship between superiors and subordinates, and completes work tasks through the coordination and communication between these departments. There are relatively fixed atomic organizations, which are usually mapped to real physical entity organizations, such as logistics network points (business departments, sorting centers, warehouses, etc.); the atomic organizations only have value increases and decreases, the hierarchy is fixed and unchanged and is the smallest hierarchy. There are physically virtual upper-level organizations above the atomic organizations. The upper-level organizations are the carriers of organizational structure changes and will change periodically (newly added, reduced, replaced, etc.). The tree structure of the organizational structure determines that there is an inevitable mapping relationship between adjacent hierarchies, and the upper-level organization: lower-level organization is 1:*, that is, a one-to-many relationship.

[0047] Step S101, in response to the triggering of a task to change the hierarchy of the organizational structure, obtain the mapping relationship between the target change hierarchy and adjacent hierarchies in the tree structure and perform parsing.

[0048] In this step, when the system receives the task of changing the organizational structure, it obtains and parses the mapping relationship between the target change level and the adjacent levels in the organizational structure, which helps the system respond correctly and effectively to the task of changing the organizational structure, maintain the accuracy, integrity and reliability of the data, and improve the performance, reliability and scalability of the system.

[0049] Before step S101, preprocessing can be carried out in the system. For example, the ID of the organizational structure is pre-transformed to have a synthesized ID of the organizational structure. The ID of the organizational structure includes prefix retrieval. The field length of the synthesized ID field of the organizational structure is at least the length from the highest level to the second lowest level of the organizational structure, and the synthesized ID field of the organizational structure is constructed by selecting several organizational structure ID fields in the data table under the field length. And the updated data table includes the update of the synthesized ID of the organizational structure, which will be illustrated with specific examples in the following instances. For example, an organizational structure domain model can also be established in advance. The organizational structure domain model includes business elements and technical elements. The business elements are specific organizational structures, including identifiers and actual values. The technical elements are the positions of the organizational structures in the tree structure, including the levels where they are located and the upper-level organizations, which will be illustrated with specific examples in the following instances.

[0050] By using the principle of the index and the characteristics of the 1:n mapping relationship between the upper and lower levels of the tree structure through prefix retrieval, a synthesized ID of the organizational structure with prefix retrieval is designed, which provides a flexible and efficient retrieval method and can be widely applied to various types of database systems, achieving compatibility. Whether it is a relational or non-relational database, whether it is row storage or column storage, the characteristics of the index can be fully utilized to improve the query efficiency. For example, when the system database uses a relational database (such as MySQL), the index is usually a B-tree or its variant, supporting the characteristic of prefix retrieval to use the index. If the database uses a non-relational database, there are two common situations: (1) Row storage database: When the index of the row storage database is a B-tree or a B-tree variant, the above-mentioned characteristic of prefix retrieval to use the index is supported. (2) Column storage database: The column storage database supports dynamic addition of fields and naturally supports dynamic adaptation, so it is also applicable to the column storage database. Moreover, during the first transformation, under the condition of sufficient field length, a certain organizational structure field in the current data table can be arbitrarily selected as the subsequent synthesized ID field of the organizational structure, and then the value of this field in the data table can be updated in full. And after the transformation, it has the configuration ability and does not require re-development, which can greatly improve the maintainability and scalability of the system. Through the synthesized ID of the organizational structure with prefix retrieval transformed, the compatibility of the existing data table with the newly added organizational structure is realized.

[0051] By pre-configuring an organizational structure domain model involving business elements and technical elements. Business elements can refer to specific organizational structures, including their identifiers and actual values, through which business operators can understand the names, locations, and other relevant details of each organizational structure. Technical elements can include the positioning of organizational structures in a tree structure, that is, their levels and upper-level organizations, and this information can help the system understand and handle changes in organizational structures. Therefore, the organizational structure domain model with these design elements can not only help business operators better understand and manage organizational structures, but also improve the flexibility and scalability of the system, enabling the system to more easily adapt to future changes.

[0052] Step S102, based on the result obtained in step S101, maintain and update the metadata table of the organizational structure with metadata reflecting the association relationship between the target change level and the upper level, where the metadata table is pre-configured for the organizational structure and stored in the system.

[0053] For example, in the tree structure of an organizational structure, it has metadata and data at the technical level and business level. Metadata is data about data, a kind of technical data, transparent to business personnel (system users), while data is business data, and it is the business data. There are various relationships between metadata and data, such as description and being described, dependence, complementarity, and hierarchy. Thus, data can be better understood, organized, and used through metadata. According to this embodiment, for an organizational structure, metadata reflects the association relationship between the levels of the organizational structure, and the underlying metadata is extracted. Therefore, after obtaining the parsing result of the previous step, the changes in the organizational structure can be reflected in the basic metadata, and through the maintenance and update of this metadata, the changes can be propagated to the data table.

[0054] Step S103, based on the description rules between metadata and data, update the data table of the organizational structure to obtain an updated data table corresponding to the changed organizational structure.

[0055] According to the data processing method with the above composition, it can respond to the need to change the organizational structure, convert the change information into the form of metadata for parsing, and obtain information such as the hierarchical relationship of the organizational structure. Each time a query, display, or other application of the organizational structure is required, according to the parsed hierarchical relationship and other information, the query, display, and other requests are converted into operations on the data table, facilitating subsequent application expansion and adjustment. Through the organizational structure change dynamic adaptation device with the hierarchical relationship of the organizational structure as metadata, the dynamic response and flexible adaptation to organizational structure changes can be achieved, improving the stability and maintainability of the entire system.

[0056] The update in step S103 may include: updating the data table in the cache area and the database respectively, and the update in the cache area and the update in the database are performed asynchronously.

[0057] According to the data processing method with the above composition, by using the combination of cache update and database update, the performance of the solution of the present invention can be improved. The cache has a small scale and can be updated quickly. Moreover, when reading data fails due to a dirty cache, the cache can be actively updated and then retried to distinguish whether the read failure is caused by a dirty cache. Specifically, during the process of the organizational structure change taking effect, the new organizational structure information can be first stored in the cache, and then the database is updated. After the database update is completed, the information in the cache is updated to ensure the consistency of the data in the cache and the database. In this way, even when the data in the cache and the database is inconsistent, the effectiveness and correctness of the organizational structure change can be guaranteed. Because before the organizational structure change takes effect, all operations are performed in the cache, and the data synchronization between the cache and the database is performed after the database update is completed, which can avoid the situation of data inconsistency. In addition, to further improve the performance, an asynchronous cache update method can also be considered. That is, when the database is updated, the cache can be updated asynchronously, which can avoid the blocking of the cache update to the database update, further improve the system performance, and ensure the effectiveness and correctness of the organizational structure change.

[0058] After step S103 is completed, the maintenance of the organizational structure change has been completed. Thereafter, the parsing and application of the organizational structure can also be performed. In response to the organizational structure field input parameter, all the original fields are replaced according to the organizational structure domain model, the input parameter conversion is performed in the second cache, the organizational structure composite ID is obtained according to the input parameter organizational structure information, and the updated data table in the database is read according to the obtained organizational structure composite ID.

[0059] According to the organizational structure domain model, the relevant attributes and data of the organizational structure are encapsulated. When the organizational structure field input parameter is received, the system will generate a corresponding organizational structure domain model instance according to the business element information and parse and populate it. Then, the system will use the technical elements (such as the organizational structure composite ID) in the organizational structure domain model instance as the query condition to read the relevant updated data table from the database and perform corresponding operations (such as updating the organizational structure tree structure). By separating the business elements and technical elements, it is possible to better focus on different aspects of business logic and technical implementation, improve the readability and maintainability of the code. At the same time, it also makes the system more flexible and extensible, and can easily cope with the demand changes in different scenarios.

[0060] Next, reference will be made to Figures 2 - 6Describe an example of the data processing method according to an embodiment of the present invention. In this example, taking the change of the logistics organization structure from the hierarchy of region - large region - province - city - network point to adding provincial management between the large region and the province as an example, the method of the above - mentioned embodiment is described.

[0061] First, refer to Figure 3 , and describe the transformation of the ID of the organization structure in this example. During the first transformation, when the field length is sufficient, any organization structure field in the current data table can be optionally selected as the subsequent organization structure composite ID field (composite_org_id), and then the values of this field in the data table can be updated in full. In Figure 3 's example, as shown in the left - hand part, the ID of the organization structure has been pre - transformed into an organization structure composite ID (composite_org_id) with a field length covering from region L1 to city L4. And by combining the letters L1~L4 at different levels, the composite_org_id can uniquely identify each node in the organization structure. For the ID "L1 - L2 - L3 - L4", it can uniquely identify the entire organization structure tree composed of L1, L2, L3, and L4. At this time, when there is an organizational structure change and it is necessary to add provincial management between the large region and the province, then retrieve the ID "composite_org_id" and dynamically adjust its fields. As Figure 3 shown on the right - hand side, it is changed to composite_org_id: L1 - L2 - L - L3 - L4, where the letter L represents a newly added level corresponding to the province. Through the transformation of the ID, the ID of the organization structure is an organization structure composite ID with prefix retrieval. And as described above, by setting the prefix as needed, it is possible to quickly locate the organization structure nodes with the same prefix, improve the retrieval efficiency, and solve the compatibility of the data table. Moreover, through this configuration, it is possible to achieve permanent benefits from one transformation, that is, after the transformation, it has the configuration ability and no need for re - development. For example, when it is necessary to reduce the levels, only need to configure the organization structure composite ID as composite_org_id: L1 - L2 - L4, which means a change in reducing the L3 level; when it is necessary to replace a level, only need to configure the organization structure composite ID as composite_org_id: L1 - L - L3 - L4, which means a change in replacing the L2 level with L, without re - development. Here, the organization structure composite ID field "composite_org_id" is not restrictive, and other fields can be customized.

[0062] Secondly, refer to Figure 4, describe the abstraction of the OrgEntity object in the organizational structure domain model and the design of metadata storage in this instance. As shown in the figure, the design of the organizational structure domain model OrgEntity reflects two aspects: business elements (specific organizational structures, including identifiers and actual values) and technical elements (the positioning of the organizational structure in the tree structure, including the level where it is located and the upper-level organization). Here, it is necessary to ensure that it is the upper-level organization. The metadata design is only for the needs of technical design, and the business application is unaware; it also reflects the association relationship between the current organization and the superior organization. In Figure 4 's specific instance, orgType and orgid are business elements. orgType is the organizational structure identifier (also known as the level identifier) of a specific organizational structure, such as "large region", and orgid is the actual value of the specific organizational structure, such as "Southwest". Through the business elements including orgType and orgid, a specific organizational structure, that is, a node in the tree structure, can be reflected. parentOrgid and level are technical elements. parentOrgid is the organizational structure identifier of the upper level of this organizational structure. For example, when orgType is "large region", parentOrgid is "region", and level is the level value of the current level of this organizational structure. 1 is the highest level, incrementing by 1 layer by layer. In this example, level is the level of the large region, so the level value of level is 2. Figure 4 On the right side in

[0063] Secondly, refer to Figure 5 , describe the cache design in this instance. In this instance, the cache design includes two caches, namely Cache 1 and Cache 2.

[0064] As Figure 5 shown, Cache 1 has key-value pairs of [atomic organization ID: organizational structure composite ID], which are used for the initialization and update of all data tables in the system. See Figure 6 's 2.1. Cache 2 has two key-value pairs of [organizational structure identifier: organizational structure level value][organizational structure level value - specific organizational structure value: prefix of organizational structure composite ID], which are used for parameter conversion during the system operation. Note that the prefix of the organizational structure composite ID ends with the specific organizational structure value of the key. The design of the two caches is to ensure the scenario where there are duplicate values for the specific organizational structure values under different organizational structure types. If it can be ensured that the actual values of the organizational structures are globally unique in actual production, the design can be simplified, and a single cache of [actual organizational structure value: prefix of organizational structure composite ID] can also be used to replace these two caches.

[0065] It should be noted that, to improve the performance of the solution of the present invention, a cache is used. Only after both the database update and the cache update are completed does it represent that the organizational structure change takes effect (is available). However, there must be a time difference between the two. Therefore, it is necessary to update the data table first and then the cache, and add failure retry at the application layer. There are two considerations for such a design: the cache size is small and the update is fast; when reading data fails due to a dirty cache, the cache can be actively updated and then retried to distinguish whether the read failure is caused by a dirty cache.

[0066] Although the cache design will definitely introduce additional resource consumption, the cache size of the present invention is not large, so the resource loss is low. The cache size is strongly correlated with the levels of the organizational structure and the number of each level. For example, if the levels and numbers of the organizational structure are as follows [L1: 10, L2: 100, L3: 1000, L4: 10000], then the size of cache 1 is the size of the L4 level, which is 10000, and the size of cache 2 is L1 + L2 + L3 + L4 = 11100. In fact, the levels of the organizational structure of an enterprise and the number of each level are relatively small numbers, so the cache size is small and the resource consumption is low.

[0067] Reference Figure 6 , shows the schematic diagram of the dynamic adaptation process of the organizational structure change of this example. For 1.1 to 2.2 therein, reference can be made to Figure 2 shown. As Figure 6 shown, the operator manually maintains the organizational structure tree structure and makes changes such as adding, reducing, and replacing the levels of the original organizational structure (in this example, adding a "large region"). After the system receives the information of this maintenance, it uses the designed organizational structure domain model based on the metadata for parsing and calculation, and can obtain the updated fact table (data table). Then, the updated fact table is used to update cache 1 and cache 2 to the organizational structure cache data, and the organizational structure composite ID is updated. Thus, during application, the organizational structure composite ID can be obtained through the input organizational structure information in cache 2, and data can be obtained according to the organizational structure composite ID. Figure 6 Steps 1.1 to 2.2 of

[0068] In this example, regarding the above-mentioned organizational structure change maintenance module, as Figure 6As shown, business operators can configure the mapping relationship between large regions and provincial regions and the mapping relationship between provincial regions and provinces through the configuration tools provided by the system (such as the front-end page). The system background parses the configuration and executes the writing of provincial region metadata and the update of province metadata. After both are executed, the new organizational structure tree structure is persistently stored. By performing a full parse of the metadata table, cache 1 can be generated; based on cache 1, the full update of the system data table can be completed through SQL scripts. By performing a full parse of the metadata table, cache 2 can be fully updated.

[0069] In this instance, regarding the above-mentioned organizational structure parsing application module, such as Figure 6 As shown, there are two changes in the application layer: (1) The change of the input parameters for the organizational structure fields. All previous specific field designs are changed to the replacement of the organizational structure domain model, that is, OrgEntity. If there are multiple organizational input parameters, they must be replaced with List <orgentity>, rather than multiple OrgEntities; (2) A proxy layer is newly added to perform organization structure parsing and processing (including cache reading / writing and database reading / writing).

[0070] Figure 7 The figure shows a schematic diagram of data reading, taking data access as an example to show how the system operates. As Figure 7 shown, it can be seen from the above example the differences between the business elements and technical elements abstracted by OrgEntity. The business elements are inputs, and the technical elements are automatically parsed and filled during the operation. For example, as Figure 7 shown, for the business elements, the organization structure identifier orgType inputs the province or region, and the specific value of the organization structure orgid inputs Guangdong and Guangxi. Then, according to the shown organization structure identifier, the hierarchy level, i.e., Guangdong and Guangxi, can be read, and cache 1 is updated. Additionally, according to the automatically filled organization structure hierarchy value level and the specific value of the organization structure, the prefix of the combined ID of the organization structure is read, and cache 2 is updated. According to the prefix of the combined ID of the organization structure, like 'prefix%', data is read. Thus, the LIKE operator can be used to match the combined ID of the organization structure starting with a specific prefix, and the organization structure data starting with a specific prefix can be quickly found during the query, so as to conveniently read and filter the data that meets the conditions. Note: When there is a List <orgentity>In input, theoretically, there should only be organization structure instances at the same level - the application scenario of multi-instance access; otherwise, it is necessary to filter and obtain the organization structure instance with the lowest level - the application scenario of single-instance access to multiple levels of input (priority relationship judgment is required).

[0071] Figure 8 It is a schematic diagram of the main units of a data processing device 800 for processing hierarchical changes of a tree structure according to an embodiment of the present invention. As Figure 8 shown, the data processing device 800 includes: an information acquisition unit 801, which, in response to the triggering of a task to change the hierarchy of the tree structure, acquires and parses the mapping relationship between the target change level and the adjacent levels in the tree structure; an information processing unit 802, which, based on the acquired result, maintains and updates the metadata table of the tree structure with metadata reflecting the association relationship between the target change level and the upper level, wherein the metadata table is pre-configured for the tree structure and stored in the system, and an information change unit 803, which updates the data table of the tree structure based on the description rule between the metadata and the data, to obtain an updated data table corresponding to the changed tree structure.

[0072] Figure 9 It shows an exemplary system architecture 900 to which the data processing method or data processing device according to the embodiment of the present invention can be applied.

[0073] As Figure 9 shown, the system architecture 900 may include terminal devices 901, 902, 903, a network 904, and a server 905 (this architecture is only an example, and the components included in the specific architecture can be adjusted according to the specific situation of the application). The network 904 is used to provide a medium for communication links between the terminal devices 901, 902, 903 and the server 905. The network 904 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0074] Users can use the terminal devices 901, 902, 903 to interact with the server 905 through the network 904 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 901, 902, 903, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0075] The terminal devices 901, 902, 903 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0076] The server 905 may be a server that provides various services. For example, it may be a back-end management server (merely an example) that supports shopping websites browsed by users using the terminal devices 901, 902, and 903. The back-end management server may analyze and process data such as product information query requests received, and feedback the processing results (such as target push information, product information - merely examples) to the terminal devices.

[0077] It should be noted that the data processing method provided by the embodiments of the present invention is generally executed by the server 905. Correspondingly, the data processing device is generally disposed in the server 905.

[0078] It should be understood that Figure 9 the numbers of the terminal devices, networks, and servers in

[0079] are merely illustrative. According to the implementation requirements, there may be any number of terminal devices, networks, and servers. Figure 10 Reference is now made to Figure 10 which shows a schematic structural diagram of a computer system 1000 of a terminal device suitable for implementing the embodiments of the present invention.

[0080] As Figure 10 shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage section 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the system 400 are also stored. The CPU 1001, ROM 1002, and RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0081] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as required. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as required, so that a computer program read from it can be installed into the storage section 1008 as required.

[0082] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment disclosed in the present invention includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009 and / or installed from the removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, the above functions defined in the system of the present invention are performed.

[0083] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in conjunction with an instruction execution system, apparatus, or device. And in the present invention, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0084] In particular, according to the embodiments disclosed by the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed by the present invention include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section 1009 and / or installed from the removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, the above functions defined in the system of the present invention are executed.

[0085] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. In the present invention, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0086] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0087] The units (or "modules") involved in the embodiments of the present invention can be implemented in software or in hardware. The described units (or "modules") can also be provided in a processor. For example, a processor can be described as including an information acquisition unit, an information processing unit, and an information modification unit. Among them, the names of these units do not constitute a limitation on the units themselves in some cases. For example, the information acquisition unit can also be described as "a unit that, in response to the triggering of a task to change the level of a tree structure, acquires and parses the mapping relationship between the target change level and the adjacent levels in the tree structure".

[0088] On the other hand, the present invention also provides a computer-readable medium, which can be included in the device described in the above embodiments; or can exist separately without being assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device includes: in response to the triggering of a task to change the level of a tree structure, acquiring and parsing the mapping relationship between the target change level and the adjacent levels in the tree structure, and based on the obtained result, maintaining and updating the metadata table of the tree structure with metadata reflecting the association relationship between the target change level and the upper level, where the metadata table is pre-configured for the tree structure and stored in the system, and updating the data table of the tree structure based on the description rule between the metadata and the data to obtain an updated data table corresponding to the changed tree structure.

[0089] According to the technical solution of the embodiment of the present invention, it is possible to achieve flexible maintenance and parsing of the tree structure, support various changes and application requirements, achieve dynamic response and flexible adaptation to the hierarchical changes of the tree structure, improve the stability and maintainability of the entire system, and is also conducive to maintaining the transparency and compatibility of the application, facilitating subsequent expansion and maintenance, so that the system can more easily adapt to future changes in technical effects.

[0090] The above specific embodiments do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.< / orgentity> < / orgentity>

Claims

1. A method for data processing, used to handle hierarchical changes of a tree structure, characterized in that, including: In response to the triggering of a task to change the hierarchy of the tree structure, obtain and parse the mapping relationship between the target changed hierarchy and the adjacent hierarchy in the tree structure Based on the obtained result, maintain and update the metadata table of the tree structure with metadata reflecting the association relationship between the target changed hierarchy and the upper hierarchy, where the metadata table is pre-configured for the tree structure and stored in the system, and Based on the description rule between metadata and data, update the data table of the tree structure to obtain an updated data table corresponding to the changed tree structure.

2. The method for data processing according to claim 1, characterized in that, It also includes: Pre-transform the ID of the tree structure to make it have a tree structure composite ID, where the tree structure composite ID includes prefix retrieval, the field length of the tree structure composite ID is at least the length from the highest hierarchy to the second lowest hierarchy of the tree structure, and the field of the tree structure composite ID is constructed from any field in the data table, and The updated data table includes the update of the tree structure composite ID.

3. The method for data processing according to claim 2, characterized in that, It also includes: Pre-establish a tree structure domain model, which includes business elements and technical elements. The business elements are the information of each node of the tree structure, including the hierarchy identifier and the actual value of each node. The technical elements are the positioning of each node in the tree structure, including the upper hierarchy identifier of the upper hierarchy of each node and the hierarchy value of the current hierarchy.

4. The method for data processing according to claim 3, characterized in that, The updating of the data table of the management tree based on the description rule between metadata and data to obtain an updated data table corresponding to the changed management tree includes: Update the data table in the cache area and the database respectively, and The update in the cache area and the update in the database are performed asynchronously.

5. The method for data processing according to claim 4, characterized in that, The asynchronous update includes: Perform a full parsing of the metadata table to generate a first cache and a second cache. The first cache has a key-value pair with the atomic organization ID as the key and the tree structure composite ID as the value. The second cache has two key-value pairs. One of the two key-value pairs has the hierarchy identifier as the key and the hierarchy value as the value. The other of the two key-value pairs has the composite value of the hierarchy value and the actual value as the key and the prefix retrieval of the tree structure composite ID as the value. Wherein, the atomic organization is the lowest hierarchy of the tree structure; Perform a full update of the data table according to the first cache to complete the initialization update of the information table, Update the data table in the database to obtain the updated data table, and Perform a full update of the data table according to the second cache to perform the input parameter conversion during application.

6. The method for data processing according to claim 4, characterized in that, The asynchronous update includes: Update the data table of the tree structure in the database and update the cache area, and The update of the cache area is configured to: perform a full parsing of the metadata table to generate a third cache, and the third cache uses the key-value pair with the actual value as the key and the prefix retrieval of the tree structure composite ID as the value to perform a full update of the data table.

7. The method for data processing according to claim 5 or 6, characterized in that, It also includes: In response to the input parameter of the tree structure field, all the original fields are replaced according to the tree structure domain model, and the input parameter conversion is performed in the second cache. According to the tree structure information of the input parameter, the tree structure composite ID is obtained, and Read the updated data table in the database according to the obtained organization composite ID.

8. An apparatus for data processing, used to handle hierarchical changes of a tree structure, characterized in that, Includes: Information acquisition unit: In response to the triggering of a task to change the level of the tree structure, obtain and parse the mapping relationship between the target change level and the adjacent levels in the tree structure. Information processing unit: Based on the obtained result, maintain and update the metadata table of the tree structure with metadata reflecting the association relationship between the target change level and the upper level, where the metadata table is pre-configured for the tree structure and stored in the system, and Information change unit: Update the data table of the tree structure based on the description rule between metadata and data to obtain an updated data table corresponding to the changed tree structure.

9. An electronic device for data processing, characterized in that, Includes: One or more processors; A storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method according to any one of claims 1-7.

Images