A data processing method and multi-source data engine

Abstract

The present invention discloses a data processing method and a multi-source data engine, wherein the method comprises: for a service object to be processed, identifying a directory of the service object under a preset data catalog; generating object information of the service object under the directory, wherein the object information is at least used to define the type and attribute set of the service object; the attribute set includes service data related to the service object; creating an object instance of the service object based on the object information, and updating the index information of the preset data catalog. The technical solution provided by the present invention can improve the access efficiency of data.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data processing method and a multi-source data engine. Background Art

[0002] With the continuous advancement of the intelligent city process, a large amount of multi-source data with different data sources, different data structures and different data forms has been generated. For example, in the CIM (City Information Modeling) platform, GIS (Geographic Information System) data, BIM (Building Information Modeling) data and IoT (Internet of Things) data will be generated.

[0003] See also Figure 1 These multi-source data can be divided according to data types, and each type of data can have its own service gateway. When a client needs to access a certain type of data, it needs to initiate a service request to the corresponding service gateway, so that the data is provided to the client through the service gateway.

[0004] However, as the number of data types continues to increase, the number of service gateways in the system will gradually increase, which will not only bring inconvenience to data management, but also for users, when they need to access different types of data, they need to switch to different service gateways, which will undoubtedly reduce the processing efficiency of service requests.

[0005] In view of this, in order to improve the query efficiency of multi-source data, we can try to use a unified data gateway to access multi-source data. However, different multi-source data usually have different data formats. In practical applications, it is difficult to use a unified data gateway to access multi-source data in different data formats, which leads to low data access efficiency. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a data processing method and a multi-source data engine, which can improve data access efficiency.

[0007] On the one hand, the present invention provides a data processing method, which includes: for a service object to be processed, identifying a directory of the service object under a preset data catalog; generating object information of the service object under the directory, wherein the object information is at least used to limit the type and attribute set of the service object; the attribute set includes service data related to the service object; creating an object instance of the service object based on the object information, and updating the index information of the preset data catalog.

[0008] In this application, service objects can be managed uniformly through preset data cataloging. For the service object to be processed, the corresponding directory can be identified under the preset data cataloging. Under this directory, the object information of the service object can be generated in a unified format. In this way, the object instance created based on the object information can have the same data format. By converting multi-source data into a unified data format, a data foundation can be provided for the subsequent unified data gateway, thereby improving the access efficiency of the data.

[0009] In one embodiment, the preset data catalog includes multiple categories divided into levels; identifying the directory of the service object under the preset data catalog includes: determining the category instance to which the service object belongs in each category in order of category range from large to small, and using the combination of the determined category instances as the directory of the service object.

[0010] By determining the category instances one by one in each category of the preset data catalog, the directory of the service object can be accurately determined.

[0011] In one embodiment, generating the object information of the service object under the directory includes: loading a specified data model, the custom fields in the specified data model include at least a type definition field and an attribute set definition field, and generating the object information of the service object according to each of the custom fields in the specified data model.

[0012] Generating object information according to a unified data model can ensure that the object information has the same data format.

[0013] In one embodiment, the method further includes: setting role information for the user, where the role information is used to represent access rights for accessing the object instance.

[0014] Accessing data based on access permissions can improve the security of data access.

[0015] In one embodiment, the object instance has a standard route; the method further includes: after receiving a service request from a client, if the routing information in the service request matches the standard route of the object instance in the path, determining that the object instance is the object instance pointed to by the service request.

[0016] For the routing information carried in the service request sent by the client, the routing information can be path-matched with the standard routes of each service instance, so as to determine the service instance corresponding to the service request. Determining the service instance by path matching can improve the query efficiency of the service instance.

[0017] In one embodiment, matching the routing information in the service request with the standard routing of the object instance on the path includes: path matching the routing information in the service request with the standard routing of the object instance to generate a path length of the object instance relative to the routing information; if the path length is longer than the path lengths of other object instances relative to the routing information, determining that the routing information matches the standard routing of the object instance on the path.

[0018] Both the routing information and the standard route may contain one or more routing identifiers arranged in order. By matching the routing information with the routing identifiers in the standard route, the path length of each object instance relative to the routing information can be determined. The path length can indicate the degree of adaptation between the routing information and the standard route, and the object instance with the longest path length can be the object instance actually pointed to by the service request. By identifying the path length, the object instance corresponding to the service request can be accurately obtained.

[0019] In one embodiment, the standard route and the routing information both include one or more routing identifiers arranged in order; path matching the routing information in the service request with the standard route of the object instance includes: starting from the first routing identifier in the routing information, determining whether the current routing identifier in the routing information is consistent with the routing identifier at the corresponding position in the standard route; if consistent, updating the path length of the object instance relative to the routing information, and continuing to determine the next routing identifier in the routing information; if inconsistent, ending the matching process of the routing identifier of the object instance, and outputting the path length of the object instance relative to the routing information.

[0020] When performing path matching, the routing information can be compared with the routing identifiers at the corresponding positions in the standard routing of the candidate service one by one. Once an inconsistent routing identifier appears, the matching process can be stopped and the corresponding path length can be output. After comparing with the standard routing of each object instance, the path length of each object instance can be generated, thereby ensuring the accuracy of the object instance obtained in subsequent queries.

[0021] In one embodiment, after creating an object instance of the service object based on the object information, the method further includes: establishing an association relationship between the object instance and one or more other object instances; wherein, when one of the associated multiple object instances is accessed, the instance identifiers of the other associated object instances are provided to the visitor; and if the instance identifier is triggered, data of the object instance corresponding to the triggered instance identifier is provided.

[0022] Users only need to initiate access to one object instance, and then they can easily obtain the data of other related object instances, which will undoubtedly greatly improve the user's access experience and improve the processing efficiency of service requests.

[0023] On the other hand, the present invention provides a multi-source data engine, which includes: a directory creation interface, which is used to create a directory of the service object to be processed under a preset data catalog; an object type definition interface, which is used to generate object information of the service object under the directory, and the object information is at least used to limit the type and attribute set of the service object; the attribute set includes service data related to the service object; an object instance creation interface, which is used to create an object instance of the service object based on the object information and update the index information of the preset data catalog.

[0024] In one embodiment, the multi-source data engine also includes: an instance association interface, which is used to establish an association relationship between the object instance and one or more other object instances; wherein, when an object instance among the associated multiple object instances is accessed, the instance identifiers of the other associated object instances are provided to the visitor; if the instance identifier is triggered, the data of the object instance corresponding to the triggered instance identifier is provided.

[0025] Another aspect of the present invention provides a computer storage medium, wherein the computer storage medium is used to store a computer program, and when the computer program is executed by a processor, the above-mentioned data processing method is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0027] Figure 1 The access process of multi-source data in the prior art is shown;

[0028] Figure 2 A schematic diagram of the structure of a system where a unified data gateway is located in one embodiment of the present invention is shown;

[0029] Figure 3 A schematic diagram showing the steps of a data processing method in one embodiment of the present invention is shown;

[0030] Figure 4 A schematic flow chart of a data processing method in one embodiment of the present invention is shown;

[0031] Figure 5 A schematic diagram of a property set of a service object in one embodiment of the present invention is shown;

[0032] Figure 6 A schematic diagram showing steps of a method for processing a service request in one embodiment of the present invention is shown;

[0033] Figure 7 A schematic diagram of a process flow of a method for processing a service request in one embodiment of the present invention is shown;

[0034] Figure 8 A schematic diagram of path matching in one embodiment of the present invention is shown;

[0035] Fig. 9 A schematic diagram of the structure of a data gateway in one embodiment of the present invention is shown. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0037] The service request processing method provided in this application can be applied to Figure 2 In the system shown. Figure 2 As shown, the system may include a data gateway, a multi-source data access engine, and a data storage cluster. Among them, the data gateway may include a routing control unit, a configuration center, an authentication center, and a service registration center. Of course, in actual applications, one or more of the components can be integrated into the same functional module as needed, or one of the functional modules can be split into multiple components, or one or more functional modules can be added or deleted according to the actual application scenario.

[0038] See also Figure 2 The routing control unit can work in coordination with the authentication center and the configuration center to perform routing analysis on the service request sent by the client, thereby determining the service identifier corresponding to the service request, and can also authenticate the identity and authority of the client.

[0039] The multi-source data access engine can be used as an access interface for various service instances. Various service instances can be created based on a specified data model, so that they have the same data structure. Through the multi-source data access engine, these service instances with the same data structure can be accessed, and data with a unified data structure can be fed back to the client. Figure 2 As shown, service instances may include spatial data services, video data services, file services, and the like.

[0040] The service registration center can provide a registration process for the service, thereby storing the registration information of the service identifier, and can also record the mapping relationship between the service identifier and the corresponding service instance, so that the multi-source data access engine can query the service identifier and service instance.

[0041] The data storage cluster can contain various types of databases (such as MySQL, Redis, MongoDB, etc.), through which the specific data of the service instance can be stored. This data can be obtained by the multi-source data access engine and provided to the client by the multi-source data access engine.

[0042] Of course, in addition to the functions described above, in actual applications, the data gateway can also monitor the service requests sent by the client and dynamically control the flow by loading the load configuration.

[0043] In one embodiment of the present application, in order to facilitate multi-source data access through a unified data gateway, the multi-source data of the service object can be uniformly encoded, thereby converting the multi-source data of the service object into data with the same data structure.

[0044] Specifically, see Figure 3 and Figure 4 An embodiment of the present application provides a data processing method, which may include the following steps.

[0045] S11: For a service object to be processed, identifying a directory of the service object under a preset data catalog.

[0046] In this embodiment, for different types of service objects, a unified preset data catalog can be used to classify the types of service objects, and a unified data model can be used to represent the data structure of the service object. Among them, the service object can be an object or project in the real world. For example, the service object can be a city component, a public event, a building space, an Internet of Things device, etc., or a camera, a construction project, a river, a highway, etc. The preset data catalog can include multiple categories divided by level. Among them, each category has its own level range. Generally speaking, the higher the level of the category, the larger the level range, and the level range of the lower level category can be included in the level range of the upper level category. For example, the preset data catalog can include multiple categories such as data categories, data categories, data categories, data categories, data categories, data groups, object types, etc. Among them, each category can contain multiple category instances. For example, under the category of the data category, category instances such as spatiotemporal basic data, resource survey data, planning and control data, building information model, engineering construction project data, public thematic data, and IoT perception data can be included. Taking spatiotemporal basic data as an example, please refer to Table 1. Under the category of major data, you can include category instances such as administrative regions, electronic maps, surveying and mapping remote sensing, and three-dimensional models; under the category of medium data, you can include many category instances such as national administrative regions, provincial administrative regions, government maps, oblique images, and site three-dimensional models.

[0047] Table 1 Schematic diagram of some hierarchical structures of preset data catalogs

[0048]

[0049] In this embodiment, for the service object to be processed uniformly, the directory of the service object can be identified under the preset data catalog. Specifically, the category instance to which the service object belongs can be determined in each category in the order of the range of each category in the preset data catalog from large to small. For example, for a building model, it can correspond to category instances such as spatiotemporal basic data, three-dimensional model, and architectural three-dimensional model. After determining the category instance to which the service object belongs in each category, the combination of the determined category instances can be used as the directory of the service object under the preset data catalog. Of course, each category instance can be arranged according to the composition rules of the directory, and the required distinguishing symbols can be added between the category instances. For example, in the directory, adjacent category instances can be distinguished by "\".

[0050] In practical applications, the preset data catalog can be represented by a tree structure. For example, the preset data catalog can be represented as a data structure such as a B+ tree or a binary tree. When identifying the directory of the service object, the data query rule in the tree structure can be used to query the object identifier of the service object as an index to identify the node corresponding to the service object in the tree structure. The directory of the node in the tree structure can be used as the directory of the service object under the preset data catalog.

[0051] S13: Generate object information of the service object under the directory, wherein the object information is at least used to define the type and attribute set of the service object; the attribute set includes service data related to the service object.

[0052] In this embodiment, after the directory of the service object is determined, the object information of the service object can be generated under the directory. In order to uniformly manage multi-source data, the object information of the service object can be generated based on a specified data model.

[0053] In a specific application example, the specified data model may include multiple pre-set custom fields, and these custom fields may describe the service object from different angles. For example, these custom fields may include an object type definition field, an attribute set definition field, an object attribute type definition field, and an object relationship definition field. Among them, the type identifier of the service object may be indicated in the object type definition field, and the type identifier may be determined by the category of the object type in the preset data catalog. In addition, in the object type definition field, the object identifier and description information of the object may be indicated as needed, and the directory and other information of the service object in the preset data catalog may also be indicated.

[0054] The attribute set definition field may include sub-definition fields for each object attribute. Among them, the object attribute can describe the service object from the business dimension. For example, for the service object of a construction project, its land red line is a GIS (Geographic Information System) surface type spatial data, then the spatial data can be used as an object attribute of the construction project. At the same time, the construction project also has three-dimensional models at different stages, then these three-dimensional models can also be used as an object attribute of the construction project. In addition, the construction project also has corresponding construction supervision data, and the construction supervision data can also be used as an object attribute of the construction project. Generally speaking, an object attribute may also contain different sub-attributes. For example, the object attributes of the construction supervision data may include character-type sub-attributes such as project name, project number, construction unit, etc., and may also include numeric-type sub-attributes such as construction scale and land area.

[0055] In this implementation, the sub-definition fields of different object attributes may contain different contents. Figure 5 For IoT monitoring devices, their object attributes may include device information (object attribute 1), spatial location information (object attribute 2) and real-time monitoring indicator data (object attribute 3), where the sub-definition field of device information may include device identification, device name, device description and other contents; the sub-definition field of spatial location information may include project name, start timestamp, end timestamp, data volume and other contents; the sub-definition field of real-time monitoring indicator data may include monitoring time, device code, data index and other contents.

[0056] In this embodiment, the object attribute type definition field can be used to describe the attribute set definition field. For example, the object attribute type definition field can describe the data type of the object attribute and the classification of the object attribute.

[0057] In this embodiment, the object relationship definition field can represent the association relationship between the current service object and other service objects. In the object relationship definition field, the associated source service object and target service object can be described, and the type of this association relationship can also be described. For example, for the service object of a manhole cover, the object relationship definition field can describe the association relationship between the manhole cover and the road, and can also describe the association relationship between the current manhole cover and other manhole covers.

[0058] It should be noted that the custom fields in the specified data model listed above are only one implementation form for a specific application scenario. In actual applications, the custom fields can be added or deleted as needed. Those skilled in the art should understand that as long as the data of the service object is uniformly managed based on the specified data model, it should fall within the technical scope of this application.

[0059] In this embodiment, the specified data model can be loaded, and then the object information of the service object can be generated according to the various custom fields contained in the specified data model. In the object information, the content of each custom field can be assigned according to the actual data of the service object. In this way, different service objects can be represented by the same data structure, but the actual content of each custom field in the data structure is different.

[0060] S15: Creating an object instance of the service object based on the object information, and updating index information of the preset data catalog.

[0061] In this embodiment, after the object information of the service object is generated, an object instance of the service object can be created according to the object information. The data structure of the object instance can be consistent with the object information, except that the data structure of the object instance includes actual application data. For example, the object instance of a camera includes monitoring data collected by the camera, and can also include optimized data generated after processing the monitoring data. For example, the monitoring data in the camera can be used to identify people, thereby obtaining optimized data labeled with person information.

[0062] In this embodiment, after the object instance of the service object is generated, the index information of the object instance can be added to the index information of the preset data catalog, thereby updating the index information of the preset data catalog. Subsequently, the data of the corresponding object instance can be queried based on the updated index information of the preset data catalog.

[0063] In actual applications, the index information of the preset data catalog can be represented by a tree structure such as a B+ tree and an R tree. After creating an object instance of a service object, the index information of the object instance can be added to the corresponding tree structure to update the index information of the preset data catalog.

[0064] In this application, service objects can be managed uniformly through preset data cataloging. For the service object to be processed, the corresponding directory can be identified under the preset data cataloging. Under this directory, the object information of the service object can be generated in a unified format. In this way, the object instance created based on the object information can have the same data format. By converting multi-source data into a unified data format, a data foundation can be provided for the subsequent unified data gateway, thereby improving the access efficiency of the data.

[0065] In one embodiment, in the configuration center of the data gateway, configuration information of resource permissions can be stored. The configuration information of resource permissions can set different data access permissions for different role information. For example, role information can be divided into ordinary users, important users, administrators, etc., wherein the data access permissions of ordinary users can be "read-only" permissions, the data access permissions of important users can be "read and write" permissions, and the data access permissions of administrators can be "read and write" and "delete" permissions. In this way, after the role information is set for the user, the user's access permissions to the object instance are determined. Subsequently, when the user initiates a service request directed to the object instance, it can first be determined whether the user's access permissions can initiate the service request. Only when the user's access permissions can initiate the service request, the data gateway will perform subsequent data processing according to the service request.

[0066] In one embodiment, after the object instance is created, an association relationship between the object instance and one or more other object instances can be established. This association relationship has been noted in the object relationship definition field when the object information of the service object is created. Based on the content in the object relationship definition field, an association relationship between multiple object instances can be established. The purpose of this processing is that when an object instance among the multiple associated object instances is accessed, the instance identifiers of other associated object instances can be provided to the visitor. For example, the visitor currently initiates data access for a manhole cover on a highway, then the data gateway can provide the manhole cover identifiers of other manhole covers adjacent to the manhole cover on this highway to the visitor. In this way, if the instance identifier is triggered, the data gateway can provide the data of the object instance corresponding to the triggered instance identifier. In other words, the user only needs to initiate access to an object instance, then the data of other associated object instances can be easily obtained, which will undoubtedly greatly improve the user's access experience and improve the processing efficiency of service requests.

[0067] The present application also provides a multi-source data engine, the multi-source data engine comprising:

[0068] A directory creation interface, used to create a directory of a service object to be processed under a preset data catalog;

[0069] An object type definition interface, used to generate object information of the service object under the directory, wherein the object information is used to define at least the type and attribute set of the service object; the attribute set includes service data related to the service object;

[0070] The object instance creation interface is used to create an object instance of the service object based on the object information and update the index information of the preset data catalog.

[0071] In one embodiment, the multi-source data engine further includes:

[0072] An instance association interface is used to establish an association relationship between the object instance and one or more other object instances; wherein, when an object instance among the associated multiple object instances is accessed, the instance identifiers of the other associated object instances are provided to the visitor; if the instance identifier is triggered, the data of the object instance corresponding to the triggered instance identifier is provided.

[0073] In actual applications, in order to cooperate with the functions of the unified data gateway, a multi-source data engine can provide multiple different types of data interfaces, which may include object type definition interfaces, object type instance interfaces, directory creation interfaces, association relationship definition interfaces, instance association interfaces, etc. It should be noted that with the continuous updating of the unified data gateway functions, the multi-source data engine can also add more data interfaces simultaneously. The multiple data interfaces listed above are only the data interfaces required in specific application scenarios. Those skilled in the art, with the understanding of the technical essence of this application, may delete, merge or add the data interfaces therein, but these methods should all fall within the technical protection scope of this application.

[0074] In this implementation, each type of data interface can be further subdivided into different sub-data interfaces according to the functions implemented, and each sub-data interface can implement a variety of different functions. For example, the object type definition interface can include the following four different sub-data interfaces:

[0075] Create an object type for a service object: POST / api / v1 / dtgw / objecttypedefs /

[0076] Modify the object type of the service object: PUT / api / v1 / dtgw / objecttypedefs /

[0077] Delete the object type of the service object: DELETE / api / v1 / dtgw / objecttypedefs /

[0078] Query the object type of the service object: GET / api / v1 / dtgw / objecttypedefs /

[0079] Similarly, other different types of data interfaces can also implement corresponding creation, modification, deletion, and query functions, which will not be repeated here.

[0080] In practical applications, each of the above data interfaces can be designed using the RESTful interface specification. After processing the above data interfaces, a service object with a unified data model can be obtained, thereby providing data support for the unified data gateway.

[0081] The present application also provides a multi-source data engine, which includes a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned data processing method is implemented.

[0082] The present application also provides a computer storage medium, which is used to store a computer program. When the computer program is executed by a processor, the above-mentioned data processing method is implemented.

[0083] In this application, after the above processing, the multi-source data of the service object can be represented by a unified data structure, and the object instances of each service object can be managed and queried through a unified data catalog. Based on this, an embodiment of this application also provides a method for processing a service request. Figure 6 and Figure 7 , the method may include the following steps.

[0084] S21: receiving a service request, wherein the service request includes routing information of a requested service, and performing path matching on the routing information to determine a service identifier corresponding to the service request.

[0085] In this embodiment, the service request can be sent by the client, and the client can be a terminal device used by the user, or an application running on the terminal device. For example, the client can be a smart phone, or a browser running on a smart phone. The service request sent by the client may include routing information of the service to be requested. Generally speaking, each service can have a standard route, and the standard route may include one or more routing identifiers arranged in order. For example, the standard route of a service can be expressed as / r1 / r2 / r3 / r4, where r1 to r4 represent four different routing identifiers, respectively. Similarly, the routing information of the service request may include one or more routing identifiers arranged in order. For example, the routing information of a service request can be expressed as / r1 / r6. By matching the routing information with the standard route, the target service to which the service request points can be determined from a large number of services.

[0086] Specifically, services that have been registered in the service registration center can be used as candidate services, and the standard routes of each candidate service can be stored in the configuration center as routing configuration items. After receiving the service request from the client, the routing control unit can read the routing configuration items from the configuration center to parse the mapping relationship between the standard routes of the candidate services and the service identifiers of the candidate services. Then, the routing information in the service request can be path-matched with the standard routes of each candidate service, so that the service identifiers of the matched candidate services are used as the service identifiers corresponding to the service request.

[0087] In actual applications, the routing control unit can identify the standard routes of each candidate service and match each routing identifier in the routing information with the routing identifier in the standard route in order, thereby generating the path length of each candidate service relative to the routing information. Figure 8 For any target candidate service among the candidate services, we can start from the first route identifier r1 in the routing information to determine whether the current route identifier in the routing information is consistent with the route identifier at the corresponding position in the standard route of the target candidate service. For example, for target candidate service 1 and target candidate service 2, the first route identifier is consistent with the first route identifier in the routing information; while for target candidate service 3 and target candidate service 4, the first route identifier is inconsistent with the first route identifier in the routing information.

[0088] In the case where the routing identifiers are consistent, the path length of the target candidate service relative to the routing information can be updated, and the next routing identifier in the routing information can be determined. For example, for target candidate service 1 and target candidate service 2, the path lengths of each relative to the routing information can be updated to 1. Then, the next routing identifier r2 in the routing information can be determined.

[0089] In the case of inconsistent routing identifiers, the matching process of the routing identifier of the target candidate service can be directly terminated, and the path length of the target candidate service relative to the routing information can be output. For example, for target candidate service 3 and target candidate service 4, the matching process of the routing identifier can be directly terminated, and the path length of the two target candidate services relative to the routing information can be output: 0.

[0090] For target candidate service 1 and target candidate service 2, when judging the next route identifier r2, since the standard route of target candidate service 1 no longer has the next route identifier, the matching process is directly terminated, and the path length corresponding to target candidate service 1 is output as 1; while the next route identifier in the standard route of target candidate service 2 is still consistent with the next route identifier r2 in the routing information, so the path length of target candidate service 2 can be updated to 2, and the matching of the next route identifier r21 can be continued. It can be seen that target candidate service 2 no longer has the next route identifier, so the matching process of the route identifier can be terminated, and the path length of target candidate service 2 is output as 2.

[0091] According to the above method, for each candidate service, the path length relative to the routing information in the service request can be generated. Finally, according to the matching principle of the longest path, the service identifier of the candidate service with the longest path length can be used as the service identifier corresponding to the service request. Figure 8 In the process, the service identifier of the target candidate service 2 can be finally used as the service identifier corresponding to the service request.

[0092] Similarly, Figure 8 The service identifiers corresponding to other service requests may also be as shown in the figure.

[0093] The above path matching strategy can be pre-configured in the rule engine. The dynamic routing module can load the path matching strategy from the rule engine and perform path matching on the routing identifier in the routing information according to the path matching strategy.

[0094] It is not difficult to find that even if multiple routing information has the same pre-order routing identifier, based on the principle of the longest path, they may eventually be routed to different services. This provides a more flexible routing mechanism for different types of services under the same data resource directory.

[0095] As can be seen from the above, for the routing information carried in the service request sent by the client, the routing information can be path matched with the standard routes of each candidate service respectively, so as to determine the service identifier corresponding to the service request. Determining the service identifier by path matching can improve the query efficiency of the service identifier.

[0096] Both the routing information and the standard route may contain one or more routing identifiers arranged in order. By matching the routing information with the routing identifiers in the standard route, the path length of each candidate service relative to the routing information can be determined. The path length can indicate the degree of adaptation between the routing information and the standard route, and the candidate service with the longest path length can be the service actually pointed to by the service request. By identifying the path length, the service identifier corresponding to the service request can be accurately obtained.

[0097] When performing path matching, the routing information can be compared with the routing identifiers at the corresponding positions in the standard routing of the candidate service one by one. Once an inconsistent routing identifier appears, the matching process of the candidate service can be stopped and the path length of the candidate service can be output. After comparing with the standard routing of each candidate service, the path length of each candidate service can be generated, thereby ensuring the accuracy of the service identifier obtained by subsequent queries.

[0098] That is, after receiving a service request from a client, if the routing information in the service request matches the standard route of the object instance in terms of path, it can be determined that the object instance is the object instance pointed to by the service request.

[0099] S23: calling a multi-source data access engine to query a service instance corresponding to the service identifier through the multi-source data access engine; wherein the service instance is created according to a specified data model.

[0100] In this embodiment, after determining the service identifier corresponding to the service request, the routing control unit can pass the service identifier to the multi-source data access engine by calling the multi-source data access engine. The multi-source data access engine can query whether the service identifier has been registered from the service registration center. If the registered service identifier is not available, a prompt message indicating an access error can be fed back to the client. In the case of having the registered service identifier, the service instance corresponding to the service identifier can be queried in the created service instance through the service registration center.

[0101] The queried service instance can be created based on the aforementioned specified data model, and the specific data of the service instance can be represented according to the data structure of the specified data model. This ensures that the data ultimately provided to the client has the same data format, thereby simplifying the data transmission process without the need for additional format conversion.

[0102] As can be seen above, the data gateway can include a service registration center, which can provide a registration process for services and store the mapping relationship between service identifiers and service instances. In this way, the multi-source data access engine can query the corresponding service instance from the service registration center based on the service identifier after routing analysis. Unified management of services through the service registration center can improve management efficiency and query efficiency.

[0103] S25: Obtain service data corresponding to the service request from the service instance.

[0104] In this embodiment, after the service instance corresponding to the service identifier is queried, the service data corresponding to the service request can be obtained from the service instance. For example, if the service request is to access monitoring data with person information, the monitoring data with person information can be queried from various data related to the service instance, and the monitoring data can be fed back to the client.

[0105] Specifically, a service instance may include multiple object attribute sets, and these object attribute sets may include multiple service data related to the service instance. For example, for an IoT monitoring device, its object attribute set may include a device information set, a spatial location information set, and a real-time monitoring indicator data set. When obtaining corresponding service data from a service instance, each object attribute set in the service instance may be parsed, and the service data in each object attribute set that matches the service request is used as the obtained service data.

[0106] In one embodiment, the routing control unit can also implement the process of identity authentication and authority authentication. Specifically, an authentication service can be deployed in the authentication center, and the authentication service can be, for example, an OAuth2 authentication service. After receiving a service request from the client, the routing control unit can work in conjunction with the authentication center to identify the second-generation authentication information in the service request. The authentication information can be, for example, an account name and password, or it can be a biometric feature that represents the user's identity (such as fingerprint features, facial features, voiceprint features, etc.). After the authentication information is authenticated, the routing control unit can issue an access token to the client. Subsequently, the client can obtain the background service data based on the access token. By verifying the authentication information in the service request, the security of data access can be improved.

[0107] In one embodiment, the routing control unit can also perform authentication operations on the client. Specifically, the routing control unit can identify the role information of the client from the service request and obtain the access rights that match the role information. If the permission required for the service request is within the access rights, the routing information in the service request can be path matched. For example, if the permission required for the service request is to delete data, but the access right corresponding to the role information of the client is only "read-only", then the service request can be directly rejected without path matching. Only when the permission required for the service request meets the access rights of the role information will the routing information in the service request be path matched. By identifying the role information of the user who initiates the service request, the access rights of the user are determined. Accessing data based on access rights can improve the security of data access.

[0108] See also Figure 2 The present application also provides a service request processing system, the system comprising:

[0109] A routing control unit, configured to receive a service request from a client, wherein the service request includes routing information of a requested service, and perform path matching on the routing information to determine a service identifier corresponding to the service request;

[0110] A multi-source data access engine is used to query the service instance corresponding to the service identifier; wherein the service instance is created according to a specified data model, and service data corresponding to the service request is obtained from the service instance, and the service data is provided to the client.

[0111] See also Fig. 9 The present application also provides a data gateway, the data gateway comprising:

[0112] Configuration center, used to store routing configuration items;

[0113] The service registration center is used to store the registration information of the service identifier and record the mapping relationship between the service identifier and the corresponding service instance;

[0114] The routing control unit is used to receive a service request from a client, wherein the service request includes routing information of the requested service; read the routing configuration item from the configuration center, and perform path matching on the routing information according to the routing configuration item to determine the service identifier corresponding to the service request.

[0115] The present application also provides a data gateway, which includes a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned service request processing method is implemented.

[0116] The present application also provides a computer storage medium, which is used to store a computer program. When the computer program is executed by a processor, the above-mentioned service request processing method is implemented.

[0117] In the present application, the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0118] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiment of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the method in the above method embodiment.

[0119] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0120] Those skilled in the art can understand that all or part of the processes in the above-mentioned implementation methods can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the implementation methods of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0121] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data processing method, characterized in that: The method comprises: For a service object to be processed, identifying a directory of the service object under a preset data catalog; Generating object information of the service object under the directory, the object information being used to at least define a type and an attribute set of the service object; the attribute set including service data related to the service object; Creating an object instance of the service object based on the object information, and updating index information of the preset data catalog; Wherein, the object instance has a standard route; and the method further comprises: After receiving a service request from a client, if the routing information in the service request matches the standard route of the object instance in terms of path, determining that the object instance is the object instance pointed to by the service request; The routing information in the service request matches the standard routing of the object instance in the path, including: Performing path matching on the routing information in the service request and the standard routing of the object instance to generate a path length of the object instance relative to the routing information; If the path length is longer than the path lengths of other object instances relative to the routing information, it is determined that the routing information matches the standard route of the object instance in terms of path.

2. The method according to claim 1, characterized in that The preset data catalog includes multiple categories divided into levels; The directory identifying the service object under the preset data catalog includes: According to the order of the category range from large to small, the category instances to which the service object belongs are determined in each category, and the combination of the determined category instances is used as the directory of the service object.

3. The method according to claim 1, characterized in that The object information of the service object generated under the directory includes: A specified data model is loaded, wherein the custom fields in the specified data model at least include a type definition field and an attribute set definition field, and object information of the service object is generated according to each of the custom fields in the specified data model.

4. The method according to claim 1, characterized in that: The method further comprises: Role information is set for the user, where the role information is used to represent access rights for accessing the object instance.

5. The method according to claim 1, characterized in that The standard route and the route information both include one or more route identifiers arranged in order; Path matching the routing information in the service request with the standard routing of the object instance includes: Starting from the first routing identifier in the routing information, determining whether the current routing identifier in the routing information is consistent with the routing identifier at the corresponding position in the standard route; If they are consistent, updating the path length of the object instance relative to the routing information, and continuing to determine the next routing identifier in the routing information; If they are inconsistent, the matching process of the routing identifier of the object instance is terminated, and the path length of the object instance relative to the routing information is output.

6. The method according to claim 1, characterized in that After creating the object instance of the service object based on the object information, the method further includes: Establish an association relationship between the object instance and one or more other object instances; wherein, when an object instance among the associated multiple object instances is accessed, the instance identifiers of the other associated object instances are provided to the visitor; if the instance identifier is triggered, provide data of the object instance corresponding to the triggered instance identifier.

7. A multi-source data engine, characterized in that: The multi-source data engine includes: A directory creation interface, used to create a directory of a service object to be processed under a preset data catalog; An object type definition interface, used to generate object information of the service object under the directory, wherein the object information is used to define at least the type and attribute set of the service object; the attribute set includes service data related to the service object; An object instance creation interface, used to create an object instance of the service object based on the object information and update the index information of the preset data catalog; The object instance has a standard route; the multi-source data engine is further used for: After receiving a service request from a client, if the routing information in the service request matches the standard route of the object instance in terms of path, determining that the object instance is the object instance pointed to by the service request; The routing information in the service request matches the standard routing of the object instance in the path, including: Performing path matching on the routing information in the service request and the standard routing of the object instance to generate a path length of the object instance relative to the routing information; If the path length is longer than the path lengths of other object instances relative to the routing information, it is determined that the routing information matches the standard route of the object instance in terms of path.

8. The multi-source data engine according to claim 7, characterized in that: The multi-source data engine also includes: An instance association interface is used to establish an association relationship between the object instance and one or more other object instances; wherein, when an object instance among the associated multiple object instances is accessed, the instance identifiers of the other associated object instances are provided to the visitor; if the instance identifier is triggered, the data of the object instance corresponding to the triggered instance identifier is provided.

Images