Data acquisition method and system based on distributed architecture, storage medium and equipment

Abstract

The invention relates to a data acquisition method and system based on a distributed architecture, a storage medium and equipment, and the method comprises the steps: receiving a first access request of a client through a gateway, and forwarding the first access request to a first service when the first access request comprises a first service identifier and a second service identifier, the first service authenticates the first access information and sends an authorization access request to the second service after the authentication succeeds to obtain authorization information returned by the second service, and the gateway receives the authorization information of the first service and generates a second access request, and the second service receives the second access request sent by the gateway, verifies the authorization information, and returns the service data to the client after successful verification, so that the security of the service data is ensured.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a data acquisition method, system, storage medium and device based on a distributed architecture. Background Art

[0002] Distributed architecture means splitting the system into multiple different services, each of which is responsible for completing a specific function. The services can communicate with each other through message queues. By splitting a large system into multiple small, independent services, system decoupling is achieved, making it easier for each service to be expanded and upgraded independently.

[0003] When the first service and the second service interact, the first service can obtain the business data of the second service by accessing the data interface of the second service, thereby realizing the flow of business data. However, since the data interface is usually not configured with a corresponding authentication mechanism, the data flow between the first service and the second service is vulnerable to attack and tampering, affecting the security of the business data. Summary of the invention

[0004] The embodiments of the present application provide a data acquisition method, system, storage medium and device based on a distributed architecture, which can improve the security of business data transmission.

[0005] In a first aspect, an embodiment of the present application provides a data acquisition system based on a distributed architecture, including: a client, a gateway, a first service, and a second service; the gateway is respectively connected to the client, the first service, and the second service in communication;

[0006] The gateway receives a first access request from a client, and when the first access request includes a first service identifier and a second service identifier, forwards the first access request to the first service; wherein the first service corresponds to the first service identifier;

[0007] The first service authenticates the first access request, sends an authorization access request to the second service after successful authentication, receives authorization information returned by the second service in response to the authorization access request, and sends the authorization information to the gateway;

[0008] The gateway generates a second access request including the authorization information, and forwards the second access request to a second service according to the second service identifier;

[0009] The second service verifies the authorization information and returns business data to the client after successful verification.

[0010] In a second aspect, an embodiment of the present application provides a data acquisition method based on a distributed architecture, comprising the following steps:

[0011] Receive a first access request from a client, and when the first access request includes a first service identifier and a second service identifier, forward the first access request to a first service, so that the first service authenticates the first access request; wherein the first access request includes the first service identifier and the second service identifier;

[0012] Receiving authorization information sent by the first service; wherein the authorization information is information returned by the second service in response to the authorization access request sent by the first service to the second service after successful authentication;

[0013] Generate a second access request including the authorization information, forward the second access request to the second service according to the second service identifier, and enable the second service to verify the authorization information and return business data to the client after successful verification.

[0014] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the data acquisition method based on a distributed architecture as described in any one of the above.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor;

[0016] When the processor executes the computer program, the steps of the data acquisition method based on the distributed architecture as described in any one of the above items are implemented.

[0017] In an embodiment of the present application, a gateway is used to receive a first access request from a client. When the first access request includes a first service identifier and a second service identifier, the first access request is forwarded to the first service. The first service authenticates the first access information and sends an authorization access request to the second service after successful authentication to obtain authorization information returned by the second service. The gateway receives the authorization information of the first service and generates a second access request. The second service receives the second access request sent by the gateway and verifies the authorization information. Only after successful verification does it return the business data to the client, thereby ensuring the security of the business data.

[0018] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of the data acquisition process of the existing distributed architecture of the present invention;

[0020] Figure 2A schematic diagram of a data acquisition process of a data acquisition system based on a distributed architecture in one embodiment of the present invention;

[0021] Figure 3 A schematic diagram of a data acquisition process of a data acquisition system based on a distributed architecture in another embodiment of the present invention;

[0022] Figure 4 A schematic diagram of a data acquisition process of a data acquisition system based on a distributed architecture in another embodiment of the present invention;

[0023] Figure 5 A schematic diagram of a data acquisition process of a data acquisition system based on a distributed architecture in another embodiment of the present invention;

[0024] Figure 6 is a flow chart of a data acquisition method based on a distributed architecture in one embodiment of the present invention;

[0025] Figure 7 The figure is a schematic diagram of the structure of an electronic device in one embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0027] It should be clear that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the embodiments of the present application.

[0028] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0029] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0030] In addition, in the description of this application, unless otherwise specified, "several" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0031] See also Figure 1 , which is an existing distributed architecture data acquisition system, the system includes a client, service A and service B; service A is connected to the client and service B in communication;

[0032] Service A and service B may be services with different business functions. In the embodiment of the present application, service A may store first business data, and service B may store second business data. A communication connection may be established between service A and service B through communication methods such as remote calls.

[0033] The first service data request is a request sent by the client to obtain service data.

[0034] After receiving the first service data request, service A may generate a second service data request and send the request to service B, thereby obtaining the second service data from service B;

[0035] In the embodiment of the present application, service A can call the data interface of service B through RPC remote call or other methods, thereby obtaining the second business data through the data interface.

[0036] Service B returns the second business data to service A in response to the second business data request. Service A obtains business data based on the first business data and the second business data and returns the business data to the client. In this embodiment of the present application, the business data may be data obtained by combining the first business data and the second business data, or the business data may also include a part of the first business data and a part of the second business data.

[0037] In the above process, the data interface of service B is usually a relatively simple data interface (such as a data interface built using middleware frameworks such as dubbo and grpc). The data interface is usually not configured with a corresponding authentication mechanism. That is, when service A passes the verification, all business data can be obtained. The security of the second business data is completely dependent on the authentication mechanism of the first service, and the security of the business data is poor.

[0038] Therefore, in response to the above problems, Figure 2 As shown, the present application provides a data acquisition system based on a distributed architecture, the system comprising: a client 110, a gateway 120, a first service 130 and a second service 140; the gateway 120 is respectively connected to the client 110, the first service 130 and the second service 140 in communication;

[0039] The client 110 is specifically the end where the person requesting business data is located. The client 110 can be a client used by business personnel and developers. The hardware pointed to by the client 110 is essentially a computer device, which can be a computer device of the type of a smart phone, a smart interactive tablet, and a personal computer.

[0040] The gateway 120 is used to process and forward requests. In the embodiment of the present application, the gateway 120 may be an apisix gateway. apisix is ​​a cloud-native, high-performance, and scalable microservice API open source gateway. Compared with traditional API gateways, it has the characteristics of dynamic routing and hot plug-in loading.

[0041] The first service 130 and the second service 140 may be services having different system functions.

[0042] The gateway 120 receives a first access request from the client 110, and when the first access request includes a first service identifier and a second service identifier, forwards the first access request to the first service 130; wherein the first service 130 corresponds to the first service identifier;

[0043] The first service 130 authenticates the first access request, sends an authorization access request to the second service 140 after successful authentication, receives authorization information returned by the second service 140 in response to the authorization access request, and sends the authorization information to the gateway 120;

[0044] The gateway 120 generates a second access request including the authorization information, and forwards the second access request to the second service 140 according to the second service identifier;

[0045] The second service 140 verifies the authorization information and returns business data to the client 110 after successful verification.

[0046] The first access request may be a request issued by the client 110 for accessing the first service 130 and the second service 140 , wherein the first service identifier corresponds to the first service 130 , and the second service identifier corresponds to the second service 140 .

[0047] The gateway 120 may store the address information of the service corresponding to each service identifier. When receiving an access request, the gateway 120 obtains the address information of the corresponding service according to the service identifier in the access request, and forwards the access request to the corresponding service according to the address information.

[0048] The first access request can be used to access the first platform. In the embodiment of the present application, the first platform needs to use the first service and the second service, so the first access request may include the first service identifier and the second service identifier. In the embodiment of the present application, the first access request may also include the master-slave relationship between the first service identifier and the second access identifier. The master-slave relationship is used to determine the master-slave relationship between the first service and the second service. In the embodiment of the present application, the first service can be the master service and the second service can be the slave service.

[0049] The gateway 120 forwards the first access request to the first service 130 for authentication. Authentication refers to the process of determining whether the identity of a user is legitimate in an information system, network or application, and determining which resources the user can access or which operations the user can perform in the system based on the user's identity and authority. Specifically, the first access request may include verification information. After receiving the first access request, the first service 130 verifies the verification information. When the verification information meets the verification conditions, it is determined that the authentication is successful. At this time, an authorized access request is sent to the second service 140. Among them, the authentication method includes username and password authentication, digital certificate authentication, biometric authentication, etc., and the verification information may include username and password, digital certificate, biometrics and other information. The verification conditions can be set according to the specific authentication method.

[0050] After receiving the authorization access request from the first service 130, the second service 140 returns authorization information to the second service 140. The authorization information can be used to determine whether the user has the authority to access the business data. After the second service 140 sends the authorization information to the gateway 120, the gateway 120 can generate a second access request based on the authorization information, so that when the second service 140 receives the second access request, it can verify the authorization information included in the second access request. Specifically, the second service 140 can verify whether the authorization information meets the authorization requirements, for example, it can verify whether the authorization information is legal or standardized, such as whether it comes from a legal user or a legal application, whether it has the correct format, header information, parameters, request body, etc. The verification method can be set according to user needs, and the business data is returned to the client 110 only after the verification is successful, thereby ensuring the security of the business data. At the same time, when other services need to access the business data of the second service 140, they only need to perform authentication and verification in the above manner. After passing, they can quickly access the business data of the second service 140, without the need to design or develop additional corresponding access programs, which can effectively reduce development costs and improve business data access efficiency.

[0051] The business data of the present application may be data of a business processing system. The business data may be stored in a database. The database may be set in a second service, or a calling interface of the database may be called by the second service. For example, when the business processing system is an enterprise management system, the business data may be enterprise information, enterprise order data and other data. When the business processing system is a digital campus management system, the business data may be teacher file data, student file data and other data.

[0052] In an embodiment of the present application, a gateway is used to receive a first access request from a client. When the first access request includes a first service identifier and a second service identifier, the first access request is forwarded to the first service. The first service authenticates the first access information and sends an authorization access request to the second service after successful authentication to obtain authorization information returned by the second service. The gateway receives the authorization information of the first service and generates a second access request. The second service receives the second access request sent by the gateway and verifies the authorization information. Only after successful verification does it return business data to the client, thereby ensuring the security of the business data.

[0053] like Figure 3 As shown, in one embodiment, the first service 130 authenticates the first access request, and returns an authentication failure request to the gateway 120 after the authentication fails;

[0054] The gateway 120 generates authentication failure information according to the authentication failure request and sends the information to the client 110 .

[0055] The authentication failure information is used to prompt the user that the authentication failed. The authentication failure information may include information such as the authentication failure identifier, the authentication failure reason, and the authentication execution object. The authentication failure identifier can be used to prompt the reason for the authentication failure. The authentication failure identifier may include letters, numbers, and a combination thereof. In an embodiment of the present application, the authentication execution object may be the first service 130, and the authentication failure identifier may be 403.

[0056] Preferably, after receiving the authentication failure information, the client 110 may display the authentication failure information on its display interface, so as to facilitate the user to understand the reason for the authentication failure and make adjustments in time, thereby improving the efficiency of business data acquisition.

[0057] like Figure 4 As shown, in one embodiment, the second service 140 verifies the authorization information, generates an error prompt message and sends it to the client 110 after the verification fails.

[0058] The error prompt information is used to prompt the user that the authorization information verification has failed. The error prompt information may include information such as the reason for the verification failure, the error prompt identifier, the verification execution object, etc. The error prompt identifier may include letters, numbers and their combinations. In the embodiment of the present application, the verification execution object is the second service 140.

[0059] Preferably, after receiving the verification failure information, the client 110 may display the verification failure information on its display interface, thereby facilitating the user to understand the reason for the verification failure and make adjustments, thereby improving the efficiency of obtaining business data.

[0060] like Figure 5 As shown, in one embodiment, the gateway 120 receives a third access request from the client 110, and if the third access request only includes the second service identifier, the third access request is forwarded to the second service 140 according to the second service identifier;

[0061] The second service 140 authenticates the third access request and returns the business data to the client 110 after successful authentication.

[0062] The third access request may be used to access a second platform; the second platform corresponds to the second service identifier.

[0063] Specifically, the third access request may include verification information. After receiving the third access request, the second service 140 verifies the verification information. When the verification information meets the verification condition, it is determined that the authentication is successful. At this time, the business data is obtained and sent to the client 110. Among them, the authentication method includes user name and password authentication, digital certificate authentication, biometric authentication, etc., and the verification information may include user name and password, digital certificate, biometrics and other information. The verification condition can be set according to the specific authentication method.

[0064] It should be noted that when the client 110 issues a fourth access request including only the first service identifier, the gateway 120 forwards the fourth access request to the first service 130, and the first service 130 authenticates the fourth access request. After successful authentication, the first service 130 returns corresponding data to the client 110. The authentication method is the same as the authentication method of the second service for the third access request. Please refer to the description of the above embodiment and will not be repeated here.

[0065] In one embodiment, the data transmission format of the data interface of the second service is the same as the data transmission format of the data interface of the first service;

[0066] The gateway adds the authorization information to the first access request to generate a second access request.

[0067] The data transmission format may refer to a protocol format corresponding to a data transmission protocol of a data interface, and the data transmission format includes but is not limited to formats such as http and rpc.

[0068] Specifically, in an embodiment of the present application, when the data transmission format of the data interface of the second service is the same as the data transmission format of the data interface of the first service, the gateway can directly add the authorization information to the first access request to generate a second access request carrying the authorization information, so that the business data of the second service can be accessed through the second access request, thereby eliminating the need to redesign and develop the corresponding access program for the data interface of the second service, thereby improving development efficiency and data acquisition efficiency.

[0069] Alternatively, in another embodiment, the data transmission format of the data interface of the second service is different from the data transmission format of the data interface of the first service;

[0070] The gateway generates a second access request corresponding to the data transmission format of the data interface of the second service according to the authorization information and the data transmission format of the data interface of the second service.

[0071] When the data transmission format of the data interface of the second service is different from that of the data interface of the first service, a corresponding second access request is generated according to the data transmission format of the data interface of the second service, thereby accessing the business data of the second service through the second access request.

[0072] See also Figure 6 The embodiment of the present application also provides a data acquisition method based on a distributed architecture, which can be applied to a gateway, where the gateway is respectively connected to a client, a first service, and a second service in communication, and the method includes the following steps:

[0073] S110: Receive a first access request from a client, and when the first access request includes a first service identifier and a second service identifier, forward the first access request to a first service, so that the first service authenticates the first access request; wherein the first access request includes the first service identifier and the second service identifier;

[0074] S120: Receive authorization information sent by the first service; wherein the authorization information is information returned by the second service in response to the authorization access request sent by the first service to the second service after successful authentication;

[0075] S130: Generate a second access request including the authorization information, forward the second access request to the second service according to the second service identifier, and enable the second service to verify the authorization information and return business data to the client after successful verification.

[0076] In one embodiment, the method further comprises:

[0077] An authentication failure request returned by the first service after authentication fails is received, authentication failure information is generated according to the authentication failure request, and is sent to the client.

[0078] In one embodiment, the method further comprises:

[0079] Receive a third access request from the client. If the third access request only includes the second service identifier, forward the third access request to the second service according to the second service identifier, so that the second service authenticates the third access request and returns the business data to the client after successful authentication.

[0080] In one embodiment, the step of generating a second access request including the authorization information specifically includes:

[0081] If the data transmission format of the data interface of the second service is the same as the data transmission format of the data interface of the first service, the authorization information is added to the first access request to generate a second access request.

[0082] Alternatively, in another embodiment, the step of generating a second access request including the authorization information specifically includes:

[0083] If the data transmission format of the data interface of the second service is different from the data transmission format of the data interface of the first service, a second access request corresponding to the data transmission format of the data interface of the second service is generated according to the authorization information and the data transmission format of the data interface of the second service.

[0084] It should be noted that the data acquisition method based on a distributed architecture provided in the above embodiment is a solution described with the gateway as the execution entity. It belongs to the same concept as the data acquisition method based on a distributed architecture in the above embodiment. The implementation process is detailed in the above embodiment and will not be repeated here.

[0085] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the data acquisition method based on a distributed architecture as described in any one of the above.

[0086] The present application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0087] See also Figure 7 , the present application also provides an electronic device 200, comprising a memory 210, a processor 220, and a computer program stored in the memory 210 and executable by the processor 220;

[0088] When the processor 220 executes the computer program, the steps of the data acquisition method based on the distributed architecture as described in any one of the above items are implemented.

[0089] The memory 210 includes a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0090] The processor 220 is the control core (Control Unit) of the electronic device 200. It uses various interfaces and lines to connect various components of the entire electronic device 200, and executes various functions and processes data of the electronic device 200 by running or executing programs or modules stored in the memory 210, and calling data stored in the memory 210. For example, when the processor 220 executes the computer program stored in the memory, it implements all or part of the steps of the data acquisition method based on the distributed architecture described in the embodiment of the present application; the processor 220 can be composed of an integrated circuit, for example, it can be composed of a single packaged integrated circuit, or it can be composed of multiple integrated circuits with the same function or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips.

[0091] The present invention is not limited to the above-mentioned embodiments. If various changes or modifications to the present invention do not depart from the spirit and scope of the present invention, and if these changes and modifications fall within the scope of the claims and equivalent technologies of the present invention, the present invention is also intended to include these changes and modifications.

Claims

1. A data acquisition system based on a distributed architecture, characterized in that: include: A client, a gateway, a first service, and a second service; The gateway is respectively connected to the client, the first service and the second service in communication; The gateway receives a first access request from a client, and when the first access request includes a first service identifier and a second service identifier, forwards the first access request to the first service; wherein the first service corresponds to the first service identifier; The first service authenticates the first access request, sends an authorization access request to the second service after successful authentication, receives authorization information returned by the second service in response to the authorization access request, and sends the authorization information to the gateway; The gateway generates a second access request including the authorization information, and forwards the second access request to a second service according to the second service identifier; The second service verifies the authorization information and returns business data to the client after successful verification.

2. The data acquisition system based on distributed architecture according to claim 1, characterized in that: The first service returns an authentication failure request to the gateway after authentication fails; The gateway generates authentication failure information according to the authentication failure request and sends the information to the client.

3. The data acquisition system based on distributed architecture according to claim 1, characterized in that: The second service generates an error prompt message after verification fails and sends it to the client.

4. The data acquisition system based on distributed architecture according to claim 1, characterized in that: The gateway receives a third access request from the client, and if the third access request only includes the second service identifier, forwards the third access request to the second service according to the second service identifier; The second service authenticates the third access request and returns the business data to the client after successful authentication.

5. The data acquisition system based on distributed architecture according to claim 1, characterized in that: The data transmission format of the data interface of the second service is the same as the data transmission format of the data interface of the first service; The gateway adds the authorization information to the first access request to generate a second access request.

6. The data acquisition system based on distributed architecture according to claim 1, characterized in that: The data transmission format of the data interface of the second service is different from the data transmission format of the data interface of the first service; The gateway generates a second access request corresponding to the data transmission format of the data interface of the second service according to the authorization information and the data transmission format of the data interface of the second service.

7. The data acquisition system based on distributed architecture according to claim 4, characterized in that: The first access request is used to access a first platform; the first platform corresponds to the first service identifier and the second service identifier; The third access request is used to access a second platform; the second platform corresponds to the second service identifier.

8. A data acquisition method based on a distributed architecture, characterized in that: The following steps are involved: Receive a first access request from a client, and when the first access request includes a first service identifier and a second service identifier, forward the first access request to a first service, so that the first service authenticates the first access request; wherein the first access request includes the first service identifier and the second service identifier; Receiving authorization information sent by the first service; wherein the authorization information is information returned by the second service in response to the authorization access request sent by the first service to the second service after successful authentication; Generate a second access request including the authorization information, forward the second access request to the second service according to the second service identifier, and enable the second service to verify the authorization information and return business data to the client after successful verification.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data acquisition method based on a distributed architecture as claimed in claim 8 are implemented.

10. An electronic device, characterized in that: comprising a memory, a processor, and a computer program stored in the memory and executable by the processor; When the processor executes the computer program, the steps of the data acquisition method based on a distributed architecture as claimed in claim 8 are implemented.

Images