A big data support method, system, computer and storage medium

Abstract

The present invention relates to the technical field of big data processing, and provides a big data support method, system, computer and storage medium, the method comprising: obtaining a number of stock SDKs, determining a number of selected big data basic platforms; obtaining a number of key files; determining a number of selected proxy modules; updating a number of docking proxy modules, and connecting the several docking proxy modules to a number of proxy service ends respectively according to the API information in the several stock SDKs; receiving HTTP requests, and extracting big data interaction SDKs; converting the big data interaction SDKs into HTTP responses. By setting the stock proxy modules and generating the docking proxy modules, the access to big data can be completed only through the HTTP protocol, reducing the difficulty and cost of various external application systems accessing the big data basic platform, and making the access more flexible and easy to change.

Description

Technical Field

[0001] The present invention relates to the field of big data processing technology, and in particular to a big data support method, system, computer and storage medium. Background Art

[0002] Big data technology is crucial to today's information industry. Big data is a huge collection of data that exceeds the processing capabilities of traditional database tools in terms of acquisition, storage, management, and analysis.

[0003] When enterprises use big data to improve their business and productivity, they need to interact with big data and access big data related services. However, building a big data platform requires huge costs in many aspects, so enterprises usually develop their own big data applications based on the existing big data basic platform.

[0004] However, the application development of enterprises is strongly coupled with the underlying big data infrastructure platform, and usually the client code of the proxy service needs to be written into the application. The big data system is huge and complex. There are many manufacturers providing big data infrastructure platform products on the market, and the proxy methods and versions of the big data infrastructure products provided by each manufacturer are different, with great differences. If enterprises directly develop applications for specific big data infrastructure platforms, there will be problems such as high difficulty and high cost in access, and it is difficult to flexibly adapt to the big data infrastructure platforms of various manufacturers. Once the manufacturer or version of the big data infrastructure platform is changed, the upper-level applications of the enterprise must be modified at the code level. Summary of the invention

[0005] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a big data support method, system, computer and storage medium, aiming to solve the problem of insufficient compatibility and difficulty in flexibly adapting various big data basic platforms when enterprises develop big data applications in the prior art.

[0006] In a first aspect, the present application provides a big data support method, comprising the following steps:

[0007] Acquire a number of stock SDKs from the external application layer, and determine a number of selected big data basic platforms from a number of big data basic platforms according to the number of stock SDKs, wherein the selected big data basic platforms include a proxy server;

[0008] Acquire a plurality of key files from the external application layer, wherein the plurality of key files correspond to a plurality of proxy servers respectively;

[0009] Acquire a first proxy service from a plurality of the proxy service terminals, acquire a second proxy service from a plurality of stock proxy modules, wherein the number of types of the second proxy services is greater than or equal to the number of types of the first proxy service, and determine a plurality of selected proxy modules from the plurality of the stock proxy modules based on the types of the first proxy service and the second proxy service;

[0010] Generate a number of standby proxy modules through a number of the selected proxy modules, update the number of the standby proxy modules to a number of docking proxy modules according to a number of the stock SDKs and a number of the key files, and connect the number of the docking proxy modules to a number of the proxy servers respectively according to the API information in the number of the stock SDKs;

[0011] Receiving an HTTP request sent by the external application layer, and calling the docking agent module according to the HTTP request, so that the docking agent module extracts a big data interaction SDK from the selected big data basic platform through the proxy server;

[0012] The big data interaction SDK is converted into a data compression package, and the data compression package is converted into an HTTP response, and the HTTP response is sent to the external application layer for use by external applications in the external application layer.

[0013] Compared with the prior art, the beneficial effect of the present invention lies in: by setting up a number of the existing proxy modules, the proxy service used by the mainstream big data basic platform on the market is adapted through the second proxy service, and according to the existing SDK and the key file, the generated backup proxy module is specifically and targetedly updated, and the docking proxy module that can interact with the proxy server in the selected big data basic platform is formed. The proxy servers in different selected big data basic platforms correspond to different types and versions of proxy services. Through the docking proxy module, the external application in the external application layer does not need to perform targeted code deployment according to the proxy server. The big data can be accessed only by sending the HTTP request and receiving the HTTP response to complete the big data function development of the external application. Enterprise development applications no longer need to be strongly coupled with a specific big data basic platform at the code level. The big data support method is beneficial to reducing the difficulty and cost of various external application systems accessing the big data basic platform, and makes the access more flexible and easy to change.

[0014] Furthermore, the step of determining a plurality of selected proxy modules from a plurality of existing proxy modules based on the types of the first proxy service and the second proxy service includes:

[0015] Different types of the second proxy services are all different, and the type of each of the first proxy services is compared with the type of each of the second proxy services;

[0016] If the type of the second proxy service is the same as the type of the first proxy service, determining the second proxy service as a designated proxy service;

[0017] The existing proxy modules corresponding to the designated proxy service are determined as the selected proxy modules, and the number of the selected proxy modules is equal to the number of the types of the first proxy services.

[0018] Furthermore, the step of generating a plurality of standby proxy modules by using a plurality of the selected proxy modules comprises:

[0019] Dividing the plurality of proxy servers into a plurality of server groups according to the type of the first proxy service, each of the server groups corresponding to one of the selected proxy modules;

[0020] According to the number of the proxy servers in the server group, the selected proxy module corresponding to the server group is replicated to form a plurality of standby proxy modules.

[0021] Furthermore, the step of updating the plurality of standby proxy modules to a plurality of docking proxy modules according to the plurality of stock SDKs and the plurality of key files includes:

[0022] Extracting, from the plurality of stock SDKs, a plurality of version numbers of the first proxy services corresponding to the plurality of proxy servers, a plurality of APIs and a plurality of tool components;

[0023] The standby agent module is updated according to the version number, the API, the tool component and the key file to form the docking agent module.

[0024] Furthermore, before the step of connecting the plurality of docking agent modules to the plurality of agent servers respectively according to the API information in the plurality of stock SDKs, there is also:

[0025] Key authentication is performed according to the key file, so that the proxy server corresponding to the key file is in a state of allowing connection.

[0026] Furthermore, the step of receiving the HTTP request sent by the external application layer, calling the docking agent module according to the HTTP request, and allowing the docking agent module to extract the big data interaction SDK from the selected big data basic platform through the proxy server also includes:

[0027] Extract the IP address, port and interaction data in the HTTP request;

[0028] An interactive proxy module is determined from a plurality of the docking proxy modules according to the IP address and the port, and the interactive data is transmitted to the interactive proxy module, where the interactive data is used to call the API of the proxy server and obtain the big data interactive SDK.

[0029] In a second aspect, an embodiment of the present application provides a big data support system, which is applied to the big data support method in the above technical solution, and the system includes:

[0030] A first acquisition module is used to acquire a number of stock SDKs from an external application layer, and determine a number of selected big data basic platforms from a number of big data basic platforms according to the number of stock SDKs, wherein the selected big data basic platforms include a proxy server;

[0031] A second acquisition module is used to acquire a plurality of key files from the external application layer, wherein the plurality of key files correspond to a plurality of proxy servers respectively;

[0032] a selection module, configured to obtain a first proxy service from a plurality of the proxy service terminals, obtain a second proxy service from a plurality of stock proxy modules, wherein the number of types of the second proxy services is greater than or equal to the number of types of the first proxy services, and determine a plurality of selected proxy modules from the plurality of stock proxy modules based on the types of the first proxy services and the second proxy services;

[0033] A generating module, used to generate a plurality of standby proxy modules through a plurality of selected proxy modules, update a plurality of the standby proxy modules to a plurality of docking proxy modules according to a plurality of the stock SDKs and a plurality of the key files, and connect a plurality of the docking proxy modules to a plurality of the proxy service terminals respectively according to the API information in the plurality of the stock SDKs;

[0034] A request module, used for receiving the HTTP request sent by the external application layer, and calling the docking agent module according to the HTTP request, so that the docking agent module extracts the big data interaction SDK from the selected big data basic platform through the proxy server;

[0035] The return module is used to convert the big data interaction SDK into a data compression package, and convert the data compression package into an HTTP response, and send the HTTP response to the external application layer for use by external applications in the external application layer.

[0036] In a third aspect, an embodiment of the present application provides a computer, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the big data support method as described in the first aspect above is implemented.

[0037] In a fourth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the big data support method as described in the first aspect above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the big data support method in the first embodiment of the present invention;

[0039] Figure 2 It is a structural block diagram of a big data support system in the second embodiment of the present invention;

[0040] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0041] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0042] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0044] See also Figure 1 The first embodiment of the present invention provides a big data support method, comprising the following steps:

[0045] S10: obtaining a number of stock SDKs from the external application layer, and determining a number of selected big data basic platforms from a number of big data basic platforms according to the number of stock SDKs, wherein the selected big data basic platforms include a proxy server;

[0046] Preferably, the external application layer may include multiple external applications developed based on big data services. The big data basic platform provides big data services. Several of the big data basic platforms include Apache open source community big data basic platform, Huawei Cloud big data basic platform, and Alibaba Cloud big data basic platform. Each of the stock SDKs corresponds to the proxy server in the selected big data basic platform. The stock SDK includes the API and toolkit of the proxy server. It can be understood that the servers of the big data basic platforms of different manufacturers need to use different APIs and toolkits when docking. The user enterprise has the corresponding stock SDKs of the corresponding manufacturers in the external application layer, so the user enterprise has the basis for docking with the big data basic platforms of the corresponding manufacturers.

[0047] S20: obtaining a plurality of key files from the external application layer, wherein the plurality of key files correspond to a plurality of the proxy servers respectively;

[0048] Preferably, the big data basic platform needs to identify the authority of the accessor, so Kerberos authentication technology is used to authenticate the services accessing it, and the key file is a keytab file. It can be understood that various data access operations can be performed on the big data basic platform only after obtaining the key file.

[0049] S30: obtaining a first proxy service from a plurality of the proxy service terminals, obtaining a second proxy service from a plurality of stock proxy modules, wherein the number of types of the second proxy services is greater than or equal to the number of types of the first proxy service, and determining a plurality of selected proxy modules from the plurality of stock proxy modules based on the types of the first proxy service and the second proxy service;

[0050] Preferably, the types of proxy services adopted by the proxy server in the mainstream big data basic platform on the market include Hive, HBase and Hadoop, the types of the first proxy services include Hive and Hadoop, and the types of the second proxy services include Hive, HBase and Hadoop. It can be understood that the types of proxy services supported by several of the existing proxy modules can cover the proxy server, and several of the existing proxy modules are adapted to the three mainstream proxy services. The big data support method increases the flexibility and adaptability of user enterprises to access the mainstream big data basic platform.

[0051] Specifically, the step S30 further includes:

[0052] S310: Different types of the second proxy services are different, and the type of each of the first proxy services is compared with the type of each of the second proxy services;

[0053] It can be understood that in order to match the proxy server, it is necessary to match the first proxy service, and it is necessary to select a module corresponding to the Hive proxy service and a module corresponding to the Hadoop proxy service from a plurality of the existing proxy modules.

[0054] S320: If the type of the second proxy service is the same as the type of the first proxy service, determining the second proxy service as a designated proxy service;

[0055] Preferably, in this embodiment, the Hive proxy service and the Hadoop proxy service in the second proxy service are determined as the designated proxy services.

[0056] S330: Determine the existing proxy modules corresponding to the designated proxy service as the selected proxy modules, and the number of the selected proxy modules is equal to the number of types of the first proxy services.

[0057] It can be understood that the two selected proxy modules are respectively a Hive proxy module and a Hadoop proxy module.

[0058] Preferably, the callable interfaces of the Hive proxy module include a Hive metadata query interface, a Hive table operation interface, a record interface and an SQL execution interface. The Hive metadata query interface is used to query the Hive metadata in the library table. The Hive metadata is used to describe the Hive table, such as the table field name, type, length, precision and field description information. The Hive table operation interface is used to complete the creation, deletion, addition and deletion of the Hive table field. The record interface is used to complete the addition, modification and deletion of the record. The SQL execution interface is used to execute SQL statements to complete the corresponding business operations. The callable interfaces of the Hadoop proxy module include an HDFS distributed file management interface and a MapReduce distributed job management interface. The HDFS distributed file management interface is used to implement the management operations of the HDFS file system, including the creation, deletion, copying and moving of the directory, and also including the uploading, downloading, moving and copying of the file. The MapReduce distributed job management interface is used to implement the management operations of the jobs of the MapReduce programming model, such as job uploading, deletion and job application startup, termination and query operations.

[0059] S40: generating a plurality of standby proxy modules through the plurality of selected proxy modules, updating the plurality of standby proxy modules to a plurality of docking proxy modules according to the plurality of stock SDKs and the plurality of key files, and connecting the plurality of docking proxy modules to a plurality of proxy service terminals respectively according to the API information in the plurality of stock SDKs;

[0060] The selected proxy module also needs to be configured, modified, and updated according to the big data basic platform of the specific manufacturer before it can meet the connection standards of the different proxy servers. Only after obtaining the information of the connection interface, that is, after obtaining the API information, can the specific path to connect to the proxy server be obtained.

[0061] Specifically, the step S40 further includes:

[0062] S410: dividing the plurality of proxy servers into a plurality of server groups according to the type of the first proxy service, each of the server groups corresponding to one of the selected proxy modules;

[0063] Preferably, they are divided into two groups, corresponding to the Hive proxy service and the Hadoop proxy service respectively.

[0064] S420: according to the number of the proxy servers in the server group, copy the selected proxy module corresponding to the server group to form a plurality of standby proxy modules;

[0065] Preferably, the total amount of the big data basic platforms provided on the market is huge and the types are rich. Different manufacturers all provide mainstream proxy services. For example, Huawei Cloud and Alibaba Cloud both have the big data basic platforms that provide Hive services. Specifically, the several selected big data basic platforms in this embodiment include Huawei Cloud Hive service platform and Alibaba Cloud Hive service platform, then there are two selected big data basic platforms corresponding to the Hive proxy service, and there are two corresponding proxy service terminals. It can be understood that the same number of Hive proxy modules need to be provided as backup. Therefore, the selected proxy modules corresponding to the Hive proxy service are copied. After copying, the total number of the selected proxy modules is two, that is, two Hive proxy modules are obtained, that is, two backup proxy modules are provided for use in subsequent processes, and the same is true for the Hadoop proxy module.

[0066] S430: extracting version numbers of several first proxy services corresponding to several proxy servers, several APIs and several tool components from several stock SDKs;

[0067] The same proxy service includes multiple versions. For example, the Hive proxy service in the standby proxy module in this embodiment may have a different version from the Hive proxy service in the proxy server. It can be understood that the version number is obtained to determine whether to update the version of the Hive proxy service in the standby proxy module. If the version numbers are different, it needs to be updated.

[0068] S440: updating the standby proxy module according to the version number, the API, the tool component and the key file to form the docking proxy module;

[0069] It can be understood that a plurality of the docking proxy modules can realize the requirement of connecting to a plurality of the proxy servers.

[0070] S450: Perform key authentication according to the key file, so that the proxy server corresponding to the key file is in a state of allowing connection.

[0071] It can be understood that the docking agent module can only interact with the agent server after passing the Kerberos authentication.

[0072] S50: receiving an HTTP request sent by the external application layer, and calling the docking agent module according to the HTTP request, so that the docking agent module extracts a big data interaction SDK from the selected big data basic platform through the proxy server;

[0073] Preferably, the information included in the HTTP request includes HTTP method, URL, query parameters, form data, etc.

[0074] Specifically, the step S50 further includes:

[0075] S510: Extracting the IP address, port and interaction data in the HTTP request;

[0076] Preferably, different IP addresses and ports correspond to different docking agent modules. The user enterprise can select the docking agent module to be called in the HTTP request issued according to the needs, and further connect to the corresponding big data basic platform through the docking agent module.

[0077] S520: Determine an interactive proxy module from the plurality of docking proxy modules according to the IP address and the port, and transmit the interactive data to the interactive proxy module, where the interactive data is used to call the API of the proxy server and obtain a big data interactive SDK.

[0078] It can be understood that after configuring several of the docking proxy modules, it is only necessary to change the port and the IP address to determine different interactive proxy modules, which is easy to replace. The big data can be called through the HTTP protocol. Even if it is necessary to switch to big data services from different manufacturers, it is only necessary to change the HTTP request. There is no need to modify the code of the external application. The big data support method greatly reduces the difficulty and cost of using big data technology by external application systems, and can play a role in shielding the software differences of the big data basic platforms of various manufacturers. User companies do not need to modify the code of self-developed applications to use big data services in the face of different proxy service types, different proxy service versions, different interfaces, and even different key authentications.

[0079] S60: Convert the big data interaction SDK into a data compression package, convert the data compression package into an HTTP response, and send the HTTP response to the external application layer for use by external applications in the external application layer.

[0080] Preferably, the user calls the API of the proxy server with different functions through the HTTP protocol, and after executing the corresponding business logic, data interaction with the big data basic platform is generated. The data interaction is encapsulated as an SDK package, and the big data interaction SDK is converted into a data compression package in JSON format or ZIP format.

[0081] Preferably, the big data interaction SDK is compressed into a ZIP format data compression package, and then the ZIP format data compression package is encapsulated to form the HTTP response. It can be understood that the data interaction process is convenient and efficient, and the HTTP protocol can be used to complete the docking of the external application with a variety of the big data basic platforms, which greatly reduces the difficulty and cost of the external application accessing big data, and can flexibly adapt to various types of the big data basic platforms.

[0082] See also Figure 2 The second embodiment of the present invention provides a big data support system, which applies the big data support method in the above-mentioned embodiment, and will not be repeated here. As used below, the terms "module", "unit", "subunit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0083] The system comprises:

[0084] The first acquisition module 10 is used to acquire a number of stock SDKs from the external application layer, and determine a number of selected big data basic platforms from a number of big data basic platforms according to the number of stock SDKs, wherein the selected big data basic platforms include a proxy server;

[0085] A second acquisition module 20 is used to acquire a plurality of key files from the external application layer, wherein the plurality of key files correspond to a plurality of proxy servers respectively;

[0086] The selection module 30 is used to obtain a first proxy service from a plurality of the proxy service terminals, obtain a second proxy service from a plurality of stock proxy modules, wherein the number of types of the second proxy service is greater than or equal to the number of types of the first proxy service, and determine a plurality of selected proxy modules from the plurality of stock proxy modules based on the types of the first proxy service and the second proxy service;

[0087] The selected module 30 includes:

[0088] a first unit, for different types of the second proxy services, comparing the type of each of the first proxy services with the type of each of the second proxy services;

[0089] a second unit, configured to determine the second proxy service as a designated proxy service if the type of the second proxy service is the same as the type of the first proxy service;

[0090] The third unit is used to determine the existing proxy modules corresponding to the designated proxy service as the selected proxy modules, and the number of the selected proxy modules is equal to the number of the types of the first proxy services.

[0091] A generating module 40, configured to generate a plurality of standby proxy modules through a plurality of selected proxy modules, update a plurality of the standby proxy modules to a plurality of docking proxy modules according to a plurality of the stock SDKs and a plurality of the key files, and connect a plurality of the docking proxy modules to a plurality of the proxy servers respectively according to the API information in the plurality of the stock SDKs;

[0092] The generating module 40 comprises:

[0093] A fourth unit, configured to divide the plurality of proxy servers into a plurality of server groups according to the type of the first proxy service, each of the server groups corresponding to one of the selected proxy modules;

[0094] A fifth unit, configured to replicate the selected proxy module corresponding to the server group according to the number of the proxy servers in the server group, so as to form a plurality of standby proxy modules;

[0095] A sixth unit is used to extract version numbers of several first proxy services corresponding to several proxy servers, several APIs and several tool components from several stock SDKs;

[0096] A seventh unit, configured to update the standby proxy module according to the version number, the API, the tool component and the key file to form the docking proxy module;

[0097] The eighth unit is used to perform key authentication according to the key file, so that the proxy server corresponding to the key file is in a state of allowing connection.

[0098] The request module 50 is used to receive the HTTP request sent by the external application layer, and call the docking agent module according to the HTTP request, so that the docking agent module extracts the big data interaction SDK from the selected big data basic platform through the proxy server;

[0099] The request module 50 comprises:

[0100] The ninth unit is used to extract the IP address, port and interaction data in the HTTP request;

[0101] The tenth unit is used to determine an interactive proxy module from a plurality of the docking proxy modules according to the IP address and the port, and transmit the interactive data to the interactive proxy module, wherein the interactive data is used to call the API of the proxy server and obtain the big data interactive SDK.

[0102] The return module 60 is used to convert the big data interaction SDK into a data compression package, and convert the data compression package into an HTTP response, and send the HTTP response to the external application layer for use by external applications in the external application layer.

[0103] The present invention also provides a computer, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the big data support method as described in the above technical solution is implemented.

[0104] The present invention also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the big data support method described in the above technical solution is implemented.

[0105] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0106] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A big data support method, characterized in that: The steps include: Acquire a number of stock SDKs from the external application layer, and determine a number of selected big data basic platforms from a number of big data basic platforms according to the number of stock SDKs, wherein the selected big data basic platforms include a proxy server; Acquire a plurality of key files from the external application layer, wherein the plurality of key files correspond to a plurality of proxy servers respectively; Acquire a first proxy service from a plurality of the proxy service terminals, acquire a second proxy service from a plurality of stock proxy modules, wherein the number of types of the second proxy services is greater than or equal to the number of types of the first proxy service, and determine a plurality of selected proxy modules from the plurality of the stock proxy modules based on the types of the first proxy service and the second proxy service; Generate a number of standby proxy modules through a number of the selected proxy modules, update the number of the standby proxy modules to a number of docking proxy modules according to a number of the stock SDKs and a number of the key files, and connect the number of the docking proxy modules to a number of the proxy servers respectively according to the API information in the number of the stock SDKs; The step of updating the plurality of standby proxy modules to a plurality of docking proxy modules according to the plurality of stock SDKs and the plurality of key files comprises: Extracting, from the plurality of stock SDKs, a plurality of version numbers of the first proxy services corresponding to the plurality of proxy servers, a plurality of APIs and a plurality of tool components; Updating the standby proxy module according to the version number, the API, the tool component and the key file to form the docking proxy module; Receiving an HTTP request sent by the external application layer, and calling the docking agent module according to the HTTP request, so that the docking agent module extracts a big data interaction SDK from the selected big data basic platform through the proxy server; The big data interaction SDK is converted into a data compression package, and the data compression package is converted into an HTTP response, and the HTTP response is sent to the external application layer for use by external applications in the external application layer.

2. The big data support method according to claim 1, characterized in that: The step of determining a plurality of selected proxy modules from a plurality of the existing proxy modules based on the types of the first proxy service and the second proxy service comprises: Different types of the second proxy services are all different, and the type of each of the first proxy services is compared with the type of each of the second proxy services; If the type of the second proxy service is the same as the type of the first proxy service, determining the second proxy service as a designated proxy service; The existing proxy modules corresponding to the designated proxy service are determined as the selected proxy modules, and the number of the selected proxy modules is equal to the number of the types of the first proxy services.

3. The big data support method according to claim 1, characterized in that: The step of generating a plurality of standby proxy modules by using a plurality of selected proxy modules comprises: Dividing the plurality of proxy servers into a plurality of server groups according to the type of the first proxy service, each of the server groups corresponding to one of the selected proxy modules; According to the number of the proxy servers in the server group, the selected proxy module corresponding to the server group is replicated to form a plurality of standby proxy modules.

4. The big data support method according to claim 1, characterized in that: Before the step of connecting the plurality of docking agent modules to the plurality of agent servers respectively according to the API information in the plurality of stock SDKs, there is also: Key authentication is performed according to the key file, so that the proxy server corresponding to the key file is in a state of allowing connection.

5. The big data support method according to claim 1, characterized in that: The step of receiving the HTTP request sent by the external application layer, calling the docking agent module according to the HTTP request, and allowing the docking agent module to extract the big data interaction SDK from the selected big data basic platform through the proxy server also includes: Extract the IP address, port and interaction data in the HTTP request; An interactive proxy module is determined from a plurality of the docking proxy modules according to the IP address and the port, and the interactive data is transmitted to the interactive proxy module, where the interactive data is used to call the API of the proxy server and obtain the big data interactive SDK.

6. A big data support system, applied to the big data support method according to any one of claims 1 to 5, characterized in that: The system comprises: A first acquisition module is used to acquire a number of stock SDKs from an external application layer, and determine a number of selected big data basic platforms from a number of big data basic platforms according to the number of stock SDKs, wherein the selected big data basic platforms include a proxy server; A second acquisition module is used to acquire a plurality of key files from the external application layer, wherein the plurality of key files correspond to a plurality of proxy servers respectively; a selection module, configured to obtain a first proxy service from a plurality of the proxy service terminals, obtain a second proxy service from a plurality of stock proxy modules, wherein the number of types of the second proxy services is greater than or equal to the number of types of the first proxy services, and determine a plurality of selected proxy modules from the plurality of stock proxy modules based on the types of the first proxy services and the second proxy services; A generating module, used to generate a plurality of standby proxy modules through a plurality of selected proxy modules, update a plurality of the standby proxy modules to a plurality of docking proxy modules according to a plurality of the stock SDKs and a plurality of the key files, and connect a plurality of the docking proxy modules to a plurality of the proxy service terminals respectively according to the API information in the plurality of the stock SDKs; The generation module comprises: A sixth unit is used to extract version numbers of several first proxy services corresponding to several proxy servers, several APIs and several tool components from several stock SDKs; A seventh unit, configured to update the standby proxy module according to the version number, the API, the tool component and the key file to form the docking proxy module; A request module, used for receiving the HTTP request sent by the external application layer, and calling the docking agent module according to the HTTP request, so that the docking agent module extracts the big data interaction SDK from the selected big data basic platform through the proxy server; The return module is used to convert the big data interaction SDK into a data compression package, and convert the data compression package into an HTTP response, and send the HTTP response to the external application layer for use by external applications in the external application layer.

7. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the big data support method according to any one of claims 1 to 5 is implemented.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the big data support method according to any one of claims 1 to 5 is implemented.

Images