Method for realizing interaction between internal and external networks through SQL proxy

Abstract

The invention discloses a method for realizing interaction between an internal network and an external network through an SQL (Structured Query Language) proxy, which comprises the following steps of: receiving request data of an external requester by a front-end processor, judging whether the request data needs to be fragmented, and if the request data needs to be fragmented, determining the fragmentation size according to a fragmentation algorithm and then fragmenting a request body; compressing the fragments, and forwarding metadata of the request data and a request body after fragment compression to a database through an SQL security isolation device; the method comprises the following steps of: reading request data from a database in batches at fixed time by a rear computer, decompressing and recombining the read fragmented request body, sending the recombined request body and the read metadata to a business system service, waiting for response information of the business system service, and writing the response information into the database; and the front-end processor polls the database to read the response information and returns the response information to the external requester. The technical problem that large file transmission cannot be carried out in existing internal and external network interaction is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a method for realizing intranet and extranet interaction through an SQL agent. Background Art

[0002] Most current office systems operate in intranet environments, isolated from the internet, ensuring data security. However, some production personnel lack access to the intranet, and many work outdoors, making mobile access to the intranet inconvenient. Currently, simple data exchange between intranets and intranets is achieved through database connections provided by secure isolation devices, but this does not allow external networks to directly access intranet software system resources. There are two main approaches currently available:

[0003] (1) Deploying a completely identical system in an external network environment has obvious disadvantages. The security of software deployed in the external network cannot be guaranteed. It is also not conducive to software maintenance, resulting in a waste of server resources and a significant increase in operating and maintenance costs.

[0004] (2) Figure 1 As shown, the request forwarding interaction is realized through the security isolation device of the SQL proxy. The basic implementation principle is to forward the user request of the external network to the intranet based on the SQL proxy provided by the security isolation device, so as to realize the purpose of the external network securely accessing the intranet system resources. This solution is completed by the cooperation of the front-end proxy service and the back-end proxy service. The user requests to the front-end proxy service, and the front-end proxy service writes the request content to the database. The back-end proxy service obtains the request content from the database and forwards it to the actual intranet system service, and writes the return value to the database again. The front-end machine obtains the response data and returns it to the user after processing, completing the HTTP / HTTPS link request. When using this solution for intranet and intranet data transmission, the large file transmission process will be restricted by the security isolation device, resulting in abnormal situations such as transmission failure or service crash. It is not suitable for large-scale content transmission. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for realizing intranet and extranet interaction through SQL proxy, so as to solve the technical problem that large files cannot be transmitted in the existing intranet and extranet interaction.

[0006] In one aspect, the present invention provides a method for implementing intranet and extranet interaction through SQL proxy, comprising the following steps:

[0007] The front-end receives the request data from the external requester and determines whether the request data needs to be fragmented. If fragmentation is required, it determines the fragment size according to the fragmentation algorithm and then fragments the request body. It compresses the fragments and forwards the request data metadata and the compressed request body to the database through the SQL security isolation device.

[0008] The backend reads request data from the database in batches at regular intervals. The request data includes the request body and metadata. The backend decompresses and reassembles the fragmented request body, sends the reassembled data along with the read metadata to the business system service, waits for the response information from the business system service, and writes the response information to the database.

[0009] The front-end processor polls the database to read the response information and returns the response information to the external requester.

[0010] In some embodiments, determining whether the requested data needs to be fragmented specifically includes:

[0011] If the request data does not contain a request body, no fragmentation is required;

[0012] If the request data includes a request body and the size of the request body is less than the remaining bandwidth after conversion to bit units, no fragmentation is required. If the size of the request body is greater than the remaining bandwidth after conversion to bit units, and the estimated time required for fragmented compression and transmission is less than the time required for direct transmission, fragmentation is performed. If the size of the request body is greater than the remaining bandwidth after conversion to bit units, and the estimated time required for fragmented compression and transmission is greater than the time required for direct transmission, whether the size of the request body data is greater than the network throughput of the SQL security isolation device is further determined. If so, fragmentation is performed; if not, fragmentation is not performed.

[0013] In some embodiments, estimating the time required for compressed transmission of fragments specifically includes:

[0014] Obtain the remaining system resources, including remaining CPU resources, remaining bandwidth resources, and remaining IO resources;

[0015] Calculate the number of fragments N1 that can be processed by the remaining CPU resources, calculate the number of fragments N2 that can be processed by the remaining I / O resources, obtain the number of logical cores N3 of the system, and select the smallest number among N1, N2, and N3 as the estimated number of fragments N;

[0016] Based on the request body size and the estimated number of shards, the estimated shard segment size is obtained, and the time T1 required for a single logical core to compress the estimated shard segment is calculated;

[0017] Obtain a compression ratio according to the compressed request body data type and a pre-stored compression mapping table, and obtain a compressed data size S' after the request body is compressed according to the compression ratio;

[0018] The time taken to transmit the request body in fragments and compression is calculated using the following formula:

[0019]

[0020] Where X' is the compressed data size, RM is the remaining bandwidth resource, and T1 is the time required for a single logical core to compress an estimated fragment.

[0021] In some embodiments, the calculation of the number N1 of fragments that can be processed by the remaining CPU resources specifically includes: calculating the resource ratio CC required for the CPU to process the request body, using the following formula:

[0022] Where S is the request body size, s1 is the shard unit size, which is the smallest data unit of a shard fragment and is a constant. Z0 is the CPU resource ratio consumed by the CPU to process one shard unit, obtained from testing under a standard hardware environment. Z0 × K1 is the CPU resource ratio consumed by the current system hardware environment to process one shard unit. K1 is the adjustment weight, which is determined and adjusted based on the ratio of the number of CPU cores in the current system to the number of CPU cores in the standard hardware environment.

[0023] According to the remaining CPU resource ratio RC and the resource ratio CC consumed by the CPU to process the request body, the number of shards N1 is obtained, N1 = RC / CC.

[0024] In some embodiments, the calculation of the number of fragments N2 that can be processed by the remaining IO resources specifically includes: calculating the proportion of resources C0 consumed by the IO processing request body, using the formula:

[0025] Where S is the request body size, s1 is the fragment unit size, which is the minimum data unit of the fragment segment and is a constant, and B O The ratio of IO resources consumed by a shard unit in IO processing obtained by testing under a standard hardware environment, B O ×K2 is the ratio of IO resources consumed by the current system hardware environment to process a shard unit, and K2 is the adjustment weight.

[0026] According to the IO remaining resource ratio RO and the resource ratio CO required to process the IO request body, the number of shards N2 is obtained, N2 = RO / CO.

[0027] In some embodiments, the sharding algorithm specifically includes:

[0028] Monitor the packet loss rate and network latency of the front-end processor in real time to obtain time series data;

[0029] Fault detection of time series data through high-dimensional topological data analysis (TDA);

[0030] The estimated fragment size and the fault detection result EXC are combined to obtain the final fragment size, S FINAL=S0-α2tanh(EXC), where S0 is the estimated fragment size and α2 is set according to the maximum fragment adjustment allowed by the business.

[0031] In some embodiments, further comprising:

[0032] After receiving the request data, the front-end extracts the request URL in the request data, searches for the target URL from the routing mapping table stored in the cache based on the request URL, and adds the target URL to the metadata of the request data. The back-end sends the request data to the business system service based on the target URL.

[0033] In some embodiments, the database includes a request table and a shard table, the request table is used to store metadata of the request data, and the shard table is used to store the request body of the request data.

[0034] In some embodiments, before the metadata of the request data is forwarded to the database through the SQL security isolation device, the metadata needs to be encrypted. The specific encryption method is:

[0035] Randomly split metadata into several groups of data segments. Each group is split differently, but each group of data segments can be restored to the original data.

[0036] Randomly select a subset as the target subset and the other subsets as non-target subsets;

[0037] Randomly splice the data segments of the non-target subset, split them again, and add random values ​​at the beginning and end to increase confusion;

[0038] Assign values ​​to the beginning and end of the data segments of the target subset according to the preset comparison table, merge all the data segments of the target subset and non-target subset, and process them in a random order;

[0039] Encrypt the merged full set of data segments, encrypt each data segment, and forward the encrypted data segments to the database through the SQL security isolation device, so that the back-end machine uses the same key to decrypt the data segments read from the database, restore the continuation chain according to the comparison table, and select the longest continuation chain as the real data to obtain metadata.

[0040] The beneficial effects of the present invention are as follows:

[0041] The front-end processor of the present invention determines whether the requested data needs to be fragmented, and then fragments the requested data according to a custom fragmentation algorithm and compresses it for transmission. This solves the problems of large single file transmission causing high pressure on system memory, increased database load, and slow system response speed, thereby further improving the stability of the system architecture.

[0042] By comprehensively considering bandwidth, SQL security isolation device network throughput, transmission time, and data size, the necessity of sharding is analyzed from multiple perspectives, rather than just from the perspective of data size. This can achieve multiple beneficial effects such as adapting to different network environments, optimizing resource utilization, reducing the risk of transmission failure, and improving transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 This is a diagram of an existing method for achieving intranet and extranet interaction through SQL proxy;

[0045] Figure 2 A flow chart of a method for implementing intranet and extranet interaction through SQL proxy provided by one embodiment of the present invention; DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0047] Please refer to Figure 2 The embodiment of the present invention provides a method for implementing intranet and extranet interaction through SQL proxy, comprising the following steps:

[0048] A method for implementing intranet and extranet interaction through SQL proxy includes the following steps:

[0049] S1. The front-end receives the request data from the external requester and determines whether the request data needs to be fragmented. If fragmentation is required, it determines the fragment size according to the fragmentation algorithm and fragments the request body. It compresses the fragments and forwards the request data metadata and the compressed request body to the database through the SQL security isolation device.

[0050] S2. The backend reads request data from the database in batches at regular intervals. The request data includes a request body and metadata. The backend decompresses and reassembles the fragmented request body, sends the reassembled data along with the read metadata to the business system service, waits for a response from the business system service, and writes the response to the database.

[0051] S3. The front-end processor polls the database to read the response information and returns the response information to the external requester.

[0052] The front-end processor of the present invention determines whether the requested data needs to be fragmented by setting a fragmentation unit, and fragments the requested data that needs to be fragmented according to a customized fragmentation algorithm, so as to solve the problems of excessive pressure on system memory, increased database load, and slow system response speed caused by excessive single file transmission, and the stability of the system architecture is further improved.

[0053] In some embodiments, determining whether the requested data needs to be fragmented specifically includes:

[0054] If the request data does not contain a request body, no fragmentation is required;

[0055] If the request data includes a request body and the size of the request body is less than the remaining bandwidth after conversion to bit units, no fragmentation is required. If the size of the request body is greater than the remaining bandwidth after conversion to bit units, and the estimated time required for fragmented compression and transmission is less than the time required for direct transmission, fragmentation is performed. If the size of the request body is greater than the remaining bandwidth after conversion to bit units, and the estimated time required for fragmented compression and transmission is greater than the time required for direct transmission, whether the size of the request body data is greater than the network throughput of the SQL security isolation device is further determined. If so, fragmentation is performed; if not, fragmentation is not performed.

[0056] By comprehensively considering the remaining bandwidth, the network throughput of the SQL security isolation device, the transmission time and the data size, the necessity of sharding is analyzed from multiple perspectives, rather than just from the perspective of data size. This can achieve multiple beneficial effects such as adapting to different network environments, optimizing resource utilization, reducing the risk of transmission failure and improving transmission efficiency.

[0057] In some embodiments, estimating the time required for compressed transmission of fragments specifically includes:

[0058] Obtain the remaining system resources, including remaining CPU resources, remaining bandwidth resources, and remaining IO resources;

[0059] By obtaining the current CPU usage, bandwidth usage, and I / O usage, you can infer the remaining CPU resources, bandwidth resources, and I / O resources. You can use the remaining CPU resource ratio and I / O resource ratio to express the corresponding remaining resource status.

[0060] Calculate the number of fragments N1 that can be processed by the remaining CPU resources, calculate the number of fragments N2 that can be processed by the remaining I / O resources, obtain the number of logical cores N3 of the system, and select the smallest number among N1, N2, and N3 as the estimated number of fragments N;

[0061] The CPU has hyperthreading technology and virtual thread technology. One core can be used as two cores, so there are 6 cores and 12 threads or 8 cores and 16 threads. Hyperthreading technology and virtual thread technology can turn 6 cores into 12 logical cores. This embodiment considers the number of logical cores of the system.

[0062] In some embodiments, the calculation of the number N1 of fragments that can be processed by the remaining CPU resources specifically includes: calculating the resource ratio CC required for the CPU to process the request body, using the following formula:

[0063] Where S is the request body size, s1 is the shard unit size, which is the smallest data unit of a shard fragment and is a constant. Z0 is the CPU resource ratio consumed by the CPU to process one shard unit, obtained from testing under a standard hardware environment. Z0 × K1 is the CPU resource ratio consumed by the current system hardware environment to process one shard unit. K1 is the adjustment weight, which is determined and adjusted based on the ratio of the number of CPU cores in the current system to the number of CPU cores in the standard hardware environment.

[0064] According to the remaining CPU resource ratio RC and the resource ratio CC consumed by the CPU to process the request body, the number of shards N1 is obtained, N1 = RC / CC.

[0065] In some embodiments, the calculation of the number of fragments N2 that can be processed by the remaining IO resources specifically includes: calculating the proportion of resources C0 consumed by the IO processing request body, using the formula:

[0066] Where S is the request body size, s1 is the fragment unit size, which is the minimum data unit of the fragment segment and is a constant. For example, it can be set to 100K. O The ratio of IO resources consumed by a shard unit in IO processing obtained by testing under a standard hardware environment, B O ×K2 is the ratio of IO resources consumed by the current system hardware environment to process a shard unit, and K2 is the adjustment weight;

[0067] According to the remaining IO resource ratio RO and the resource ratio CO required to process the IO request body, the number of shards N2 is obtained, N2 = RO / CO;

[0068] Based on the request body size and the estimated number of shards, the estimated shard segment size is obtained, and the time T1 required for a single logical core to compress the estimated shard segment is calculated;

[0069] Estimated shard segment size Where N is the estimated number of shards and S is the request body size.

[0070] The present invention compresses an estimated slice segment for each logical core respectively, obtains the time required for a logical core to compress a slice unit size through testing, and then combines the estimated slice segment size to linearly obtain the time T1 required for the current system to compress an estimated slice segment.

[0071] Obtain a compression ratio according to the compressed request body data type and a pre-stored compression mapping table, and obtain a compressed data size S' after the request body is compressed according to the compression ratio;

[0072] The pre-stored compression mapping table can find the compression ratio according to the request body data type and the compression algorithm used.

[0073] The time taken to compress and transmit the request body fragments T2 is calculated using the following formula:

[0074]

[0075] Where X' is the compressed data size, RM is the remaining bandwidth resource, and T1 is the time required to compress an estimated fragment segment.

[0076] The time T3 used to directly transmit the request body is calculated by dividing the request body size by the remaining bandwidth.

[0077] In some embodiments, the sharding algorithm specifically includes:

[0078] Monitor the packet loss rate and network latency of the front-end processor in real time to obtain time series data;

[0079] Fault detection of time series data through high-dimensional topological data analysis (TDA);

[0080] When TAS detects an anomaly, the estimated fragment size and the fault detection result EXC are combined to obtain the final fragment size, S FINAL = S0 - α2tanh(EXC), where S0 is the estimated fragment size and α2 is set based on the maximum fragment adjustment allowed by the service. The front-end processor fragments the request body data based on the final fragment size.

[0081] Fault detection of time series data is performed through high-dimensional topological data analysis (TDA), specifically including:

[0082] First, collect the multi-dimensional time series data of the front-end, which is recorded as: Where X represents the time series data set collected; N4 represents the total number of time series data collected; each x(t a ) indicates that at time t a The measured 2D state vector includes packet loss rate and network delay.

[0083] Perform Z-Score normalization on the original time series data to eliminate the dimensional differences between packet loss rate and delay;

[0084] According to Takens embedding theory, the standardized time series data is constructed into an embedded dataset by selecting the embedding dimension m and the delay time τ, and the dimension of each embedding point is 2m:

[0085] X E ={(x(t f ),x(t f +τ),…,x(t f +(m-1)τ))|f=1,…,N′}

[0086] Where N′ is the number of embedded points, N′=N4-(m-1)τ;

[0087] By computing the embedding dataset X E The Euclidean distance between the embedded points is used to construct a distance matrix;

[0088] Based on the distance matrix, the Vietoris-Rips method is used to establish a simplex complex. Its construction process depends on a filtering parameter ∈. During the filtering process, when the distance between any two embedded points is less than or equal to ∈, they are connected into edges to construct a higher-dimensional simplex. On this basis, by changing the value range of ∈, a series of complexes are obtained, thus constructing a filtering process. The persistent homology of the filtering process is calculated, and the persistence graph D under different homology dimensions k is obtained. k , where each point in the figure (b k,o ,d k,o ) represent the scales at which the oth topological feature appears and disappears during the filtering process; the persistence of each topological feature is defined as:

[0089] p k,o =d k,o -b k,o

[0090] The persistence of all features is counted to obtain a global persistence index:

[0091]

[0092] Among them, n k represents the number of features under dimension k, where K is the highest homology dimension considered; dk,o with b k,o denote the birth and disappearance scales of the oth topological feature respectively; P denotes the global persistence index; p k,o Persistence indicators of characteristics;

[0093] In order to compare the topological characteristics of the current state with the preset normal state, a reference persistence graph D is constructed. ref , and adopt the bottleneck distance d B Calculate the current persistence graph D1 and D ref The difference between , which is defined as:

[0094]

[0095] Among them, γ1 represents the difference between D1 and D ref All possible bijections between ||·|| ∞ represents the infinite norm, d B (D1,D ref ) represents the bottleneck distance between the current persistence graph and the reference persistence graph; x represents a point in the persistence graph D1;

[0096] To further quantify the geometric complexity of the state trajectory, the topological anomaly score (TAS) is defined as:

[0097]

[0098] in, α1 and β1 are weight coefficients determined by calibration of historical data;

[0099] Calculate the TAS mean μ of historical normal data 正常 and standard deviation σ 正常 , if the current TAS satisfies TAS-μ 正常 >λσ 正常 or d B (D1,D ref ) exceeds the preset threshold ∈ th , it is determined to be abnormal, and an abnormality detection result EXC is generated, where EXC is a scalar value;

[0100] When an anomaly is detected by TAS, the scalar value EXC of the TDA anomaly detection result is converted into the fragment size adjustment value ΔS through the nonlinear mapping function g(·). The nonlinear mapping function g(·) can be designed as a hyperbolic tangent function (tanh). The final fragment size calculation formula is:

[0101] S FINAL =S0-α2ΔS, where S0 is the estimated fragment size and α2 is set according to the maximum fragment adjustment allowed by the business.

[0102] The fragmentation algorithm of the present invention obtains fragmentation estimates through system resources. Combined with the TDA detection results of packet loss rate and network delay, it triggers dynamic adjustment of fragmentation segments when anomalies are detected to maximize resource utilization and adapt to different network environments. At the same time, the hyperbolic tangent function (tanh) is used as the nonlinear mapping function g(·), and its output range is limited to (-1, 1). This ensures that the fragmentation adjustment amount ΔS changes smoothly and avoids transmission jitter caused by sudden changes in fragment size.

[0103] In some embodiments, the present invention further comprises:

[0104] After receiving the request data, the front-end extracts the request URL in the request data, searches for the target URL from the routing mapping table stored in the cache based on the request URL, and adds the target URL to the metadata of the request data. The back-end sends the request data to the business system service based on the target URL.

[0105] In some embodiments, the database includes a request table and a shard table, the request table is used to store metadata of the request data, and the shard table is used to store the request body of the request data.

[0106] In some embodiments, before the metadata of the request data is forwarded to the database through the SQL security isolation device, the metadata needs to be encrypted. The specific encryption method is:

[0107] Randomly split metadata into several groups of data segments. Each group is split differently, but each group of data segments can be restored to the original data.

[0108] Randomly select a subset as the target subset and the other subsets as non-target subsets;

[0109] Randomly splice the data segments of the non-target subset, split them again, and add random values ​​at the beginning and end to increase confusion;

[0110] Assign values ​​to the beginning and end of the data segments of the target subset according to the preset comparison table, merge all the data segments of the target subset and non-target subset, and process them in a random order;

[0111] Encrypt the merged full set of data segments, encrypt each data segment, and forward the encrypted data segments to the database through the SQL security isolation device, so that the back-end machine uses the same key to decrypt the data segments read from the database, restore the continuation chain according to the comparison table, and select the longest continuation chain as the real data to obtain metadata.

[0112] The backend is designed with the same sharding unit as the frontend, determining whether to shard the message and determining the fragment size using a custom sharding algorithm. This will not be detailed here. For responses that require sharding, the backend segments them according to the sharding algorithm, compresses them, and writes them to the database. The frontend reads the response, decompresses it, concatenates it, and sends it to the external requester.

[0113] The above describes in detail the method provided by the present invention for implementing intranet and extranet interaction through a SQL proxy. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above examples is intended only to facilitate understanding of the method and core concepts of the present invention. It should be noted that those skilled in the art may make various improvements and modifications to the present invention without departing from the principles of the present invention, and such improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0114] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiments.

Claims

1. A method for realizing intranet and extranet interaction through SQL proxy, characterized in that: The steps include: The front-end receives the request data from the external requester and determines whether the request data needs to be fragmented. If fragmentation is required, it determines the fragment size according to the fragmentation algorithm and then fragments the request body. It compresses the fragments and forwards the request data metadata and the compressed request body to the database through the SQL security isolation device. The backend reads request data from the database in batches at regular intervals. The request data includes the request body and metadata. The backend decompresses and reassembles the fragmented request body, sends the reassembled data along with the read metadata to the business system service, waits for the response information from the business system service, and writes the response information to the database. The front-end processor polls the database to read the response information and returns the response information to the external requester.

2. The method for realizing intranet and extranet interaction through SQL proxy according to claim 1, characterized in that: Determining whether the requested data needs to be fragmented specifically includes: If the request data does not contain a request body, no fragmentation is required; If the request data includes a request body and the size of the request body is smaller than the remaining bandwidth after conversion to bit units, fragmentation is not required. If the size of the request body is larger than the remaining bandwidth after conversion to bit units, and the estimated time required for fragmented compression and transmission is smaller than the time required for direct transmission, fragmentation is performed. If the size of the request body is larger than the remaining bandwidth after conversion to bit units, and the estimated time required for fragmented compression and transmission is larger than the time required for direct transmission, whether the size of the request body data is larger than the network throughput of the SQL security isolation device is further determined. If so, fragmentation is performed; if not, fragmentation is not performed.

3. The method for realizing intranet and extranet interaction through SQL proxy according to claim 2, characterized in that: Estimate the time required for fragment compression and transmission, including: Obtain the remaining system resources, including remaining CPU resources, remaining bandwidth resources, and remaining IO resources; Calculate the number of fragments N1 that can be processed by the remaining CPU resources, calculate the number of fragments N2 that can be processed by the remaining I / O resources, obtain the number of logical cores N3 of the system, and select the smallest number among N1, N2, and N3 as the estimated number of fragments N; Based on the request body size and the estimated number of shards, the estimated shard segment size is obtained, and the time T1 required for a single logical core to compress the estimated shard segment is calculated; Obtain a compression ratio according to the compressed request body data type and a pre-stored compression mapping table, and obtain a compressed data size S' after the request body is compressed according to the compression ratio; The time taken to transmit the request body in fragments and compression is calculated using the following formula: Where X' is the compressed data size, RM is the remaining bandwidth resource, and T1 is the time required for a single logical core to compress an estimated fragment.

4. The method for realizing intranet and extranet interaction through SQL agent according to claim 3, characterized in that: The calculation of the number of fragments N1 that can be processed by the remaining CPU resources specifically includes: calculating the resource ratio CC required for the CPU to process the request body, and the formula is as follows: Where S is the request body size, s1 is the shard unit size, which is the smallest data unit of a shard fragment and is a constant. Z0 is the CPU resource ratio consumed by the CPU to process one shard unit, obtained from testing under a standard hardware environment. Z0 × K1 is the CPU resource ratio consumed by the current system hardware environment to process one shard unit. K1 is the adjustment weight, which is determined and adjusted based on the ratio of the number of CPU cores in the current system to the number of CPU cores in the standard hardware environment. According to the remaining CPU resource ratio RC and the resource ratio CC consumed by the CPU to process the request body, the number of shards N1 is obtained, N1 = RC / CC.

5. The method for realizing intranet and extranet interaction through SQL agent according to claim 3, characterized in that: The calculation of the number of fragments N2 that can be processed by the remaining IO resources specifically includes: calculating the resource ratio C0 required for IO processing request body, the formula is: Where S is the request body size, s1 is the fragment unit size, which is the minimum data unit of the fragment segment and is a constant, and B O The ratio of IO resources consumed by a shard unit in IO processing obtained by testing under a standard hardware environment, B O ×K2 is the ratio of IO resources consumed by the current system hardware environment to process a shard unit, and K2 is the adjustment weight. According to the IO remaining resource ratio RO and the resource ratio CO required to process the IO request body, the number of shards N2 is obtained, N2 = RO / CO.

6. The method for realizing intranet and extranet interaction through SQL agent according to claim 3, characterized in that: The sharding algorithm specifically includes: Monitor the packet loss rate and network latency of the front-end processor in real time to obtain time series data; Fault detection of time series data through high-dimensional topological data analysis (TDA); The estimated fragment size and the fault detection result EXC are combined to obtain the final fragment size, S FINAL =S0-α2tanh(EXC), where S0 is the estimated fragment size and α2 is set according to the maximum fragment adjustment allowed by the business.

7. The method for realizing intranet and extranet interaction through SQL agent according to claim 1, characterized in that: Also includes: After receiving the request data, the front-end extracts the request URL in the request data, searches for the target URL from the routing mapping table stored in the cache based on the request URL, and adds the target URL to the metadata of the request data. The back-end sends the request data to the business system service based on the target URL.

8. The method for realizing intranet and extranet interaction through SQL agent according to claim 1, characterized in that: The database includes a request table and a shard table. The request table is used to store metadata of request data, and the shard table is used to store a request body of the request data.

9. The method for realizing intranet and extranet interaction through SQL agent according to claim 1, characterized in that: Before forwarding the metadata of the requested data to the database through the SQL security isolation device, the metadata needs to be encrypted. The specific encryption method is as follows: Randomly split metadata into several groups of data segments. Each group is split differently, but each group of data segments can be restored to the original data. Randomly select a subset as the target subset and the other subsets as non-target subsets; Randomly splice the data segments of the non-target subset, split them again, and add random values ​​at the beginning and end to increase confusion; Assign values ​​to the beginning and end of the data segments of the target subset according to the preset comparison table, merge all the data segments of the target subset and non-target subset, and process them in a random order; Encrypt the merged full set of data segments, encrypt each data segment, and forward the encrypted data segments to the database through the SQL security isolation device, so that the back-end machine uses the same key to decrypt the data segments read from the database, restore the continuation chain according to the comparison table, and select the longest continuation chain as the real data to obtain metadata.

Images