An optimization processing method, device and equipment for high-concurrency data requests, and a medium

Abstract

The application belongs to the technical field of optimizing network requests, and specifically provides a high-concurrency data request optimization processing method, device, equipment and medium, the method comprises the following steps: sending an asynchronous network request, setting token and a timestamp in the interface Body of a request test question; after the data request is successful, obtaining Json data conversion and writing into a local database; when data is loaded, judging whether the test question id exists in the local cache; when the test question id exists, loading data from the local database; when the test question id does not exist, executing the following steps: setting token and a timestamp in the interface Body of a request test question when sending network request data. The mobile terminal is more smooth in user experience, and is not disturbed by the lag and crash phenomenon caused by network instantaneous request and data loading, and the running efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of network request optimization technology, and specifically to an optimization processing method, apparatus, device, and medium for high-concurrency data requests. Background Technology

[0002] While mobile training has brought convenience to both on-site training and mobile learning, some problems have also emerged over time. Because the learning center contains a large amount of exam question data, while improving the user experience, it has also increased the client's workload, leading to a surge in instantaneous requests and causing some instances of lag and stuttering.

[0003] The existing methods for requesting and loading exam data on mobile training platforms have the following drawbacks: First, too many requests or too frequent requests can cause transmission interruptions due to network issues; Second, when re-entering the exam (mock exam) midway through, data will be requested again, making it impossible to guarantee fast and efficient loading and display; Third, when there are too many questions, the interface loading becomes sluggish, and data conversion and display are slow, affecting the user experience; Fourth, when loading questions, performing multiple question transcoding and loading operations simultaneously on the main thread can cause rapid memory growth and a sudden increase in CPU load.

[0004] In summary, existing technologies cannot meet the requirements for requesting, transcoding, and loading exam questions. Therefore, it is essential to provide an optimized method, apparatus, device, or medium for handling high-concurrency data requests, addressing the aforementioned shortcomings of existing technologies. Summary of the Invention

[0005] Existing technologies cannot meet the needs of requesting, transcoding, and loading exam questions. In addition, considering the issues of internal and external network data transmission in mobile training, this invention aims to make the mobile user experience smoother, eliminating the problems of lag and crashes caused by instantaneous network requests and data loading, improving operating efficiency, directly accelerating the release of mobile phone memory, and maximizing the efficiency of network requests and data loading to meet business scenario requirements. To achieve this goal, this invention provides an optimized processing method, apparatus, device, and medium for high-concurrency data requests.

[0006] In a first aspect, the present invention provides an optimized processing method for high-concurrency data requests, comprising the following steps:

[0007] Send an asynchronous network request, setting the token and timestamp in the interface body that requests the test questions;

[0008] After a successful data request, the obtained JSON data is encoded and written to the local database.

[0009] When loading data, check if the question ID exists in the local cache;

[0010] If the question ID exists, load the data from the local database;

[0011] When the question ID does not exist, the following steps are executed: When sending network request data, set the token and timestamp in the interface body that requests the question.

[0012] Network data transmission is prone to sudden surges in requests for large amounts of data, which can lead to time-consuming and resource-intensive client operations and server responses, impacting user experience and potentially causing client memory overflow and crashes in severe cases. This invention solves the thread blocking problem caused by the instantaneous transmission and loading of large amounts of data, reducing client memory usage and server response time, thereby significantly improving the client's resilience against high concurrency and enhancing the user experience.

[0013] As a further limitation of the technical solution of the present invention, after the data request is successful, the steps of obtaining the JSON data, transcoding it, and writing it into the local database include:

[0014] After the data request is successful, the JSON data is obtained and transcoded.

[0015] Determine if the local database exists;

[0016] If so, the obtained JSON data will be transcoded and written to the local database;

[0017] If not, create a local database and perform the following steps: transcode the obtained JSON data and write it to the local database.

[0018] As a further limitation of the technical solution of the present invention, the step of loading data from a local database includes:

[0019] Retrieve cached data from the local cache pool, obtain the data corresponding to different question IDs, retrieve the chunk storage block corresponding to each table, convert it into binary JSON data, assign values ​​according to the unique identifier UUID of each question, and then display it on the page.

[0020] As a further limitation of the technical solution of the present invention, before the step of setting the token and timestamp in the interface body of the request for test questions, the following is included:

[0021] Different databases are set up for data storage based on different request types;

[0022] Configure multiple servers for data storage.

[0023] As a further limitation of the technical solution of the present invention, the method also includes:

[0024] Assign a Go goroutine as the execution body for each request.

[0025] Secondly, the present invention also provides an optimized processing device for high-concurrency data requests, including a first processing module, a cache module, a first judgment module, and a loading module;

[0026] The first processing module is used to send asynchronous network requests, setting the token and timestamp in the interface body that requests the test questions;

[0027] The caching module is used to obtain JSON data, transcode it, and write it to the local database after a successful data request.

[0028] The second judgment module is used to determine whether the question ID exists in the local cache when data is loaded;

[0029] The loading module is used to load data from the local database when the question ID exists.

[0030] It solves the thread blocking problem caused by the instantaneous transmission and loading of large amounts of data, reduces the utilization of client running memory and the response time of server, thereby greatly improving the client's resistance to high concurrency and enhancing the user experience.

[0031] As a further limitation of the technical solution of the present invention, the device also includes a second judgment module and a database creation module;

[0032] The second judgment module is used to determine whether the local database exists;

[0033] The database creation module is used to create a local database if the second judgment module determines that the local database does not exist.

[0034] As a further limitation of the technical solution of the present invention, the loading module is specifically used to retrieve cached data from the local cache pool, obtain the data corresponding to different question IDs, retrieve the chunk storage block corresponding to each table, convert it into binary JSON data, assign values ​​according to the unique identifier UUID of each question, and then display it on the page.

[0035] As a further limitation of the technical solution of the present invention, the device also includes a preprocessing module, which is used to set up different databases for data storage according to different request types; and to set up multiple servers for data storage.

[0036] As a further limitation of the technical solution of the present invention, the device also includes a request setting module, which is used to allocate a Go coroutine as an executor for each request.

[0037] Thirdly, the present invention also provides an electronic device, the electronic device comprising:

[0038] At least one processor; and,

[0039] A memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores computer program instructions that can be executed by at least one processor to enable the at least one processor to perform the optimized processing method for high-concurrency data requests as described in the first aspect.

[0041] Fourthly, the present invention also provides a non-transitory computer-readable storage medium that stores computer instructions that cause the computer to execute the optimized processing method for high-concurrency data requests as described in the first aspect.

[0042] As can be seen from the above technical solutions, the present invention has the following advantages: it makes the mobile terminal user experience smoother, without being troubled by lag and crashes caused by instantaneous network requests and data loading, improves operating efficiency, directly accelerates the release process of mobile phone running memory, maximizes the efficiency of network requests and data loading, and meets the needs of business scenarios.

[0043] Furthermore, the design principle of this invention is reliable, the structure is simple, and it has a very wide range of application prospects.

[0044] Therefore, it is evident that the present invention has outstanding substantive features and significant progress compared with the prior art, and the beneficial effects of its implementation are also obvious. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0047] Figure 2 This is a schematic block diagram of an apparatus according to an embodiment of the present invention. Detailed Implementation

[0048] The existing methods for requesting and loading exam data on mobile training platforms have the following drawbacks: First, too many requests or too frequent requests can cause transmission interruptions due to network issues; Second, when re-entering the exam (mock exam) midway through, data will be requested again, making it impossible to guarantee fast and efficient loading and display; Third, when there are too many questions, the interface loading becomes sluggish, and data conversion and display are slow, affecting the user experience; Fourth, when loading questions, performing multiple question transcoding and loading operations simultaneously on the main thread can cause rapid memory growth and a sudden increase in CPU load.

[0049] In summary, existing technologies cannot meet the requirements for requesting, transcoding, and loading exam questions. This invention provides a smoother user experience on mobile devices, eliminating the stuttering and crashes caused by instantaneous network requests and data loading. It improves operational efficiency, directly accelerates the release of mobile phone RAM, and maximizes the efficiency of network requests and data loading to meet business scenario needs. To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0050] like Figure 1 As shown, this embodiment of the invention provides an optimized processing method for high-concurrency data requests, including the following steps:

[0051] Step 1: Send an asynchronous network request, setting the token and timestamp in the interface body that requests the test questions;

[0052] Step 2: After the data request is successful, the obtained JSON data is transcoded and written to the local database;

[0053] Step 3: When loading data, check if the question ID exists in the local cache;

[0054] If the question ID exists, proceed to step 4; if the question ID does not exist, proceed to step 1.

[0055] Step 4: Load data from the local database;

[0056] Network data transmission is prone to sudden surges in requests for large amounts of data, which can lead to time-consuming and resource-intensive client operations and server responses, impacting user experience and potentially causing client memory overflow and crashes in severe cases. This invention solves the thread blocking problem caused by the instantaneous transmission and loading of large amounts of data, reducing client memory usage and server response time, thereby significantly improving the client's resilience against high concurrency and enhancing the user experience.

[0057] This invention provides an optimized processing method for high-concurrency data requests, comprising the following steps:

[0058] S1: Send an asynchronous network request, setting the token and timestamp in the interface body that requests the test questions;

[0059] S2: After the data request is successful, the JSON data is obtained and transcoded;

[0060] S3: Determine if the local database exists;

[0061] If yes, proceed to step S4; otherwise, proceed to step S5.

[0062] S4: Convert the obtained JSON data and write it to the local database;

[0063] S5: Create a local database, proceed with step S4;

[0064] S6: When loading data, check if the question ID exists in the local cache;

[0065] If the question ID exists, proceed to step S7; if the question ID does not exist, proceed to step S1.

[0066] S7: Load data from the local database;

[0067] Specifically, this includes: retrieving cached data from the local cache pool, obtaining the data corresponding to different question IDs, retrieving the chunk storage block corresponding to each table, converting it into binary JSON data, assigning a value according to the unique identifier UUID of each question, and then displaying it on the page.

[0068] It should be noted that in this application, the server processes network requests asynchronously after receiving them. Specifically, a thread is created to receive the network request from the client and create a task for it, adding the task to a task queue. An execution thread retrieves the task from the task queue and executes it. When the task is completed, the execution thread sends the execution result (the data requested by the client) back to the client. Here, any idle execution thread in the execution thread pool is invoked to retrieve a task from the task queue. This allows for orderly processing even when a large number of network requests are received, minimizing the risk of blocking. Furthermore, even if blocking occurs, it will not affect the execution of tasks by the execution thread.

[0069] This invention provides an optimized processing method for high-concurrency data requests, comprising the following steps:

[0070] S01: Set different databases for data storage based on different request types;

[0071] S02: Configure multiple servers for data storage;

[0072] S1: Send an asynchronous network request, setting the token and timestamp in the interface body that requests the test questions;

[0073] S2: After the data request is successful, the JSON data is obtained and transcoded;

[0074] S3: Determine if the local database exists;

[0075] If yes, proceed to step S4; otherwise, proceed to step S5.

[0076] S4: Convert the obtained JSON data and write it to the local database;

[0077] S5: Create a local database, proceed with step S4;

[0078] S6: When loading data, check if the question ID exists in the local cache;

[0079] If the question ID exists, proceed to step S7; if the question ID does not exist, proceed to step S1.

[0080] S7: Load data from the local database;

[0081] Specifically, this includes: retrieving cached data from the local cache pool, obtaining the data corresponding to different question IDs, retrieving the chunk storage block corresponding to each table, converting it into binary JSON data, assigning a value according to the unique identifier UUID of each question, and then displaying it on the page.

[0082] In this embodiment of the invention, horizontal scaling is divided into horizontal scaling at the reverse proxy layer, horizontal scaling at the site layer, horizontal scaling at the service layer, and horizontal scaling at the data layer. The reverse proxy layer scaling used in this embodiment involves splitting a system into multiple subsystems, with each subsystem connecting to a database, thus transforming a single database into multiple databases. Another method used is data layer scaling, which, when dealing with large amounts of data, involves horizontally scaling the data layer (caching, database). This splits the data (caching, database) originally stored on one server across different servers to expand system performance, thereby accelerating network request response time, reducing network response time, optimizing network request QPS, and ultimately achieving high concurrency resistance.

[0083] Vertical scaling improves single-machine processing power, increases single-machine architecture performance, reduces I / O blocking time, and improves CPU utilization efficiency. For the HTTP standard library, a Go goroutine is allocated as the execution body for each request. That is, one goroutine is allocated for each request to reduce I / O blocking time. Furthermore, goroutines are more lightweight execution bodies (compared to threads and processes), which improves CPU efficiency because the time for thread / process scheduling and context switching is much longer than the time for goroutines.

[0084] The test questions that are loaded and successfully parsed for the first time are cached. When the first loading is completed, the system will retrieve the cached data from the local cache pool by default, obtain the Data corresponding to different test question IDs, and retrieve the chunk storage block corresponding to each table. It is then converted into binary JsonData, and the value is assigned according to the unique identifier UUID of each question, and then displayed on the page.

[0085] When the client sends a request, it sends an asynchronous network request. While the network request is being sent to the background, it does not affect the operation of the main thread, thus improving the user experience.

[0086] like Figure 2 As shown, this embodiment of the invention also provides an optimized processing device for high-concurrency data requests, including a first processing module, a caching module, a first judgment module, and a loading module;

[0087] The first processing module is used to send asynchronous network requests, setting the token and timestamp in the interface body that requests the test questions;

[0088] The caching module is used to obtain JSON data after a successful data request, encode it, and write it to the local cache. The second judgment module is used to determine whether the question ID exists in the local cache when loading data.

[0089] The loading module is used to load data from the local database when the question ID exists.

[0090] It solves the thread blocking problem caused by the instantaneous transmission and loading of large amounts of data, reduces the utilization of client running memory and the response time of server, thereby greatly improving the client's resistance to high concurrency and enhancing the user experience.

[0091] This invention also provides an optimized processing device for high-concurrency data requests, including a first processing module, a caching module, a first judgment module, and a loading module;

[0092] The first processing module is used to send asynchronous network requests, setting the token and timestamp in the interface body that requests the test questions;

[0093] The caching module is used to obtain JSON data, transcode it, and write it to the local database after a successful data request.

[0094] The second judgment module is used to determine whether the question ID exists in the local cache when data is loaded;

[0095] The loading module is used to load data from the local database when the question ID exists.

[0096] The device also includes a second judgment module and a database creation module;

[0097] The second judgment module is used to determine whether the local database exists;

[0098] The database creation module is used to create a local database if the second judgment module determines that the local database does not exist.

[0099] The loading module is specifically used to retrieve cached data from the local cache pool, obtain the data corresponding to different question IDs, retrieve the chunk storage block corresponding to each table, convert it into binary JSON data, assign values ​​according to the unique identifier UUID of each question, and then display it on the page.

[0100] The device also includes a preprocessing module for setting up different databases for data storage based on different request types; and for setting up multiple servers for data storage.

[0101] The device also includes a request setting module for assigning a Go goroutine as the executor for each request.

[0102] This invention also provides an electronic device, comprising: a processor, a communication interface, a memory, and a bus, wherein the processor, communication interface, and memory communicate with each other via the bus. The bus can be used for information transmission between the electronic device and sensors. The processor can call logical instructions in the memory to execute the following methods: S1: Send an asynchronous network request, setting a token and timestamp in the interface body requesting the test question; S2: After a successful data request, obtain and transcode the JSON data; S3: Determine if the local database exists; if yes, execute step S4; otherwise, execute step S5; S4: Transcode the obtained JSON data and write it to the local database; S5: Create the local database and execute step S4; S6: When loading data, determine if the test question ID exists in the local cache; if the test question ID exists, execute step S7; if the test question ID does not exist, execute step S1; S7: Load data from the local database.

[0103] It solves the thread blocking problem caused by the instantaneous transmission and loading of large amounts of data, reduces the utilization of client running memory and the response time of server, thereby greatly improving the client's resistance to high concurrency and enhancing the user experience.

[0104] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] This invention provides a non-transitory computer-readable storage medium storing computer instructions that cause a computer to execute the method provided in the above-described method embodiments, including, for example: S1: sending an asynchronous network request and setting a token and timestamp in the interface body requesting test questions; S2: after a successful data request, obtaining and transcoding the JSON data; S3: determining whether a local database exists; if yes, proceeding to step S4, otherwise proceeding to step S5; S4: transcoding the obtained JSON data and writing it to the local database; S5: creating the local database and proceeding to step S4; S6: when loading data, determining whether the test question ID exists in the local cache; if the test question ID exists, proceeding to step S7; if the test question ID does not exist, proceeding to step S1; S7: loading data from the local database.

[0106] It solves the thread blocking problem caused by the instantaneous transmission and loading of large amounts of data, reduces the utilization of client running memory and the response time of server, thereby greatly improving the client's resistance to high concurrency and enhancing the user experience.

[0107] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the invention should also be covered within the protection scope of the invention. Therefore, the protection scope of the invention should be determined by the scope of the claims.

Claims

1. An optimized processing method for high-concurrency data requests, characterized in that, Includes the following steps: Send an asynchronous network request, setting the token and timestamp in the interface body that requests the test questions; After a successful data request, the obtained JSON data is encoded and written to the local database. When loading data, check if the question ID exists in the local cache; If the question ID exists, load the data from the local database; When the question ID does not exist, the following steps are executed: When sending network request data, set the token and timestamp in the interface body that requests the question; The methods also include: The data that is loaded and successfully parsed for the first time will be cached in the local cache pool; After a successful data request, the steps for obtaining, transcoding, and writing the JSON data to the local database include: After the data request is successful, the JSON data is obtained and transcoded. Determine if the local database exists; If so, the obtained JSON data will be transcoded and written to the local database; If not, create a local database and perform the following steps: transcode the obtained JSON data and write it to the local database; The steps for loading data from a local database include: Retrieve cached data from the local cache pool, obtain the data corresponding to different question IDs, retrieve the chunk storage block corresponding to each table, convert it into binary JSON data, assign values ​​according to the unique identifier UUID of each question, and then display it on the page.

2. The optimized processing method for high-concurrency data requests according to claim 1, characterized in that, Before sending an asynchronous network request and setting the token and timestamp in the API body for requesting test questions, the following steps are included: Different databases are set up for data storage based on different request types; Configure multiple servers for data storage.

3. The optimized processing method for high-concurrency data requests according to claim 2, characterized in that, The method also includes: Assign a Go goroutine as the execution body for each request.

4. An optimized processing apparatus for high-concurrency data requests, characterized in that, It includes a first processing module, a cache module, a first judgment module, and a loading module; The first processing module is used to send asynchronous network requests, setting the token and timestamp in the interface body that requests the test questions; The caching module is used to obtain JSON data, transcode it, and write it to the local database after a successful data request. The second judgment module is used to determine whether the question ID exists in the local cache when data is loaded; The loading module is used to load data from the local database when the question ID exists; The system is configured to cache the data that is loaded and successfully parsed for the first time to a local cache pool; After the data request is successful, the JSON data is obtained, transcoded, and written to the local database, including: After the data request is successful, the JSON data is obtained and transcoded. Determine if the local database exists; If so, the obtained JSON data will be transcoded and written to the local database; If not, create a local database and perform the following steps: transcode the obtained JSON data and write it to the local database; The loading module is specifically used to retrieve cached data from the local cache pool, obtain the data corresponding to different question IDs, retrieve the chunk storage block corresponding to each table, convert it into binary JSON data, assign values ​​according to the unique identifier UUID of each question, and then display it on the page.

5. The optimized processing apparatus for high-concurrency data requests according to claim 4, characterized in that, The device also includes a preprocessing module for setting up different databases for data storage based on different request types; and for setting up multiple servers for data storage.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions executable by at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the optimized processing method for high-concurrency data requests as described in any one of claims 1 to 3.

7. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the optimized processing method for high-concurrency data requests as described in any one of claims 1 to 3.

Images