A method and device for obtaining module data, a storage medium, and an electronic device

Abstract

The present application provides a method and device for obtaining module data, a storage medium, and an electronic device. The method calls an AI engine to obtain module-related data, forms module IDs belonging to the same module type into the same ID set, creates a data resource acquisition task corresponding to each module type, then parallelly calls the threads corresponding to each data resource acquisition task, executes each data resource acquisition task to obtain the resource data of each module ID, and creates a data packaging task corresponding to each data type, parallelly executes each data packaging task to perform data packaging on the resource data of each module ID. Finally, sorts the packaged data according to the display sequence number and feeds the sorted packaged data back to the application program. It realizes real-time acquisition and packaging of resource data without the need to pre-store the packaged data in the middleware, thus saving a large amount of memory and having strong scalability and business flexibility.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a method and device for obtaining module data, a storage medium, and an electronic device. Background Art

[0002] The channel page is the home page of the application (APP), that is, the information that is first presented visually when the user opens the APP. To achieve personalized content for each user, it is necessary to adapt to the channel pages that each user is interested in. Currently, through an AI engine, the module types and module IDs that the user is interested in are calculated from a large amount of data, and the business user then packages other required information (such as pictures, badges, titles, etc.) according to the business characteristics.

[0003] In the prior art, to quickly respond to the data requirements of the application, a data model is often established, and all resource data is pre-packaged offline in full volume to obtain the complete business data resources required by the application. The result data is stored in the middleware. When the client request arrives, after the AI engine obtains the data result, the result is quickly retrieved from the middleware according to the module type and module ID of the result, and then returned to the client. This leads to the need to occupy a large amount of memory, and the pre-processed result data may only be applicable to the channel page of the current application, with poor flexibility. Summary of the Invention

[0004] This application provides a method and device for obtaining module data, a storage medium, and an electronic device, aiming to solve the problems in the prior art that storing the processed data in the middleware leads to the need to occupy a large amount of memory, and the pre-processed result data may only be applicable to the channel page of the current application, with poor flexibility.

[0005] To achieve the above object, this application provides the following technical solutions:

[0006] A method for obtaining module data includes:

[0007] Receiving a fetch request sent by an application; the fetch request is used to request to obtain the module data of each module included in the channel page of the application;

[0008] Based on the service parameters included in the fetch request, calling an AI engine to obtain module-related data corresponding to the service parameters; the module-related data includes at least one module type, a list of module IDs, and the display sequence number and data type of the data to be supplemented corresponding to each module ID included in the list of module IDs;

[0009] Grouping the module IDs belonging to the same module type among the various module IDs included in the list of module IDs into the same ID set to obtain the ID set corresponding to each module type;

[0010] Create data resource acquisition tasks corresponding to each module type based on the ID set corresponding to each module type;

[0011] Parallelly call the threads corresponding to each data resource acquisition task in the data supplement thread pool, execute each data resource acquisition task, and obtain the resource data of all module IDs corresponding to each module type;

[0012] Create data packaging tasks corresponding to each data type;

[0013] Execute each data packaging task in parallel, perform data packaging on the resource data of each module ID, and obtain the packaged data corresponding to each module ID;

[0014] Sort each packaged data according to the display sequence number corresponding to each module ID, and feedback the sorted packaged data to the application.

[0015] In the above method, optionally, the step of executing each data resource acquisition task and obtaining the resource data of all module IDs corresponding to each module type includes:

[0016] For each module ID corresponding to each module type, determine whether there is resource data within the validity period corresponding to the module ID in the preset cache. If so, obtain the resource data in the cache. If not, based on the module ID, send a resource data acquisition request to the data provider corresponding to the module ID, and determine whether the resource data acquisition request has an exception. If there is no exception, obtain the resource data corresponding to the module ID feedback by the data provider.

[0017] In the above method, optionally, it further includes:

[0018] If the resource data acquisition request has an exception, determine whether there is resource data within the validity period corresponding to the module ID in the preset cache;

[0019] If so, obtain the resource data in the cache;

[0020] If not, determine whether it is necessary to resend the resource data acquisition request. If it is necessary to resend the resource data acquisition request, return to execute the step of sending a resource data acquisition request to the data provider corresponding to the module ID based on the module ID.

[0021] In the above method, optionally, after obtaining the resource data corresponding to the module ID feedback by the data provider, it further includes:

[0022] Based on the resource data corresponding to the module ID feedback by the data provider, update the resource data corresponding to the module ID stored in the cache, and set the validity period of the updated resource data.

[0023] For the above method, optionally, after obtaining the resource data of all module IDs corresponding to each module type, it further includes:

[0024] Create a temporary cache;

[0025] Store the resource data of all module IDs corresponding to each obtained module type into the temporary cache.

[0026] For the above method, optionally, when parallelly executing each data packaging task to perform data packaging on the resource data of each module ID to obtain the packaged data corresponding to each module ID, it includes:

[0027] Obtain the resource data of each module ID from the temporary cache;

[0028] Perform data packaging on the resource data of each module ID to obtain the packaged data corresponding to each module ID.

[0029] For the above method, optionally, when calling the AI engine based on the service parameters included in the acquisition request to obtain the module-related data corresponding to the service parameters, it includes:

[0030] Call the AI engine based on the service parameters included in the acquisition request;

[0031] Determine whether the call to the AI engine is abnormal;

[0032] If the call is abnormal, obtain the module-related data corresponding to the service parameters from a preset fallback cache based on the service parameters included in the acquisition request;

[0033] If the call is not abnormal, obtain the module-related data corresponding to the service parameters feedback by the AI engine.

[0034] A module data acquisition device, including:

[0035] A receiving unit, configured to receive an acquisition request sent by an application program; the acquisition request is used to request to acquire the module data of each module included in the channel page of the application program;

[0036] An acquisition unit, configured to call an AI engine based on the service parameters included in the acquisition request to obtain the module-related data corresponding to the service parameters; the module-related data includes at least one module type, a list of module IDs, and the display sequence number and the data type of the data to be supplemented corresponding to each module ID included in the list of module IDs;

[0037] A combined unit, configured to form the module IDs belonging to the same module type among the respective module IDs included in the module ID list into the same ID set, so as to obtain an ID set corresponding to each module type;

[0038] A first creation unit, configured to create a data resource acquisition task corresponding to each module type based on the ID set corresponding to each module type;

[0039] A first execution unit, configured to parallelly call the threads corresponding to each data resource acquisition task in a data supplement thread pool to execute each data resource acquisition task, so as to obtain the resource data of all module IDs corresponding to each module type;

[0040] A second creation unit, configured to create a data packaging task corresponding to each data type;

[0041] A second execution unit, configured to parallelly execute each data packaging task to perform data packaging on the resource data of each module ID, so as to obtain the packaged data corresponding to each module ID;

[0042] A feedback unit, configured to sort the respective packaged data according to the display sequence number corresponding to each module ID, and feed back the sorted respective packaged data to the application program.

[0043] A storage medium storing an instruction set, wherein when the instruction set is executed by a processor, the module data acquisition method as described above is implemented.

[0044] An electronic device, comprising:

[0045] A memory, configured to store at least one set of instruction sets;

[0046] A processor, configured to execute the instruction sets stored in the memory, and implement the module data acquisition method as described above by executing the instruction sets.

[0047] Compared with the prior art, the present application has the following advantages:

[0048] The present application provides a method and device for obtaining module data, a storage medium, and an electronic device. The method calls an AI engine to obtain module-related data corresponding to service parameters, forms module IDs belonging to the same module type among the module IDs included in the module ID list into the same ID set, and creates a data resource acquisition task corresponding to each module type. Then, it parallelly calls the threads corresponding to each data resource acquisition task in the data supplementation thread pool to execute each data resource acquisition task, obtains the resource data of all module IDs corresponding to each module type, and creates a data packaging task corresponding to each data type. It parallelly executes each data packaging task to perform data packaging on the resource data of each module ID to obtain the packaged data corresponding to each module ID. Finally, it sorts each packaged data according to the display sequence number corresponding to each module ID and feeds the sorted packaged data back to the application program. It can be seen that the solution of the present application realizes real-time acquisition and packaging of resource data without the need to pre-store the result data, that is, the packaged data, in the middleware, thus saving a large amount of memory, and having strong scalability and business flexibility, as well as parallel resource data and parallel data packaging of resource data, improving the efficiency of data acquisition and packaging, and further improving the efficiency of module data acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to the provided drawings.

[0050] Figure 1 It is a flowchart of a method for obtaining module data provided by the present application;

[0051] Figure 2 It is another flowchart of a method for obtaining module data provided by the present application;

[0052] Figure 3 It is another flowchart of a method for obtaining module data provided by the present application;

[0053] Figure 4 It is another flowchart of a method for obtaining module data provided by the present application;

[0054] Figure 5 It is another flowchart of a method for obtaining module data provided by the present application;

[0055] Figure 6 It is an example diagram of a method for obtaining module data provided by the present application;

[0056] Figure 7 Another example diagram of a method for obtaining module data provided by this application;

[0057] Figure 8 Another example diagram of a method for obtaining module data provided by this application;

[0058] Figure 9 Another example diagram of a method for obtaining module data provided by this application;

[0059] Figure 10 Structural schematic diagram of a device for obtaining module data provided by this application;

[0060] Figure 11 Structural schematic diagram of an electronic device provided by this application. Detailed implementation manners

[0061] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0062] The term "including" and its variations used herein are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0063] It should be noted that the concepts such as "first" and "second" mentioned in the disclosure of this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependent relationship of the functions performed by these devices, modules or units.

[0064] It should be noted that the modification of "one" and "multiple" mentioned in the disclosure of this application is illustrative rather than restrictive. Those skilled in the art should understand that unless clearly indicated otherwise in the context, it should be understood as "one or more".

[0065] The embodiments of this application provide a method for obtaining module data. This method can be applied to a data real-time supplementary packaging system. The flowchart of this method is as Figure 1 shown and specifically includes:

[0066] S101. Receive an acquisition request sent by an application program.

[0067] In this embodiment, a fetch request sent by an application is received. The fetch request is used to request the module data of each module included in the channel page of the application.

[0068] Among them, the fetch request includes service parameters. Exemplarily, the service parameters include a channel ID, a device identifier, and indication information on whether recommendations are allowed, etc. The device identifier is the device identifier of the terminal device on which the application is installed.

[0069] In this embodiment, the fetch request sent by the application is a request sent by the application after receiving the start instruction of the holding user. The holding user is the user who holds the terminal device on which the application is installed.

[0070] S102. Based on the service parameters included in the fetch request, call the AI engine to obtain the module-related data corresponding to the service parameters.

[0071] In this embodiment, after receiving the fetch request of the application, the fetch request is parsed to obtain the service parameters included in the fetch request, and based on the service parameters included in the fetch request, the AI engine is called to obtain the module-related data corresponding to the service parameters. Among them, the module-related data includes at least one module type, a list of module IDs, and the display sequence numbers and data types of the data to be supplemented corresponding to each module ID included in the list of module IDs.

[0072] Refer to Figure 2 , the process of calling the AI engine based on the service parameters included in the fetch request to obtain the module-related data corresponding to the service parameters specifically includes the following steps:

[0073] S201. Based on the service parameters included in the fetch request, call the AI engine.

[0074] In this embodiment, based on the service parameters included in the fetch request, calling the AI engine specifically means sending the service parameters included in the fetch request to the AI engine, and the AI engine calculates the module-related data of interest to the holding user corresponding to the service parameters.

[0075] S202. Determine whether the call to the AI engine is abnormal. If so, execute S203; if not, execute S204.

[0076] In this embodiment, determining whether the call to the AI engine is abnormal specifically means determining whether the AI engine fails to return data within a preset fetch duration, or returns data within the preset fetch duration but the returned data is empty data;

[0077] Among them, if the AI engine fails to return data within the preset acquisition duration, or returns data within the preset acquisition duration but the returned data is empty data, it is determined that the AI engine call is abnormal.

[0078] If the AI engine returns data within the preset acquisition duration and the returned data is not empty data, it is determined that the AI engine call is not abnormal.

[0079] S203. Based on the service parameters included in the acquisition request, obtain the module-related data corresponding to the service parameters from the preset fallback cache.

[0080] In this embodiment, if the AI engine call is abnormal, based on the service parameters included in the acquisition request, obtain the module-related data corresponding to the service parameters from the preset fallback cache. Specifically, search for the module-related data corresponding to the service parameters in the preset fallback cache, and the found module-related data is the module-related data corresponding to the service parameters. Among them, the module-related data corresponding to the service parameters existing in the fallback cache is the data successfully obtained through the AI engine for the previous time corresponding to the service parameters.

[0081] S204. Obtain the module-related data corresponding to the service parameters fed back by the AI engine.

[0082] In this embodiment, if the AI engine call is not abnormal, obtain the module-related data corresponding to the service parameters fed back by the AI engine.

[0083] Optionally, based on the module-related data corresponding to the service parameters currently fed back by the AI engine, update the corresponding module-related data in the fallback cache, that is, update the module-related data corresponding to the service parameters in the fallback cache to the module-related data corresponding to the service parameters currently fed back by the AI engine.

[0084] In this embodiment, through the preset fallback cache, it is possible to ensure obtaining the module-related data corresponding to the service parameters when the AI call is abnormal, providing a prerequisite for subsequent data packaging.

[0085] S103. Combine the module IDs belonging to the same module type among the various module IDs included in the module ID list to form the same ID set, and obtain the ID set corresponding to each module type.

[0086] In this embodiment, combine the module IDs belonging to the same module type among the various module IDs included in the module ID list to form the same ID set, thereby obtaining the ID set corresponding to each module type. That is to say, the various module IDs included in each ID set belong to the same module type.

[0087] S104. Based on the ID set corresponding to each module type, create a data resource acquisition task corresponding to each module type.

[0088] In this embodiment, based on the ID set corresponding to each module type, a data resource acquisition task corresponding to each module type is created. Specifically, for each module type, a Function function for the data resource acquisition task corresponding to this module type is created, and the ID set corresponding to the module type is used as the parameter of the Function function.

[0089] S105. Parallelly call the threads corresponding to each data resource acquisition task in the data supplement thread pool, execute each data resource acquisition task, and obtain the resource data of all module IDs corresponding to each module type.

[0090] In this embodiment, parallelly call the threads corresponding to each data resource acquisition task in the data supplement thread pool, execute each data resource acquisition task, and obtain the resource data of all module IDs corresponding to each module type. That is, execute each data resource acquisition task in parallel. Specifically, batch submit all assembled Functions to the data supplement thread pool for concurrent execution, and each Function inside will complete the corresponding task according to its respective responsibilities.

[0091] Refer to Figure 3 , the process of executing each data resource acquisition task and obtaining the resource data of all module IDs corresponding to each module type specifically includes the following steps:

[0092] S301. For each module ID corresponding to each module type, determine whether there is resource data within the validity period corresponding to the module ID in the preset cache. If so, execute S302; if not, execute S303.

[0093] In this embodiment, for each module ID corresponding to each module type, determine whether there is resource data within the validity period corresponding to the module ID in the preset cache. Among them, after the resource data stored in the preset cache is written into the cache, the validity period of each resource data is set in advance.

[0094] Preferably, the validity period is 1 minute, that is, starting from the time when the resource data is written into the cache, it is within the validity period within 1 minute, and if it exceeds 1 minute, it is determined to be outside the validity period.

[0095] S302. Obtain the resource data in the cache.

[0096] In this embodiment, if there is resource data within the validity period corresponding to the module ID in the cache, then obtain the resource data in the cache, that is, obtain the resource data within the validity period corresponding to the module ID in the cache.

[0097] S303. Based on the module ID, send a resource data acquisition request to the data provider corresponding to the module ID.

[0098] In this embodiment, if the resource data corresponding to the module ID and within the validity period does not exist in the cache, a resource data acquisition request is sent to the data provider corresponding to the module ID based on the module ID, so as to obtain the resource data corresponding to the module ID through the feedback of the data provider.

[0099] Optionally, the resource data acquisition request is an Http request.

[0100] S304. Determine whether an exception occurs in the resource data acquisition request. If not, execute S305; if so, execute SS306.

[0101] In this embodiment, after sending the resource data acquisition request to the data provider corresponding to the module ID, it is determined whether an exception occurs in the resource data acquisition request. Specifically, it is determined whether the data corresponding to the module ID feedback by the data provider is not obtained within a preset duration, or the data corresponding to the module ID feedback by the data provider is obtained within a preset duration, but the feedback data is empty data.

[0102] Among them, if the data corresponding to the module ID feedback by the data provider is not obtained within the set duration, or the data corresponding to the module ID feedback by the data provider is obtained within the preset duration, but the feedback data is empty data, it is determined that an exception occurs in the resource data acquisition request; otherwise, it is determined that no exception occurs in the resource data acquisition request.

[0103] S305. Obtain the resource data corresponding to the module ID feedback by the data provider.

[0104] In this embodiment, if no exception occurs in the resource data acquisition request, the resource data corresponding to the module ID feedback by the data provider is obtained.

[0105] Optionally, after obtaining the resource data corresponding to the module ID feedback by the data provider, it further includes: updating the resource data corresponding to the module ID stored in the cache based on the resource data corresponding to the module ID feedback by the data provider, and setting the validity period of the updated resource data.

[0106] In this embodiment, after obtaining the resource data corresponding to the module ID feedback by the data provider, the resource data corresponding to the module ID stored in the cache is updated to the resource data feedback by the data provider, and the validity period of the updated resource data is set.

[0107] S306. Determine whether there is resource data corresponding to the module ID and within the validity period in the preset cache. If so, execute S302; if not, execute S307.

[0108] In this embodiment, if an exception occurs in the resource data acquisition request, it is determined again whether there is resource data corresponding to the module ID in the preset cache and within the validity period.

[0109] In this embodiment, during the period of requesting the data provider, the cache may be updated. Therefore, it is determined again whether there is resource data corresponding to the module ID in the preset cache and within the validity period.

[0110] In this embodiment, if there is resource data corresponding to the module ID in the preset cache and within the validity period, step S302 is executed; otherwise, S307 is executed.

[0111] S307: Determine whether it is necessary to resend the resource data acquisition request. If so, return to execute S303; if not, execute S308.

[0112] In this embodiment, if there is still no resource data corresponding to the module ID in the preset cache and within the validity period, it is further determined whether it is necessary to resend the resource data acquisition request.

[0113] It should be noted that a retransmission count threshold is pre-configured. By determining whether the retransmission count is not greater than the retransmission count threshold, it is determined whether it is necessary to resend the resource data acquisition request. Among them, if the retransmission count is not greater than the retransmission count threshold, it is determined that the resource data acquisition request needs to be resent; if the retransmission count is greater than the retransmission count threshold, it is determined that the resource data acquisition request does not need to be resent.

[0114] In this embodiment, if it is necessary to resend the resource data acquisition request, return to execute S303; if it is not necessary to resend the resource data acquisition request, execute step S308.

[0115] S308: Return an error prompt message.

[0116] In this embodiment, if it is not necessary to resend the resource data acquisition request, an error prompt message is returned, and the error prompt message is used to prompt that the resource data corresponding to the module ID has not been successfully acquired.

[0117] In the method provided by the embodiment of the present application, by pre-storing the acquired resource data in the cache and setting a validity period, the efficiency of resource data acquisition is improved, and by sending the resource data acquisition request multiple times, it is ensured to acquire the resource data corresponding to the module ID.

[0118] S106: Create a data packaging task corresponding to each data type.

[0119] In this embodiment, based on the data type corresponding to each module ID, a data packaging task Supplier corresponding to each data type is created. Among them, each module ID corresponds to one Supplier, and one Supplier is used to perform data packaging on one module ID or multiple module IDs.

[0120] S107. Execute each data packaging task in parallel to perform data packaging on the resource data of each module ID, and obtain the packaged data corresponding to each module ID.

[0121] In this embodiment, each data packaging task is executed in parallel to package the resource data of each module ID, and the packaged data corresponding to each module ID is obtained. Specifically, each data packaging task is sent to a data packaging thread pool, and the packaging threads corresponding to each data packaging task in the packaging thread pool execute each data packaging task in parallel, so as to implement data packaging on the resource data of each module ID and obtain the packaged data corresponding to each module ID.

[0122] It should be noted that the packaged data corresponding to each module ID is the module data of the modules included in the channel page.

[0123] S108. Sort each piece of packaged data according to the display sequence number corresponding to each module ID, and feed the sorted pieces of packaged data back to the application program.

[0124] In this embodiment, each piece of packaged data is sorted according to the display sequence number corresponding to each module ID to ensure that the assembled data is in the same order as the data returned by the intelligent AI engine, and the sorted pieces of packaged data are fed back to the application program.

[0125] In this embodiment, if the resource data of some module IDs is not successfully obtained in step S105, that is, it is impossible to perform data packaging on the resource data of the module ID, the display sequence number corresponding to the module ID is deleted, that is, the packaged data corresponding to the module ID is not displayed.

[0126] In this embodiment, it is judged whether the channel ID is a preset value, that is, it is judged whether pageNum is equal to 1. If it is equal to 1, it is judged whether the minimum data volume requirement for filling the modules included in the channel page is met. If not, an error code is sent to trigger the application program to resend a data acquisition request; if the minimum quantity requirement is met, the sorted pieces of packaged data are directly fed back to the application program for the application program to display the rich data views of various module types on the channel page.

[0127] The module data acquisition method provided by the embodiments of the present application obtains module-related data corresponding to service parameters by calling an AI engine, forms module IDs belonging to the same module type among the module IDs included in the module ID list into the same ID set, and creates a data resource acquisition task corresponding to each module type. Then, it parallelly calls the threads corresponding to each data resource acquisition task in the data supplement thread pool to execute each data resource acquisition task, obtains the resource data of all module IDs corresponding to each module type, and creates a data packaging task corresponding to each data type. It parallelly executes each data packaging task to perform data packaging on the resource data of each module ID to obtain the packaged data corresponding to each module ID. Finally, it sorts each packaged data according to the display sequence number corresponding to each module ID, and feeds the sorted packaged data back to the application program. It can be seen that the solution of the present application realizes real-time acquisition and packaging of resource data without the need to pre-store the result data, that is, the packaged data, in the middleware, thus saving a large amount of memory, and having strong scalability and business flexibility, as well as parallel resource data and parallel data packaging of resource data, improving the efficiency of data acquisition and packaging, and further improving the efficiency of module data acquisition.

[0128] Refer to Figure 4 For the module data acquisition method provided by the above embodiments of the present application, after step S105, the following steps may further be included:

[0129] S401: Create a temporary cache.

[0130] In this embodiment, after obtaining the resource data of all module IDs corresponding to each module type, a temporary cache is created. Optionally, the temporary cache may be a context Context.

[0131] S402: Store the resource data of all module IDs corresponding to each module type obtained into the temporary cache.

[0132] In this embodiment, storing the resource data of all module IDs corresponding to each module type obtained into the temporary cache specifically means storing the resource data of all module IDs corresponding to each module type obtained into the context Context.

[0133] The module data acquisition method provided by the embodiments of the present application improves the efficiency of subsequent data packaging by creating a temporary cache and storing the resource data of all module IDs corresponding to each module type obtained into the temporary cache.

[0134] Refer to Figure 5 For the module data acquisition method provided by the above embodiments of the present application, step S107 specifically includes the following steps:

[0135] S501. Obtain the resource data of each module ID from the temporary cache.

[0136] S502. Perform data packaging on the resource data of each module ID to obtain the packaged data corresponding to each module ID.

[0137] In the method provided by the embodiments of the present application, the packaging threads corresponding to each data packaging task in the packaging thread pool obtain the resource data of each module ID in parallel from the temporary cache, and perform data packaging on the resource data of each module ID to obtain the packaged data corresponding to each module ID. Since the resource data is stored in the temporary cache and the data packaging is performed in parallel, the efficiency of data packaging is improved.

[0138] Refer to Figure 6 , and the specific implementation process of the module data acquisition method provided by the embodiments of the present application is illustrated as follows:

[0139] Step 1. The client APP (i.e., the application mentioned above) initiates a service request (i.e., the data acquisition request mentioned above), and the request reaches the service thread of the data real-time supplement packaging system;

[0140] Step 2. Inside the service thread, a call is initiated to the third-party real-time data module to obtain the recommended data set of the intelligent AI engine (i.e., the AI engine mentioned above) (i.e., the module-related data mentioned above);

[0141] Step 3. The third-party real-time data module uses HttpClient to call the intelligent AI engine to obtain the list of module categories and module ID sets returned by the intelligent AI engine interface; the third-party real-time data module then feeds back the obtained data to the service thread;

[0142] Step 4. The service thread calls the data supplement module for data supplement. A data supplement thread pool is maintained inside the data supplement module. The threads in the data supplement thread pool call the third-party real-time data module. Each thread pool initiates a request to obtain data resources of a specified type for one data source, and each data source returns the corresponding data resources (i.e., the resource data mentioned above); after receiving the data resources returned by the data source, the third-party real-time data module feeds back the data resources to the data supplement module, and the data supplement module stores the set data in the memory cache corresponding to the current context according to the type and ID;

[0143] Step 5. After the service thread obtains the data resources after data supplement, it calls the data packaging module. The data packaging module has one Supplier for each type, and each Supplier processes one or a batch of IDs internally;

[0144] Step 6. The data wrapped by multiple Suppliers returns to the business thread. The business thread combines them in sequence to form a multi-module view in response to the client and returns the data to the client.

[0145] See Figure 7 , and Figure 6 the process of obtaining the resource data corresponding to the module ID by the third-party implementation data module mentioned above is illustrated as follows:

[0146] Step 1. The purpose of the third-party real-time data module is to obtain the interface data of the third party, which includes timeout control, data degradation fallback, memory caching mechanism, etc. When the business calls the third-party real-time data stream module, the business cache key and the business parameters for requesting the third party are assembled according to the business parameters.

[0147] Step 2. Read the memory cache according to the cache key assembled in the previous step, and judge whether there is valid data in the memory cache. If there is data in the memory cache, judge whether the cached data has expired. If the data has not expired, return it directly. If the cached data does not exist or the cached data exists but has expired, go to Step 3.

[0148] Step 3. Initiate an Http request with the third-party business parameters assembled in Step 1. When the request returns data normally without exception, parse the Cache-Control value in the Header, set the cache and business validity periods, and return the data. If the interface call fails, try to read the memory cache again. Similarly, if the cached data is read and the cached data has not expired, return the data in the cache. If the cached data does not exist, enter the fallback logic in Step 4.

[0149] Step 4. In the fallback logic, read the set parameters of the current business, and use different strategies according to different configurations. The options include: returning null, retrying the Http request, returning an error code, etc. If retrying the Http request, the effective cache will also be updated according to the result returned by the interface.

[0150] See Figure 8 , and Figure 6 the process of the data supplement module mentioned above for obtaining resource data is illustrated as follows:

[0151] Step 1. After the business thread obtains the data type and data ID set returned by the intelligent AI engine from the third-party real-time data module, it enters the business data supplement module and parses and splits the intelligent AI result set data in JSON format.

[0152] Step 2. Perform concurrent processing according to the module type. Each module type assembles its own request parameters and finally forms its own Function function.

[0153] Step 3: Batch submit the Functions from the previous step to the data supplement thread pool and concurrently execute the call to obtain the third-party real-time data stream module;

[0154] Step 4: After the thread pool finishes execution within the specified time, temporarily store the module resource data in the Context context.

[0155] See Figure 9 , for Figure 6 the process of performing data packaging on the data packaging module mentioned is illustrated as follows:

[0156] Step 1: Parse the data type and data ID set returned by the intelligent AI engine again and split them again according to the module type;

[0157] Step 2: Each module type respectively obtains the corresponding data resources of its own module from the Context context and finally assembles them into Supplier functions;

[0158] Step 3: Batch submit the Supplier functions from the previous step to the data packaging thread pool for concurrent execution; if the context cannot be obtained inside the Supplier, skip the data module according to the business rules (hide the display of the other end);

[0159] Step 4: Re-sort and assemble the result set after the thread pool finishes execution in the order when the intelligent AI engine returns to form new packaged data.

[0160] It should be noted that although the instructions are depicted in a specific order, this should not be construed as requiring these instructions to be executed in the specific order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous.

[0161] It should be understood that the various steps recorded in the method embodiments disclosed in this application can be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the disclosure of this application is not limited in this regard.

[0162] Corresponding to Figure 1 the method described above, the embodiment of this application also provides a module data acquisition device for Figure 1 the specific implementation of the method in Figure 10 as shown, which specifically includes:

[0163] A receiving unit 1001, configured to receive an acquisition request sent by an application; the acquisition request is used to request to acquire the module data of each module included in the channel page of the application;

[0164] An acquisition unit 1002, configured to call an AI engine based on service parameters included in the acquisition request to obtain module-related data corresponding to the service parameters; the module-related data includes at least one module type, a list of module IDs, and for each module ID included in the list of module IDs, a display sequence number and a data type of data to be supplemented;

[0165] A combination unit 1003, configured to form module IDs belonging to the same module type among the respective module IDs included in the list of module IDs into the same ID set, to obtain an ID set corresponding to each module type;

[0166] A first creation unit 1004, configured to create a data resource acquisition task corresponding to each module type based on the ID set corresponding to each module type;

[0167] A first execution unit 1005, configured to parallelly call threads corresponding to each data resource acquisition task in a data supplementation thread pool to execute each data resource acquisition task, to obtain resource data of all module IDs corresponding to each module type;

[0168] A second creation unit 1006, configured to create a data packaging task corresponding to each data type;

[0169] A second execution unit 1007, configured to parallelly execute each data packaging task to perform data packaging on the resource data of each module ID, to obtain packaged data corresponding to each module ID;

[0170] A feedback unit 1008, configured to sort the respective packaged data according to the display sequence number corresponding to each module ID, and feedback the sorted respective packaged data to the application program.

[0171] The module data acquisition device provided by the embodiment of the present application realizes real-time acquisition and packaging of resource data without the need to pre-store result data, that is, packaged data, into a middleware, thereby saving a large amount of memory, and has strong scalability and business flexibility, as well as parallel resource data and parallel data packaging of resource data, improving the efficiency of data acquisition and packaging, and further improving the efficiency of module data acquisition.

[0172] In an embodiment of the present application, based on the foregoing solution, the first execution unit 1005 is specifically configured to:

[0173] For each module ID corresponding to each module type, determine whether there is resource data within the validity period corresponding to the module ID in a preset cache. If so, obtain the resource data in the cache. If not, send a resource data acquisition request to the data provider corresponding to the module ID based on the module ID, and determine whether an exception occurs in the resource data acquisition request. If no exception occurs, obtain the resource data corresponding to the module ID feedback by the data provider.

[0174] In an embodiment of the present application, based on the foregoing solution, the first execution unit 1005 is further configured to:

[0175] If an exception occurs in the resource data acquisition request, determine whether there is resource data within the validity period corresponding to the module ID in the preset cache;

[0176] If so, obtain the resource data in the cache;

[0177] If not, determine whether it is necessary to resend the resource data acquisition request. If it is necessary to resend the resource data acquisition request, return to execute the step of sending a resource data acquisition request to the data provider corresponding to the module ID based on the module ID.

[0178] In an embodiment of the present application, based on the foregoing solution, the first execution unit 1005 is further configured to:

[0179] Based on the resource data corresponding to the module ID feedback by the data provider, update the resource data corresponding to the module ID stored in the cache, and set the validity period of the updated resource data.

[0180] In an embodiment of the present application, based on the foregoing solution, it can also be configured as:

[0181] The third creation unit is used to create a temporary cache;

[0182] The storage unit is used to store the resource data of all module IDs corresponding to each module type obtained into the temporary cache.

[0183] In an embodiment of the present application, based on the foregoing solution, the second execution unit 1007 is specifically configured to:

[0184] Obtain the resource data of each module ID from the temporary cache;

[0185] Perform data packaging on the resource data of each module ID to obtain the packaged data corresponding to each module ID.

[0186] In one embodiment of the present application, based on the foregoing solution, the obtaining unit 1002 is specifically configured to:

[0187] Call the AI engine based on the service parameters included in the obtaining request;

[0188] Determine whether the call to the AI engine is abnormal;

[0189] If the call is abnormal, obtain the module-related data corresponding to the service parameters from a preset fallback cache based on the service parameters included in the obtaining request;

[0190] If the call is not abnormal, obtain the module-related data corresponding to the service parameters feedback by the AI engine.

[0191] An embodiment of the present application further provides a storage medium, which stores an instruction set. When the instruction set runs, it executes the module data obtaining method disclosed in any of the above embodiments.

[0192] An embodiment of the present application further provides an electronic device, the structural schematic diagram of which is as Figure 11 shown, and specifically includes a memory 1101 for storing at least one set of instruction sets; a processor 1102 for executing the instruction sets stored in the memory by executing the module data obtaining method disclosed in any of the above embodiments.

[0193] Although the subject matter has been described in language specific to structural features and / or methodological act logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms of implementing the claims.

[0194] Although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the disclosure of the present application. Certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment can also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0195] The above description is only a preferred embodiment of the disclosure of the present application and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the disclosure involved in the present application is not limited to the technical solution formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, a technical solution formed by mutually replacing the above features with technical features having similar functions disclosed in the present application (but not limited to).

Claims

1. A method for obtaining module data, characterized in that, Including: Receiving a fetch request sent by an application; the fetch request is used to request to fetch module data of each module included in the channel page of the application; Based on the service parameters included in the fetch request, invoking an AI engine to obtain module-related data corresponding to the service parameters; the module-related data includes at least one module type, a list of module IDs, and the display sequence number and the data type of the data to be supplemented corresponding to each module ID included in the list of module IDs; Forming the module IDs belonging to the same module type among the various module IDs included in the list of module IDs into the same ID set, and obtaining the ID set corresponding to each module type; Based on the ID set corresponding to each module type, creating a data resource fetch task corresponding to each module type; Parallelly invoking the threads corresponding to each data resource fetch task in a data supplementation thread pool to execute each data resource fetch task, and obtaining the resource data of all module IDs corresponding to each module type; Creating a data packaging task corresponding to each data type; Parallelly executing each data packaging task to perform data packaging on the resource data of each module ID, and obtaining the packaged data corresponding to each module ID; Sorting the various packaged data according to the display sequence number corresponding to each module ID, and feeding back the sorted various packaged data to the application.

2. The method according to claim 1, characterized in that, The step of executing each data resource fetch task to obtain the resource data of all module IDs corresponding to each module type includes: For each module ID corresponding to each module type, determining whether there is resource data within the validity period corresponding to the module ID in a preset cache. If so, obtaining the resource data in the cache; if not, based on the module ID, sending a resource data fetch request to the data provider corresponding to the module ID, and determining whether the resource data fetch request has an exception. If there is no exception, obtaining the resource data corresponding to the module ID fed back by the data provider.

3. The method according to claim 2, wherein Also including: If the resource data fetch request has an exception, determining whether there is resource data within the validity period corresponding to the module ID in a preset cache; If so, obtaining the resource data in the cache; If not, determining whether it is necessary to resend the resource data fetch request. If it is necessary to resend the resource data fetch request, returning to execute the step of sending a resource data fetch request to the data provider corresponding to the module ID based on the module ID.

4. The method according to claim 2, wherein After obtaining the resource data corresponding to the module ID fed back by the data provider, further including: Based on the resource data corresponding to the module ID fed back by the data provider, updating the resource data corresponding to the module ID stored in the cache, and setting the validity period of the updated resource data.

5. The method according to claim 1, wherein After obtaining the resource data of all module IDs corresponding to each module type, further including: Creating a temporary cache; Storing the resource data of all module IDs corresponding to each module type obtained into the temporary cache.

6. The method according to claim 5, wherein Performing the respective data packaging tasks in parallel to perform data packaging on the resource data of each module ID to obtain the packaged data corresponding to each module ID, including: Obtaining the resource data of each module ID from the temporary cache; Performing data packaging on the resource data of each module ID to obtain the packaged data corresponding to each module ID.

7. The method according to claim 1, wherein Based on the service parameters included in the acquisition request, calling an AI engine to obtain the module-related data corresponding to the service parameters, including: Calling an AI engine based on the service parameters included in the acquisition request; Determining whether the call to the AI engine is abnormal; If the call is abnormal, obtaining the module-related data corresponding to the service parameters from a preset fallback cache based on the service parameters included in the acquisition request; If the call is not abnormal, obtaining the module-related data corresponding to the service parameters fed back by the AI engine.

8. A module data acquisition device, characterized in that, Including: A receiving unit, configured to receive an acquisition request sent by an application; The acquisition request is used to request to obtain the module data of each module included in the channel page of the application; An acquisition unit, configured to call an AI engine based on the service parameters included in the acquisition request to obtain the module-related data corresponding to the service parameters; the module-related data includes at least one module type, a module ID list, and the display sequence number and the data type of the data to be supplemented corresponding to each module ID included in the module ID list; A combining unit, configured to form the module IDs belonging to the same module type among the respective module IDs included in the module ID list into the same ID set to obtain the ID set corresponding to each module type; A first creating unit, configured to create a data resource acquisition task corresponding to each module type based on the ID set corresponding to each module type; A first execution unit, configured to parallelly call the threads corresponding to each data resource acquisition task in a data supplementation thread pool to execute each data resource acquisition task to obtain the resource data of all module IDs corresponding to each module type; A second creating unit, configured to create a data packaging task corresponding to each data type; A second execution unit, configured to perform the respective data packaging tasks in parallel to perform data packaging on the resource data of each module ID to obtain the packaged data corresponding to each module ID; A feedback unit, configured to sort the respective packaged data according to the display sequence number corresponding to each module ID and feedback the sorted respective packaged data to the application.

9. A storage medium, characterized in that, The storage medium stores an instruction set, wherein when the instruction set is executed by a processor, the module data acquisition method described in any one of claims 1-7 is implemented.

10. An electronic device, characterized in that, Including: A memory, configured to store at least one set of instruction sets; A processor, configured to execute the instruction sets stored in the memory and implement the module data acquisition method described in any one of claims 1-7 by executing the instruction sets.

Images