Data processing method and device, terminal equipment and computer readable storage medium

Abstract

The invention is suitable for the technical field of data processing, and provides a data processing method and device, terminal equipment and a computer readable storage medium, and the method comprises the steps: obtaining a plurality of first modeling requests of a user, the first modeling requests comprising a plurality of first modeling parameters required by modeling; performing data screening on the plurality of first modeling parameters to obtain screened second modeling parameters; and calling a modeling engine according to the plurality of second modeling parameters to obtain first modeling data corresponding to the first modeling request of the user. According to the method, the network delay can be reduced in the modeling process, the calling speed is increased, and the user experience is improved.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular, relates to a data processing method, apparatus, terminal equipment and computer-readable storage medium. Background Art

[0002] In modern cloud computing and big data processing, modeling engines are an important technical tool for handling complex computing tasks and data analysis. However, with the increase in user demand and the increase in application complexity, traditional modeling cloud engines have encountered some performance bottlenecks during the calling process, such as slow calling speed and high system latency, which directly affect the user experience. Therefore, an efficient modeling method is urgently needed to provide users with a smoother and more efficient cloud service experience. Summary of the invention

[0003] The embodiments of the present application provide a data processing method, apparatus, terminal device and computer-readable storage medium, which can reduce network latency, increase call speed and improve user experience.

[0004] In a first aspect, an embodiment of the present application provides a data processing method, including:

[0005] Acquire multiple first modeling requests from users, where the first modeling requests include multiple first modeling parameters required for modeling;

[0006] Performing data screening on the plurality of the first modeling parameters to obtain screened second modeling parameters;

[0007] The modeling engine is called according to the plurality of the second modeling parameters to obtain the first modeling data corresponding to the first modeling request of the user.

[0008] In an embodiment of the present application, a user can submit multiple modeling requests, each request includes multiple modeling parameters (first modeling parameters), and the received modeling parameters are screened. After the key requests or parameters are screened, the modeling engine is called using the screened second modeling parameters, and the modeling engine calculates and generates the modeling data requested by the user according to the second modeling parameters. By screening the data of the first modeling parameters, unnecessary parameters or requests can be effectively reduced, and the amount of data transmission can be reduced, thereby reducing network delays, increasing call speeds, and improving user experience.

[0009] In a possible implementation manner of the first aspect, the step of performing data screening on the plurality of first modeling parameters to obtain screened second modeling parameters includes:

[0010] Identify the plurality of first modeling parameters according to the identification model, and identify repeated first modeling parameters and invalid first modeling parameters;

[0011] The repeated and invalid first modeling parameters are filtered out to obtain a plurality of screened second modeling parameters.

[0012] In the embodiments of the present application, redundant information and invalid parameters are intelligently identified and removed, thereby effectively reducing the amount of data transmission, thereby reducing network latency and improving call speed.

[0013] In a possible implementation manner of the first aspect, calling a modeling engine according to the plurality of second modeling parameters to obtain first modeling data corresponding to the first modeling request of the user includes:

[0014] Identifying the first modeling requests corresponding to the plurality of the second modeling parameters, and identifying the plurality of second modeling requests that can be processed in parallel and the plurality of third modeling requests that need to be processed in series;

[0015] The modeling engine is called according to the plurality of the second modeling requests and the plurality of the third modeling requests to obtain the first modeling data corresponding to the first modeling request of the user.

[0016] In the embodiment of the present application, by identifying parallel processing requests, multiple requests can be sent to the modeling engine at the same time, thereby significantly reducing the total processing time. Parallel processing can make full use of multi-core processors and distributed computing resources to improve computing efficiency.

[0017] In a possible implementation manner of the first aspect, calling the modeling engine according to the plurality of second modeling requests and the plurality of third modeling requests to obtain first modeling data corresponding to the first modeling request of the user includes:

[0018] Determine a first request order corresponding to each of the plurality of second modeling requests and a second request order corresponding to each of the plurality of third modeling requests;

[0019] Sending a plurality of second modeling requests and a plurality of the third modeling requests to the modeling engine according to the first request sequence and the second request sequence to obtain a plurality of second modeling data returned by the modeling engine;

[0020] The first modeling data corresponding to the first modeling request of the user is obtained according to the plurality of second modeling data returned by the modeling engine.

[0021] In the embodiment of the present application, by splitting the tasks that can be processed in parallel and assigning them to multiple computing nodes for parallel computing, a significant improvement in data processing speed is achieved.

[0022] In a possible implementation manner of the first aspect, acquiring, according to the plurality of second modeling data returned by the modeling engine, first modeling data corresponding to the first modeling request of the user includes:

[0023] Cache each of the second modeling data in the first cache area;

[0024] If the first number of the second modeling data in the first cache area is equal to the number of first modeling requests corresponding to the second modeling parameter, deduplication processing is performed on the multiple second modeling data in the first cache area to obtain the deduplication-processed first modeling data.

[0025] In an embodiment of the present application, by checking whether the amount of data in the cache area is equal to the number of modeling requests, it can be ensured that all necessary data have been collected to avoid data missing. Deduplication processing is performed only after all data has been collected, which can avoid invalid processing when the data is incomplete, thereby improving the accuracy and efficiency of processing.

[0026] In a possible implementation manner of the first aspect, the method further includes:

[0027] splitting the plurality of the second modeling requests into a plurality of first tasks; wherein each of the first tasks includes a different number of second modeling requests;

[0028] The plurality of first tasks are respectively sent to a computing unit including the modeling engine corresponding to each of the first tasks.

[0029] In the embodiment of the present application, by splitting the tasks that can be processed in parallel and assigning them to multiple computing nodes for parallel computing, a significant improvement in data processing speed is achieved.

[0030] In a possible implementation manner of the first aspect, the first tasks corresponding to the computing units are matched according to corresponding performances of the computing units.

[0031] In a second aspect, an embodiment of the present application provides a data processing device, including:

[0032] An acquisition module, configured to acquire a plurality of first modeling requests from a user, wherein the first modeling requests include a plurality of first modeling parameters required for modeling;

[0033] An identification module, used for performing data screening on a plurality of the first modeling parameters to obtain screened second modeling parameters;

[0034] A calling module is used to call a modeling engine according to a plurality of the second modeling parameters to obtain first modeling data corresponding to the user's first modeling request.

[0035] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements a data processing method as described in any one of the first aspects above.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data processing method as described in any one of the above-mentioned first aspects is implemented.

[0037] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the data processing method described in any one of the above-mentioned first aspects.

[0038] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0041] Figure 2 It is a flow chart of parameter processing provided by an embodiment of the present application;

[0042] Figure 3 It is a flowchart of calling a modeling engine provided by an embodiment of the present application;

[0043] Figure 4 It is a schematic diagram of a process for obtaining modeling results provided in an embodiment of the present application;

[0044] Figure 5 It is a schematic diagram of a process for providing an optimized modeling result in an embodiment of the present application;

[0045] Figure 6 is a schematic block diagram of a data processing method provided in an embodiment of the present application;

[0046] Figure 7 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0047] Figure 8 It is a structural diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0049] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0050] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0051] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0052] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0053] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0054] In modern cloud computing and big data processing, modeling engines are an important technical tool for handling complex computing tasks and data analysis. However, with the increase in user demand and the increase in application complexity, traditional modeling cloud engines have encountered some performance bottlenecks during the calling process, such as slow calling speed and high system latency, which directly affect the user experience. For example, traditional data processing technologies often rely on single-threaded or limited parallel processing methods, or traditional virtualization technologies (such as VMware, Hyper-V, etc.) may have deficiencies in dynamic resource allocation and scheduling, making it difficult to quickly adjust resource allocation according to actual needs.

[0055] In order to solve the problems in the above-mentioned related technologies, the embodiment of the present application provides a data processing method. In the present application,

[0056] See also Figure 1 , is a flow chart of a data processing method provided in an embodiment of the present application. As an example but not a limitation, the method may include the following steps:

[0057] S101, obtaining a plurality of first modeling requests from a user, where the first modeling requests include a plurality of first modeling parameters required for modeling.

[0058] In the embodiment of the present application, the user can send multiple modeling requests through the Web interface, each request is a specific request that the user wants to perform geometric modeling, and each request can contain multiple parameters, which are necessary for geometric modeling, and these parameters may include but are not limited to geometric parameters, user information, etc. These parameters and data are used to define the shape, size, position, etc. of the geometric model, such as user ID: used to identify the user who sent the request, to facilitate user management and authority control; edge length: used to define the length of the edge in the geometric model, such as the edge length of a cube; session ID: used to identify the user's session, to ensure that multiple requests of the user are consistent in one session, etc.

[0059] The web interface can send the request to the server for further processing.

[0060] S102, performing data screening on the plurality of first modeling parameters to obtain screened second modeling parameters.

[0061] In the embodiment of the present application, after obtaining the modeling parameters of the user, the modeling parameters of the user need to be further processed to remove unnecessary parameters.

[0062] In one embodiment, see Figure 2 , is a flow chart of parameter processing provided by an embodiment of the present application, step S102 includes:

[0063] S201, identifying a plurality of the first modeling parameters according to a recognition model, and identifying repeated first modeling parameters and invalid first modeling parameters.

[0064] In the embodiment of the present application, the system receives multiple modeling requests submitted by the user, each request includes multiple first modeling parameters, uses the recognition model to detect and mark duplicate parameters, and uses the recognition model to detect and mark invalid parameters, which may include format errors, out-of-range values, abnormal values, etc. Parameters marked as duplicates are removed, and parameters marked as invalid are removed. After the above recognition and filtering steps, second modeling parameters that meet the requirements are generated, and these parameters will be used to call the modeling engine.

[0065] S202: Filter out repeated and invalid first modeling parameters to obtain a plurality of screened second modeling parameters.

[0066] In the embodiment of the present application, the identified repeated parameters are filtered out, wherein the repeated parameters can be placed in a cache area, including the API name, parameters, and return data, so as to reduce repeated requests.

[0067] Exemplarily, the first modeling parameter submitted by the user is:

[0068] Parameter 1: 10, parameter 2: 20, parameter 3: 10, parameter 4: 30, parameter 5: 50 and parameter 6: "abc";

[0069] The tag information obtained after recognition is:

[0070] Parameter 1:10, Parameter 2:20, Parameter 3:10 (repeated), Parameter 4:30, Parameter 5:50, and Parameter 6: "abc" (invalid parameter);

[0071] After identification and filtering of the recognition model, the second modeling parameters are obtained:

[0072] Parameter 1: 10, parameter 2: 20, parameter 4: 30, parameter 5: 50.

[0073] It should be noted that repeated parameters can be placed in the cache module, which can reduce the number of times the engine is called with repeated data, reduce the amount of data input, and improve system efficiency.

[0074] In the above method, redundant information and invalid parameters are intelligently identified and removed, thereby effectively reducing the amount of data transmission, thereby reducing network latency and improving call speed.

[0075] S103: Calling a modeling engine according to the plurality of second modeling parameters to obtain first modeling data corresponding to the first modeling request of the user.

[0076] In an embodiment of the present application, the system receives multiple second modeling parameters after data screening, uses these second modeling parameters to call the modeling engine, performs modeling calculations, and obtains calculation results from the modeling engine, that is, the first modeling data corresponding to the user's first modeling request.

[0077] In one embodiment, see Figure 3 , is a flow chart of calling a modeling engine provided by an embodiment of the present application, step S103 includes:

[0078] S301, identifying the first modeling requests corresponding to the plurality of the second modeling parameters, and identifying the plurality of second modeling requests that can be processed in parallel and the plurality of third modeling requests that need to be processed in series.

[0079] In the embodiment of the present application, a mapping relationship with the first modeling request is established according to the second modeling parameter to ensure that each second modeling parameter can be traced back to the corresponding first modeling request. Then, before calling the engine according to multiple second modeling parameters, a certain strategy or model (such as task-based dependencies, resource requirements, etc.) can be used to identify modeling requests that can be processed in parallel (second modeling requests) and modeling requests that need to be processed serially (third modeling requests).

[0080] Parallel processing can significantly increase the processing speed of tasks and reduce the total processing time.

[0081] S302: Call the modeling engine according to the plurality of the second modeling requests and the plurality of the third modeling requests to obtain first modeling data corresponding to the first modeling request of the user.

[0082] In one embodiment, step S302 is implemented as follows:

[0083] Split the plurality of second modeling requests into a plurality of first tasks; wherein each of the first tasks includes a different number of second modeling requests; and send the plurality of first tasks to a computing unit including the modeling engine corresponding to each of the first tasks, respectively.

[0084] In the embodiment of the present application, considering the parallel processing capability of the computing unit, tasks can be reasonably allocated to maximize the parallel processing efficiency. If the computing unit supports multi-threaded processing, multiple simple second modeling requests can be combined into one first task to fully utilize the parallel processing capability.

[0085] Specifically, understand the resource situation of each computing unit, including CPU, memory, storage, etc. For example, computing unit 1 has 2CPUs and 4GB memory, and computing unit 2 has 4CPUs and 8GB memory. According to the resource situation of the computing unit and the resource requirements of each first task, assign the first task to the appropriate computing unit, and assign the first task with higher resource requirements to the computing unit with more abundant resources. An intelligent task scheduling algorithm can be used, which can dynamically assign tasks according to the real-time load and performance indicators of the computing nodes.

[0086] Assume that there are five first modeling tasks that can be processed in parallel corresponding to the second request parameter, the first task 1 includes request A, the second task 2 includes request B, and the third task 3 includes requests C, D, and E. There are three computing units (computing unit 1, computing unit 2, computing unit 3), and their resources are (2CPU, 4GB memory), (4CPU, 8GB memory), (2CPU, 4GB memory), respectively. The first task can be assigned as follows by using the intelligent task scheduling algorithm combined with the computing unit performance: the first task 1 (request A) is assigned to computing unit 1, the second task (request B) is assigned to computing unit 2, and the third task (requests C, D, and E) is assigned to computing unit 3.

[0087] In the above method, the data processing speed is greatly improved by splitting the tasks that can be processed in parallel and assigning them to multiple calculation points for parallel calculation.

[0088] In one embodiment, the first tasks corresponding to the computing units are matched according to the corresponding performances of the computing units.

[0089] In an embodiment of the present application, when using an intelligent task scheduling algorithm to assign tasks to multiple computing units (nodes) of parallel computing, the real-time load and performance indicators of all nodes can be collected, and the performance score Si(t) of each node can be updated based on the collected data. A task Tj is selected from the task queue T and assigned to the computing unit or computing node with the highest performance score. The load situation of each node is detected. If a node is overloaded, task migration is performed, and the real-time load data and task queue of the new node are collected. When the task queue T is empty, the task assignment process is terminated.

[0090] Through this dynamic task allocation algorithm, the system can achieve better load balancing and improve overall computing efficiency and resource utilization.

[0091] In one embodiment, see Figure 4 , is a schematic diagram of a flow chart of obtaining modeling results provided in an embodiment of the present application, such as Figure 4 As shown, step S302 includes:

[0092] S401, determining a first request order corresponding to each of the plurality of second modeling requests and a second request order corresponding to each of the plurality of third modeling requests.

[0093] In the embodiment of the present application, after obtaining the processing methods of multiple modeling requests (parallel or serial), it is necessary to clarify the dependency relationship of each request, that is, which other requests each request depends on, to determine the order of processing the requests, and design the parallel processing and serial processing processes according to the dependency relationship. Parallel processing means that multiple requests can be processed at the same time, but the premise is that their dependent requests have been completed, and serial processing means that the requests must be processed in sequence, and the next request will be started after one request is completed.

[0094] S402: Send multiple second modeling requests and multiple third modeling requests to the modeling engine according to the first request sequence and the second request sequence, so as to obtain multiple second modeling data returned by the modeling engine.

[0095] In an embodiment of the present application, the request order of multiple requests is determined according to the dependency relationship of the multiple requests, and the modeling engine is called according to the request order to obtain the modeling result corresponding to the modeling engine.

[0096] Specifically, the modeling requests include A, B, C, and D, where C and D can be processed in parallel, and A and B can be processed in series, but C depends on A and can start processing after A is completed, and D depends on A and B. If A is the first request and does not depend on other requests, the request sequence can be ABC, D. The modeling engines are called in sequence.

[0097] S403: Acquire first modeling data corresponding to the first modeling request of the user according to the plurality of second modeling data returned by the modeling engine.

[0098] In the embodiment of the present application, the user's modeling request is sent to the modeling engine according to the parallel and serial processing methods respectively, and the modeling request processed in parallel is sent to different computing units for parallel processing. After the computing unit calculates, the calculation result corresponding to each modeling request, i.e., the second modeling data, is returned. However, since the calculation result returned by the modeling engine may store unnecessary information, it is necessary to further process the returned calculation result. See steps S501-S502.

[0099] In the above method, the data processing speed is greatly improved by splitting the tasks that can be processed in parallel and assigning them to multiple computing nodes for parallel computing.

[0100] In one embodiment, see Figure 5 , is a flow chart of optimizing modeling results provided by the embodiment of the present application, such as Figure 5 As shown, step S403 includes:

[0101] S501: Cache each of the second modeling data into a first cache area.

[0102] In an embodiment of the present application, after obtaining each calculation result (second modeling data), each calculation result is temporarily stored in a cache area. Through caching, it can be ensured that data accessed multiple times in a short period of time is consistent, avoiding inconsistency problems caused by frequent data changes.

[0103] S502: If the first number of the second modeling data in the first cache area is equal to the number of first modeling requests corresponding to the second modeling parameter, deduplication processing is performed on the plurality of second modeling data in the first cache area to obtain the deduplication-processed first modeling data.

[0104] In the embodiment of the present application, the cache area can also be used to record the status and quantity of each task. When the number of cached data is equal to the number corresponding to the modeling request, it means that the task is completed. At this time, multiple modeling data need to be optimized uniformly.

[0105] Specifically, compression technology and data optimization algorithms can be used to compress and optimize the call results to remove redundant information or duplicate invalid data. In addition, relevant algorithms such as natural language processing algorithms can be used to identify and analyze the modeling data, extract key information from the modeling data, and return the extracted key information to the user as the calculation result (first modeling data).

[0106] In the above method, by checking whether the number of data in the cache area is equal to the number of modeling requests, it can be ensured that all necessary data have been collected to avoid data missing. Deduplication processing is performed only after all data has been collected, which can avoid invalid processing when the data is incomplete, thereby improving the accuracy and efficiency of processing.

[0107] See also Figure 6 , is a schematic block diagram of a data processing method provided in an embodiment of the present application, such as Figure 6 As shown, the data processing steps include:

[0108] ① The user sends a geometric modeling request (first modeling request) to the server through the client (web front end), including the parameters required for modeling (first modeling parameters);

[0109] ② After receiving the modeling request from the user, the server processes the modeling parameters and removes redundant information using the recognition model to obtain the second modeling parameters without redundant information;

[0110] ③ Identifying the first modeling request, and obtaining a plurality of modeling requests that can be processed in parallel (second modeling requests) and a plurality of modeling requests that can be processed in series (third modeling requests);

[0111] ④ Calling the modeling engine according to the multiple second modeling parameters and the requests corresponding to the parameters, and performing concurrent calls in parallel;

[0112] ⑤ Calculate the user's modeling request by dynamically loading multiple first calculation units containing modeling engines, and return the calculation results to the cache area of ​​the server. The calculation results of concurrent calls are also returned to the server concurrently;

[0113] ⑥ When the task execution is completed, the calculation results in the cache area are equal to the called requests (when the number of the second modeling request and the third modeling request is equal), all the calculation results are optimized, redundant information is removed, and key data (first modeling data) is extracted;

[0114] ⑦ Send the processed first modeling data to the client.

[0115] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0116] Corresponding to the data processing method described in the above embodiment, Figure 7 This is a structural block diagram of a data processing device provided in an embodiment of the present application. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0117] Reference Figure 7 , the device comprises:

[0118] An acquisition module 71 is used to acquire a plurality of first modeling requests from a user, wherein the first modeling requests include a plurality of first modeling parameters required for modeling;

[0119] An identification module 72, configured to perform data screening on a plurality of the first modeling parameters to obtain screened second modeling parameters;

[0120] The calling module 73 is used to call the modeling engine according to the plurality of the second modeling parameters to obtain the first modeling data corresponding to the first modeling request of the user.

[0121] Optionally, the identification module 72 is further used for:

[0122] Identify the plurality of first modeling parameters according to the identification model, and identify repeated first modeling parameters and invalid first modeling parameters;

[0123] The repeated and invalid first modeling parameters are filtered out to obtain a plurality of screened second modeling parameters.

[0124] Optionally, calling module 73 is further used to:

[0125] Identifying the first modeling requests corresponding to the plurality of the second modeling parameters, and identifying the plurality of second modeling requests that can be processed in parallel and the plurality of third modeling requests that need to be processed in series;

[0126] The modeling engine is called according to the plurality of the second modeling requests and the plurality of the third modeling requests to obtain the first modeling data corresponding to the first modeling request of the user.

[0127] Optionally, calling module 73 is further used to:

[0128] Determine a first request order corresponding to each of the plurality of second modeling requests and a second request order corresponding to each of the plurality of third modeling requests;

[0129] Sending a plurality of second modeling requests and a plurality of the third modeling requests to the modeling engine according to the first request sequence and the second request sequence to obtain a plurality of second modeling data returned by the modeling engine;

[0130] The first modeling data corresponding to the first modeling request of the user is obtained according to the plurality of second modeling data returned by the modeling engine.

[0131] Optionally, calling module 73 is further used to:

[0132] Cache each of the second modeling data in the first cache area;

[0133] If the first number of the second modeling data in the first cache area is equal to the number of first modeling requests corresponding to the second modeling parameter, deduplication processing is performed on the multiple second modeling data in the first cache area to obtain the deduplication-processed first modeling data.

[0134] Optionally, calling module 73 is further used to:

[0135] splitting the plurality of the second modeling requests into a plurality of first tasks; wherein each of the first tasks includes a different number of second modeling requests;

[0136] The plurality of first tasks are respectively sent to a computing unit including the modeling engine corresponding to each of the first tasks.

[0137] Optionally, calling module 73 is further used to:

[0138] The plurality of first tasks are respectively sent to a computing unit including the modeling engine corresponding to each of the first tasks.

[0139] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0140] in addition, Figure 7 The data processing device shown may be a software unit, a hardware unit, or a combination of software and hardware units built into an existing terminal device, or may be integrated into the terminal device as an independent accessory, or may exist as an independent terminal device.

[0141] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0142] Figure 8 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Figure 8 As shown, the terminal device 8 of this embodiment includes: at least one processor 80 ( Figure 8 Only one is shown in the figure) a processor, a memory 81, and a computer program 82 stored in the memory 81 and executable on the at least one processor 80, and when the processor 80 executes the computer program 82, the steps in any of the above-mentioned data processing method embodiments are implemented.

[0143] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 8It is only an example of the terminal device 8 and does not constitute a limitation on the terminal device 8. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0144] The processor 80 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0145] In some embodiments, the memory 81 may be an internal storage unit of the terminal device 8, such as a hard disk or memory of the terminal device 8. In other embodiments, the memory 81 may also be an external storage device of the terminal device 8, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device 8. Further, the memory 81 may also include both an internal storage unit of the terminal device 8 and an external storage device. The memory 81 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as the program code of the computer program, etc. The memory 81 may also be used to temporarily store data that has been output or is to be output.

[0146] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0147] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0148] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0149] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0150] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0151] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0153] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire multiple first modeling requests from users, where the first modeling requests include multiple first modeling parameters required for modeling; Performing data screening on the plurality of the first modeling parameters to obtain screened second modeling parameters; The modeling engine is called according to the plurality of the second modeling parameters to obtain the first modeling data corresponding to the first modeling request of the user.

2. The data processing method according to claim 1, characterized in that: The step of screening the plurality of first modeling parameters to obtain screened second modeling parameters includes: Identify the plurality of first modeling parameters according to the identification model, and identify repeated first modeling parameters and invalid first modeling parameters; The repeated and invalid first modeling parameters are filtered out to obtain a plurality of screened second modeling parameters.

3. The data processing method according to claim 1, characterized in that: The calling of the modeling engine according to the plurality of the second modeling parameters to obtain the first modeling data corresponding to the first modeling request of the user includes: Identifying the first modeling requests corresponding to the plurality of the second modeling parameters, and identifying the plurality of second modeling requests that can be processed in parallel and the plurality of third modeling requests that need to be processed in series; The modeling engine is called according to the plurality of the second modeling requests and the plurality of the third modeling requests to obtain the first modeling data corresponding to the first modeling request of the user.

4. The data processing method according to claim 3, characterized in that: The calling the modeling engine according to the plurality of the second modeling requests and the plurality of the third modeling requests to obtain the first modeling data corresponding to the first modeling request of the user includes: Determine a first request order corresponding to each of the plurality of second modeling requests and a second request order corresponding to each of the plurality of third modeling requests; Sending a plurality of second modeling requests and a plurality of the third modeling requests to the modeling engine according to the first request sequence and the second request sequence to obtain a plurality of second modeling data returned by the modeling engine; The first modeling data corresponding to the first modeling request of the user is obtained according to the plurality of second modeling data returned by the modeling engine.

5. The data processing method according to claim 4, characterized in that: The acquiring the first modeling data corresponding to the first modeling request of the user according to the plurality of second modeling data returned by the modeling engine includes: Cache each of the second modeling data in the first cache area; If the first number of the second modeling data in the first cache area is equal to the number of first modeling requests corresponding to the second modeling parameter, deduplication processing is performed on the multiple second modeling data in the first cache area to obtain the deduplication-processed first modeling data.

6. The data processing method according to claim 4, characterized in that: The method further comprises: splitting the plurality of the second modeling requests into a plurality of first tasks; wherein each of the first tasks includes a different number of second modeling requests; The plurality of first tasks are respectively sent to a computing unit including the modeling engine corresponding to each of the first tasks.

7. The data processing method according to claim 6, characterized in that: The first tasks corresponding to the computing units are matched according to the corresponding performances of the computing units.

8. A data processing device, characterized in that: include: An acquisition module, configured to acquire a plurality of first modeling requests from a user, wherein the first modeling requests include a plurality of first modeling parameters required for modeling; An identification module, used for performing data screening on a plurality of the first modeling parameters to obtain screened second modeling parameters; A calling module is used to call a modeling engine according to a plurality of the second modeling parameters to obtain first modeling data corresponding to the first modeling request of the user.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images