A method and system for online analysis of massive data based on multiple computing engines

Abstract

The present invention proposes a method and system for online analysis of massive data based on multiple computing engines. The method comprises: responding to job requests initiated by third-party applications and submitting the job requests to a web-based online engine job queue; an engine job submission executor (EJSR) polling a session computing engine job queue, extracting job information (JCI), and submitting it to a session computing engine (WebActor); starting the session computing engine (WebActor) and, upon receiving the job information, publishing job execution conditions (JEC) to a remote message queue (MQT); automatically starting the session computing engine (Actor) after the web service is started and subscribing to the remote message queue topic (MQT); and simultaneously starting a job executor (JER), which continuously polls the job queue and calls a specific engine for processing. Calling a multi-computing engine-based online analysis interface for massive data not only improves data processing efficiency and computing reliability, but also enhances system flexibility and scalability, and helps reduce system maintenance costs.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a method and system for online analysis of massive data based on multiple computing engines. Background Art

[0002] In the big data era, efficiently processing massive amounts of data is a challenge every data scientist and engineer faces. Distributed computing, as a means of efficiently processing massive amounts of data, plays a vital role in big data computing engines. In big data applications, conversational computing is a highly effective technical tool. By analyzing user behavior data, companies can deeply understand user needs and behavior patterns and, based on these findings, optimize product design, operational strategies, and service experience.

[0003] However, conversational computing also faces challenges and difficulties. For example, how to accurately define a conversation, how to identify similar or overlapping conversations, and how to perform real-time processing and analysis. To overcome these difficulties, conversational computing requires the integration of technologies such as distributed computing, machine learning, and data visualization. These technologies can help companies better understand user behavior data and leverage this information to develop more intelligent and efficient products and services. By continuously exploring and applying these technologies, companies can better serve their customers, enhance their market competitiveness, and maximize their business value.

[0004] Akka is an excellent distributed computing framework, offering multiple advantages, including high scalability, high concurrency, high reliability, distributed computing, and responsive programming. Based on the actor model, Akka divides applications into independent computing units that can asynchronously receive and send messages to each other and create more actors as needed, making it easy to scale to multiple nodes and achieve highly scalable applications. Akka also enables efficient concurrent programming, improving application performance and throughput. It ensures high application reliability through monitoring and recovery mechanisms, preventing system crashes and data loss. It supports the deployment of actors on different nodes to build complex distributed applications. It also supports a responsive programming model, enabling non-blocking I / O operations, improving application responsiveness and scalability. Therefore, combining Akka with Spark to build a conversational computing engine architecture can improve data processing efficiency, computational reliability, system flexibility, and scalability, while also reducing system maintenance costs.

[0005] Currently, online analysis of massive data based on multiple computing engines faces some important problems. One of the more prominent problems is that due to the complexity of distributed computing, achieving efficient data communication and synchronization in online analysis methods of massive data based on multiple computing engines remains a challenge.

[0006] In view of this, it is very meaningful to propose a massive data online analysis method and system based on multiple computing engines. Summary of the Invention

[0007] In order to solve the problem of low data communication and synchronization efficiency faced by the existing online analysis method of massive data based on multiple computing engines, the present invention provides an online analysis method of massive data based on multiple computing engines and a system thereof to solve the above-mentioned technical defects.

[0008] In a first aspect, the present invention proposes a method for online analysis of massive data based on multiple computing engines, the method comprising the following steps:

[0009] Responding to job requests initiated by third-party applications and submitting the job requests to the web-side online engine job queue;

[0010] The engine job submission executor EJSR polls the session computing engine job queue, extracts the job information JCI and submits it to the session computing engine WebActor;

[0011] Start the session computing engine WebActor, and after receiving the job information, publish the job execution condition JEC to the remote message queue MQT. After the Web service is started, the session computing engine Actor is automatically started and subscribes to the remote message queue topic MQT.

[0012] At the same time, the job executor JER is started. The job executor JER continuously polls the job queue to check whether there is job information JCI in the job queue. When the job information JCI is polled, it creates and submits the job information JCI to the job executor JERA. The job executor JERA calls a specific engine for processing.

[0013] Preferably, it also includes:

[0014] After receiving the published job execution condition JEC, the remote message queue topic MQT broadcasts it to the session computing engine actor that subscribes to the MQT topic. The session computing engine actor-SeEActor that meets the job execution condition JEC sends a message to the session computing engine WebActor that it can receive the job information JCI and creates a first timer to receive confirmation information. After receiving the confirmation information, the session computing engine WebActor closes the first timer. It then sends the job information JCI to the session computing engine actor-SeEActor-1 that first receives the confirmation information that meets the conditions, and creates a second timer to wait for the returned confirmation information. After receiving the job information JCI, SeEActor-1 closes the second timer and sends the job information JCI to the job queue. It also sends a message confirming receipt of the JCI to the session computing engine WebActor-WseEActor. At this time, WSeEActor sends a message to the remaining SeEActors that the broadcast message has expired. After receiving the confirmation information, WSeEActor closes the third timer, sends a confirmation message to SeEActor-1, and ends.

[0015] More preferably, it also includes:

[0016] If the first timer, the second timer, or the third timer times out, an exception is thrown and the data is persisted to the distributed storage database. The third-party application periodically pulls the corresponding persistence table of the distributed storage database to obtain relevant request result information.

[0017] Further preferably, the job executor JERA calls a specific engine to perform processing, specifically including:

[0018] The job executor JERA calls the initialization engine method, then calls the pre-processing method before executing the job. After completion, it calls the execution method to start the job. After the job ends, it calls the cleanup operation method and persists the result to the distributed storage database. If an error is reported during job execution, the corresponding exception handling method is called and the exception information is persisted to the distributed storage database. Finally, the third-party application will periodically pull the corresponding persistence table of the distributed storage database to obtain relevant request result information, and the entire request and response is completed.

[0019] More preferably, it also includes:

[0020] A third-party application initiates a job request through the Web, which calls the engine executor control layer interface (EECI). After receiving the job request, the engine executor control layer interface (EECI) calls an engine executor service class interface (EESI). The passed parameters are checked. If the job ID is empty, an exception is thrown. Otherwise, the job information (JCI) is submitted to the Web session computing engine job queue based on the engine's operating mode.

[0021] More preferably, it also includes:

[0022] Get the corresponding job ID according to the request parameters and start a session computing engine WebActor instance WSeEActor named with the job ID.

[0023] In a second aspect, an embodiment of the present invention further provides a massive data online analysis system based on multiple computing engines, the system comprising:

[0024] A job request module configured to respond to job requests initiated by third-party applications and submit the job requests to the web-side online engine job queue;

[0025] The polling session module is configured to poll the session computing engine job queue of the engine job submission executor EJSR, and retrieve the job information JCI and submit it to the session computing engine WebActor;

[0026] The session computing engine WebActor module is configured to start the session computing engine WebActor, publish the job execution condition JEC to the remote message queue MQT after receiving the job information, automatically start the session computing engine Actor after the Web service is started, and subscribe to the remote message queue topic MQT;

[0027] The job executor JER module is configured to start the job executor JER. The job executor JER continuously polls the job queue to check whether there is job information JCI in the job queue. When the job information JCI is polled, it creates and submits the job information JCI to the job executor JERA. The job executor JERA calls a specific engine for processing.

[0028] Preferably, it also includes:

[0029] The session computing engine actor module is configured to automatically start the session computing engine actor after the web service is started and subscribe to the remote message queue topic MQT;

[0030] a timer module configured to receive confirmation information;

[0031] The database module is configured to throw an exception if the first timer, the second timer, or the third timer times out, and then persist the data to the distributed storage database. The third-party application will periodically pull the corresponding persistent table of the distributed storage database to obtain relevant request result information;

[0032] The parameter verification module is configured to perform parameter verification on the passed parameters.

[0033] In a third aspect, an embodiment of the present invention provides an electronic device comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0034] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] (1) In the actual modeling process, the present invention not only improves data processing efficiency and computational reliability by invoking an interface for a massive data online analysis method based on multiple computing engines, but also enhances system flexibility and scalability, while also helping to reduce system maintenance costs. Furthermore, the present invention can better serve customers, enhance market competitiveness, and maximize commercial value. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and, together with the description, serve to explain the principles of the present invention. Other embodiments and many of the expected advantages of the embodiments will be readily apparent as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with respect to each other. Like reference numerals designate corresponding similar parts.

[0038] Figure 1 1 is a flow chart of a method for online analysis of massive data based on multiple computing engines according to an embodiment of the present invention;

[0039] Figure 2 Schematic diagram of the overall framework of a method for online analysis of massive data based on multiple computing engines according to an embodiment of the present invention;

[0040] Figure 3 Schematic diagram of a massive data online analysis system based on multiple computing engines according to an embodiment of the present invention;

[0041] Figure 4 It is a schematic structural diagram of a computer device suitable for implementing an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In the following detailed description, reference is made to the accompanying drawings, which form a part of the detailed description and are illustrated by illustrative specific embodiments in which the present invention may be practiced. To this end, directional terms, such as "top," "bottom," "left," "right," "up," "down," etc., are used with reference to the orientation of the figures being described. Because the components of the embodiments may be positioned in several different orientations, directional terms are used for illustrative purposes and are in no way limiting. It should be understood that other embodiments may be utilized or logical changes may be made without departing from the scope of the present invention. Therefore, the following detailed description should not be adopted in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0043] Figure 1 The embodiment of the present invention discloses a method for online analysis of massive data based on multiple computing engines, such as Figure 1 As shown, the method includes the following steps:

[0044] S1. Responding to a job request initiated by a third-party application and submitting the job request to the web-side online engine job queue;

[0045] S2. The engine job submission executor EJSR polls the session computing engine job queue, extracts the job information JCI, and submits it to the session computing engine WebActor.

[0046] S3. Start the session computing engine WebActor. After receiving the job information, publish the job execution condition JEC to the remote message queue MQT. After the Web service is started, the session computing engine Actor is automatically started and subscribes to the remote message queue topic MQT.

[0047] S4. At the same time, the job executor JER is started. The job executor JER continuously polls the job queue to check whether there is job information JCI in the job queue. When the job information JCI is polled, it creates and submits the job information JCI to the job executor JERA. The job executor JERA calls a specific engine for processing.

[0048] Specifically, Figure 2 Shows the overall framework schematic diagram of the present invention. Figure 2 The implementation method of the present invention is mainly divided into two stages, and the specific method is described as follows:

[0049] In step 1, a third-party application initiates a job request through the web, which calls the Engine Executor Control Interface (EECI).

[0050] Step 2: After receiving the job request, the Engine Executor Control Layer Interface (EECI) will call an Engine Executor Service Class Interface (EESI);

[0051] Step 3: Verify the parameters passed in. If the job ID is empty, an exception will be thrown.

[0052] In step 4, if the job ID is not empty, the job information (JCI) will be submitted to the Web-side session computing engine job queue according to the engine's operating mode;

[0053] In step 5, the Engine Job Submission Executor (EJSR) polls the Session Computing Engine job queue, retrieves the job information (JCI), and submits it to the Session Computing Engine WebActor (WSeEActor).

[0054] Step 6: Get the corresponding job ID according to the request parameter and start a session computing engine WebActor instance (WSeEActor) named after this job ID;

[0055] Step 7: After receiving the job information, the session computing engine WebActor instance will publish the job execution condition (JEC) to the remote message queue (MQT) (such as Figure 2 7.1.1 Send JEC), and the WSeEActor instance creates a timer (such as Figure 2 (Created in 7.1.2) is used to receive confirmation information;

[0056] In step 8, when the Web service starts, it automatically starts the session computing engine actor (SeEActor) and subscribes to the remote message queue topic (MQT).

[0057] Specifically, after receiving the published JEC, MQT will broadcast it to SeEActors that have subscribed to the MQT topic; SeEActors that meet the JEC conditions will send a message to WSeEActor (such as Figure 2 8.1.1 Send and receive JCI), and create a timer for receiving confirmation information (such as Figure 2 8.1.2 created), WSeEActor will turn off the timer after receiving the confirmation message (such as Figure 2 8.1.3 closed); then send JCI to SeEActor-1 which first receives the confirmation message of sending the qualified condition (such as Figure 2 8.2.1 Send JCI);

[0058] At the same time, create a timer to wait for the confirmation information returned (such as Figure 2 After receiving the JCI, SeEActor-1 will turn off the timer (e.g. Figure 2 8.2.3 closed), and sends the JCI to the job queue (such as Figure 2 8.2.4 Send JCI), and also send a confirmation message to WSeEActor (such as Figure 2 8.3.1 Send confirmation of receipt of JCI), at this time WSeEActor will send the broadcast message expired information to other SeEActors (such as Figure 2 8.2.5 Send notification expiration information); WSeEActor will turn off the timer after receiving the confirmation message (such as Figure 2 8.3.2 Close), then send a confirmation message to SeEActor-1 and end itself.

[0059] During the entire core interaction period, if the timer waits for a timeout, an exception will be thrown, and then the data will be persisted to the distributed storage database. The third-party application will periodically pull the corresponding persistence table of the distributed storage database to obtain relevant request result information.

[0060] In step 9, when the Web service is started, the job executor (JER) is started. After starting, the job executor (JER) will continuously poll the job queue to check whether there is job information (JCI) in the job queue. Once the job information (JCI) is polled, the job information (JCI) is created and submitted to the job executor (JERA);

[0061] In step 10, the job executor (JERA) calls the initialization engine method, then calls the pre-job processing method before executing the job. After completion, it calls the execution method to start the job. After the job ends, it calls the cleanup method and persists the results to the distributed storage database. If an error occurs during job execution, the corresponding exception handling method is called and the exception information is persisted to the distributed storage database. Finally, the third-party application periodically pulls the corresponding persistent table in the distributed storage database to obtain the relevant request result information, completing the entire request and response process.

[0062] The method of the present invention is implemented in the Tiansu application of the Qiankun big data operating system and has been repeatedly verified in recent years in the practical modeling work of the ministry and provincial level, such as smart cities, public health event prevention and control, and anti-fraud. In the actual modeling process, by calling the interface for online analysis of massive data based on multiple computing engines, not only can the data processing efficiency and calculation reliability be improved, but also the flexibility and scalability of the system can be improved, while also helping to reduce system maintenance costs. At the same time, it can also better serve customers, enhance market competitiveness, and maximize commercial value.

[0063] In a second aspect, the embodiment of the present invention further discloses a massive data online analysis system based on multiple computing engines, such as Figure 3 As shown, the system includes: a job request module 31, a polling session module 32, a session computing engine WebActor module 33, a job executor JER module 34, a session computing engine Actor module 35, a timer module 36, a database module 37 and a parameter verification module 38.

[0064] In a specific embodiment, the job request module 31 is configured to respond to a job request initiated by a third-party application and submit the job request to the online engine job queue on the web side; the polling session module 32 is configured for the engine job submission executor EJSR to poll the session computing engine job queue, and take out the job information JCI and submit it to the session computing engine WebActor; the session computing engine WebActor module 33 is configured to start the session computing engine WebActor, and after receiving the job information, publish the job execution condition JEC to the remote message queue MQT, automatically start the session computing engine Actor after the Web service is started, and subscribe to the remote message queue topic MQT.

[0065] The job executor JER module 34 is configured to start the job executor JER. The job executor JER continuously polls the job queue to check whether there is job information JCI in the job queue. When the job information JCI is polled, it creates and submits the job information JCI to the job executor JERA. The job executor JERA calls a specific engine for processing; the session computing engine Actor module 35 is configured to automatically start the session computing engine Actor after the Web service is started, and subscribe to the remote message queue topic MQT; the timer module 36 is configured to receive confirmation information; the database module 37 is configured to throw an exception if the first timer, the second timer or the third timer waits for timeout, and then uniformly persists it to the distributed storage database. The third-party application will periodically pull the corresponding persistence table of the distributed storage database to obtain relevant request result information; the parameter verification module 38 is configured to perform parameter verification on the parameter.

[0066] Reference below Figure 4 , which shows an electronic device (eg Figure 1 A schematic structural diagram of a computer device 600 (a server or terminal device as shown). Figure 4 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0067] like Figure 4As shown, the computer device 600 includes a central processing unit (CPU) 601 and a graphics processing unit (GPU) 602, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 603 or the program loaded from the storage part 609 to the random access memory (RAM) 606. Various programs and data required for the operation of the device 600 are also stored in the RAM 604. The CPU 601, GPU 602, ROM 603 and RAM 604 are connected to each other via a bus 605. An input / output (I / O) interface 606 is also connected to the bus 605.

[0068] The following components are connected to the I / O interface 606: an input section 607 including a keyboard, a mouse, and the like; an output section 608 including a display such as a liquid crystal display (LCD), a speaker, and the like; a storage section 609 including a hard disk and the like; and a communication section 610 including a network interface card such as a LAN card or a modem. The communication section 610 performs communication processing via a network such as the Internet. A drive 611 may also be connected to the I / O interface 606 as needed. A removable medium 612, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like, is installed in the drive 611 as needed, so that a computer program read therefrom can be installed into the storage section 609 as needed.

[0069] In particular, according to an embodiment disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment disclosed in the present invention includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 610, and / or installed from a removable medium 612. When the computer program is executed by a central processing unit (CPU) 601 and a graphics processing unit (GPU) 602, the above-mentioned functions defined in the method of the present invention are performed.

[0070] It should be noted that the computer-readable medium described in the present invention may be a computer-readable signal medium or a computer-readable medium, or any combination thereof. Computer-readable media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, apparatuses, or components, or any combination thereof. More specific examples of computer-readable media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, apparatus, or component. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution apparatus, device, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0071] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0072] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based device that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0073] The modules involved in the embodiments of the present invention may be implemented in software or hardware, and may also be set in a processor.

[0074] As another aspect, the present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently and not be assembled into the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device: responds to a job request initiated by a third-party application and submits the job request to a web-side online engine job queue; the engine job submission executor EJSR polls the session computing engine job queue and extracts job information JCI and submits it to the session computing engine WebActor; the session computing engine WebActor is started, and after receiving the job information, the job execution condition JEC is published to the remote message queue MQT, and after the web service is started, the session computing engine Actor is automatically started and subscribed to the remote message queue topic MQT; and the job executor JER is started at the same time, and the job executor JER continuously polls the job queue to check whether there is job information JCI in the job queue, and when the job information JCI is polled, creates and submits the job information JCI to the job executor JERA, which calls a specific engine for processing.

[0075] The above description is merely an illustration of the preferred embodiments of the present invention and the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the present invention.

Claims

1. A method for online analysis of massive data based on multiple computing engines, characterized in that: The method comprises the following steps: Responding to job requests initiated by third-party applications and submitting the job requests to the web-side online engine job queue; The engine job submission executor EJSR polls the session computing engine job queue, extracts the job information JCI and submits it to the session computing engine WebActor; Start the session computing engine WebActor, and after receiving the job information, publish the job execution condition JEC to the remote message queue MQT. After the Web service is started, the session computing engine Actor is automatically started and subscribes to the remote message queue topic MQT. At the same time, the job executor JER is started. The job executor JER continuously polls the job queue to check whether there is job information JCI in the job queue. When the job information JCI is polled, it creates and submits the job information JCI to the job executor JERA. The job executor JERA calls a specific engine for processing.

2. The method for online analysis of massive data based on multiple computing engines according to claim 1, characterized in that: Also includes: After receiving the published job execution condition JEC, the remote message queue topic MQT broadcasts it to the session computing engine Actor that subscribes to the remote message queue topic MQT; The conversation computing engine actor-SeEActor that meets the job execution condition JEC will send a message to the conversation computing engine WebActor to receive the job information JCI and create a first timer for receiving confirmation information. After receiving the confirmation information, the conversation computing engine WebActor will close the first timer. Then the job information JCI is sent to the session computing engine Actor-SeEActor-1 that first receives the confirmation information that meets the conditions, and a second timer is created to wait for the returned confirmation information; After receiving the job information JCI, SeEActor-1 will turn off the second timer and send the job information JCI to the job queue. At the same time, it will also send a confirmation message of receiving the JCI to the session computing engine WebActor-WseEActor. At this time, WSeEActor will send a broadcast message expiration information to other SeEActors. After receiving the confirmation message, WSeEActor will stop the third timer, then send a confirmation message to SeEActor-1 and end.

3. The method for online analysis of massive data based on multiple computing engines according to claim 2, characterized in that: Also includes: If the first timer, the second timer, or the third timer times out, an exception is thrown and the data is persisted to the distributed storage database. The third-party application periodically pulls the corresponding persistence table of the distributed storage database to obtain relevant request result information.

4. The method for online analysis of massive data based on multiple computing engines according to claim 3, characterized in that: The job executor JERA calls a specific engine to process the following: The job executor JERA calls the initialization engine method, then calls the pre-processing method before executing the job. After the job is completed, it calls the execution method of the start job. After the job is completed, it calls the cleanup operation method and persists the results to the distributed storage database. If an error occurs during job execution, the corresponding exception handling method is called and the exception information is persisted to the distributed storage database; Finally, the third-party application will periodically pull the corresponding persistent table of the distributed storage database to obtain relevant request result information, and the entire request and response will be completed.

5. The method for online analysis of massive data based on multiple computing engines according to claim 4, characterized in that: Also includes: The third-party application initiates a job request through the Web, which calls the engine executor control layer interface EECI; After receiving a job request, the engine executor control layer interface EECI will call an engine executor service class interface EESI; Parameter verification is performed on the passed parameters. If the job ID is empty, an exception will be thrown. Otherwise, the job information JCI will be submitted to the Web-side session computing engine job queue based on the engine's operating mode.

6. The method for online analysis of massive data based on multiple computing engines according to claim 5, characterized in that: Also includes: Get the corresponding job ID according to the request parameters and start a session computing engine WebActor instance WSeEActor named with the job ID.

7. A massive data online analysis system based on multiple computing engines, characterized by: The system includes: A job request module configured to respond to job requests initiated by third-party applications and submit the job requests to the web-side online engine job queue; The polling session module is configured to poll the session computing engine job queue of the engine job submission executor EJSR, and retrieve the job information JCI and submit it to the session computing engine WebActor; The session computing engine WebActor module is configured to start the session computing engine WebActor, publish the job execution condition JEC to the remote message queue MQT after receiving the job information, automatically start the session computing engine Actor after the Web service is started, and subscribe to the remote message queue topic MQT; The job executor JER module is configured to start the job executor JER. The job executor JER continuously polls the job queue to check whether there is job information JCI in the job queue. When the job information JCI is polled, it creates and submits the job information JCI to the job executor JERA. The job executor JERA calls a specific engine for processing.

8. The massive data online analysis system based on multiple computing engines according to claim 7, characterized in that: Also includes: The session computing engine actor module is configured to automatically start the session computing engine actor after the web service is started and subscribe to the remote message queue topic MQT; a timer module configured to receive confirmation information; The database module is configured to throw an exception if the first timer, the second timer, or the third timer times out, and then persist the data to the distributed storage database. The third-party application will periodically pull the corresponding persistent table of the distributed storage database to obtain relevant request result information; The parameter verification module is configured to perform parameter verification on the passed parameters.

9. An electronic device comprising: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images