Data parallel collection method and device, equipment and medium

Abstract

The invention discloses a data parallel collection method and device, equipment and a medium. The method comprises the steps of obtaining a data instruction set; identifying the data type of each data instruction in the data instruction set, and performing parallel processing on each data instruction according to an identification result to obtain each data processing result; and updating the data instruction set according to each data processing result, and obtaining a data collection result based on the updated data instruction set in response to an input instruction of a user. By means of the technical scheme, parallel collection of various types of data can be achieved, manpower consumption of data collection operation is reduced, and data collection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method, device, equipment and medium for parallel data collection. Background Art

[0002] In today's highly digital business and technology environment, various systems are extensively and deeply integrated into business processes across all domains. However, no matter how rigorously tested the initial phases, unexpected flaws are inevitably exposed after a system goes live. To ensure smooth business operations, operations and maintenance personnel must navigate across multiple systems to identify and correct data anomalies.

[0003] Currently, the mainstream operations and maintenance model relies heavily on manpower. On the one hand, operations and maintenance personnel must set up monitoring tasks in various business systems, obtain monitoring data, and proactively identify anomalies. In this process, monitoring data, as a crucial record of system operating status and operations, is invaluable. However, under this approach, the retention of monitoring data depends entirely on individual operating habits. Furthermore, after discovering anomalies, each piece of data must be manually calculated. This process is not only time-consuming and labor-intensive, but also extremely inefficient and prone to errors. Furthermore, given the diversification of enterprise businesses and the interweaving of multiple systems, operations and maintenance personnel must frequently switch databases to perform data comparisons and handle anomalies. Furthermore, the current operations and maintenance model suffers from serious deficiencies in data collection, with data collection often fragmented and serialized, failing to meet the requirements for rapidly locating and processing anomalies. Faced with the massive amounts of data generated by multiple systems, traditional methods are not only slow to collect data but also struggle to integrate key information quickly.

[0004] In summary, existing data collection methods are unable to achieve parallel collection in multi-system scenarios, and the data collection operation consumes a lot of manpower and has low data collection efficiency. Summary of the Invention

[0005] The present invention provides a parallel data collection method, device, equipment and medium, which can solve the problems of existing data collection methods that cannot achieve parallel collection in multi-system scenarios, and at the same time, the data collection operation consumes a lot of manpower and has low data collection efficiency.

[0006] In a first aspect, an embodiment of the present invention provides a method for parallel data collection, the method comprising:

[0007] Get data instruction set;

[0008] Identify the data type of each data instruction in the data instruction set, and perform parallel processing on each data instruction according to the identification result to obtain each data processing result;

[0009] The data instruction set is updated according to each data processing result, and the data collection result is obtained based on the updated data instruction set in response to the user's input instruction.

[0010] In a second aspect, an embodiment of the present invention provides a parallel data collection device, the device comprising:

[0011] A data acquisition module, used for acquiring a data instruction set;

[0012] The parallel processing module is used to identify the data type of each data instruction in the data instruction set, and to perform parallel processing on each data instruction according to the identification result to obtain each data processing result;

[0013] The result acquisition module is used to update the data instruction set according to each data processing result, and acquire the data collection result based on the updated data instruction set in response to the user's input instruction.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device, comprising:

[0015] at least one processor; and:

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform a parallel data collection method according to any embodiment of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a parallel data collection method described in any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention obtains a data instruction set, then identifies the data type of each data instruction in the data instruction set, and processes each data instruction in parallel according to the identification result to obtain each data processing result. Finally, the data instruction set is updated according to each data processing result, and the data collection result is obtained based on the updated data instruction set in response to the user's input instruction. This solves the problems of the existing data collection method that it is impossible to achieve parallel collection in a multi-system scenario, and the data collection operation consumes a lot of manpower and the data collection efficiency is low. It realizes the parallel collection of multiple types of data, reduces the manpower consumption of data collection operation, and improves the efficiency of data collection.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 This is a flow chart of a parallel data collection method provided according to the first embodiment of the present invention;

[0023] Figure 2 This is a flow chart of a parallel data collection method provided according to the second embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of a parallel data collection device provided according to the third embodiment of the present invention;

[0025] Figure 4 The present invention is a schematic structural diagram of an electronic device for implementing a parallel data collection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," and the like in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, any variations of the terms "including" and "having" are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0028] Example 1

[0029] Figure 1 This is a flowchart of a method for parallel data collection provided in Example 1 of the present invention. This embodiment is applicable to the situation where multiple types of data are collected in parallel in a multi-system scenario. The method can be executed by a device for parallel data collection. The device for parallel data collection can be implemented in the form of hardware and / or software. The device for parallel data collection can be configured in a terminal or server with a parallel data collection function.

[0030] like Figure 1 As shown, the method includes:

[0031] S110: Obtain a data instruction set.

[0032] The data instruction includes: a data node address, a data type and a data parameter, and the data type includes: a query type, a request type or a calculation type.

[0033] Furthermore, a data instruction is the basic unit of a data instruction set and includes key elements such as a data node address, data type, and data parameters. The data node address is used to precisely locate the system that generates the data processing result matching the data instruction. In this embodiment, the system types can include: data node, client, and executor.

[0034] The data type further clarifies the nature and purpose of the data instruction, which can be categorized as query, request, and calculation types. A query-type data instruction indicates that the data parameters in the instruction are used to query data in a specific data node. For example, in a large database system, when an administrator needs to understand the operational data of a specific user within a certain time period, they initiate a query-type data instruction. The data parameters in this instruction may include information such as the query time range and user ID. The system then uses these parameters to filter and search through the vast amount of data to obtain the required data information. Furthermore, a request-type data instruction means that the corresponding data parameters are used to request a web page. The data parameters in this instruction may include the requested URL, the request method (e.g., GET, POST), and request header information. After receiving the data instruction, the client processes the request based on the parameter information and returns the corresponding web page to the user. Furthermore, in a calculation-type data instruction, the corresponding data parameters are used as input parameters, and a specialized script performs calculations to obtain the calculation results. For example, in a data analysis system, when statistical analysis of large amounts of sales data is required, calculation-type data instructions are used. The data parameters in these instructions may include information such as the storage location of the sales data, the analysis time range, and the analysis metric (such as sales revenue, sales volume, etc.). Based on the rules, the system passes these data parameters to a specially written data analysis script. The script then processes and calculates the data based on these input parameters, ultimately deriving calculation results such as sales growth trends and the sales share of different products.

[0035] It should be noted that, in this embodiment, the specific data content of the data instruction can be edited and set by relevant personnel according to the needs of the actual implementation scenario, and this embodiment does not impose any restrictions on this.

[0036] S120 , identifying the data type of each data instruction in the data instruction set, and performing parallel processing on each data instruction according to the identification result to obtain each data processing result.

[0037] Among them, each data instruction is processed in parallel according to the recognition result to obtain each data processing result, including: if the data type in the current data instruction is recognized as a query type, the data parameters are parsed using a pre-configured document object model parsing technology to obtain a target query language that matches the data instruction; according to the data node address in the data instruction, the target query language is sent to a target node that matches the data node address; the target query language is processed by the target node, and the processing result is bound to the data type that matches the data instruction to obtain a data processing result that matches the data instruction.

[0038] For example, when the data type in the current data instruction is identified as a query type, the data parameters are parsed using pre-configured Document Object Model (DOM) parsing technology. DOM parsing technology is a standard programming interface for processing XML or HTML documents. It parses documents into a tree structure, allowing developers to easily access and manipulate individual elements within the document. For example, in an e-commerce system, if a query-type data instruction records a product inventory query operation, its data parameters may include information such as the product ID and the query time range. This information is stored in a specific format, such as XML. In this case, DOM parsing technology will parse the data parameters according to XML syntax rules into a form that can be understood and processed by the system, thereby obtaining a target query language that matches the data instruction. Then, based on the data node address in the data instruction, the target query language is sent to the target node that matches the address. For example, in a distributed database system, different database server nodes have their own addresses. Once the data node address of the data instruction is determined, the system will send the target query language to the database node corresponding to the data node address. Upon receiving the target query language, the target node will process it according to its own database management system rules. For example, in a MySQL database, it executes the target query language. If the target query language is used to query the inventory of the target product, it retrieves the inventory quantity data of the product that meets the conditions from the database. Finally, the target node binds the processing result to the data type that matches the data instruction, obtaining the data processing result that matches the data instruction. Assuming the query result is that the inventory quantity of the product is 100 pieces, the final data processing result may be an information pair containing the data type (query type) and the processing result (100 pieces), that is, {"data type":"query type","processing result":100}.

[0039] Furthermore, each data instruction is processed in parallel according to the identification result to obtain each data processing result, which also includes: if the data type in the current data instruction is identified as a request type, a pull-up operation is performed on the target client that matches the data node address according to the data node address in the data instruction; based on the data parameters in the data instruction, a positioning request is initiated to the target client, and the positioning result and the data type that matches the data instruction are bound to obtain a data processing result that matches the data instruction.

[0040] Specifically, if the data type in the current data instruction is identified as a request type, the system first launches the target client matching the data node address in the data instruction. For example, in a web application system, the data node address may correspond to a client program on a specific user terminal device, such as a browser. After determining the target client's address, the system launches the corresponding program or service. For example, in the case of a browser, the system wakes up the browser program and prepares for subsequent operations. Then, based on the data parameters in the data instruction, a location request is initiated for the target client. In a request-type data instruction, the data parameters contain detailed request information, such as the requested URL. For example, if the URL included in the data parameters is "https: / / www.123," the system will initiate a location request on the target client (e.g., a browser) based on this URL, attempting to access the webpage. After receiving the request, the target client sends the request information to the corresponding server and retrieves the webpage data returned by the server. Finally, the location result (e.g., webpage content or access status information) is bound to the data type matching the data instruction to obtain the data processing result matching the data instruction. If the web page is successfully accessed, the data processing result may be {"data type":"request type","processing result":"web page content:..."}.

[0041] Furthermore, each data instruction is processed in parallel according to the recognition result to obtain each data processing result, which also includes: if the data type in the current data instruction is recognized as a calculation type, the data parameters in the data instruction are processed by a preset independent code tool to obtain independent parameters matching the data instruction; according to the data node address in the data instruction, a target executor matching the data node address is pulled up; the independent parameter is calculated by the target executor, and the calculation result is bound to the data type matching the data instruction, and output as a data processing result matching the data instruction.

[0042] In a specific implementation scenario of this embodiment, when the data type in the current data instruction is identified as a computational type, a pre-set independent code tool is used to process the data parameters in the data instruction. The pre-set independent code tool is a software component developed specifically for processing computational tasks. For example, in some data analysis systems, Python data analysis libraries (such as Pandas and NumPy) can be used as independent code tools. Assuming that the data parameters in the data instruction are a set of sales data, such as [100, 200, 300, 400], and a calculation rule (such as calculating an average), the independent code tool will process the data according to these rules to obtain independent parameters that match the data instruction. In this example, the calculated independent parameter may be the average value of 300. Then, based on the data node address in the data instruction, a target executor matching the address is launched. The target executor is the component responsible for executing the computational task. In a distributed computing environment, the target executor can be, for example, a Python interpreter pre-configured with a specific computational script. The system will find the corresponding target executor based on the data node address and launch it. Finally, the target executor uses the preset script to perform calculation operations on independent parameters based on the script rules, and binds the calculation results to the data type that matches the data instruction and outputs them as data processing results that match the data instruction.

[0043] S130: updating the data instruction set according to each data processing result, and acquiring a data collection result based on the updated data instruction set in response to a user input instruction.

[0044] Specifically, the update operation involves replacing the data instruction that matches the processing result with the generated data processing result. For example, in a resource planning system, after processing a query-type data instruction, assuming that this data instruction is about querying the sales data of a specific product within a certain time period, after processing the data parameters using DOM parsing technology and executing the query operation on the target node, a specific sales data result is obtained (e.g., 500 units of the product were sold within a specific time period). At this point, the system will replace the original data instruction (containing information such as the data node address, data type, and data parameters) with a new record containing the processing result. In the new record, the data type remains unchanged (still the query type), and the data parameter portion will be updated to the processing result (i.e., sales data of 500 units).

[0045] The technical solution of the embodiment of the present invention obtains a data instruction set, then identifies the data type of each data instruction in the data instruction set, and processes each data instruction in parallel according to the identification result to obtain each data processing result. Finally, the data instruction set is updated according to each data processing result, and the data collection result is obtained based on the updated data instruction set in response to the user's input instruction, thereby realizing the parallel collection of multiple types of data, reducing the manpower consumption of data collection operations, and improving the efficiency of data collection.

[0046] Example 2

[0047] Figure 2 This is a flowchart of a parallel data collection method provided in Example 2 of the present invention. This embodiment is refined based on the above embodiment. In this embodiment, the method of obtaining each data processing result based on the updated data instruction set in response to the user's input instructions is specifically refined.

[0048] like Figure 2 As shown, the method includes:

[0049] S210: Obtain a data instruction set.

[0050] S220 , identifying the data type of each data instruction in the data instruction set, and performing parallel processing on each data instruction according to the identification result to obtain each data processing result.

[0051] S230, updating the data instruction set according to each data processing result, and obtaining the user's input instruction;

[0052] If the input instruction is a termination instruction, execute S240;

[0053] If the input instruction is an execution instruction, execute S250.

[0054] It should be noted that in this embodiment, the user can enter instructions in a variety of ways, such as directly entering text instructions in the system's command line interface, or entering instructions indirectly by clicking buttons, selecting menu options, etc. in the graphical operation interface. This embodiment does not impose any restrictions here.

[0055] S240: Output the updated data instruction set as the data collection result.

[0056] S250. According to the user's input operation, the data node address of each data instruction in the data instruction set is set to update the data instruction set, and then S260 is executed.

[0057] Specifically, if the input instruction is an execution instruction, it indicates that the user wants the system to continue subsequent operations to obtain data information that better meets the requirements. In this case, the method described in this embodiment can set the data node address of each data instruction in the data instruction set according to the user's input operation, thereby further updating the data instruction set.

[0058] It is easy to understand that, based on the above steps, the set data instruction includes the data node address, data type and data parameters, which can be used to obtain the data processing results matching the data instruction based on the method described in this embodiment.

[0059] S260. Return and execute an operation of identifying the data type of each data instruction in the data instruction set based on the updated data instruction set.

[0060] Specifically, based on the above steps, after completing the update of the data instruction set, an operation of identifying the data type of each data instruction in the data instruction set is executed based on the updated data instruction set. This step is to restart the data acquisition process in the above steps. The method described in this embodiment will parse the updated data instruction set again and re-identify the data type of each data instruction. Regardless of the query type, request type or calculation type, it will be processed according to the corresponding processing logic. Through this cyclic processing method, the system can continuously respond to changes in user needs, and use the data collection results obtained from the previous process as the data instruction set for the next process, providing users with more accurate and comprehensive data collection results, thereby better supporting operation and maintenance monitoring and system optimization.

[0061] The technical solution of the embodiment of the present invention obtains a data instruction set, then identifies the data type of each data instruction in the data instruction set, and processes each data instruction in parallel based on the identification result to obtain each data processing result. Finally, the data instruction set is updated based on each data processing result, and the user's input instruction is obtained. If the input instruction is a termination instruction, the updated data instruction set is output as the data collection result. If the input instruction is an execution instruction, the data node address of each data instruction in the data instruction set is set according to the user's input operation to update the data instruction set, and the operation of identifying the data type of each data instruction in the data instruction set is returned based on the updated data instruction set. This achieves the parallel collection of multiple types of data, reduces the manpower consumption of data collection operations, and improves the efficiency of data collection.

[0062] Example 3

[0063] Figure 3This is a schematic diagram of the structure of a parallel data collection device provided by the third embodiment of the present invention. Figure 3 As shown, the device includes:

[0064] The data acquisition module 310 is used to acquire a data instruction set;

[0065] The parallel processing module 320 is used to identify the data type of each data instruction in the data instruction set, and perform parallel processing on each data instruction according to the identification result to obtain each data processing result;

[0066] The result acquisition module 330 is configured to update the data instruction set according to each data processing result, and acquire data collection results based on the updated data instruction set in response to a user input instruction.

[0067] The technical solution of the embodiment of the present invention obtains a data instruction set, then identifies the data type of each data instruction in the data instruction set, and processes each data instruction in parallel according to the identification result to obtain each data processing result. Finally, the data instruction set is updated according to each data processing result, and the data collection result is obtained based on the updated data instruction set in response to the user's input instruction, thereby realizing the parallel collection of multiple types of data, reducing the manpower consumption of data collection operations, and improving the efficiency of data collection.

[0068] Based on the above embodiment, the parallel processing module 320 includes:

[0069] an object parsing unit, configured to parse the data parameters using a pre-configured document object model parsing technology to obtain a target query language matching the data instruction if the data type in the current data instruction is recognized as a query type;

[0070] a first address query unit, configured to send the target query language to a target node matching the data node address according to the data node address in the data instruction;

[0071] The first binding unit is configured to process the target query language through the target node, and bind the processing result to the data type matching the data instruction to obtain a data processing result matching the data instruction.

[0072] Based on the above embodiment, the parallel processing module 320 further includes:

[0073] A second address query unit is configured to, if it is recognized that the data type in the current data instruction is the request type, perform a pull-up operation on a target client matching the data node address according to the data node address in the data instruction;

[0074] The second binding unit is used to initiate a positioning request to the target client based on the data parameters in the data instruction, and bind the positioning result and the data type matching the data instruction to obtain a data processing result matching the data instruction.

[0075] Based on the above embodiment, the parallel processing module 320 further includes:

[0076] an independent code unit, configured to process data parameters in the data instruction using a preset independent code tool to obtain independent parameters matching the data instruction if the data type in the current data instruction is identified as a calculation type;

[0077] a third address query unit, configured to perform a pull-up operation on a target executor matching the data node address according to the data node address in the data instruction;

[0078] The third binding unit is used to perform a calculation operation on the independent parameter through the target executor, and bind the calculation result with the data type matching the data instruction, and output it as a data processing result matching the data instruction.

[0079] On the basis of the above embodiment, the result acquisition module 330 is specifically configured to: if the input instruction is a termination instruction, output the updated data instruction set as the data collection result.

[0080] Based on the above embodiment, the result acquisition module 330 is also used to: if the input instruction is an execution instruction, set the data node address of each data instruction in the data instruction set according to the user's input operation to update the data instruction set; based on the updated data instruction set, return to execute the operation of identifying the data type of each data instruction in the data instruction set.

[0081] A parallel data collection device provided by an embodiment of the present invention can execute a parallel data collection method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects of the execution method.

[0082] Example 4

[0083] Figure 4A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0084] like Figure 4 As shown, electronic device 10 includes at least one processor 11 and memory, such as read-only memory (ROM) 12 and random access memory (RAM) 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by the at least one processor. Processor 11 can perform various appropriate actions and processes based on the computer programs stored in ROM 12 or loaded from storage unit 18 into RAM 13. RAM 13 can also store various programs and data required for the operation of electronic device 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. An input / output (I / O) interface 15 is also connected to bus 14.

[0085] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0086] Processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any other suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as a parallel data collection method.

[0087] Accordingly, the method includes:

[0088] Get data instruction set;

[0089] Identify the data type of each data instruction in the data instruction set, and perform parallel processing on each data instruction according to the identification result to obtain each data processing result;

[0090] The data instruction set is updated according to each data processing result, and the data collection result is obtained based on the updated data instruction set in response to the user's input instruction.

[0091] In some embodiments, a method for parallel data collection can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method for parallel data collection described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform a method for parallel data collection in any other suitable manner (e.g., via firmware).

[0092] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0093] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0094] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0095] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0096] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0097] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0098] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

Claims

1. A parallel data collection method, characterized in that: include: Obtaining a data instruction set, wherein the data instruction includes: a data node address, a data type, and data parameters, wherein the data type includes: a query type, a request type, or a calculation type; Identify the data type of each data instruction in the data instruction set, and perform parallel processing on each data instruction according to the identification result to obtain each data processing result; updating the data instruction set according to each data processing result, and acquiring a data collection result based on the updated data instruction set in response to a user input instruction; Among them, each data instruction is processed in parallel according to the recognition result to obtain each data processing result, including: if the data type in the current data instruction is recognized as a query type, the data parameters in the data instruction are parsed using a pre-configured document object model parsing technology to obtain a target query language that matches the data instruction; according to the data node address in the data instruction, the target query language is sent to a target node that matches the data node address; the target query language is processed by the target node, and the processing result is bound to the data type that matches the data instruction to obtain a data processing result that matches the data instruction.

2. The method according to claim 1, characterized in that Processing each data instruction in parallel according to the recognition result to obtain each data processing result also includes: If it is recognized that the data type in the current data instruction is the request type, a pull-up operation is performed on a target client that matches the data node address according to the data node address in the data instruction; Based on the data parameters in the data instruction, a positioning request is initiated to the target client, and the positioning result is bound to the data type matching the data instruction to obtain a data processing result matching the data instruction.

3. The method according to claim 1, characterized in that Processing each data instruction in parallel according to the recognition result to obtain each data processing result also includes: If it is recognized that the data type in the current data instruction is a calculation type, the data parameters in the data instruction are processed by a preset independent code tool to obtain independent parameters matching the data instruction; Performing a pull-up operation on a target executor matching the data node address according to the data node address in the data instruction; The target executor performs a calculation operation on the independent parameter, and binds the calculation result to the data type matching the data instruction, and outputs it as a data processing result matching the data instruction.

4. The method according to claim 1, wherein Acquiring data collection results based on the updated data instruction set in response to a user input instruction, including: If the input instruction is a termination instruction, the updated data instruction set is output as a data collection result.

5. The method according to claim 1, wherein Responding to the user's input instruction, obtaining each data processing result based on the updated data instruction set, further comprising: If the input instruction is an execution instruction, the data node address of each data instruction in the data instruction set is set according to the user's input operation to update the data instruction set; An operation of identifying the data type of each data instruction in the data instruction set is returned based on the updated data instruction set.

6. A parallel data collection device, characterized in that: include: A data acquisition module is used to acquire a data instruction set, wherein the data instruction includes: a data node address, a data type, and a data parameter, wherein the data type includes: a query type, a request type, or a calculation type; The parallel processing module is used to identify the data type of each data instruction in the data instruction set, and to perform parallel processing on each data instruction according to the identification result to obtain each data processing result; a result acquisition module, configured to update the data instruction set according to each data processing result, and acquire a data collection result based on the updated data instruction set in response to a user input instruction; Among them, the parallel processing module includes: an object parsing unit, which is used to parse the data parameters using a pre-configured document object model parsing technology if it is identified that the data type in the current data instruction is a query type, so as to obtain a target query language that matches the data instruction; a first address query unit, which is used to send the target query language to a target node that matches the data node address according to the data node address in the data instruction; and a first binding unit, which is used to process the target query language through the target node, and bind the processing result to the data type that matches the data instruction, so as to obtain a data processing result that matches the data instruction.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the parallel data collection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a parallel data collection method according to any one of claims 1 to 5 when executed.

Images