An intelligent data processing system based on service computing

Abstract

The present invention provides an intelligent data processing system based on service computing, including an input module, an output module, a service module, a calling module, and a storage module. The input module is used to receive the original data submitted by the user. The output module is used to send the result data after service computing. The service module is used to perform computing processing on the original data. The storage module is used to save the original data and the intermediate data generated during the service computing process. The calling module is used to manage the calling of the data in the storage module by the service module. Through the calling module, this system can reasonably arrange the order of sending user data to the service module for computing processing, reduce the data storage volume in the storage module, and improve the simultaneous processing ability of the system.

Description

Technical Field

[0001] The present invention relates to the field of digital information transmission, and particularly to an intelligent data processing system based on service computing. Background Art

[0002] Data center as a service is a public computing service, in which the hosting company provides its data center infrastructure and facilities leased or rented to customers. Users submit raw data to the hosting company for data processing services, and the hosting company then feeds back the processing results to the users. The computing service can reduce the cost of users' data processing and improve the efficiency of output data. However, with the increase in the number of users using the computing service, the hosting company providing the computing service needs to improve the data processing system to ensure the data processing efficiency.

[0003] The foregoing discussion of the background art is only intended to facilitate the understanding of the present invention. This discussion does not recognize or admit that any of the materials mentioned is part of common general knowledge.

[0004] Now, many data processing systems have been developed. After a large amount of retrieval and reference by us, it is found that existing systems such as the system disclosed in the publication number CN112860675B. These systems generally include ensuring that the noise filtering thread has better noise data recognition and filtering performance when using the noise filtering thread, ensuring that the noise filtering thread fully considers the business requirements corresponding to the actual cloud service environment when processing the content of the target service item, and being able to accurately and reliably distinguish different noise sources when removing noise from the content of the target service item so as to perform corresponding noise removal processing according to different noise sources. However, this system only optimizes the data itself, but does not improve the data calling process, and the efficiency of the computing service will decrease when the number of users increases. Summary of the Invention

[0005] The object of the present invention is to propose an intelligent data processing system based on service computing in view of the existing deficiencies.

[0006] The present invention adopts the following technical solutions:

[0007] An intelligent data processing system based on service computing, including an input module, an output module, a service module, a calling module, and a storage module. The input module is used to receive the raw data submitted by the user, the output module is used to send the result data after service computing, the service module is used to perform computing processing on the raw data, the storage module is used to save the raw data and the intermediate data generated during the service computing process, and the calling module is used to manage the calling of the data in the storage module by the service module.

[0008] The service module includes several functional calculators, each of which has a separate calculation function. The calling module creates a corresponding process tree according to the received user service information. The process tree is formed by connecting nodes, and each node corresponds to a functional calculator. The storage module organizes the received user raw data and the processing results returned by the functional calculators into several data packets;

[0009] The nodes in the process tree include immediate nodes and sluggish nodes. The calling module creates a first processing sequence and a second processing sequence. The first processing sequence is arranged by the immediate nodes of the same user. The second processing sequence is arranged by the same-type immediate nodes of different users. The first processing sequence is sorted from largest to smallest according to the first processing exponent of the immediate nodes. The second processing sequence is sorted from largest to smallest according to the second processing exponent of the immediate nodes. The calculation formula for the first processing exponent P1 is:

[0010]

[0011] where S0 is the reference size, i is the number of the data packet required by the functional calculator corresponding to the immediate node, C(i) is the durability in the data packet, and S(i) is the size of the data packet;

[0012] The calculation formula for the second processing exponent P2 is:

[0013]

[0014] where t0 is the reference time, and Δt is the time difference between the time when the second processing sequence receives the information of the immediate node and the current time;

[0015] The functional calculator obtains the information of the first immediate node from the second processing sequence and sends a data application to the calling module. The calling module arranges the data applications according to the first processing sequence to obtain an application sequence. The calling module sends the corresponding data packets to the corresponding functional calculators for processing according to the application sequence;

[0016] Further, the calling module is provided with a pointer. The pointer moves between each node according to the process tree. The pointer can read the information of the functional calculator corresponding to the node, and the pointer can establish a one-time data channel between the storage module and the corresponding functional calculator according to the information;

[0017] Further, the durability of the data packet is used to represent the number of times the data packet needs to be sent to the functional calculator. Each time the data packet is sent to the functional calculator, its durability is reduced by one. When the durability of the data packet is 0, the storage module will delete the data packet;

[0018] Further, the storage module sends data to the function calculator through a one-time data channel, and the function calculator feeds back the processing result to the storage module through the one-time data channel. The one-time data channel is automatically destroyed after two data transmissions;

[0019] Further, when the immediate node and the sluggish node are connected in the process tree, the call module also adjusts the first processing exponent of the immediate node according to the information of the sluggish node according to the following formula:

[0020]

[0021] where P1′ is the adjusted first processing exponent, j is the number of the connected sluggish node, and V(j) represents the size of all data packets of this user required by the function calculator corresponding to the sluggish node numbered j;

[0022] The call module reorders the first processing sequence according to the adjusted first processing exponent.

[0023] The beneficial effects achieved by the present invention are as follows:

[0024] This system divides data into data packets, then sorts the nodes according to the number of times the data packets need to be used and the size of the data packets, and then arranges the processing order of the data packets according to the sorting of the nodes, so that the data stored in the storage module can be released as soon as possible, reducing the average space occupancy of each user in the storage module, increasing the number of users that the system can process simultaneously, and ensuring the data processing efficiency.

[0025] To enable a further understanding of the features and technical content of the present invention, please refer to the following detailed description of the present invention and the attached drawings. However, the attached drawings are only provided for reference and illustration, and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the overall structural framework of the present invention;

[0027] Figure 2 It is a schematic diagram of the process tree of the present invention and the function of the pointer on the process tree;

[0028] Figure 3 It is a schematic diagram of node classification on the process tree of the present invention;

[0029] Figure 4 It is a schematic diagram of the generation process of the first processing sequence of the present invention;

[0030] Figure 5 It is a schematic diagram of the generation process of the second processing sequence of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0031] The following are specific embodiments to illustrate the implementation manners of the present invention. Those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the spirit of the present invention. Additionally, the drawings of the present invention are only for simple schematic illustration and are not drawn according to actual dimensions, which is hereby declared in advance. The following embodiments will further detail the related technical content of the present invention, but the disclosed content is not intended to limit the protection scope of the present invention.

[0032] Embodiment 1.

[0033] This embodiment provides an intelligent data processing system based on service computing, combined with Figure 1 , including an input module, an output module, a service module, a call module, and a storage module. The input module is used to receive the original data submitted by the user. The output module is used to send the result data after service computing. The service module is used to perform computational processing on the original data. The storage module is used to save the original data and the intermediate data generated during the service computing process. The call module is used to manage the call of the data in the storage module by the service module;

[0034] The service module includes several functional calculators, and each functional calculator has a separate computational function. The call module creates a corresponding process tree according to the received user service information. The process tree is composed of nodes connected together, and each node corresponds to a functional calculator. The storage module organizes the received user original data and the processing results returned by the functional calculators into several data packets;

[0035] The nodes in the process tree include immediate nodes and sluggish nodes. The call module creates a first processing sequence and a second processing sequence. The first processing sequence is obtained by arranging the immediate nodes of the same user. The second processing sequence is obtained by arranging the same-type immediate nodes of different users. The first processing sequence is sorted from largest to smallest according to the first processing index of the immediate nodes. The second processing sequence is sorted from largest to smallest according to the second processing index of the immediate nodes. The calculation formula for the first processing index P1 is:

[0036]

[0037] where S0 is the reference size, i is the number of the data packet required by the functional calculator corresponding to the immediate node, C(i) is the durability in the data packet, and S(i) is the size of the data packet;

[0038] The calculation formula for the second processing index P2 is:

[0039]

[0040] Among them, t0 is the reference time, and Δt is the time difference between the time when the second processing sequence receives the instant node information and the current moment;

[0041] The function calculator obtains the information of the first instant node from the second processing sequence and sends a data application to the call module. The call module arranges the data application according to the first processing sequence to obtain an application sequence, and the call module sends the corresponding data packet to the corresponding function calculator for processing according to the application sequence;

[0042] The call module is provided with a pointer. The pointer moves between each node according to the process tree. The pointer can read the information of the function calculator corresponding to the node, and the pointer can establish a one-time data channel between the storage module and the corresponding function calculator according to the information;

[0043] The durability of the data packet is used to represent the number of times the data packet needs to be sent to the function calculator. Each time the data packet is sent to the function calculator, its durability is reduced by one. When the durability of the data packet is 0, the storage module will delete the data packet;

[0044] The storage module sends the data to the function calculator through the one-time data channel, and the function calculator feeds back the processing result to the storage module through the one-time data channel. The one-time data channel is automatically destroyed after two data transmissions;

[0045] When the instant node and the sluggish node are connected in the process tree, the call module also adjusts the first processing index of the instant node according to the information of the sluggish node according to the following formula:

[0046]

[0047] Among them, P1′ is the adjusted first processing index, j is the number of the connected sluggish node, and V(j) represents the size of all data packets of this user required by the function calculator corresponding to the sluggish node numbered j;

[0048] The call module reorders the first processing sequence according to the adjusted first processing index.

[0049] Embodiment 2.

[0050] This embodiment includes all the content of Embodiment 1 and provides an intelligent data processing system based on service computing, including an input module, an output module, a service module, a call module, and a storage module. The input module is used to receive the original data submitted by the user. The output module is used to send the result data after service computing. The service module is used to perform computing processing on the original data. The storage module is used to save the original data and the intermediate data generated during the service computing process. The call module is used to manage the call of the data in the storage module by the service module;

[0051] Combined with Figure 2 , the service module includes a number of functional calculators, each of which has an independent computing function. The call module can create a process tree according to the service type submitted by the user. The process tree includes a number of nodes, and each node corresponds to a functional calculator. The call module is provided with a pointer, and the pointer moves between the nodes according to the process tree. When the pointer points to a node, the pointer reads the information of the functional calculator corresponding to the node. The information includes the data type to be processed and the address of the functional calculator. The pointer establishes a one-time data channel between the storage module and the corresponding functional calculator according to the information. The storage module sends the data to the functional calculator through the one-time data channel, and the functional calculator feeds back the processing result to the storage module through the one-time data channel. The one-time data channel is automatically destroyed after two data transmissions;

[0052] The storage module creates a dedicated space for each user to store data. The data stored in the dedicated space is divided into several data packets according to integrity. The integrity of the data means that the data will be sent to the functional calculator for processing together. For example, if all the functional calculators that process data A need data B, and all the functional calculators that process data B need data A, then data A and data B have integrity and are integrated into one data packet. The processing results returned by the functional calculator become one or more data packets according to the data integrity. When a data packet in the dedicated space no longer needs to be sent to the functional calculator corresponding to the unprocessed node in the process tree, the data packet will be deleted. The size of the dedicated space is variable. When the call module moves the pointer to control the processing of the stored data by the service module, it should make the size of the dedicated space decrease as fast as possible, so that the entire system can process the data of more users at the same time;

[0053] The data packet is marked with durability, which is used to indicate the number of times the data packet needs to be sent to the function calculator. Each time the data packet is sent to the function calculator, its durability is decreased by one. When the durability of the data packet is 0, the storage module deletes the data packet. When the processing result returned by the function calculator is the final service result, the data packet generated by the final service result is used as a special data packet, and its durability is set to a negative value. The data packet with a negative durability value will not be deleted until the entire calculation service of the user ends;

[0054] Combined with Figure 3 , the nodes in the process tree are divided into two categories. One is the immediate node, and the data packets required by the function calculator corresponding to the immediate node already exist in the exclusive space of the corresponding user. The other is the sluggish node, and the data packets required by the function calculator corresponding to the sluggish node do not completely exist in the exclusive space of the corresponding user;

[0055] Combined with Figure 4 , the call module obtains the first processing sequence of all immediate nodes in the process tree according to the following steps:

[0056] S1. The call module moves the pointer to the immediate node and obtains the information of the function calculator corresponding to the immediate node;

[0057] S2. The call module finds the corresponding data packet in the exclusive space of the user in the storage module according to the data type in the information obtained in step S1;

[0058] S3. The call module obtains the durability C(i) and the size S(i) of the data packet, where i is the number of the data packet found in step S2;

[0059] S4. The call module calculates the first processing index P1 of the immediate node according to the following formula:

[0060]

[0061] where S0 is the reference size;

[0062] The reference size is set by itself according to the actual situation and experience;

[0063] S5. The call module repeats steps S1 to S4 to obtain the first processing indices of all immediate nodes in the process tree;

[0064] S6. The call module sorts the immediate nodes in the process tree in descending order according to the first processing index to obtain the first processing sequence;

[0065] Combined with Figure 5, the calling module processes the same type of instant nodes of different users according to the following steps to obtain a second processing sequence:

[0066] S21. The calling module creates a buffer queue for each function calculator;

[0067] S22. The calling module sends the processing exponent of the newly generated instant node in the first processing sequence to the buffer queue of the corresponding function calculator;

[0068] S23. The calling module calculates the second processing exponent P2 of each instant node in the buffer queue according to the following formula:

[0069]

[0070] where t0 is the reference time, and Δt is the time difference between the time when the buffer queue receives the information of this instant node and the current moment;

[0071] The reference time is set by itself according to the actual situation and experience;

[0072] S24. The calling module continues to sort the instant nodes in the buffer queue from largest to smallest according to the second processing exponent to obtain a second processing sequence;

[0073] The function calculator obtains an instant node in order from the second processing sequence, and sends a data application to the calling module according to the information of this instant node. The calling module arranges the multiple data applications received by each user in the order of the first processing sequence to obtain an application sequence. The calling module moves the pointer to the instant node corresponding to the first data application in the application sequence and creates a one-time data channel. The function calculator obtains data from the one-time data channel for calculation processing, and then obtains an instant node from the second processing sequence after processing, and repeats this process continuously;

[0074] When the instant node is connected to the sluggish node in the process tree, the calling module also adjusts the first processing exponent of this instant node according to the information of the sluggish node according to the following formula:

[0075]

[0076] where P1′ is the adjusted first processing exponent, j is the number of the connected sluggish node, and V(j) represents the size of all data packets of this user required by the function calculator corresponding to the sluggish node numbered j.

[0077] The content disclosed above is only the preferred and feasible embodiment of the present invention, and does not limit the protection scope of the present invention. Therefore, all equivalent technical changes made by using the content of the specification and drawings of the present invention are included in the protection scope of the present invention. In addition, with the development of technology, the elements therein can be updated.

Claims

1. An intelligent data processing system based on service computing, characterized in that It includes an input module, an output module, a service module, a call module, and a storage module. The input module is used to receive the original data submitted by the user. The output module is used to send the result data after service calculation. The service module is used to perform calculation processing on the original data. The storage module is used to save the original data and the intermediate data generated during the service calculation. The call module is used to manage the call of the data in the storage module by the service module; The service module includes several function calculators, and each function calculator has a single calculation function. The call module creates a corresponding process tree according to the received user service information. The process tree is formed by connecting nodes, and each node corresponds to a function calculator. The storage module organizes the received user original data and the processing results returned by the function calculators into several data packets; The nodes in the process tree include immediate nodes and sluggish nodes. The call module creates a first processing sequence and a second processing sequence. The first processing sequence is obtained by arranging the immediate nodes of the same user. The second processing sequence is obtained by arranging the same type of immediate nodes of different users. The first processing sequence is sorted from largest to smallest according to the first processing exponent of the immediate nodes. The second processing sequence is sorted from largest to smallest according to the second processing exponent of the immediate nodes. The calculation formula for the first processing exponent P1 is: Where S0 is the reference size, and i is the number of the data packet required by the function calculator corresponding to this immediate node, C(i) is the durability of the data packet, and S(i) is the size of the data packet; The calculation formula for the second processing exponent P2 is: Where t0 is the reference time, and Δt is the time difference between the time when the second processing sequence receives the information of this immediate node and the current time; The function calculator obtains the information of the first immediate node from the second processing sequence and sends a data application to the call module. The call module arranges the data application according to the first processing sequence to obtain an application sequence. The call module sends the corresponding data packet to the corresponding function calculator for processing according to the application sequence.

2. The intelligent data processing system based on service computing according to claim 1, characterized in that, The call module is provided with a pointer. The pointer moves between each node according to the process tree. The pointer can read the information of the function calculator corresponding to the node. The pointer can establish a one-time data channel between the storage module and the corresponding function calculator according to the information.

3. An intelligent data processing system based on service computing according to claim 2, characterized in that, The durability of the data packet is used to represent the number of times the data packet needs to be sent to the function calculator. Each time the data packet is sent to the function calculator, its durability is decreased by one. When the durability of the data packet is 0, the storage module will delete the data packet.

4. An intelligent data processing system based on service computing according to claim 3, characterized in that, The storage module sends the data to the function calculator through a one-time data channel. The function calculator feeds back the processing result to the storage module through the one-time data channel. The one-time data channel is automatically destroyed after two data transmissions.

5. An intelligent data processing system based on service computing according to claim 4, characterized in that, When an immediate node is connected to a sluggish node in the process tree, the call module also adjusts the first processing exponent of this immediate node according to the information of the sluggish node according to the following formula: Wherein, P1′ is the adjusted first processing index, j is the number of the connected sluggish node, and V(j) represents the size of all data packets of this user required by the function calculator corresponding to the sluggish node numbered j; The calling module reorders the first processing sequence according to the adjusted first processing index.

Images