Computer system and data transmission control method
The described system addresses inefficiencies in load balancing for multiplexed data flow applications by optimizing data distribution based on communication indices and link tolerance values, enhancing throughput in diverse computing environments.
Patent Information
- Application Number
- JP2021146630
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-09-09
- Publication Date
- 2025-07-08
- Estimated Expiration
- 2041-09-09
AI Technical Summary
Existing load balancing techniques for multiplexed data flow type applications are ineffective in managing data processing throughput due to varying computing resources and communication environments, particularly in systems with distributed applications where data storage and retrieval are not evenly utilized.
A computer system that includes a load distribution control unit to evaluate communication indices and calculate link tolerance values for data allocation, generating a transmission plan to optimize data distribution across multiple processing applications, thereby improving data processing throughput.
The system enhances data processing throughput by optimizing data allocation based on communication performance and processing capabilities, preventing throughput limitations and ensuring efficient data handling across diverse environments.
Smart Images

Figure 0007704623000001 
Figure 0007704623000002 
Figure 0007704623000003
Abstract
Description
Technical Field
[0001] The present invention relates to a load balancing technique for multiplexed data flow type applications.
Background Art
[0002] As a method of utilizing IoT (Internet of Things), there is a form of distributed computing in which data processing is performed at various sites from a machine on a cloud service to a device arranged at an edge such as a factory site. In recent years, the movement toward microservices, in which a monolithic application is subdivided into functions for data processing and packaged, has been accelerating. By packaging, the limitations of hardware and the OS (Operating System) are eliminated, and thus applications subdivided in various environments can be operated. In the following description, the subdivided applications are simply referred to as applications.
[0003] An application does not always operate on a cloud service with abundant computing resources, and may operate on a device with scarce computing resources existing at the edge. When an application is operated on a device with scarce computing resources, the amount of data that can be processed per unit time of the application decreases. Therefore, there is a technique called scale-out. In scale-out, replicas of an application are generated, and a plurality of applications execute processing in parallel. In the case of a system adopting a microservices architecture, replicas need only be created for applications with a high execution load and a decreased amount of data that can be processed per unit time. By causing a plurality of applications including replicas to execute processing in parallel, the throughput of data processing of the entire system can be improved compared to the case of operating a single application.
[0004] The application and its replicas can be deployed not only on cloud services and devices on the edge, but also on devices located in the communication layer such as Multi-access Edge Computing servers. Also, by enabling software-based communication control, the communication bandwidth and path can be easily changed.
[0005] In this way, in a situation where there are diverse options for deploying applications, the communication environment of the application changes constantly. In order to improve the throughput of data processing for the entire application that constitutes the microservice, each application needs to follow the changes in the environment where it is deployed.
[0006] As one solution, a method for selecting multiplexed and distributed applications is devised to follow environmental changes. In a system where multiplexed applications are distributed, it is necessary to select the application that executes the processing. Although the result of data processing does not change regardless of which application is used, the throughput of data processing may vary depending on the computing resources and communication environment. Also, in a data flow type application, different from an application with a clear distinction between server / client, it is necessary to select an application considering the computing resources and communication environment both in data storage and retrieval while being aware of the connection of the data flow.
[0007] As techniques for selecting an application for data processing from among distributed applications, the techniques described in Patent Document 1 and Patent Document 2 are known.
[0008] Patent Document 1 describes "a storage management device capable of communicating with a plurality of edge servers each including a storage unit that stores data and access log information for the data. The storage management device acquires access log information from each edge server, and based on the acquired access log information, determines whether the access situation for the data stored in the data storage unit of each edge server meets a preset criterion. Data for which it is determined that the access situation does not meet the above criterion is selected as a transfer target, and control is performed to transfer the data to another storage unit different from the storage unit in which the data was stored."
[0009] Patent Document 2 describes "When there are a plurality of candidate destination servers for a received request, the request is sent to a server among them whose number of requests waiting for a response has not reached a threshold. When all candidates for the destination server have reached the threshold of the number of requests waiting for a response, the request is stored in a buffer, and transfer is awaited until the threshold of the number of requests waiting for a response falls below that of any server. Further, the feasibility of transferring a request to each individual server is determined in fine granularity in units of requests or responses. Further, when there are a plurality of servers whose number of requests waiting for a response has not reached the threshold, a server is selected based on the number of requests waiting for a response of each server."
Prior Art Documents
Patent Documents
[0010]
Patent Document 1
Patent Document 2
Summary of the Invention
Problems to be Solved by the Invention
[0011] The technology described in Patent Document 1 can achieve improved efficiency in data retrieval and effective utilization of storage. However, in the case of data flow type applications, it is assumed that the stored data is retrieved evenly without any difference in usage frequency, so the technology described in Patent Document 1 is not effective. The technology described in Patent Document 2 can prevent data overflow during data storage. However, since the load during data output is not considered, it is not always effective in a system where applications are distributed.
[0012] An object of the present invention is to provide a technology for realizing data allocation for a multiplexed data flow type application that improves the performance of data processing.
Means for Solving the Problems
[0013] A typical example of the invention disclosed in the present application is as follows. That is, a computer system including at least one computer having a processor, a storage device connected to the processor, and a network interface connected to the processor, wherein the processor acquires data from a source object, executes a predetermined process on the data, and transmits it to a destination object. For each of a plurality of processing applications, a communication index for evaluating the data transmission and reception performance of the processing application is acquired, a link tolerance value representing the processing performance of each of the plurality of processing applications is calculated based on the communication index, and based on the link tolerance value, a transmission plan regarding the amount of data to be allocated to the plurality of processing applications is generated, and based on the transmission plan, the transmission of data to the plurality of processing applications is controlled.
Effects of the Invention
[0014] According to the present invention, it is possible to realize data allocation for a multiplexed data flow type application that improves the throughput of data processing of the entire system. Problems, configurations, and effects other than those described above will be clarified by the description of the following embodiments.
Brief Description of the Drawings
[0015]
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12A
Figure 12B
Figure 12C
Figure 13A
Figure 13B
Figure 13C
Figure 14
Figure 15A
Figure 15B
Figure 16
Figure 17A
Figure 17B
Figure 17C
Figure 18
Figure 19A
Figure 19B
Figure 19C
Figure 20
Mode for Carrying Out the Invention
[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not construed as being limited to the description of the embodiments shown below. It will be easily understood by those skilled in the art that the specific configuration can be changed without departing from the spirit or gist of the present invention.
[0017] In the configuration of the invention described below, the same or similar configurations or functions are denoted by the same reference numerals, and duplicate descriptions are omitted.
[0018] In the drawings and the like, the positions, sizes, shapes, and ranges of the respective components shown may not represent the actual positions, sizes, shapes, and ranges in order to facilitate understanding of the invention. Therefore, the present invention is not limited to the positions, sizes, shapes, and ranges disclosed in the drawings and the like.
Example
[0019] FIG. 1 is a diagram showing an example of the configuration of the system of Example 1. FIG. 2 is a diagram showing an example of a computer constituting the control system of Example 1.
[0020] The system is composed of a control system 1, a transmission application 2, a processing application 3, and a reception application 4. The transmission application 2, the processing application 3, and the reception application 4 may serve multiple roles among the transmission application 2, the processing application 3, and the reception application 4 according to the usage method of the system data. For example, the transmission application 2 may function as the reception application 4.
[0021] In this specification, when the transmission application 2, the processing application 3, and the reception application 4 are not distinguished, they are described as applications. Also, a set of the transmission application 2, the processing application 3, and the reception application 4 on the path through which data flows is described as a link.
[0022] The control system 1 is composed of a computer 200 as shown in FIG. 2. Note that the control system 1 may include a storage system, a network switch, and the like.
[0023] The computer 200 has a processor 201, a main memory device 202, a secondary storage device 203, and a network interface 204. Note that the computer 200 may have an input device and an output device.
[0024] Processor 201 executes a program stored in main memory device 202. By executing processing according to the program, it operates as a functional unit (module) that realizes a specific function. In the following description, when explaining the processing with the functional unit as the subject, it indicates that processor 202 is executing a program that realizes the functional unit. Main memory device 202 is a memory or the like, and stores a program executed by processor 202 and data used by the program. Auxiliary storage device 203 is an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like, and stores data permanently. The program and data stored in main memory device 202 may be stored in auxiliary storage device 203. In this case, processor 202 reads the program and data from auxiliary storage device 203 and loads them into main memory device 202. Network interface 204 communicates with other devices via a network.
[0025] Control system 1 includes, as a functional configuration, a load distribution control unit 11, a load distribution evaluation unit 12, a user input unit 13, a user output unit 14, and a storage unit 15.
[0026] Load distribution control unit 11 includes a distribution control planning unit 111 and a distribution control execution unit 112. Distribution control planning unit 111 generates a transmission plan for controlling the transmission of data received from transmission application 2 to processing application 3. Distribution control execution unit 112 transmits the data received from transmission application 2 to processing application 3 based on the transmission plan generated by distribution control planning unit 111.
[0027] The load distribution evaluation unit 12 includes measurement units 121, 122, and a feedback unit 123. The measurement unit 121 acquires an index (communication index) related to communication between the transmission application 2 and the processing application 3, and the measurement unit 122 acquires an index (communication index) related to communication between the processing application 3 and the reception application 4. As an example of the communication index, the number of transferred messages per unit time, the transferred data size per unit time, the transfer time, Round Trip Time, and the number of hops, etc. can be considered. However, the communication index is not limited to the above. Also, the measurement units 121 and 122 acquire the number of replicas of the application and the connection state between the applications.
[0028] The user input unit 13 includes a training execution schedule specifying unit 131, a status confirmation schedule specifying unit 132, an index specifying unit 133, and an end command execution unit 134. The training execution schedule specifying unit 131 receives an input of a training schedule for generating a transmission plan. The status confirmation schedule specifying unit 132 receives an input of a status confirmation schedule. Here, the status confirmation means confirming whether the transmission plan generated by the distributed control planning unit 111 is appropriate, that is, whether the stable state S3 (see FIG. 5) can be continued. The index specifying unit 133 receives an input of a communication index used for generating a transmission plan. The end command execution unit 134 receives a stop instruction for the control system 1 and executes a command to stop the control system 1.
[0029] The user output unit 14 includes a topology display unit 141 and a distributed control status display unit 142. The topology display unit 141 displays a topology indicating the connection between applications to the user. The distributed control status display unit 142 displays the operation status of the distributed control planning unit 111 and the distributed control execution unit 112 to the user.
[0030] The memory unit 15 manages user setting information 151, measurement result information 152, and link tolerance value information 153. The user setting information 151 stores the set values input by the user input unit 13. The measurement result information 152 stores the communication metrics measured by the measurement unit 121. The link tolerance value information 153 stores the tolerance values of the communication metrics specified via the metric specifying unit 133.
[0031] FIG. 3 is a diagram showing an example of an interface presented by the user input unit 13 in the first embodiment.
[0032] The user input unit 13 presents an input screen 300 as shown in FIG. 3. The input screen 300 includes an operation bar 310, an operation bar 320, an operation bar 330, and a stop button 340.
[0033] The operation bar 310 includes an input field 311 and an apply button 312. The input field 311 is a field for inputting the training schedule to the training execution schedule specifying unit 131. When the user presses the apply button 312, the schedule input in the input field 311 is transmitted to the distributed control execution unit 112 via the training execution schedule specifying unit 131. The distributed control execution unit 112 performs training according to the schedule as described later.
[0034] The operation bar 320 includes an input field 321 and an apply button 312. The input field 321 is a field for inputting the status check schedule to the status check schedule specifying unit 132. When the user presses the apply button 322, the schedule input in the input field 321 is transmitted to the distributed control execution unit 112 via the status check schedule specifying unit 132. The distributed control execution unit 112 performs a status check according to the schedule as described later.
[0035] The operation bar 330 includes an input field 331 and an apply button 332. The input field 331 is a field for inputting the communication metrics used by the distributed control planning unit 111, that is, the types of communication metrics for evaluating the data transmission and reception performance of the processing application 3. When the user presses the apply button 332, the type of communication metric input in the input field 331 is transmitted to the distributed control planning unit 111 via the metric specifying unit 133.
[0036] In the first embodiment, the amount of data (data transfer amount) received by the processing application 3 from the transmission application 2 and the amount of data (data transfer amount) transmitted by the processing application 3 to the reception application 4 are used as communication metrics for evaluating the data transmission and reception performance of the processing application 3.
[0037] The stop button 340 is a button for instructing the stop of the control system 1. When the user presses the stop button 340, the end command execution unit 134 stops the control system 1.
[0038] FIG. 4 is a diagram showing an example of the interface presented by the user output unit 14 of the first embodiment.
[0039] The user output unit 14 presents an output screen 400 as shown in FIG. 4. The output screen 400 includes a display column 410 and a display column 420.
[0040] The display column 410 is a column for displaying the information generated by the topology display unit 141. The topology display unit 141 displays, in the display column 410, the connections between the applications managed by the control system 1 and the communication metrics measured by the measurement unit 121.
[0041] The display column 420 is a column for displaying the information generated by the distributed control status display unit 142. The distributed control status display unit 142 displays, in the display column 420, the type of communication metric specified via the metric specifying unit 133, the execution log of the distributed control execution unit 112, and instructions to the user, etc.
[0042] FIG. 5 is a diagram showing the transition of the operating state of the control system 1 according to the first embodiment.
[0043] The initial state of the control system 1 is the stop state S1. When the control system 1 is activated, it transitions to the training state S2.
[0044] In the training state S2, the control system 1 executes training processing. When the training processing is completed, the control system 1 transitions to the stable state S3.
[0045] In the stable state S3, the control system 1 periodically checks the state and the opportunity to execute training.
[0046] If the opportunity to check the state and execute training is not detected, the control system 1 maintains the stable state S3. If the opportunity to execute training is detected, the control system 1 transitions to the training state S2. If the opportunity to check the state is detected, the control system 1 determines the appropriateness of the transmission plan. As a result of this determination, if there is no need to change the transmission plan, the control system 1 maintains the stable state S3. As a result of this determination, if there is a need to change the transmission plan, the control system 1 transitions to the high load state S4.
[0047] In the high load state S4, the control system 1 changes the transmission plan. If the change of the transmission plan is successful, the control system 1 transitions to the stable state S3. If the change of the transmission plan is not successful, the control system 1 transitions to the warning state S5.
[0048] In the warning state S5, the control system 1 presents to the user that distributed control satisfying the conditions is impossible via the distributed control status display unit 142.
[0049] Even in the warning state S5, the control system 1 may execute data transmission and reception according to the transmission plan. However, since there may be a backlog in data processing, the user needs to increase the number of replicas, reduce the amount of data transmitted from the transmission application 2, etc.
[0050] In the training state S2, the stable state S3, the high load state S4, and the warning state S5, when the stop button 340 is pressed, an end command is executed by the end command execution unit 134, and the control system 1 transitions to the stop state S1.
[0051] FIG. 6 is a diagram showing an example of the data structure of the measurement result information 152 of Example 1.
[0052] The measurement result information 152 is information in a table format and stores entries including a transmission application T101, a processing application T102, a reception application T103, a data transfer amount (1) T104, and a data transfer amount (2) T105. There is one entry for the measurement result of one link. Note that the fields included in the entry are not limited to those described above.
[0053] The transmission application T101 is a field for storing the identification information of the transmission application 2. The processing application T102 is a field for storing the identification information of the processing application 3. The reception application T103 is a field for storing the identification information of the reception application 4. The data transfer amount (1) T104 is a field for storing the data transfer amount between the transmission application 2 and the processing application 3. The data transfer amount (2) T105 is a field for storing the data transfer amount between the processing application 3 and the reception application 4.
[0054] Note that the measurement result information 152 is not limited to the table format, and may be in CSV, XML format, etc.
[0055] FIG. 7 is a diagram showing an example of the data structure of the link tolerance value information 153 of Example 1.
[0056] The link tolerance value information 153 is information in a table format and stores entries including a transmission application T201, a processing application T202, and a link tolerance value T203. There is one entry for one link. Note that the fields included in the entry are not limited to those described above.
[0057] The transmission application T201 is a field that stores the identification information of the transmission application 2. The processing application T202 is a field that stores the identification information of the processing application 3. The link tolerance value T203 is a field that stores the tolerance value (maximum value) of the data transfer amount in the link. In this embodiment, the maximum value of the data transfer amount between the transmission application 2 and the processing application 3 is stored.
[0058] Note that the link tolerance value information 153 is not limited to the table format and may also be in the CSV format, XML format, etc.
[0059] FIG. 8 is a flowchart for explaining the processing executed by the control system 1 of the first embodiment in the training state S2.
[0060] The distributed control execution unit 112 of the control system 1 selects one processing application 3 from among the processing applications 3 to be monitored (step S21).
[0061] The distributed control execution unit 112 sets the data transmission amount (step S22). When setting the data transmission amount for the first time, the distributed control execution unit 112 sets an initial value, and when the data transmission amount is already set, the distributed control execution unit 112 adds an increment to the current value. The initial value and the increment are assumed to be set in advance. Note that the initial value and the increment may be updated.
[0062] The distributed control execution unit 112 instructs the transmission of training packets of the data volume set in the transmission application 2, and transmits the received training packets to the selected processing application 3 (step S23). The training packets reach the receiving application 4 via the processing application 3.
[0063] The measurement units 121 and 122 of the load distribution evaluation unit 12 measure the values of the communication metrics specified via the metric specifying unit 133 during the transmission period of the training packets, and output the measurement results to the distributed control planning unit 111 via the feedback unit 123. The distributed control planning unit 111 stores the measurement results in the measurement result information 152 (step S24). Note that the distributed control planning unit 111 can identify each application based on the IP address etc. included in the training packets.
[0064] After the measurement by the measurement units 121 and 122 is completed, the distributed control execution unit 112 determines whether to complete the measurement (step S25).
[0065] For example, when the data transfer volume between the processing application 3 and the receiving application 4 is smaller than the data transfer volume between the transmission application 2 and the processing application 3, the distributed control execution unit 112 completes the measurement.
[0066] If the measurement is not completed, the distributed control execution unit 112 returns to step S22 and executes the same processing.
[0067] When the measurement is completed, the distributed control execution unit 112 determines whether the measurement of all the processing applications 3 is completed (step S26).
[0068] If the measurement of all the processing applications 3 is not completed, the distributed control execution unit 112 returns to step S21 and executes the same processing.
[0069] When the measurement of all the processing applications 3 is completed, the distributed control execution unit 112 instructs the distributed control planning unit 111 to generate a transmission plan. The distributed control planning unit 111 calculates the link tolerance value of each link based on the measurement results (step S27). For example, the following processing is executed.
[0070] The distributed control planning unit 111 selects a link, refers to the values of the data transfer amount (1) T104 and the data transfer amount (2) T105 of the measurement result of the selected link, and calculates the maximum value of the data transfer amount (1) T104 that is less than or equal to the value of the data transfer amount (2) as the link tolerance value. The distributed control planning unit 111 executes the same processing for each link. Note that the method for calculating the tolerance value is an example and is not limited to this.
[0071] The state where the value of the data transfer amount (1) T104 is less than or equal to the value of the data transfer amount (2) indicates that no data retention has occurred. Therefore, the maximum value of the data transfer amount (1) without data retention can be used as an index for evaluating the series of processing performances of the input, data processing, and output of the processing application 3.
[0072] The distributed control planning unit 111 calculates the allocation ratio of the data amount of each link based on the link tolerance value (step S28). Then, the distributed control planning unit 111 outputs the allocation ratio of the data amount of each link to the distributed control execution unit 112 as a transmission plan.
[0073] In the first embodiment, the distributed control planning unit 111 calculates the ratio of the reciprocals of the link tolerance values of each link as the allocation ratio. Note that the method for calculating the allocation ratio is an example and is not limited to this.
[0074] After receiving the transmission plan from the distributed control planning unit 111, the distributed control execution unit 112 transitions to the stable state S3 (step S29).
[0075] The distributed control execution unit 112 distributes the data received from the transmission application 2 to each processing application 3 based on the transmission plan.
[0076] FIG. 9 is a flowchart for explaining the process executed by the control system 1 of Example 1 in the stable state S3. The control system 1 periodically executes the processes described below in the stable state S3.
[0077] The distributed control execution unit 112 of the control system 1 determines whether it has detected an opportunity to execute state confirmation (step S31). For example, the distributed control execution unit 112 detects the elapse of a certain period of time after the transition to the stable state S3 or after the previous process as an opportunity to execute state confirmation.
[0078] When detecting an opportunity to execute state confirmation, the distributed control execution unit 112 acquires the data transfer amount of each link from the measurement unit 121 and the measurement unit 122, and determines whether the data transfer amount of all links is equal to or less than the link allowable value (step S32).
[0079] When the data transfer amount of at least one link is greater than the link allowable value, the distributed control execution unit 112 transitions to the high load state S4 (step S33).
[0080] When the data transfer amount of all links is equal to or less than the link allowable value, the distributed control execution unit 112 maintains the stable state S3 (step S34).
[0081] In step S31, when not detecting an opportunity to execute state confirmation, the distributed control execution unit 112 determines whether it has detected an opportunity to execute training (step S35). For example, the distributed control execution unit 112 detects the elapse of a certain period of time after the previous process, or changes in the number of replicas of the application and the connection state of the application as an opportunity to execute training. Changes in the number of replicas of the application and the connection state of the application can be detected via the measurement units 121 and 122.
[0082] When not detecting an opportunity to execute training, the distributed control execution unit 112 maintains the stable state S3 (step S34).
[0083] When the opportunity to execute training is detected, the distributed control execution unit 112 transitions to the training state S2 (step S36).
[0084] In addition to the above processing, the distributed control execution unit 112 controls the amount of data to be transmitted to the processing application 3 according to the transmission plan.
[0085] FIG. 10 is a flowchart for explaining the processing executed by the control system 1 of the first embodiment in the high load state S4.
[0086] The distributed control execution unit 112 acquires the data transfer amount of each link from the measurement units 121 and 122, and determines whether the data transfer amount of all links is equal to or less than the link allowable value (step S41).
[0087] If the data transfer amount of all links is equal to or less than the link allowable value, the distributed control execution unit 112 transitions to the stable state S3 (step S44).
[0088] If the data transfer amount of at least one link is greater than the link allowable value, the distributed control execution unit 112 determines whether there is a link whose data transfer amount can be changed (step S42). Specifically, the distributed control execution unit 112 refers to the data transfer amount of each link acquired from the measurement unit 121, and determines whether there is a link whose data amount is less than the link allowable value.
[0089] If there is a link whose data transfer amount can be changed, the distributed control execution unit 112 instructs the distributed control planning unit 111 to change the transmission plan. When the distributed control planning unit 111 receives the instruction, it changes the transmission plan (step S43).
[0090] Specifically, the distributed control planning unit 111 changes the data transfer amount of a link whose data transfer amount can be changed, and calculates the allocation ratio of the data amount of each link based on the changed data transfer amount and the current data transfer amounts of other links. The distributed control planning unit 111 outputs the calculated allocation ratio to the distributed control execution unit 112 as a transmission plan. When receiving the transmission plan, the distributed control execution unit 112 controls the data amount based on the transmission plan. After a certain period of time has elapsed, the distributed control execution unit 112 returns to step S41.
[0091] When there is no link whose data transfer amount can be changed, the distributed control execution unit 112 transitions to the warning state S5 (step S45).
[0092] As described above, according to the first embodiment, the control system 1 evaluates a series of processing performances of data input, processing, and output in the processing application 3 which is a data flow type application, and controls the data amount of the data processed by each processing application 3. Thereby, it is possible to prevent the throughput of the processing of the entire system from being limited by the processing performance of a specific processing application 3. Therefore, the throughput of the processing of the entire system can be improved.
[0093] Next, the operation and control of the control system 1 will be described using a specific example.
[0094] First, the training state S2 will be described. FIG. 11 is a diagram showing a configuration example of an application to be controlled by the system of the first embodiment. FIGS. 12A, 12B, and 12C are diagrams for explaining a method of measuring communication metrics of the control system 1 of the first embodiment. FIGS. 13A, 13B, and 13C are diagrams showing specific examples of the measurement result information 152 of the first embodiment. FIG. 14 is a diagram showing a specific example of the link tolerance value information 153 of the first embodiment.
[0095] Hereinafter, assume a service with an application configuration as shown in FIG. 11. Specifically, the service is composed of one transmission application (Data Source A) 2, a processing application (1) (App(1)) 3-1, a processing application (2) (App(2)) 3-2, a processing application (3) (App(3)) 3-3, and a reception application (Data Sink A) 4. Note that the present invention exhibits the same effect even with a configuration other than the application configuration shown in FIG. 11.
[0096] In the following description, a set of the transmission application 2, the processing application (1) 3-1, and the reception application 4 is described as link (1), a set of the transmission application 2, the processing application (2) 3-2, and the reception application 4 is described as link (2), and a set of the transmission application 2, the processing application (3) 3-3, and the reception application 4 is described as link (3).
[0097] In the training state S2, the communication index (data transfer amount) is measured by the following procedure. First, the distributed control execution unit 112 instructs the transmission application 2 to transmit a training packet to the processing application (1) 3-1 (FIG. 12A). The processing application (1) 3-1 transmits the training packet to the reception application 4. A plurality of training packets with different data transmission amounts are transmitted multiple times. The measurement units 121 and 122 measure the data transmission amount between applications. As a result, a measurement result as shown in FIG. 13A is stored in the measurement result information 152.
[0098] The distributed control execution unit 112 instructs the transmission application 2 to transmit a training packet to the processing application (2) 3-2 (FIG. 12B). The processing application (2) 3-2 transmits the training packet to the reception application 4. A plurality of training packets with different data transmission amounts are transmitted multiple times. The measurement units 121 and 122 measure the data transmission amount between applications. As a result, a measurement result as shown in FIG. 13B is stored in the measurement result information 152.
[0099] The decentralized control execution unit 112 instructs the transmission application 2 to transmit a training packet to the processing application (3) 3-3 (FIG. 12C). The processing application (3) 3-3 transmits the training packet to the reception application 4. A plurality of training packets with different data transmission amounts are transmitted multiple times. The measurement units 121 and 122 measure the data transmission amount between applications. As a result, measurement results such as those in FIG. 13C are stored in the measurement result information 152.
[0100] When measurement results as shown in FIGS. 13A to 13C are stored in the measurement result information 152, in step S27, the decentralized control planning unit 111 calculates the link allowable value of the link (1) as "10 topic / sec", the link allowable value of the link (2) as "20 topic / sec", and the link allowable value of the link (3) as "11 topic / sec". As a result, link allowable value information 153 as shown in FIG. 14 is generated.
[0101] Next, the stable state S3 and the high load state S4 will be described. FIGS. 15A and 15B are diagrams showing an example of the communication state of the application to be controlled in the first embodiment.
[0102] When a trigger for executing the state check is detected, the decentralized control execution unit 112 acquires the data transfer amount of each link from the measurement unit 121 and the measurement unit 122.
[0103] In the case of the measurement results as shown in FIG. 15A, the data transfer amounts of the respective links (1), (2), and (3) are below the link allowable values. Therefore, the decentralized control execution unit 112 maintains the stable state S3 (step S34).
[0104] In the case of the measurement results as shown in FIG. 15B, the data transfer amount between the transmission application 2 and the processing application 3 of the link (3) is larger than the link tolerance value. Therefore, the distributed control execution unit 112 transitions to the high load state S4 (step S33). Since the data transmission amounts between the transmission application 2 and the processing application 3 of the links (1) and (2) have reached the link tolerance value, there is no link for which the transfer amount can be changed. Therefore, the distributed control execution unit 112 transitions to the warning state S5 (step S45).
Embodiment
[0105] In Embodiment 2, the transfer time from the transmission application 2 via the processing application 3 to the reception application 4 is used as a communication index for evaluating the data transmission and reception performance of the processing application 3. Hereinafter, Embodiment 2 will be described centering on the differences from Embodiment 1 and the like.
[0106] The system configuration of Embodiment 2 is the same as that of Embodiment 1. The configuration of the control system 1 of Embodiment 2 is the same as that of Embodiment 1.
[0107] In Embodiment 2, the data structure of the measurement result information 152 is partially different. FIG. 16 is a diagram showing an example of the data structure of the measurement result information 152 of Embodiment 2.
[0108] In Embodiment 2, the entries stored in the measurement result information 152 include a transmission application T101, a processing application T102, a reception application T103, a data transfer amount T111, and a transfer time T112. The data transfer amount T111 is the same field as the data transfer amount (1) T104. The transfer time T112 is a field for storing the transfer time until the data reaches the reception application 4 via the processing application 3 from the transmission application 2.
[0109] The data structure of the link tolerance value information 153 of Embodiment 2 is the same as that of Embodiment 1. However, the transfer time is stored in the link tolerance value T203.
[0110] The processing executed by the control system 1 in the training state S2 in Example 2 is the same as that in Example 1. However, in Example 2, the processing contents of steps S25 and S27 are different from those in Example 1.
[0111] In step S25, when the transfer time increases rapidly, the distributed control execution unit 112 determines that the measurement is completed. For the rapid increase in the transfer time, for example, methods using the average value of the transfer time, the change amount of the transfer time, and the transfer time change rate can be considered. Note that the determination method is not limited to this. In Example 2, when the transfer time fluctuates by 10% or more compared to the previous time, it is determined that there has been a rapid increase in the transfer time.
[0112] In step S27, the distributed control execution unit 112 calculates the second largest transfer time after the transfer time at the time when the transfer time increases rapidly as the link tolerance value. Note that the link tolerance value may be calculated by other methods. The transfer time can be used as an index for evaluating the series of processing performances of the input, data processing, and output of the processing application 3.
[0113] The processing executed by the control system 1 in the stable state S3 and the high load state S4 in Example 2 is the same as that in the example.
[0114] Next, the operation and control of the control system 1 will be described using a specific example.
[0115] First, the training state S2 will be described. Assume that the configuration of the application to be controlled by the system is the configuration shown in FIG. 11. FIGS. 17A, 17B, and 17C are diagrams showing specific examples of the measurement result information 152 in Example 2. FIG. 18 is a diagram showing a specific example of the link tolerance value information 153 in Example 2.
[0116] In the training state S2, the communication index (transfer time) is measured according to the following procedure. First, the distributed control execution unit 112 instructs the transmission application 2 to transmit a training packet to the processing application (1) 3-1 (Fig. 12A). The processing application (1) 3-1 transmits the training packet to the receiving application 4. A plurality of training packets with different data transmission amounts are transmitted multiple times. The measurement units 121 and 122 measure the data transfer time. As a result, measurement results as shown in Fig. 17A are stored in the measurement result information 152.
[0117] The distributed control execution unit 112 instructs the transmission application 2 to transmit a training packet to the processing application (2) 3-2 (Fig. 12B). The processing application (2) 3-2 transmits the training packet to the receiving application 4. A plurality of training packets with different data transmission amounts are transmitted multiple times. The measurement units 121 and 122 measure the data transfer time. As a result, measurement results as shown in Fig. 17B are stored in the measurement result information 152.
[0118] The distributed control execution unit 112 instructs the transmission application 2 to transmit a training packet to the processing application (3) 3-3 (Fig. 12C). The processing application (3) 3-3 transmits the training packet to the receiving application 4. A plurality of training packets with different data transmission amounts are transmitted multiple times. The measurement units 121 and 122 measure the data transfer time. As a result, measurement results as shown in Fig. 17C are stored in the measurement result information 152.
[0119] When the measurement results as shown in Figs. 17A to 17C are stored in the measurement result information 152, in step S27, the distributed control planning unit 111 calculates the link tolerance value of link (1) as "0.93 sec", the link tolerance value of link (2) as "0.67 sec", and the link tolerance value of link (3) as "1.01 sec". As a result, link tolerance value information 153 as shown in Fig. 18 is generated.
[0120] Next, the stable state S3 and the high load state S4 will be described. FIGS. 19A, 19B, and 19C are diagrams showing an example of the communication state of the application to be controlled in the second embodiment. FIG. 20 is a diagram showing an example of the communication state of the application to be controlled in the second embodiment.
[0121] When the trigger for executing the state confirmation is detected, the distributed control execution unit 112 acquires the data transfer amount of each link from the measurement unit 121 and the measurement unit 122.
[0122] In the case of the measurement results shown in FIG. 19A, the transfer times of the links (1), (2), and (3) are within the link allowable values. Therefore, the distributed control execution unit 112 maintains the stable state S3 (step S34).
[0123] In the case of the measurement results shown in FIG. 19B, the transfer time of the link (3) is greater than the link allowable value. Therefore, the distributed control execution unit 112 transitions to the high load state S4 (step S33). Since the transfer times of the links (1) and (2) are within the link allowable values, there are links whose transfer amounts can be changed. Therefore, the distributed control execution unit 112 changes the transmission plan (step S43).
[0124] For example, the distributed control planning unit 111 changes the transmission plan so as to transmit a part of the data of the link (3) to at least one of the links (1) and (2).
[0125] As a result of changing the transmission plan, when the communication state is as shown in FIG. 20, the distributed control execution unit 112 transitions to the stable state S3 (step S44).
[0126] In the case of the measurement results shown in FIG. 19C, the transfer time of the link (1) is greater than the link allowable value. Therefore, the distributed control execution unit 112 transitions to the high load state S4 (step S33). Since the transfer times of the links (2) and (3) have reached the link allowable values, there are no links whose transfer amounts can be changed. Therefore, the distributed control execution unit 112 transitions to the warning state S5 (step S45).
Description of Symbols
[0127] 1 Control System 2 Transmission Application 3 Processing Application 4 Reception Application 11 Load Balancing Control Unit 12 Load Balancing Evaluation Unit 13 User Input Unit 14 User Output Unit 15 Storage Unit 111 Distributed Control Planning Unit 112 Distributed Control Execution Unit 121, 122 Measurement Units 123 Feedback Unit 131 Training Execution Schedule Designation Unit 132 Status Confirmation Schedule Designation Unit 133 Index Designation Unit 134 End Command Execution Unit 141 Topology Display Unit 142 Distributed Control Status Display Unit 151 User Setting Information 152 Measurement Result Information 153 Link Tolerance Value Information 200 Computer 201 Processor 202 Main Memory Device 203 Auxiliary Memory Device 204 Network Interface 300 Input Screen 400 Output Screen
Claims
1. A computer system comprising: at least one computer having a processor, a storage device connected to the processor, and a network interface connected to the processor; wherein the processor: obtains data from a source object, performs a predetermined process on the data, and transmits the data to a destination object, and obtains a communication metric for evaluating the data transmission and reception performance of the processing applications from each of a plurality of processing applications; calculates a link tolerance value representing the processing performance of each of the plurality of processing applications based on the communication metric; generates a transmission plan regarding the amount of data to be allocated to the plurality of processing applications based on the link tolerance value; and controls the transmission of data to the plurality of processing applications based on the transmission plan.
2. The computer system according to claim 1, wherein the processor: calculates a ratio of the amount of data to be transmitted to each of the plurality of processing applications using the link tolerance value of each of the plurality of processing applications; and generates the transmission plan including the ratio of the amount of data to be transmitted to each of the plurality of processing applications.
3. The computer system according to claim 2, wherein the processor: for each of the plurality of processing applications, transmits test data having different amounts of data a plurality of times, and obtains, as the communication metric, a first data transfer amount between the source object and the processing application and a second data transfer amount between the processing application and the destination object; and calculates, as the link tolerance value, the largest first data transfer amount that is less than or equal to the second data transfer amount.
4. The computer system according to claim 2, wherein the processor: for each of the plurality of processing applications, transmits test data having different amounts of data a plurality of times, and obtains, as the communication metric, a transfer time from when data is transmitted from the source object until it reaches the destination object via the processing application. A computer system, characterized in that when the transfer time increases rapidly, the second largest transfer time after the time when the transfer time increases rapidly is calculated as the link tolerance value.
5. The computer system according to claim 2, wherein, during the control of data transmission for the plurality of processing applications using the transmission plan, when the communication index is detected to be greater than the link tolerance value for a target processing application, the processor changes the transmission plan based on the link tolerance values of the plurality of processing applications other than the target processing application. A computer system characterized by this.
6. The computer system according to claim 2, wherein the processor, during the control of data transmission for the plurality of processing applications using the transmission plan, monitors an opportunity for regenerating the transmission plan, and when an opportunity for regenerating the transmission plan is detected, regenerates the transmission plan. A computer system characterized by this.
7. The computer system according to claim 2, wherein the processor provides an interface for receiving a designation of the communication index to be acquired and an interface for displaying a status of control of data transmission for the plurality of processing applications using the transmission plan. A computer system characterized by this.
8. A data transmission control method executed by a computer system, wherein the computer system includes at least one computer having a processor, a storage device connected to the processor, and a network interface connected to the processor, and the data transmission control method includes: a first step in which the processor acquires a communication index for evaluating the data transmission and reception performance of each of the plurality of processing applications, wherein the processor acquires data from a source object, performs a predetermined process on the data, and transmits the data to a destination object; a second step in which the processor calculates a link tolerance value representing the processing performance of each of the plurality of processing applications based on the communication index; a third step in which the processor generates a transmission plan regarding the amount of data to be allocated to the plurality of processing applications based on the link tolerance value. A fourth step in which the processor controls the transmission of data to the plurality of processing applications based on the transmission plan, and a data transmission control method characterized by including the same.
9. The data transmission control method according to claim 8, wherein the third step includes a step in which the processor calculates a ratio of the amount of data to be transmitted to each of the plurality of processing applications using the link tolerance value of each of the plurality of processing applications, and a step in which the processor generates the transmission plan including the ratio of the amount of data to be transmitted to each of the plurality of processing applications, and a data transmission control method characterized by including the same.
10. The data transmission control method according to claim 9, wherein the first step includes a step in which the processor transmits test data having different amounts of data a plurality of times for each of the plurality of processing applications, and obtains, as the communication metrics, a first data transfer amount between the source object and the processing application and a second data transfer amount between the processing application and the destination object, and the second step includes a step in which the processor calculates, as the link tolerance value, the largest first data transfer amount that is less than or equal to the second data transfer amount, and a data transmission control method characterized by including the same.
11. The data transmission control method according to claim 9, wherein the first step includes a step in which the processor transmits test data having different amounts of data a plurality of times for each of the plurality of processing applications, and obtains, as the communication metrics, a transfer time from when data is transmitted from the source object until it reaches the destination object via the processing application, and the second step includes a step in which the processor calculates, as the link tolerance value, the next largest transfer time after the time when the transfer time rapidly increases, and a data transmission control method characterized by including the same.
12. The data transmission control method according to claim 9, When the processor detects a target processing application for which the communication metric is greater than the link tolerance value during the control of data transmission to the plurality of processing applications using the transmission plan, the method includes changing the transmission plan based on the link tolerance values of the plurality of processing applications other than the target processing application. A data transmission control method characterized by that.
13. The data transmission control method according to claim 9, The processor monitors an opportunity for regenerating the transmission plan during the control of data transmission to the plurality of processing applications using the transmission plan; When an opportunity for regenerating the transmission plan is detected, the method includes the step of the processor regenerating the transmission plan again. A data transmission control method characterized by that.
14. The data transmission control method according to claim 9, The processor provides an interface for receiving a designation of the communication metric to be acquired; The processor provides an interface for displaying a status of control of data transmission to the plurality of processing applications using the transmission plan. A data transmission control method characterized by that.
Citation Information
Patent Citations
Coflow collaborative job flow scheduling perception data flow division method and device
CN110321223A
Server load distribution method and system, load distribution device for use with the system, and server
JP2005010983A
Load distribution control device and method
JP2008040718A
Communication path management program, communication path management method, and communication path management device
JP2018207285A
Storage management device, method, and program
JP2020091706A