Information processing system, information processing method, and program
The information processing system addresses performance issues by identifying and reducing replicas of lower-priority containers based on load status, ensuring balanced operation and performance across nodes.
Patent Information
- Application Number
- JP2022016010
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-02-04
- Publication Date
- 2025-12-23
- Estimated Expiration
- 2042-02-04
AI Technical Summary
Existing technologies face issues with performance degradation when mission-critical containers are in high load states or moved to new nodes, leading to potential impairment of processing performance.
An information processing system that acquires load status information, identifies containers with lower priorities for replica reduction based on priority, and adjusts the number of replicas on nodes to manage load balance.
The system ensures appropriate operation of containers by reducing replicas of lower-priority containers, maintaining performance and load balance across nodes.
Smart Images

Figure 0007790181000001 
Figure 0007790181000002 
Figure 0007790181000003
Abstract
Description
[Technical Field]
[0001] The present disclosure relates to an information processing system, an information processing method, and a program. [Background technology]
[0002] Patent Document 1 describes that when a container that has a large impact on the host (a container in a high load state) is detected, communication with that container is restricted.
[0003] Patent document 2 also describes the movement of a group of containers that are connected by a call relationship from one called container to a container that does not call other containers, based on topology information, to another information processing device.
[0004] Patent Document 3 also describes that importance is calculated for each host based on the number of callers for each container and configuration information (information indicating the placement relationship between containers, VMs (Virtual Machines), and PMs (Physical Machines)), and containers are moved so that importance is equalized between hosts. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] Patent Publication No. 2021-149808 [Patent Document 2] Japanese Patent Publication No. 2021-150876 [Patent Document 3] Patent Publication No. 2021-117806 Summary of the Invention [Problem to be solved by the invention]
[0006] However, the technology described in Patent Document 1 restricts communication of a container that is in a high load state, which may cause performance problems if the container is mission-critical.
[0007] Furthermore, in the techniques described in Patent Documents 2 and 3, for example, when a mission-critical container is moved to another node (information processing device), there is a possibility that the processing performance of the container may be impaired.
[0008] In view of the above-mentioned problems, an object of the present disclosure is to provide an information processing device, an information processing method, and a program that can operate containers appropriately. [Means for solving the problem]
[0009] In a first aspect of the present disclosure, there is provided an information processing system having an acquisition unit that acquires information indicating a load status of a first node, an identification unit that, when the load status of the first node on which multiple containers are running satisfies certain conditions, identifies a container for which the number of replicas is to be reduced on the first node based on the priority of each container, and a control unit that reduces the number of replicas on the first node of the container identified by the identification unit.
[0010] In addition, a second aspect of the present disclosure provides an information processing method that acquires information indicating the priority of each container and the load status of a first node, and when the load status of a first node on which multiple containers are running satisfies certain conditions, identifies a container for which the number of replicas should be reduced on the first node based on the priority of each container, and reduces the number of replicas of the identified container on the first node.
[0011] In addition, a third aspect of the present disclosure provides a program that causes a computer to execute the following steps: a process of acquiring information indicating the priority of each container and the load status of a first node; a process of identifying a container for which the number of replicas is to be reduced on the first node based on the priority of each container when the load status of the first node on which multiple containers are running satisfies certain conditions; and a process of reducing the number of replicas of the identified container on the first node. [Effects of the Invention]
[0012] According to one aspect, the container can be operated appropriately. [Brief explanation of the drawings]
[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Figure 2] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Figure 3] FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing apparatus according to an embodiment. [Figure 4] 10 is a flowchart illustrating an example of processing by the information processing apparatus according to the embodiment. [Figure 5] FIG. 4 is a diagram illustrating an example of a load condition DB according to the embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of a priority DB according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION
[0014] The principles of the present disclosure will be described with reference to some exemplary embodiments. It should be understood that these embodiments are set forth for illustrative purposes only, to aid those skilled in the art in understanding and practicing the present disclosure, without implying any limitation on the scope of the disclosure. The disclosure described herein may be implemented in various ways other than those described below. In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0015] (Embodiment 1) <Configuration> The configuration of an information processing system 1 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the information processing system 1 according to an embodiment. The information processing system 1 has an acquisition unit 11, an identification unit 12, and a control unit 13. Each of these units may be realized by cooperation between one or more programs installed in an information processing device 10 included in the information processing system 1 and hardware such as a processor 101 and a memory 102 of the information processing device 10.
[0016] The acquisition unit 11 acquires information indicating the load status of a first node. When the load status of a first node on which multiple containers are running satisfies a specific condition, the identification unit 12 identifies a container for reducing the number of replicas in the first node based on the priority of each container. Note that a container is, for example, one or more software (application) objects for providing a certain service. Note that, for example, by packaging an application and a library required to run the application in a container, it is possible to provide an independent operating environment for the application.
[0017] A container replica is an instance of the same container that is generated (created) and maintained for purposes such as redundancy. An instance is, for example, an object in which software is deployed in memory and ready to run. If the number of replicas of a container is 0, this means that the container is stopped.
[0018] Container-based virtualization is known as one method for achieving highly available services on cloud systems. Container-based virtualization is a type of virtualization in which multiple "compartments that behave like independent servers," called containers, are created on a single operating system (OS) and assigned to individual users or services. Using container-based virtualization makes it possible to increase the number of containers according to the number of users, etc., and flexibly increase capacity. This makes it easier to achieve highly available services.
[0019] The control unit 13 reduces the number of replicas of the container identified by the identification unit 12 in the first node.
[0020] (Embodiment 2) Next, the configuration of the information processing system 1 according to the embodiment will be described with reference to FIG. <System configuration> FIG. 2 is a diagram illustrating an example of the configuration of an information processing system 1 according to an embodiment. In the example of FIG. 2, the information processing system 1 includes an information processing device 10. The information processing system 1 also includes a cluster environment 20A and a cluster environment 20B (hereinafter, when there is no need to distinguish between them, they will also be simply referred to as "cluster environments 20"). The cluster environment 20A includes nodes 21A and 21B. The cluster environment 20B includes nodes 21C and 21D. Hereinafter, when there is no need to distinguish between the nodes 21A to 21D, they will also be simply referred to as "nodes 21"). In the example of FIG. 2, the information processing device 10 and the cluster environment 20 are connected to each other so as to be able to communicate with each other via a network N. Note that the numbers of the information processing devices 10, the cluster environments 20, and the nodes 21 are not limited to those in the example of FIG. 2.
[0021] Examples of the network N include, for example, the Internet, a LAN (Local Area Network), a mobile communication system, a wireless LAN, etc. Examples of the mobile communication system include, for example, a fifth generation mobile communication system (5G), a sixth generation mobile communication system (6G, Beyond 5G), a fourth generation mobile communication system (4G), a third generation mobile communication system (3G), etc.
[0022] The information processing device 10 is, for example, a device such as a server, a cloud, or a personal computer. The information processing device 10 may have the function of an orchestration tool for a container-type virtual environment. In this case, the information processing device 10 may manage a cluster environment 20 (container cluster) that is a collection of nodes 21 that execute containerized applications, and when a container is newly created, the information processing device 10 may operate (start) the container on a node 21 that meets certain conditions.
[0023] The node 21 is, for example, a device such as a server, a cloud, a personal computer, etc. The node 21 operates a container to provide, for example, a predetermined service to a user or the like.
[0024] <Hardware configuration> 3 is a diagram illustrating an example of the hardware configuration of the information processing device 10 and the node 21 according to the embodiment. In the following, the information processing device 10 will be described as an example, but the hardware configuration of the node 21 may also be the same as the hardware configuration of the information processing device 10.
[0025] 3, the information processing device 10 (computer 100) includes a processor 101, a memory 102, and a communication interface 103. These components may be connected via a bus or the like. The memory 102 stores at least a portion of a program 104. The communication interface 103 includes an interface required for communication with other network elements.
[0026] When the program 104 is executed by the processor 101, memory 102, and other components in cooperation with each other, the computer 100 performs at least some of the processing of the embodiments of the present disclosure. The memory 102 may be of any type suitable for a local technology network. The memory 102 may be, by way of non-limiting example, a non-transitory computer-readable storage medium. The memory 102 may also be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. While only one memory 102 is shown in the computer 100, several physically distinct memory modules may be present in the computer 100. The processor 101 may be of any type. The processor 101 may include one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), and, by way of non-limiting example, a processor based on a multi-core processor architecture. The computer 100 may have multiple processors, such as application-specific integrated circuit chips time-slaved to a clock that synchronizes the main processor.
[0027] Embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device.
[0028] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, that execute on a target real or virtual processor or device to perform the processes or methods of the present disclosure. Program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or divided among program modules as desired in various embodiments. The machine-executable instructions of the program modules may be executed in local or distributed devices. In a distributed device, the program modules may be located in both local and remote storage media.
[0029] The program code for executing the methods of the present disclosure may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus. When the program code is executed by the processor or controller, the functions / acts in the flowcharts and / or implementing block diagrams are performed. The program code may be executed entirely on the machine, partly on the machine, as a standalone software package, partly on the machine and partly on a remote machine, or entirely on a remote machine or server.
[0030] The program can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible recording media. Examples of non-transitory computer-readable media include magnetic recording media, magneto-optical recording media, optical disk media, and semiconductor memory. Magnetic recording media include, for example, flexible disks, magnetic tapes, and hard disk drives. Magneto-optical recording media include, for example, magneto-optical disks. Optical disk media include, for example, Blu-ray discs, CD (Compact Disc)-ROMs (Read Only Memory), CD-Rs (Recordable), and CD-RWs (Rewritable). Semiconductor memory includes, for example, solid-state drives, mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory). The program may also be provided to a computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.
[0031] <Processing> Next, an example of processing of the information processing device 10 according to the embodiment will be described with reference to Fig. 4 to Fig. 6. Fig. 4 is a flowchart showing an example of processing of the information processing device 10 according to the embodiment. Fig. 5 is a diagram showing an example of a load condition DB (database) 501 according to the embodiment. Fig. 6 is a diagram showing an example of a priority DB 601 according to the embodiment.
[0032] The following describes an example in which the following processing is performed by the information processing device 10 of the information processing system 1. Note that, in the technology disclosed herein, the following processing may be performed by one or more nodes 21 instead of or in addition to the information processing device 10.
[0033] The information processing device 10 may perform the following process, for example, periodically.
[0034] In step S1, the acquisition unit 11 of the information processing device 10 acquires information indicating the load status of each node 21. The load status of the node 21 may include, for example, the CPU usage rate of the node 21, the memory usage rate of the node 21, the free memory capacity of the node 21, etc.
[0035] Next, the specifying unit 12 of the information processing device 10 refers to the load condition DB 501 and determines whether the load status of at least one node 21 of the multiple nodes 21 satisfies a specific condition (step S2). In the example of FIG. 5, the load condition DB 501 records (registers, sets) a CPU usage rate threshold and a memory usage rate threshold in association with a node ID. The node ID is identification information of the node 21. The load condition DB 501 may be stored in a storage device inside the information processing device 10, or may be stored in a storage device (for example, a DB server) external to the information processing device 10. The information recorded in the load condition DB 501 may be set in advance by an operator (administrator) or the like.
[0036] When the CPU usage rate of a certain node 21 is equal to or greater than a CPU usage rate threshold, the identifying unit 12 may determine that the load situation of the certain node 21 satisfies a specific condition. Furthermore, when the memory usage rate of the certain node 21 is equal to or greater than a memory usage rate threshold, the identifying unit 12 may determine that the load situation of the certain node 21 satisfies a specific condition.
[0037] If none of the nodes 21 satisfies the condition (NO in step S2), the process is terminated. On the other hand, if any of the nodes 21 satisfies the condition (YES in step S2), the identifying unit 12 of the information processing device 10 identifies a container for which the number of replicas is to be reduced (decreased, reduced) in a node 21 (hereinafter also referred to as the "first node 21" as appropriate) whose load situation satisfies the specific condition, based on the priority of each container running in the node 21 (step S3). In the example of FIG. 6, priorities are recorded (registered, set) in the priority DB 601 in association with container IDs. The container ID is identification information of a container. The priority DB 601 may be stored in a storage device inside the information processing device 10 or in a storage device (e.g., a DB server) external to the information processing device 10. The information recorded in the priority DB 601 may be set in advance by an operator (administrator), etc.
[0038] The identifying unit 12 may identify a container with a lower priority among the first container and the second container operating in the first node 21 as a container for reducing the number of replicas in the node 21. Furthermore, the identifying unit 12 may identify a container with the lowest priority among the containers operating in the first node 21 as a container for reducing the number of replicas in the node 21.
[0039] Next, the control unit 13 of the information processing device 10 reduces the number of replicas of the container identified by the identification unit 12 in the node 21 (first node 21) whose load situation satisfies a specific condition (step S4). By repeating the processes from step S1 to step S4 one or more times, the load situation of the node 21 in which the container with a relatively high priority is operating can be made smaller than the threshold value.
[0040] Here, the control unit 13 may, for example, reduce the number of replicas of the container in the first node 21 by moving at least some of the one or more replicas of the container identified by the identification unit 12 from the first node 21 to a node 21 other than the first node 21 (hereinafter, also referred to as "another node 21" as appropriate). Furthermore, the control unit 13 may, for example, reduce the number of replicas of the container in the first node 21 by stopping at least some of the one or more replicas of the container identified by the identification unit 12 without moving them to another node 21.
[0041] (Example of stopping a low priority container on node 1 21) The control unit 13 may stop the container identified by the identification unit 12. This allows, for example, a container with a high priority to operate appropriately. In this case, for example, when the priority of the container identified by the identification unit 12 is equal to or lower than a first threshold, the control unit 13 may reduce the number of replicas of the container in the first node 21 by a predetermined number (for example, 1) without moving the container to another node 21.
[0042] Then, for example, the control unit 13 may decrease the number of replicas of the container in the first node 21 by a predetermined number (for example, 1) as long as the load condition of the first node 21 satisfies a specific condition. Then, for example, if the number of replicas of the container in the first node 21 is a predetermined number (for example, 1) but the load condition of the first node 21 satisfies a specific condition, the control unit 13 may stop the container by setting the number of replicas of the container in the first node 21 to 0.
[0043] In addition, the control unit 13 may reduce the number of replicas of the container in the first node 21 without moving the container to another node 21 only when the load status of each node 21 in the same cluster environment 20 is above a threshold, for example.
[0044] (Example of moving a container to another node 21 in the same cluster environment 20) The control unit 13 may move the container identified by the identification unit 12 to another node 21. This allows, for example, a container with a high priority to be operated appropriately. In this case, for example, when the priority of the container identified by the identification unit 12 is higher than a first threshold, the control unit 13 may move the container to another node 21. Furthermore, the control unit 13 may select, for example, a node 21 with a relatively low load status from among the nodes 21 in the same cluster environment 20 as the other node 21.
[0045] (Example of moving a container to another cluster environment 20) The control unit 13 may move the container identified by the identification unit 12 from the node 21 in the on-premise cluster environment 20A to the node 21 in the public cloud cluster environment 20B. This allows, for example, a container with a high priority to operate appropriately. In this case, for example, when the priority of the container identified by the identification unit 12 is higher than a second threshold, the control unit 13 may move the container to the public cloud.
[0046] Furthermore, for example, when the load status of each node 21 in the same cluster environment 20 is equal to or greater than a threshold, the control unit 13 may move the container identified by the identification unit 12 to a node 21 in another cluster environment 20. In this case, for example, the identification unit 12 may identify the container with the highest priority in the first node 21, and the control unit 13 may move the container to a public cloud. This makes it possible to appropriately operate a container with a high priority even when the load status of each node 21 in the on-premise cluster environment 20 including the first node 21 is equal to or greater than a threshold.
[0047] (An example in which a container with a low priority in another node 21 is stopped and moved to the other node 21) When the load situation of the first node 21 satisfies a specific condition, the identifying unit 12 may identify a third container for which the number of replicas is to be reduced in the first node 21 and a fourth container for which the number of replicas is to be reduced in the second node 21, based on the priority of each container. Then, the control unit 13 may reduce the number of replicas of the fourth container in the second node 21 and move the third container from the first node 21 to the second node 21. This makes it possible, for example, to stop a container with a fourth lower priority in another node 21 and move the third container from the first node 21 to the other node 21. Note that the other node 21 may be a node 21 in the same cluster environment 20 as the first node 21, or may be a node 21 in a cluster environment 20 different from that of the first node 21.
[0048] (Example of determining the priority of each container) In the above example, the priority of each container is set in advance by an operator (administrator) or the like. Alternatively, or in addition, the identification unit 12 may identify the priority of each container based on at least one of the current time, the frequency of access to each container, and the cluster environment in which each container operates. This allows, for example, the priority of the container to be set more appropriately.
[0049] In this case, for example, the specifying unit 12 may determine that a container that undergoes batch processing at night or the like has a higher priority at night than during the day. Also, for example, the specifying unit 12 may determine that the higher the frequency of access from users to a certain container, the higher the priority of the container.
[0050] Furthermore, for example, when a certain container operates in a public cloud cluster environment, the identifying unit 12 may determine a lower priority for the container than when the container operates on-premise. This allows the number of replicas to be reduced when the container operates in a public cloud, for example, when the processing volume of the container is relatively high and the importance of the container to the business is relatively low. Therefore, when the usage fee for the public cloud is a pay-per-use fee, the usage fee paid to the public cloud provider can be reduced.
[0051] <Modification> The information processing device 10 may be a device contained in a single housing, but the information processing device 10 of the present disclosure is not limited to this. Each unit of the information processing device 10 may be realized, for example, by cloud computing configured with one or more computers. Furthermore, the information processing device 10 and the node 21 may be housed in the same housing and configured as an integrated information processing device. Furthermore, the node 21 may execute at least a portion of the processing of each functional unit of the information processing device 10. Such information processing devices 10 are also included in examples of the "information processing device" of the present disclosure.
[0052] The present invention is not limited to the above-described embodiment, and can be modified as appropriate within the scope of the invention.
[0053] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) an acquisition unit that acquires information indicating a load status of the first node; an identification unit that identifies a container whose number of replicas is to be reduced in the first node based on the priority of each container when a load condition of the first node on which a plurality of containers are running satisfies a specific condition; a control unit that reduces the number of replicas of the container identified by the identification unit in the first node; An information processing system having the above. (Appendix 2) The identification unit When the load condition of a first node on which multiple containers are running satisfies a specific condition, a container for which the number of replicas is to be reduced is identified on the first node based on the priority of each container set by an operator. 10. The information processing system of claim 1. (Appendix 3) the identification unit identifies, from a first container having a first priority and a second container having a second priority lower than the first priority, the second container as a container that reduces the number of replicas in the first node; 3. The information processing system according to claim 1 or 2. (Appendix 4) The control unit stops the container identified by the identification unit. 4. An information processing system according to any one of claims 1 to 3. (Appendix 5) the control unit migrates the container identified by the identification unit from the on-premise first node to a public cloud; 5. An information processing system according to any one of claims 1 to 4. (Appendix 6) the identifying unit identifies, when a load condition of a first node on which a plurality of containers are running satisfies a specific condition, a third container for reducing the number of replicas in the first node and a fourth container for reducing the number of replicas in the second node, based on a priority of each container; the control unit reduces the number of replicas of the fourth container in the second node and moves the third container from the first node to the second node. 6. An information processing system according to any one of appendices 1 to 5. (Appendix 7) The identification unit identifies a priority of each container based on at least one of a current time, an access frequency to each container, and a cluster environment in which each container operates. 7. An information processing system according to any one of appendices 1 to 6. (Appendix 8) Obtain information indicating the priority of each container and the load status of the first node, When a load condition of a first node on which a plurality of containers are running satisfies a specific condition, a container for which the number of replicas is to be reduced is identified on the first node based on the priority of each container; Decreasing the number of replicas of the identified container on the first node; Information processing methods. (Appendix 9) A process of acquiring information indicating the priority of each container and the load status of the first node; When a load condition of a first node on which a plurality of containers are running satisfies a specific condition, a process of identifying a container for which the number of replicas is to be reduced on the first node based on the priority of each container; reducing the number of replicas of the identified container on the first node; A program that causes a computer to execute the following. [Explanation of symbols]
[0054] 1. Information Processing Systems 10. Information processing equipment 11 Acquisition Department 12 Specific part 13 Control Unit 20 Cluster Environment 21 nodes
Claims
1. an acquisition unit that acquires information indicating a load status of the first node; an identification unit that identifies a container whose number of replicas is to be reduced in the first node based on a priority of each container when a load condition of the first node on which a plurality of containers are running satisfies a specific condition; a control unit that reduces the number of replicas of the container identified by the identification unit in the first node; and the identification unit identifies, from among a first container having a first priority and a second container having a second priority lower than the first priority, the second container as a container that reduces the number of replicas in the first node; when the priority of the container identified by the identification unit is equal to or lower than a first threshold, the control unit reduces the number of replicas of the container in the first node without moving the container to another node; Information processing system.
2. The control unit stops the container identified by the identification unit. The information processing system according to claim 1 .
3. The identification unit identifies a priority of each container based on at least one of a current time, an access frequency to each container, and a cluster environment in which each container operates.
3. The information processing system according to claim 1 or 2.
4. Obtaining information indicating the priority of each container and the load status of the first node; When a load condition of a first node on which a plurality of containers are running satisfies a specific condition, a container for which the number of replicas is to be reduced is identified on the first node based on the priority of each container; Decreasing the number of replicas of the identified container on the first node; Furthermore, of a first container having a first priority and a second container having a second priority lower than the first priority, the second container is identified as a container that reduces the number of replicas in the first node; Furthermore, if the priority of the identified container is equal to or less than a first threshold, the container is not moved to another node, and the number of replicas of the container in the first node is reduced. The computer performs the processing, Information processing methods.
5. A process of acquiring information indicating the priority of each container and the load status of the first node; When a load condition of a first node on which a plurality of containers are running satisfies a specific condition, a process of identifying a container for which the number of replicas is to be reduced on the first node based on the priority of each container; reducing the number of replicas of the identified container on the first node; on the computer, In the process of identifying the container, of a first container having a first priority and a second container having a second priority lower than the first priority, the second container is identified as a container that reduces the number of replicas in the first node; In the process of reducing the number of replicas, if the priority of the identified container is equal to or less than a first threshold, the number of replicas of the identified container in the first node is reduced without moving the identified container to another node. program.
Citation Information
Patent Citations
Virtualization environment management system and virtualization environment management method
JP2017138895A
Real-time communication processing system and real-time communication processing method
JP2021060858A
Container arranging method and container arranging program
JP2021117806A
CPU status display method and CPU status display program
JP2021149808A
Movement target container determination method and movement target container determination program
JP2021150876A