Method, apparatus and computer-readable medium for resource control for system logging

Abstract

To provide a device, a system, and a method that identify the status of a plurality of synchronized devices at a particular time.SOLUTION: A method includes the steps of identifying the state of a plurality of synchronized devices at a specific time, allocating a set of time-synchronization resources for transmitting time-synchronization information from at least one device of the plurality of devices on the basis of the identified state of each of the plurality of synchronized devices, and transmitting a control message to at least one of the plurality of devices indicating an allocated set of time synchronization resources determined for transmitting time synchronization information from the at least one device.SELECTED DRAWING: Figure 17

Description

[Technical Field]

[0001] The present disclosure generally relates to collaborative systems in which humans and machines (devices, robots, mechatronics, etc.) cooperate. [Background technology]

[0002] Control systems that support social infrastructure have traditionally been constructed in closed spaces, physically and electronically isolated from humans. However, recent advances in mechatronics, control technology, information technology, etc. have led to the goal of cooperation between control systems and humans. This trend is being driven by labor shortages in developed countries and the demand for flexibility in response to diversifying social needs.

[0003] One example of such a collaborative control system is a collaborative robot. Collaborative robots that meet certain conditions can work in the same space as humans. To meet the needs mentioned above, collaborative robots may eventually provide labor and realize flexibility in control systems. Although such collaborative systems still have many limitations, in the future, advances in AI technology and autonomous and flexible behavior based on human behavior testing are expected to lead to the realization of collaborative systems that work in harmony with humans.

[0004] In realizing a human-collaborative control system, one of the challenges is ensuring safety and improving efficiency as a control system. Because a human-collaborative control system is designed to work in cooperation with humans, the most important premise, unlike conventional automated systems, is that it does not harm humans. Therefore, it is essential to guarantee safety.

[0005] Furthermore, compared to conventional automation systems, human collaborative control systems generally tend to sacrifice efficiency and performance in order to prioritize safety. For example, the operating speed of a collaborative robot is generally limited when a human is nearby for safety reasons. In such cases, the performance of the control system is lower than that of conventional automation control systems. Summary of the Invention [Problem to be solved by the invention]

[0006] In recent years, 5G has been considered as a wireless standard intended for application to control systems. The introduction of 5G to control systems is expected to promote the improvement of the quality of control systems composed of autonomous entities. In particular, multiple autonomous entities such as automated guided vehicles (AGVs) and autonomous mobile robots (AMRs) in factories, as well as autonomous driving, are attracting attention as control targets. Considering the societal demand for human-cooperative control systems, the challenge is how to realize such human-cooperative control systems that include both autonomous entities and humans. [Means for solving the problem]

[0007] Implementations described herein include innovative methods for dynamically allocating communications and / or calculating resources for communication with a control system based on the state of a controlled (terminal) device, which in some embodiments ensure the safety and improve the efficiency of human-cooperative control systems involving multiple autonomous entities.

[0008] Aspects of the present disclosure include a method that may include identifying states of a plurality of synchronized devices at a particular time. The method may further include determining a frequency for transmitting time synchronization information from at least one device of the plurality of devices based on the identified states of each of the plurality of synchronized devices, and allocating a set of time synchronization resources for transmitting the time synchronization information according to the determined frequency. The method may also include transmitting a control message to the at least one device of the plurality of devices indicating the determined assigned set of time synchronization resources for transmitting the time synchronization information from the at least one device. The state may include at least one of the following: the type of the device, the speed or acceleration of the device, an indication of whether the device is in a dynamic state or a stationary state, information about the device's environment, the location of the device, and information about objects in the vicinity of the device. The information about the objects may include at least one of the type of the object, the distance to the device, the relative speed to the device, and the relative acceleration to the device.

[0009] Aspects of the present disclosure include a non-transitory computer-readable medium storing instructions for execution by a processor, the medium may include instructions for identifying a state of a plurality of synchronized devices at a particular time. The instructions may further include instructions for determining a frequency for transmitting time synchronization information from at least one device of the plurality of devices based on the identified state of each of the plurality of synchronized devices, and allocating a set of time synchronization resources for transmitting the time synchronization information according to the determined frequency. The instructions may also include instructions for transmitting a control message to the at least one device of the plurality of devices indicating the determined assigned set of time synchronization resources for transmitting the time synchronization information from the at least one device. The state may include at least one of the following: the type of the device, the speed or acceleration of the device, an indication of whether the device is in a dynamic state or a stationary state, information about the device's environment, the location of the device, and information about objects in the vicinity of the device. The information about the objects may include at least one of the type of the object, the distance to the device, the relative speed to the device, and the relative acceleration to the device.

[0010] An aspect of the present disclosure may include a system that includes: means for identifying a state of a plurality of synchronized devices at a particular time; means for determining a frequency for transmitting time synchronization information from at least one device of the plurality of devices based on the identified state of each of the plurality of synchronized devices and allocating a set of time synchronization resources for transmitting the time synchronization information according to the determined frequency; and means for transmitting a control message to the at least one device of the plurality of devices indicating the allocated set of time synchronization resources determined for transmitting time synchronization information from the at least one device. The state may include at least one of the following: the type of the device, the speed or acceleration of the device, an indication of whether the device is in a dynamic state or a stationary state, information about the device's environment, the location of the device, and information about objects in the vicinity of the device. The information about the objects may include at least one of the type of the object, the distance to the device, the relative speed to the device, and the relative acceleration to the device.

[0011] Aspects of the present disclosure include an apparatus that may include a processor configured to identify states of a plurality of synchronized devices at a particular time. The processor may be further configured to determine a frequency for transmitting time synchronization information from at least one device of the plurality of devices based on the identified states of each of the plurality of synchronized devices, and to allocate a set of time synchronization resources for transmitting the time synchronization information according to the determined frequency. The processor may also be configured to transmit a control message to the at least one device of the plurality of devices indicating the allocated set of time synchronization resources determined for transmitting the time synchronization information from the at least one device. The state may include at least one of the following: the type of the device, the speed or acceleration of the device, an indication of whether the device is in a dynamic state or a stationary state, information about the device's environment, the location of the device, and information about objects in the vicinity of the device. The information about the objects may include at least one of the type of the object, the distance to the device, the relative speed to the device, and the relative acceleration to the device. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a control system including a resource control device according to an implementation example. [Figure 2] FIG. 2 illustrates the hardware structure of a device according to one implementation example. [Figure 3] FIG. 2 illustrates the hardware structure of a device according to one implementation example. [Figure 4] FIG. 1 illustrates a functional structure of a device according to an example implementation. [Figure 5] FIG. 1 illustrates a functional structure of a device according to an example implementation. [Figure 6] FIG. 1 is a flow diagram illustrating a method according to some aspects of the present disclosure. [Figure 7] FIG. 10 is a call flow diagram illustrating an example of a latency calculation process related to resource control in accordance with certain aspects of the present disclosure. [Figure 8] FIG. 1 is a flow diagram illustrating a method for log registration according to some aspects of the present disclosure. [Figure 9] FIG. 1 illustrates a first set of logged events associated with a local time and a second set of logged events associated with a system time (e.g., a synchronized time). [Figure 10]1 is a set of diagrams illustrating exemplary relationships of (potential) synchronization errors as a function of time associated with reception between different frequencies of synchronization packets and / or synchronization messages; [Figure 11] FIG. 1 illustrates a system in an exemplary environment including a collaborative robot, an AGV, and a worker in a factory, in accordance with some aspects of the present disclosure. [Figure 12] FIG. 1 illustrates a system for transporting parts from a belt conveyor to a table through coordination between two industrial robotic arms. [Figure 13] FIG. 1 is a flow diagram illustrating a method for causal analysis associated with a fault in a system. [Figure 14] FIG. 14 is a flow diagram of a method for implementing the updated configuration determined for a simulated device (or environment) described in connection with FIG. 13 with respect to a physical device (or environment). [Figure 15] FIG. 10 illustrates an example of a time correction that may be performed as part of recording an updated configuration in accordance with some aspects of the present disclosure. [Figure 16] A set of diagrams of drones flying in different factory environments and configurations related to different areas or conditions. [Figure 17] FIG. 1 is a flow diagram illustrating a method according to some aspects of the present disclosure. [Figure 18] FIG. 1 is a flow diagram illustrating a method according to some aspects of the present disclosure. [Figure 19] FIG. 1 is a flow diagram illustrating a method according to some aspects of the present disclosure. [Figure 20] FIG. 1 illustrates an exemplary computing environment with an exemplary computing device suitable for use in some implementations. DETAILED DESCRIPTION OF THE INVENTION

[0013] The following detailed description provides details of the figures and example implementations of the present application. Reference numerals and descriptions of redundant elements between figures have been omitted for clarity. Terms used throughout this specification are provided by way of example and are not intended to be limiting. For example, the use of the term "automatic" may include fully automatic or semi-automatic implementations with user or administrator control over certain aspects of the implementation, depending on the desired implementation of those skilled in the art practicing the embodiments of the present application. Selection may be performed by a user through a user interface or other input means, or may be performed by a desired algorithm. The example implementations described herein may be utilized either singly or in combination, and the functionality of the example implementations may be implemented by any means according to the desired implementation.

[0014] Implementation examples described herein include an innovative method for dynamically allocating and / or calculating communication resources for communication with a control system based on the state of a controlled (terminal) device. Aspects of the present disclosure include a method that may include identifying the state of a plurality of synchronized devices at a particular time. The method may further include allocating a set of time synchronization resources for transmitting time synchronization information from at least one device of the plurality of devices based on the identified state of each of the plurality of synchronized devices. The method may also include transmitting a control message to at least one device of the plurality of devices indicating the determined set of allocated time synchronization resources for transmitting the time synchronization information from the at least one device.

[0015] 1 illustrates an example configuration of a control system including a resource control device according to one implementation. Resource control device 120 may communicate with one or more elements of a communication network 122. The communication network may include one or more network relay devices, such as network relay device 121a, network relay device 121b, network relay device 121c, or network relay device 121d. The communication network may include one or more wireless base stations, such as wireless base station 124a or wireless base station 124b, input / output (I / O) control devices, such as I / O control device 125a, I / O control device 125b, I / O control device 125c, or I / O control device 125d, and control devices, such as control device 127. The resource control device 120 may communicate with one or more terminal devices, e.g., terminal device 123a, terminal device 123b, or terminal device 123c, via a control network 122 and / or a wireless network (e.g., a network including network relay devices 121a-121d and wireless base station 124a or 124b).

[0016] The resource control device 120 may send and / or receive (generally "communicate") communication packets with the network relay devices 121a-121d, the terminal devices 123a-123c, the wireless base stations 124a or 124b, the I / O control devices 125a-125d, and / or the control device 127 to control (allocate) resources associated with the terminal devices 123a-123c and / or the I / O control devices 125a-125d. The resource control device 120 may further communicate with one or more elements of the control network 122, the I / O control devices 125a-125d, and / or the terminal devices 123a-123c to obtain control commands, measurements, and / or sensor information from the devices to determine the status of the terminal devices 123a-123c and / or the I / O control devices 125a-125d.

[0017] In some aspects, resource control device 120 may be implemented as a virtualized control device and / or as a software container that provides control device functionality. In some aspects, an application or operating system (OS) may be managed as a container through virtualization. Depending on the desired implementation, the physical implementation of resource control device 120 may include a dedicated controller, an industrial personal computer (PC), a distributed control system (DCS) controller, a multi-access edge computing (MEC) device, a computer cloud, a server computer, a supervisory control and data acquisition (SCADA) server, etc. Alternatively, a configuration in which such a device is located in a destination network (DN) defined in the 5G standard is also illustrated.

[0018] In some aspects, the network relay devices 121a-121d may be relay devices within the control network 122. Packets communicated between the resource control device 120, the terminal devices 123a-123c, the I / O control devices 125a-125d, the control device 127, and the network relay devices 121a-121d may be routed and forwarded in some aspects.

[0019] Depending on the desired implementation, the physical implementation of the network relay devices 121a-121d may be in the form of a Time Sensitive Network (TSN) compatible switch, a network switch including an L2 switch or an L3 switch, a Software Defined Network (SDN) compatible device, a Network Function Virtualization (NFV) compatible device, a bridge, a router, an IEEE 1588TC (Transparent Clock) or BC (Boundary Clock), an OpenFlow (trademark) switch, a RedBox or QuadBox as defined by IEC 62439-3, an optical switch, an optical multiplexer / demultiplexer, and / or various other network relay devices.

[0020] In some aspects, the control network 122 may be a network for connecting the resource control device 120, the network relay devices 121a-121d, the terminal devices 123a-123c, and the wireless base stations 124a and / or 124b, the I / O control devices 125a-125d, and the control device 127. The physical implementation of the control network 122 may be based on any of the IEEE 802.3 (Ethernet) standards, the IEC 61784 standard, the IEC 61784-2 Communication Profile Family 12 (hereinafter referred to as EtherCAT™) standard, IEEE communication standards related to TSN, the High availability Seamless Redundancy (HSR) and Parallel Redundancy Protocol (PRP) of the IEC 62439-3 standard, OpenFlow™, other control communications, and / or various wireless communications, wireless networks such as 4G, 5G, and / or 6G core networks, the Distributed Network Protocol (DNP) 3 standard, the IEC 61970 standard, redundant networks, ring networks, the IEEE 802.17 RPR standard, the CAN™ (Controller Area Network), DeviceNet, the RS-232C standard, the RS-422 standard, the RS-485 standard, the ZigBee™ standard, the Bluetooth™ standard, the IEEE 802.15 standard, the IEEE 802.1 standard, mobile communications, and ECHONET. It may be compatible with any standard including various wireless communications such as Lite™, OpenADR™, and / or Wi-Fi™ and / or industrial wireless networks.

[0021] Furthermore, examples of upper layer protocols include IEC 61850, OPC UA (Unified Architecture), DDS (Data Distribution Service), IEC 61850-7-420, IEC 60870-5-104, communication protocols based on Service-Based Interface (SBI), REST API, HTTP / 2, OpenAPI, and JSON exchange. Alternatively, the above protocols may be layered. For example, the data content included in TSN packets may conform to the OPC UA standard.

[0022] In some embodiments, the terminal devices 123a-123c may be connected to I / O control devices 125a-125d, such as sensors or actuators. The terminal devices 123a-123c may control and configure the I / O control devices 125a-125d according to resource control commands received from the resource control device 120 via the control network 122. The terminal devices 123a-123c may also control and configure the I / O control devices 125a-125d according to control commands received from the control device 127 via the control network 122. Furthermore, the terminal devices 123a-123c may transmit information about their own status and the status of the I / O control devices 125a-125d to the resource control device 120 via the control network 122. The terminal devices 123a-123c and / or I / O control devices 125a-125d may be configured to provide information as sensors (e.g., input to the resource control device 120 and / or control device 127), to receive information as actuators (e.g., output from the resource control device 120 and / or control device 127), or may combine both transmitting and / or receiving (e.g., input and / or output functions) according to the desired implementation.

[0023] The terminal devices 123a-123c may be implemented as dedicated controllers, industrial PCs, control computers, DCS controllers, SCADA devices, programmable logic controllers (PLCs), terminal devices such as smartphones or UEs with wireless communication interfaces, intelligent electronic devices (IEDs), merging units (MUs), and / or protection relays. For convenience, the terminal devices 123a-123c may be described as single devices, but may be comprised of controllers, sensors, actuators, and communication devices such as UEs used in mobile communications.

[0024] In some aspects, the wireless base stations 124a and / or 124b may be base stations for wireless communication and may communicate with the terminal devices 123a-123c and / or the I / O control devices 125a-125d as mobile units without wired connections via a wireless network. Furthermore, the wireless base stations 124a and / or 124b may communicate with the resource control device 120 and the control device 127 via the control network 122. Examples of the wireless base stations 124a and / or 124b may include base stations for 4G networks, 5G networks, 6G networks, industrial wireless communications such as WirelessHART and ISA100, and the like. The wireless base stations 124a and / or 124b may include multiple devices, such as a baseband unit (BU), a remote radio head (RRH), a central unit (CU), and / or a distributed unit (DU), depending on configuration options related to functional division (FS).

[0025] In some embodiments, the I / O control devices 125a-125d may be devices such as sensors, actuators, or both. The I / O control devices 125a-125d may be connected to and controlled by the terminal devices 123a-123c. Examples of the I / O control devices 125a-125d, depending on the desired implementation, may be industrial robots such as mobile robots, humanoid robots, robot arms, automated guided vehicles (AGVs), and autonomous mobile robots (AMRs), remote-controlled mobile bodies, chip mounters, machine tool tables, processing equipment, machine tools, semiconductor manufacturing equipment, motors, inverters for manufacturing devices, power equipment such as circuit breakers or disconnectors, various sensors (encoders, temperature sensors, pressure sensors, etc.), and so on.

[0026] The wireless network, in some aspects, is a wireless network for connecting wireless base stations 124a and / or 124b and terminal devices 123b-123c. Depending on the desired implementation, the physical implementation of the wireless network may be in the form of mobile wireless communication such as 3G, 4G (e.g., LTE), 5G (including new wireless systems), and 6G, or wireless communication such as Wi-Fi™, ZigBee™, Bluetooth™, and / or IEEE 802.15. While FIG. 1 illustrates a wireless network used to connect control network 122 to terminal devices 123b-123c, in some aspects, terminal devices 123a-123c may be connected to control network 122 by wired communication, such as Ethernet, USB, and / or other wired communication.

[0027] In some embodiments, the control device 127 may communicate with one or more of the terminal devices 123a-123c and the I / O control devices 125a-125d via the control network 122 (e.g., a network including the network relay device 121) and a wireless network. The control device 127 may send and / or receive communication packets to and from the terminal devices 123a-123c and / or the I / O control devices to (1) send control command values ​​for controlling the terminal devices 123a-123c and / or the I / O control devices 125a-125d, (2) obtain measurements and / or sensor information from components such as sensors or actuators of the I / O control devices 125a-125d, or (3) obtain and / or configure various settings of the terminal devices 123a-123c and / or the I / O control devices 125a-125d. Depending on the desired implementation, the physical implementation of the control device 127 may include a PC, a dedicated controller, an industrial PC, a control computer, a DCS controller, a multi-access edge computing (MEC) device, a computer cloud, a server computer, a supervisory control and data acquisition (SCADA) server, a programmable logic controller (PLC), an intelligent electronic device (IED), a power system protection relay, etc. Alternatively, a configuration in which such devices are located in a destination network (DN) defined by the 5G standard is also illustrated. These devices may exchange sampling data, control commands, and / or status signals in a control system. Data within the control system may be aggregated into packets. In some aspects, the control device 127 may be implemented as a virtualized control device and / or a software container that provides control device functionality. In some aspects, an application or operating system (OS) may be managed as a container through virtualization.

[0028] Although the communication path between the control device 127 and the network relay device 121c is illustrated as a single connection, any number of paths may be used to achieve redundancy. Each communication link may connect any of the resource control device 120, the network relay devices 121a-121d, the wireless base stations 124a and / or 124b, the terminal devices 123a-123c, and the control device 127, and constitute the control network 122. In this specification, the term "communication link" may refer to a communication link between adjacent communication devices (e.g., the network relay devices 121a-121d, the wireless base stations 124a and 124b, the terminal devices 123a-123c, and / or the control device 127). The term "communication path" may also refer to a route formed by a collection of communication links.

[0029] In some aspects, the number and configuration of network relay devices, terminal devices, and / or radio base stations may differ from the number and configuration of network relay devices, terminal devices, and / or radio base stations illustrated in FIG. 1 based on a desired implementation. Similarly, the number of communication paths between resource control device 120 and terminal devices 123a-123c may differ from each other and from the number of communication paths illustrated in FIG. 1. Although resource control device 120 and terminal devices 123a-123c are illustrated as separate devices in FIG. 1, in some aspects, a communication device may assume both roles in one device depending on a desired implementation. In some aspects, there may be one or more resource control devices (e.g., control device 120) in a control system.

[0030] 1 may include DCSs for factory automation and power plant automation, remote control systems using wireless communications, power grids, industrial equipment, semiconductor manufacturing equipment, in-vehicle systems, control systems for construction machinery and railroad vehicles, railway ground signal systems, aviation control systems, etc. In some embodiments, an Internet of Things (IoT) system may be used to improve the performance of the control system by applying artificial intelligence to information collected by resource control devices 120 and / or control devices 127 via control network 122, and / or cloud or computer applications may also be applicable. These examples may include local 5G, private 5G, and public 5G systems.

[0031] 2 illustrates a hardware structure of a device 200 according to an implementation example. The device 200 may be a resource control device and / or a control device. In some embodiments, the hardware structure may include a central processing unit (CPU) 201 that may be configured to transfer programs from a non-volatile storage medium 205 to a memory 204 and execute the programs. Examples of executable programs include an OS and application programs running on the OS. The programs running on the CPU 201 manipulate the settings of a communication control integrated circuit (IC) 202 and obtain status information.

[0032] The communications control IC 202 may receive transmission requests and transmission data from software running on the CPU 201 and transmit the data to the control network 122 using a physical layer (PHY) 203. The communications control IC 202 may also transfer data received from the control network 122 to the CPU 201, memory 204, and / or non-volatile storage medium 205 via a bus 206. One implementation of the communications control IC 202 is an IC such as a field programmable gate array (FPGA), a complex programmable logic device (CPLD), an application specific integrated circuit (ASIC), or a gate array, depending on the desired implementation. In some embodiments, the communications control IC 202 may be integrated with or implemented by the CPU 201. The communications control IC 202 may be an IEEE 802.3 communications device including a media access control (MAC) layer and / or a PHY layer. The communications control IC 202 may be implemented as an IEEE 802.3 MAC chip, a PHY chip, and / or a combined MAC / PHY chip.

[0033] In some embodiments, the communication control IC 202 may be included in the CPU 201 or a chipset that controls information paths within the computer. Furthermore, although only a single communication control IC 202 is shown in the structure of FIG. 2 , multiple communication control ICs may be present. In some embodiments, the PHY 203 may be a transceiver IC that implements communication functionality with the control network 122. The communication functionality provided by the PHY 203 may be based on the IEEE 802.3 communication standard. In the configuration of FIG. 2 , the PHY 203 and the communication control IC 202 are connected, and the IEEE 802.3 MAC processing is included in the communication control IC 202. However, in some embodiments, a MAC function IC may be located between the communication control IC 202 and the PHY 203, or the PHY 203 capable of MAC functionality may be incorporated into the communication control IC 202.

[0034] While FIG. 2 shows a single PHY 203, other embodiments may include multiple PHY components. In some embodiments, memory 204 may be a temporary storage area for CPU 201 to operate. For example, memory 204 may store an OS or application programs transferred from non-volatile storage medium 205. In some embodiments, non-volatile storage medium 205 may be information storage used to store programs for operating CPU 201, such as an OS, applications, and device drivers, among other data, and may also store the results of program execution. Examples of non-volatile storage medium 205 may include a hard disk drive (HDD), a solid-state drive (SSD), and / or flash memory. Furthermore, non-volatile storage medium 205 may be implemented as an external removable storage medium, such as a floppy disk (FD), CD, DVD, Blu-ray™, USB memory, or Compact Flash.

[0035] The bus 206 connects the CPU 201, the communication control IC 202, the memory 204, and the non-volatile storage medium 205. Examples of the bus 206 may include a peripheral component interconnect (PCI) bus, a PCI express bus, an on-chip bus, etc. The device 200 may include multiple communication interfaces for the communication control IC 202 and the PHY 203 for high reliability based on redundancy of the communication path.

[0036] FIG. 3 illustrates a hardware structure of a device 300 according to an example implementation. The device 300 may be a terminal device (e.g., one of the terminal devices 123a-123c) and / or one of the I / O control devices (e.g., one of the I / O control devices 125a-125d). The I / O unit 307 may be an input / output interface for controlling an actuator associated with the device 300 (e.g., the I / O control device) or for acquiring a detected value from a sensor associated with the device 300. Examples of the I / O unit 307 may include a digital I / O interface, an analog I / O IC, and a driver IC. Although a signal line from the I / O unit 307 is shown in singular, multiple lines may exist depending on the device 300 (e.g., the terminal device or the I / O control device). Also, multiple I / O units 307 may be included in the device 300 (e.g., the terminal device or the I / O control device).

[0037] The bus 306 connects the CPU 301, the communication control IC 302, the memory 304, and the non-volatile storage medium 305. Examples of the bus 306 may include a peripheral component interconnect (PCI) bus, a PCI express bus, an on-chip bus, etc. The device 300 may include multiple communication interfaces for the communication control IC 302 and the PHY 303 for high reliability based on redundancy of the communication path.

[0038] The I / O unit 307, in some aspects, may be configured to input information from connected and / or associated sensors, output commands to connected and / or associated actuators, or combine both input and output functions, according to a desired implementation. The baseband processing units 308a and / or 308b may be configured to convert analog and digital signals using the radio frequency (RF) processing units 309a and / or 309b and perform signal processing on the digital data. Examples of signal processing include fast Fourier transform (FFT) for orthogonal frequency division multiplexing (OFDM), matrix calculations for MIMO signals, and error correction.

[0039] Depending on the desired implementation, the physical implementation of the baseband processing units 308a and / or 308b may include ICs such as dedicated ICs, FPGAs, CPLDs, ASICs, and gate arrays. Alternatively, they may be integrated with the CPU 201 and / or RF processing units 309a and / or 309b. In some embodiments, the RF processing units 309a and / or 309b may perform high-frequency processing such as modulation of analog RF signals, amplification of high-frequency signals, high-frequency switching, and / or filtering. For example, the RF processing units 309a and / or 309b may be implemented as dedicated ICs. The antennas 310a and / or 310b may be antenna devices conforming to wireless network standards. Some embodiments may include more than two of each of the baseband processing units 308, RF processing units 309a and / or 309b, and antennas 310a and / or 310b illustrated in the configuration of FIG. 3 .

[0040] In some aspects, multiple terminal devices (e.g., terminal devices 123a-123c) having baseband processing unit 308, RF processing units 309a and / or 309b, and antennas 310a and / or 310b are connected and coordinated via inter-chip communication means (not shown). In some aspects, such multiple terminal devices (e.g., terminal devices 123a-123c) may be connected to integrated hardware (e.g., the structures of FIGS. 2 and 3) via inter-chip communication means, and the integrated hardware may then control them. In some aspects, the hardware components shown in FIG. 3 (e.g., I / O unit 307, baseband processing unit 308, RF processing units 309a and / or 309b, and / or antennas 310a and / or 310b) may be implemented as separate components connected to other components of the terminal devices (e.g., terminal devices 123a-123c) by a network via PHY 303 or by inter-chip communication means (not shown).

[0041] In some aspects, if wireless communication is not required, any of the baseband processing unit 308, the RF processing units 309a and / or 309b, and the antennas 310a and / or 310b may be omitted. A terminal device (e.g., terminal devices 123a to 123c) may have communication means for connecting with an I / O control device (e.g., I / O control devices 125a to 125d). In some aspects, a terminal device (e.g., terminal devices 123a to 123c) has multiple sets of communication control ICs 202 and PHYs 203, one of which is for connecting with the control network 122 and the other is for connecting with an I / O control device (e.g., one of the I / O control devices 125a to 125d). Alternatively, the I / O unit 307 may be connection means with an I / O control device (e.g., one of the I / O control devices 125a to 125d).

[0042] FIG. 4 illustrates a functional structure of a device 400 according to an example implementation. The device 400 may be a terminal device (e.g., one of the terminal devices 123a-123c) and / or one of the I / O control devices (e.g., one of the I / O control devices 125a-125d). The communication unit 430 may connect to the control network 122 or a wireless network and communicate according to a communication protocol of the control network 122 or the wireless network. In some aspects, the communication unit 430 connects to the PHY 303 to transmit and / or receive data and processes data or packets received from the bus 306. Processing in the communication unit 430 may include generating frames from data and / or calculating and adding abnormality diagnostic data such as a CRC. Upon receiving a packet, the communication unit 430 may be configured to forward the received packet to any other function. Although FIG. 4 illustrates a single communication unit 430, a terminal device (e.g., one of the terminal devices 123a-123c) may include multiple communication units 430. The communication unit 430 may be implemented by one or more of an application running on the CPU 201 , the communication control IC 202 and the PHY 203 .

[0043] The time synchronization unit 431 synchronizes the time of a terminal device (e.g., one of the terminal devices 123a to 123c) and / or the I / O control devices 125a to 125d with a reference time. The time synchronization means may include time synchronization means using wireless communication such as IEEE802.1AS, IEEE1588, NTP, GPS, or 5G. The time synchronization unit 431 may be realized by one or more of the CPU 201, an application running on the CPU 201, the communication control IC 202, and the PHY 203.

[0044] In some aspects, the processing unit 432 may perform processing such as control operations and information processing. In one example of a control operation, the processing unit 432 in a terminal device (e.g., one of the terminal devices 123a-123c) or an I / O control device (e.g., one of the I / O control devices 125a-125d) outputs command values ​​to actuators using the I / O unit 307 according to command values ​​received from the resource control device 120 and / or the control device 127, or according to a control algorithm by the terminal device (e.g., one of the terminal devices 123a-123c) and / or the I / O control device 125a-125d itself. To manipulate these command values, the processing unit 432 may use sensor signals received from the I / O unit 307. In some embodiments, a processing unit 432 within a terminal device (e.g., one of terminal devices 123a-123c) and / or I / O control device 125a-125d may obtain state information or sensor information via I / O unit 307, apply processing to the information, and transmit the processed information to resource control device 120 and / or control device 127. For example, the processing may include filtering operations, statistical processing, and / or statistical calculations of a set of sensor information for a particular time in the past.

[0045] Other example processes may include control processes based on control theory, such as feedback control theory and / or feedforward control theory, or control operations based on artificial intelligence (AI) or machine learning. For example, integral control or a state machine-based control method that changes state depending on past information and inputs may be used. Other example processes may include asset management, such as fault detection and risk state estimation, device management, software version management (e.g., software version management and updates), predictive and preventive maintenance, condition-based maintenance (CBM), and statistical calculations using AI or machine learning for remaining life prediction of an I / O control device (e.g., any of I / O control devices 125a-125d).

[0046] The processing unit 432 may also be configured to calculate command values ​​necessary to control an I / O control device (e.g., one of the I / O control devices 125a-125d). For example, if the I / O control device (e.g., one of the I / O control devices 125a-125d) is a moving body, the I / O control device (e.g., one of the I / O control devices 125a-125d) may calculate a movement direction and target position, speed control, acceleration control, stopping, deceleration, etc. For example, if the I / O control device (e.g., one of the I / O control devices 125a-125d) is an industrial robot arm, the I / O control device (e.g., one of the I / O control devices 125a-125d) may calculate control commands for the motors of each joint, the end position of the robot arm, and trajectory control of the robot arm. The I / O control device (e.g., one of I / O control devices 125a-125d) may be a wide variety of devices, from complex devices such as industrial robots to simple sensors or actuators, and the processing unit may be configured to perform control according to the type of I / O control device (e.g., one of I / O control devices 125a-125d). The processing unit 432 may be implemented by the CPU 201 or by one or more applications running on the CPU 201.

[0047] The input unit 433 may be a sensor device in some aspects. For example, the input unit 433 acquires physical information of the environment in which the terminal device (e.g., one of the terminal devices 123a-123c) and / or the I / O control device (e.g., one of the I / O control devices 125a-125d) operates, or internal information of the terminal device (e.g., one of the terminal devices 123a-123c) and / or the I / O control device (e.g., one of the I / O control devices 125a-125d), and then the input unit 433 provides the information to each functional unit. The input unit 433 may be implemented as a sensor by the I / O unit 307 or the I / O control device (e.g., one of the I / O control devices 125a-125d).

[0048] In some embodiments, the output unit 434 may be an actuator device. For example, the output unit 434 outputs based on a control command processed by the processing unit 432 to act on the physical environment in which the terminal device (e.g., one of the terminal devices 123a-123c) and / or the I / O control device (e.g., one of the I / O control devices 125a-125d) operates. The output unit 434 may be implemented as an actuator by the I / O unit 307 or the I / O control device (e.g., one of the I / O control devices 125a-125d).

[0049] In some embodiments, the log registration unit 435 may record events having synchronized times provided by the time synchronization unit 431 as logs in the log storage 436. The registered events may be operations performed by the processing unit 432, sensing by the input unit 433, or output to an actuator by the output unit 434. Alternatively, the events may be control commands received from a control device via the control network 122. The events may be represented in a format that can identify the events. For example, the format may be a character string and / or a numeric code representing the event. When a numeric code is used, the correspondence between the numeric code and the event is defined separately. The synchronized time used in the recording may, for example, follow a time format defined by a time synchronization protocol or the like. For example, IEEE 1588 defines a format of 10 bytes, consisting of 6 bytes per second and 4 bytes per nanosecond. The synchronized time used in the recording may be absolute time or not, as long as it is the synchronized time in the control system of FIG. 1. The log registration unit 435 may be implemented by the CPU 201 or by one or more applications running on the CPU 201 .

[0050] In some embodiments, the log storage 436 may store logs, each log combining a synchronized time with the events registered by the log registration unit 435. The log storage 436 may be implemented as a database, such as a relational database. The log storage 436 may delete data, starting with the oldest data, when the amount of data is full or exceeds a threshold. The log storage 436 may also delete information other than information that meets a predetermined condition, such as the importance of the information. The log storage 436 may output data to external storage. The log storage 436 may be implemented by one or more of the CPU 201, an application running on the CPU 201, the memory 204, and the non-volatile storage medium 205.

[0051] In some embodiments, the log acquisition unit 437 may extract logs from the log storage 436 according to specified conditions and provide them to functional units that require the information. The number of logs is not limited to one. The log acquisition unit 437 may process the acquired log information. Processing may include correcting, supplementing, and / or excluding data, preprocessing data, and / or adding or deleting specific times to or from the synchronized time of the log information. The synchronized time recorded in the logs may be changed to maintain the time difference between the logs. The log acquisition unit 437 may be implemented by one or more of the CPU 201, an application running on the CPU 201, the memory 204, and the non-volatile storage medium 205.

[0052] In some embodiments, one or more of the log registration unit 435, the log storage 436, and the log acquisition unit 437 may be implemented as standalone devices connected to the control network 122. For example, the log storage 436 may be implemented as an independent NAS (Network Attached Storage), and the log storage 436 may record log information generated in a terminal device and transmitted from a terminal device (e.g., one of the terminal devices 123a-123c) or an I / O control device (e.g., one of the I / O control devices 125a-125d) via the control network 122. In some embodiments, the log registration unit 435 and the log storage 436 may be implemented as multiple standalone devices connected to the control network 122. The log registration unit 435 may receive an event from the processing unit 432, the input unit 433, or the output unit 434, and register the event in the log storage 436. During registration, the synchronized time included in the log information may be the synchronized time provided by the time synchronization unit 431 in the terminal device (e.g., one of the terminal devices 123a to 123c) or the I / O control device (e.g., one of the I / O control devices 125a to 125d), or an independent device having the time synchronization unit 431 may provide the synchronized time.

[0053] FIG. 5 illustrates a functional structure of a device 500 according to an exemplary embodiment. The device 500 may be the resource control device 120. In some aspects, the control system state determination unit 540 may determine the state of the control system illustrated in FIG. 1. The determined state is not limited to the overall state of the system, but also includes the state of each terminal device (e.g., one of the terminal devices 123a-123c) or I / O control device (e.g., one of the I / O control devices 125a-125d). In some aspects, the state refers to the state of a terminal device, an I / O control device, and / or a control system configured by the terminal device and / or the I / O control device, which is related to a control method of log registration in the terminal device and / or the I / O control device.

[0054] Although the control system state determination unit 540 is shown in the functional structure of the device 500, the control system state determination unit 540 may be implemented within a terminal device or an I / O control device (e.g., as shown in FIG. 4). Alternatively, the control system state determination unit 540 may be distributed among the terminal device, the I / O control device, and the device 500 (e.g., the resource control device 120), with the state being determined by cooperation of the distributed control system state determination units 540. The control system state determination unit 540 may be implemented by the CPU 201 or one or more applications running on the CPU.

[0055] 1 and one or more terminal devices and I / O control devices. In some embodiments, the control system information storage 541 may store information about the control system and one or more terminal devices and I / O control devices shown in FIG. 1 . Such information may, in some embodiments, include information about a plurality of pieces of information necessary to communicate with the set of terminal devices and I / O control devices (e.g., IP addresses and MAC addresses). In some embodiments, the information stored in the control system information storage 541 may include information about the mobility of each I / O control device in the set of I / O control devices (e.g., whether each I / O control device is mobile), information about sensors and actuators associated with each terminal device or I / O control device, information about the computing capabilities of each terminal device or I / O control device, information about communication capabilities (e.g., which protocol to use, latency, or bandwidth), the operating state (e.g., on / off state, standby mode, etc.) of each terminal device or I / O control device (or the communication unit 430 of the terminal device or I / O control device), an operating schedule, etc. In some aspects, the control system information storage 541 may additionally store information regarding one or more of a set of resource control devices (e.g., resource control device 120), a set of network relay devices (network relay devices 121a-121d), the control network 122, and / or a set of wireless base stations (e.g., wireless base stations 124a and / or 124b).

[0056] The stored information may include computer resource performance, communication performance, network topology, etc. In some aspects, the information may include identifiers (e.g., IP addresses) required for communication and parameters related to communication settings of one or more components of the control network, the installation location and coverage area of ​​a wireless base station (e.g., one of wireless base stations 124a and / or 124b). In some aspects, the information may include physical information about the environment in which the terminal device and / or I / O control device operates (e.g., aisle width and length, layout information within a factory, etc.). The control system information storage 541 may be configured as a database, such as a relational database. The control system information storage 541 may be implemented by one or more of the CPU 201, an application (including a software container) running on the CPU 201, the memory 204, and / or the non-volatile storage medium 205.

[0057] In some aspects, the resource control unit 542 may control resources of the control system, where the resources are related to logs stored in the terminal device and the I / O control device. The resource control unit 542 may include a communication resource control unit 543, a storage resource control unit 544, a computer resource control unit 545, and an I / O resource control unit 546. In some aspects, the resource control unit 542 may not include all of the communication resource control unit 543, the storage resource control unit 544, the computer resource control unit 545, and the I / O resource control unit 546.

[0058] The communication resource control unit 543 may, in some aspects, control communication resources related to log storage in the terminal device and the I / O control device. The resources controlled by the communication resource control unit 543 include communications for a time synchronization protocol executed by the terminal device and / or the I / O control device, and communications for setting and / or obtaining configurations related to one or more of the log registration unit 435, the log storage 436, and the log acquisition unit 437. In some aspects, the resources controlled in some aspects by the communication resource control unit 543 include communications for notifying one or more of the log registration unit 435, the log storage 436, and the log acquisition unit 437 of log information when any of the log registration unit 435, the log storage 436, and the log acquisition unit 437 is configured as a standalone device.

[0059] For these communications, the communication resource control unit 543 may allocate resources from one or more of a network relay device, the control network 122, a wireless base station, a resource control device, an end device, an I / O control device, and a control device (e.g., as illustrated in the system of FIG. 1 ). In some aspects, the resources may include a communication path (or route) from the I / O control device to the resource control device, and communication resources within the control network 122 and / or the wireless network. The communication resources within the control network 122 and / or the wireless network may be one or more of a communication band, a frequency, a communication priority, an IEEE 802.1Qbv (TSN) time slot, and a communication path within the network. The communication path is configured by routing packets in the network relay device, the control network, the end device, and / or the wireless base station. Examples of route configurations include setting routing rules in a routing table in a network router, configuring packet forwarding rules in an OpenFlow controller, and specifying a destination port (such as a VLAN) in a network switch.

[0060] The storage resource control unit 544 may, in some aspects, control storage resources within the terminal device and / or the I / O control device. The resources may be allocated capacity of a storage device such as an SSD, HDD, RAM memory, etc., a sampling period of the stored events, and / or sampling precision (i.e., sampling resolution, or the number of bits for formatting the events).

[0061] The computer resource control unit 545 may, in some aspects, control computer resources in a resource control device, a network relay device, a terminal device, a wireless base station, an I / O control device, and / or a control device. Such computer resources may be the number of processor cores allocated for a particular task in a multiprocessor, the scheduling priority and / or the number of allocated processor cores for an application performing communication, the allocated time of a processor core, the priority and / or the allocated communication bandwidth on the bus 206.

[0062] The I / O resource control unit 546 may, in some aspects, control I / O resources within the terminal device and / or the I / O control device. Such I / O resources may be set points and / or limits for position, direction of movement, velocity, and / or acceleration. The resource control unit 542, communication resource control unit 543, storage resource control unit 544, computer resource control unit 545, and I / O resource control unit 546 may be implemented in the CPU 201 or by one or more applications running on the CPU 201.

[0063] FIG. 6 is a flow diagram 600 illustrating a method according to some aspects of the present disclosure. In some aspects, the method is performed by a resource controller of a control system that includes, in some aspects, device 500 (e.g., resource controller 120), or computing device 2005 of FIG. 20, or a distributed computing device (e.g., a distributed set of computing devices) having similar components. At 601, the resource controller (e.g., control system state determination unit 540) may determine and / or detect that a trigger event (e.g., an event that triggers resource control) has occurred. In some aspects, the trigger event determined and / or detected at 601 may be based on a change in the type of at least one device, the surrounding environment of the at least one device, or a relationship between the at least one device and at least one other device of the plurality of devices. For example, the trigger event may be related to the presence of a human (e.g., the introduction of a human into the environment), a change in distance to at least one other device or human, a change in relative velocity between the at least one device and at least one other device or human, or a change in relative acceleration between the at least one device and at least one other device or human. To determine and / or detect a trigger event, the control system may monitor (eg, execute a monitoring loop) for the trigger event.

[0064] Once a trigger event is determined and / or detected in 601, the resource control device may determine 602 the state of the control system (e.g., determine and / or identify the state of multiple synchronized devices associated with the control system at a particular time). In some aspects, the state of the control system determined in 602 may include the state of a set of terminal devices and the state of the communication network providing communication between the control system and the terminal devices (e.g., the state of a set of network relay devices, wireless base stations, I / O control devices, etc.). The state of the terminal device may, in some aspects, include an identification of the type of device (e.g., a localized robotic agent, autonomous object, or other controllable device), a variable characteristic of the terminal device such as speed or acceleration, an indication of whether the terminal device is in a dynamic state or a steady state, information about the environment of the terminal device, the location of the terminal device, information about other objects near the terminal device (e.g., object type, distance, relative speed, relative acceleration, and / or other relative information). The state of the communication network may include an identification of available paths and / or routes through the communication network to different terminal devices and resources (e.g., used resources and available resources) associated with each path and / or route through the communication network.

[0065] At 603, the resource control device determines whether the state determined at 602 indicates a resource control operation. The state determined at 602 may include one or more state variables that reflect a change from a previously determined state (e.g., a set of state variables determined based on a previously determined and / or detected trigger event). The change in the state variables may, in some aspects, be used to determine at 603 whether the state determined at 602 indicates for a resource control operation. For example, a state variable related to one of the distance from the terminal device to a human, or the position or velocity of the autonomous device, indicated and / or included in the state determined at 602, may have changed from the previously determined state.

[0066] If, based on the changes to the state variables, the resource controller determines at 603 that the state determined at 602 indicates a resource control operation, the resource controller proceeds to execute the resource control operation at 604. For example, if a state variable related to the distance separating the terminal device and a person indicates that a distance threshold has been exceeded (e.g., from an above-threshold state to a below-threshold state, or vice versa), the resource controller may determine at 603 that the state determined at 602 indicates a resource control operation (e.g., suggests that a resource control operation should be performed). Additionally, or alternatively, if a state variable related to the position or velocity of the autonomous device indicates that the autonomous device has entered or exited an area associated with more frequent logging, or indicates that the velocity (e.g., either absolute velocity or velocity relative to other objects or autonomous devices) has exceeded a threshold velocity for more frequent logging, the resource controller may determine at 603 that the state determined at 602 suggests a resource control operation. The resource control operations performed by the control system at 604 include allocating a set of time synchronization resources for transmitting time synchronization information from at least one device of the plurality of devices based on the state determined at 602 (e.g., the identified state of each of the plurality of synchronized devices). The set of resource control operations performed by the control system at 604 may further include transmitting a control message to at least one device of the plurality of devices indicating the allocated set of time synchronization resources determined for transmitting time synchronization information from the at least one device.

[0067] In some aspects, the set of time synchronization resources may include one or more of time resources within a set of slots, frequency resources in a frequency range, bandwidth resources, or communication route resources. The set of time resources may, in some aspects, be associated with one or more of a wired communication network or a wireless communication network. As described above, the allocation of the set of time synchronization resources may be based on one or more of a type of at least one device, a surrounding environment of the at least one device, or a change in a relationship between at least one device of the plurality of devices and at least one other device as indicated by the conditions determined in 602. In some aspects, the allocation may include updating a previous allocation of time synchronization resources.

[0068] In some embodiments, the set of time-synchronized resources includes a set of logging parameters for recording sensor data associated with the at least one device, and the at least one device is configured to transmit information recorded by the at least one device to the resource control device based on the logging parameters. In some embodiments, the set of logging parameters relates to one or more of: a duration of each data recording event of the plurality of data recording events; a frequency of the data recording events in the plurality of data recording events; a resolution of sensor data collected during each data recording event of the plurality of data recording events; or a data storage allocation for sensor data collected during one or more data recording events in the plurality of data recording events. In some embodiments, the resource control operation at 604 may be associated with multiple devices, and a first set of logging parameters for a first device may indicate that the first device begins data recording for the data recording event at the same time that at least one other device of the plurality of devices, e.g., a second device, begins data recording for the data recording event. The second set of logging parameters for the second device, in some embodiments, indicates a frequency of data recording events that is equal to or an integer multiple of the frequency of data recording events indicated in the first set of logging parameters.

[0069] If, based on the changes to the state variables, the resource control device determines at 603 that the condition determined at 602 does not suggest a resource control operation, or if a resource control operation has been executed by the resource source control device, the method may end. For example, if a state variable related to the distance separating the terminal device from the human indicates that a distance threshold has not been exceeded (e.g., the distance remains within the same range as a pre-determined and / or identified distance), the control system may determine at 603 that the condition determined at 602 suggests a resource control operation (e.g., indicates that a resource control operation is not triggered by the current condition). Additionally, or alternatively, if a state variable related to the position or velocity of the autonomous device indicates that the autonomous device is in the same area associated with the current logging configuration, or if the velocity (e.g., either absolute velocity or velocity relative to other objects or autonomous devices) does not exceed a threshold velocity associated with more frequent and / or less frequent logging, the control system may determine at 603 that the condition determined at 602 does not suggest a resource control operation. The method then ends. In some aspects, following completion of the method illustrated in flow diagram 600, there may be monitoring for a trigger event that restarts the method.

[0070] In some aspects, parameters related to resources controlled by the resource controller may control the transmission frequency of packets in a time synchronization protocol, the sampling frequency of events, the sampling resolution of events, and / or storage capacity allocation for events, at 604. Resources controlled by the resource controller, in some aspects, may include a set of time slots and / or frequency bands in wireless communication, time slots for time-division communication such as time-sensitive networking (TSN), and / or route selection of synchronization packets in terms of synchronization accuracy and communication latency.

[0071] 7 is a call flow diagram 700 illustrating an example of a latency calculation process associated with resource control according to some aspects of the present disclosure. Call flow diagram 700 corresponds in some aspects to the components of resource control 604 of FIG. 6 and in some aspects conforms to a High Precision Clock Synchronization Protocol for Networked Measurement and Control Systems standard (e.g., as defined in IEEE 1588). In some aspects, resource control may be based on designating a first device (e.g., a terminal device) as a master 710 and a second device (e.g., another terminal device) as a slave 720.

[0072] The master 710 may begin the process by sending a first (e.g., Sync) message to the slave and recording the send time t1 of the first message, at 701. The slave 720 may receive the first message and record the receive time t2, at 702. The slave 720 may also receive the send time t1 from the master 710. For example, the master 710 may send an indication of the send time t1 in the first message or in a subsequent message related to the first message (e.g., a Follow_Up message).

[0073] The slave may then send a second (e.g., Delay_Req) ​​message at 703 and record the sending time t3. The master 710 may receive the second message at 704 and record the receiving time t4 of the second message. Based on the second message, the master 710 may send a third (e.g., Delay_Resp) message at 705 indicating the receiving time t4 of the second message. The slave 720 may then calculate the communication latency between the master 710 and the slave 720 at 706 using the times t1, t2, t3, and t4. For example, the average latency is calculated by averaging the difference between t1 and t2 and the difference between t3 and t4 (e.g., t d= ((t4-t3)+(t2-t1)) / 2). In some embodiments, the slave 720 may also calculate a time difference (e.g., timing offset) between a time maintained at the slave 720 (e.g., local time) and a time maintained at the master 710 (e.g., system time). For example, the time difference t diff is t if the average latency has already been calculated. diff =t1+t d -t2, or t regardless of whether the average latency has been calculated. diff = ((t1+t4)-(t2+t3)) / 2. The system time is calculated by adding a time offset (t diff ) in 708. Then, the slave 720 calculates the calculated t diff 7. For example, referring to FIG. 9, a slave may perform the process shown in FIG. 9 based on a calculated time difference associated with a particular log and / or record. The calculated time difference may be periodically re-evaluated and associated with a set of logs and / or records to perform the process shown in FIG. 9, since local and system times may drift over time. While FIG. 7 illustrates a particular method for calculating latency time and / or time difference between the master 710 and the slave 720, this example is not limiting, and other methods, including the peer delay method defined in IEEE 1588, may be used in other embodiments or according to other standards.

[0074] FIG. 8 is a flow diagram 800 illustrating a method of log registration according to some aspects of the present disclosure. In some aspects, the method may be performed by a terminal device (e.g., terminal device 123) and / or an I / O control device (e.g., I / O control device 125). A log registration unit (e.g., log registration unit 435 of device 400) may determine 801 to acquire event information for logging. The log registration unit may determine to acquire the event information based on an event logging timing configuration. In some aspects, the events may be acquired periodically and / or the acquisition may be triggered by a specified event. For example, the periodic acquisition of the event information may include periodic acquisition from a sensor associated with the log registration unit, and the event-triggered acquisition of the event information may include activation of an interrupt signal of a processing unit, a CPU, or an application running on the CPU, triggered by a change in a sensor signal.

[0075] In some embodiments, the log registration unit may acquire event information for logging an event and / or determine the timing for acquiring information for logging an event based on a request from an external device in 801. In addition, an external device may request the log registration unit 435 to acquire an event when an incident occurs in the control system, when a maintenance inspection is performed to check a specific operation, and / or when a log function is confirmed, etc.

[0076] Based on the determination at 801 to acquire event information, the log registration unit 435 may acquire event information for logging at 802. For example, acquiring event information may include acquiring sensor values. In some aspects, the log registration unit (e.g., the log registration unit 435) acquires synchronized time from a time synchronization unit (e.g., the time synchronization unit 431) at 803. The synchronized time (e.g., system time) may be acquired at 803 based on the method shown in call flow diagram 700 of FIG. 7 (e.g., by looking up a stored value or by calculating the current value of the synchronized time). In some aspects, acquiring the synchronized time at 803 may be performed for each event information acquisition performed by the log registration unit in a series of processes employing the method of flow diagram 800.

[0077] Finally, at 804, the log registration unit 435 registers the acquired event information and the synchronized time as a log in a log storage (e.g., the log storage 436 in FIG. 4 ). FIG. 9 is a diagram 900 illustrating a first set 910 of logged events associated with a local time 911 and a second set 920 of logged events associated with a system time 921 (e.g., the synchronized time). For example, a first entry 901 in the set of logged events 910 may be updated based on a time difference 931 between the local time 911 and the system time 921 to generate a first entry 901′ in the second set of logged events 920. The time difference 931 between the local time 911 and the system time 921 may, in some aspects, be a dynamic value. Thus, the local time 911 of a logged event may be updated to the system time 921 with the currently calculated time difference upon acquisition of the event information. In some aspects, the time difference 931 between the local time 911 of the event and the system time 921 may be recorded along with the event information, and the local time 911 may be updated (e.g., by batch processing) to the system time 921 at the current time or a later time.

[0078] In some aspects, synchronization error may be related to the frequency of synchronization packets and / or messages. FIG. 10 is a diagram 1010 and a diagram 1020 illustrating the relationship between (potential) synchronization error as a function of time for receiving synchronization packets and / or synchronization messages at different frequencies. For example, diagram 1010 shows that for a first set of synchronization messages 1012 (including individual synchronization messages 1011) transmitted less frequently from a master (e.g., master 710 of FIG. 7), the maximum (potential) synchronization error 1013 is 50 units, where units are arbitrary units provided to illustrate the relative difference between the maximum (potential) synchronization error associated with different frequencies of synchronization message transmission. Diagram 1020 shows that for a second set of synchronization messages 1022 (including individual synchronization messages 1021) transmitted more frequently (e.g., 2.5 times) from a master (e.g., master 710 of FIG. 7), the maximum (potential) synchronization error 1023 may be 20 units (e.g., a 2.5-fold reduction).

[0079] In the descriptions of FIGS. 1010 and 1020, reference is made to potential synchronization errors to indicate the possibility of various synchronization errors between synchronization events (e.g., transmission of a set of synchronization messages 1012 or 1022). In FIGS. 1010 and 1020, synchronization errors over time between synchronization events are illustrated as having the same slope based on the assumption that they are associated with a slave (e.g., terminal or I / O control) device; other devices may have different rates or nonlinear synchronization error accumulation. Based on the same device assumption and linear synchronization error accumulation, increasing the synchronization message rate by a certain factor (e.g., 2.5 times in the examples of FIGS. 1010 and 1020) reduces the maximum synchronization error by that factor. While the specific implementation of the synchronized time estimation method may also affect the error in some aspects, FIG. 10 conceptually illustrates that, in a particular environment and / or for a particular slave device, more frequent synchronization messages result in more accurate transmissions (e.g., reduced synchronization error) compared to less frequent synchronization messages.

[0080] FIG. 11 illustrates a system 1100 in an exemplary environment including a collaborative robot 1151, an AGV 1157, and a worker 1155 in a factory, in accordance with some embodiments of the present disclosure. The system 1100 illustrated in FIG. 11 is described in the context of a process performed by an industrial robotic arm 1151, controlled by a robot controller 1150, picking up parts 1154 traveling along a belt conveyor 1153 and handing them off to a worker 1155 or an AGV 1157. The worker's 1155 entry into a collaboration area 1158 may be detected by a curtain sensor 1156 in some embodiments. The industrial robotic arm 1151 may, in some embodiments, include multiple joint motors 1152. Control performed by the robot controller 1150 for the joint motors 1152 may, in some embodiments, enable the industrial robotic arm 1151 to perform desired movements or control a tool center point (TCP). The robot controller 1150, in some embodiments, corresponds to a terminal device 123.

[0081] The robot controller 1150 and the curtain sensor 1156, in some embodiments, may connect with a time synchronization master 1159, a resource control device 120, and a control device 127 via the control network 122a, the UE 1160, the wireless network 126, the wireless base station 124, and the control network 122b. The AGV 1157, in some embodiments, communicates with the time synchronization master 1159, the resource control device 120, and the control device 127 via the wireless network 126. The robot controller 1150, the curtain sensor 1156, and the AGV 1157, in some embodiments, may be synchronized based on a time synchronization protocol (e.g., a protocol based on IEEE 1588).

[0082] The robot controller 1150 and curtain sensor 1156 may, in some embodiments, synchronize via wireless and wired communication paths. In such embodiments, the UE 1160, wireless network 126, and / or wireless base station 124 may be configured to support IEEE 1588. The robot controller 1150 performs control of the industrial robot arm 1151, which may, in some embodiments, rely on fine-grained time granularity. In some embodiments, coarse-grained control commands may be communicated by the control device 127.

[0083] The robot controller 1150 and the AGV 1157 may, in some embodiments, be configured to notify other devices of the stored logs. For example, the system may be configured to accurately record control system and entity logs, e.g., with fine time granularity, each time the worker 1155 approaches the industrial robotic arm 1151. In some embodiments, the fine time granularity may be used to ensure and / or verify the safety of the worker 1155. On the other hand, when the worker 1155 is away from the collaboration area 1158 and the AGV 1157 approaches the industrial robotic arm 1151, the system may be configured to record control system and entity logs with less precision, e.g., with coarser time granularity. The coarser granularity used when the AGV 1157 enters the collaboration area 1158 compared to when the worker 1155 enters the collaboration area 1158 may, in some embodiments, be based on a reduced safety concern associated with the AGV 1157 compared to a safety concern associated with the worker 1155. In some embodiments, the system may accurately record logs regarding the AGV 1157 entering the collaboration area 1158, for example, with fine time granularity, based on a judgment and / or determination that a collision between the AGV 1157 and the industrial robotic arm 1151 is considered a dangerous event in accordance with an operational policy.

[0084] For example, whenever the worker 1155 enters the collaboration area 1158 via the curtain sensor 1156, the resource control device 120 is notified by the curtain sensor 1156 that it has detected the worker 1155 entering the collaboration area 1158 (e.g., corresponding to determining and / or detecting that a trigger event has occurred at 601 in FIG. 6 ). Based on the notification, the resource control device 120 (e.g., the device 500 or the control system state determination unit 540 of the device 500) may determine that the worker 1155 has entered the collaboration area 1158 and is near the industrial robot arm 1151 (e.g., corresponding to determining the state of the control system at 602 in FIG. 6 ). Based on the state of the system (e.g., the proximity of the worker 1155 to the robotic arm 1151), the resource controller (e.g., the control system state determination unit 540 of the resource controller 120) may determine one or more resource control operations to be performed (e.g., corresponding to determining that the state determined in 602 indicates a resource control operation to be performed in 603 of FIG. 6). For example, the resource controller may determine to update a logging rate based on the state of the system so that behavior of the industrial robotic arm 1151 is logged and / or recorded with fine time granularity to ensure the safety of the worker 1155.

[0085] Based on the determination, the resource control device 120 may then control resources related to logging in the robot controller 1150 (e.g., corresponding to executing a resource control operation in 604 of FIG. 6 ). For example, the resource control device 120 may increase the transmission frequency of Sync messages and / or Follow_Up messages sent from the time synchronization master 1159 to the robot controller 1150. Increasing the transmission frequency improves the synchronization accuracy of the robot controller 1150 in some embodiments. The resource control device 120 may further decrease the sampling interval (e.g., increase the sampling frequency) of the logging in the log registration unit of the robot controller 1150 (e.g., corresponding to the log registration unit 435 of the device 400). The updated synchronization message frequency and sampling frequency may be notified to the time synchronization master 1159 and the robot controller 1150, respectively, to configure different components to record the operation of the robot controller 1150 at fine time granularity.

[0086] The series of sensing, logging, and / or recording events related to the resource control described above may include a first sensing event from the curtain sensor 1156 related to the worker 1155 crossing a boundary into the collaboration area 1158, a logging and / or recording event related to a control command from the robot controller 1150 to the joint motors 1152, and an associated sensing event from the joint motors 1152. For example, at a first time, it may be detected that the worker 1155 or the AGV 1157 passes through the curtain sensor 1156 (e.g., a decision is made to acquire event information at 801 in FIG. 8 ). Based on the detection, a log registration unit of the curtain sensor 1156 may identify an event upon detection of the curtain sensor 1156 (e.g., corresponding to acquiring event information at 802 in FIG. 8 ). To report an event, the log registration unit of the curtain sensor 1156 obtains a synchronized time (e.g., as described in connection with 803 of FIG. 8 ) and registers the event in the log storage 436 of the curtain sensor 1156 (e.g., as described in connection with 804 of FIG. 8 ). In some aspects, the log registration unit and the log storage of the curtain sensor 1156 are implemented by the terminal device 123 of the curtain sensor 1156.

[0087] In some embodiments, control commands to joint motors 1152 by robot controller 1150 may be calculated by a processing unit within robot controller 1150 (e.g., corresponding to processing unit 432 in FIG. 4 ). The commands and values ​​associated with the commands may be obtained by a log registration unit (e.g., corresponding to obtaining event information in 802 in FIG. 8 performed for the robot controller), which may then output the commands and values ​​associated with the commands to an output unit (e.g., corresponding to output unit 434 in FIG. 4 ). The log registration unit associated with the robot controller may then obtain a synchronized time and register events in log storage based on the synchronized time. In some embodiments, outputting and logging control commands may be performed at regular intervals set by the resource control device or by default.

[0088] Similarly, for sensing at the joint motors 1152, a logging process may be initiated by an interrupt to the input unit by an internal encoder and / or by a distance sensor or human presence sensor detecting or sensing a worker 1155 or an AGV 1157. The log registration unit may acquire the number of notifications from the encoder, the conversion result from the number of notifications to a motor angle, and / or the conversion result from the angle to a posture, or TCP information (e.g., a homogeneous transformation matrix) of the industrial robot arm 1151, along with distance information to the detected object (e.g., status information for the system). The log registration unit may acquire a synchronized time and register the event in log storage along with the synchronized time. In some embodiments, logging of sensing information from the joint motors 1152 may be performed at regular intervals configured by the resource control device or by default.

[0089] The distance to the worker 1155 and the speed and acceleration limits associated with that distance are important safety considerations when operating a collaborative robot (a robot operating in close proximity to a human worker). Therefore, the sampling frequency for logging when a human approaches a collaborative robot is related to the control period (or control message frequency) allowed for the collaborative robot's joint motors. The sampling period can be configured to enable observation (e.g., by a control system) of the transient responses of the joint motors 1152 and verify safety for the worker 1155. To accomplish this, the industrial robot arm 1151 and / or the collaboration area 1158, in some embodiments, may be equipped with sensing capabilities to measure the distance between the industrial robot arm 1151 and the worker 1155. Such sensors, in some embodiments, may include a distance sensor associated with the industrial robot arm 1151 and an image analysis system with a camera associated with the collaboration area 1158 as a sensor, or other set of sensors capable of monitoring the location of (and distance from) different objects in the collaboration area 1158.

[0090] In some aspects, the time synchronization master 1159 may execute an application that cooperates with the resource control device 120 to enable the resource control device 120 to control communication resources. The time synchronization master 1159 may change the transmission frequency of the Sync message and / or the Follow_Up message according to an instruction (based on a resource control operation) from the resource control device 120 to change the transmission frequency. The instruction may include an indication to change the transmission frequency and the new transmission frequency to use. In some aspects, the instruction may indicate either an increase or decrease in frequency, and the time synchronization master 1159 may determine the actual frequency.

[0091] In some embodiments, resource controller 120 may indicate a difference from the current frequency in the instruction. For example, the instruction may be based on a multiplier (.5 or 2) applied to the frequency or period, or an additive factor (+1 Hz to go from 4 Hz to 5 Hz).

[0092] In the above-described example, assume that the worker 1155 transfers the part 1154 to or from the industrial robotic arm 1151 and then exits the collaboration area 1158. As described above, in some embodiments, the exit is detected by the curtain sensor 1156 (601 in FIG. 6 ), and the control system state determination unit 540 then determines the state of the collaboration area 1158 (602 in FIG. 6 ). The control system state determination unit 540 determines that no risk exists because the worker 1155 has left the collaboration area 1158, and as a result, the control system state determination unit 540 may release resources (e.g., sensing and / or communication resources) associated with the resource allocation augmentation that were based on the proximity of the human worker to the robotic arm 1151 because the state associated with the resource allocation augmentation is no longer a state of the system. Specifically, the transmission interval (e.g., period) of the Sync message and / or the Follow_Up message may be configured to be longer (e.g., configured to be transmitted less frequently) to conserve communication resources. As another example, the sampling period and / or the interval between recordings to the log storage 436 may be set longer to reduce the computational load on the processing unit 432 and the log registration unit 435 and reduce the storage capacity used by the log storage 436. In some embodiments, if there are adequate resources available in the system to maintain the enhanced resource allocation, the allocated resources may not be released.

[0093] At a second time after the worker 1155 exits the collaboration area 1158, if the AGV 1157 enters the collaboration area 1158 via the curtain sensor 1156, the control system state determination unit of the resource controller 120 may determine the state of the collaboration area 1158 according to the operational policy. The resource controller, in some embodiments, may decrease resource allocation, may not change resource allocation, or may prohibit allocation of enhanced resources (e.g., large amounts of resources) based on a determination that the risk of danger to humans is deemed low. In some embodiments, the resource controller may increase resource allocation to prevent or reduce the likelihood of collisions between machines. The amount of resources (synchronization and / or reporting resources) allocated to prevent collisions between machines may, in some embodiments, be less than the amount of resources allocated when a worker is present in the collaboration area 1158, but more than the amount of resources allocated when neither the AGV nor humans are present in the collaboration area 1158.

[0094] In general, a set of configured events may be associated with different hazard levels and / or risks and may be associated with a corresponding configured set of resource allocations. The system may use a set of predefined hazard levels associated with different potential hazards. For example, a first, lowest hazard level and corresponding low-granularity resource allocation (e.g., minimum resource allocation) may be associated with a low risk of damage to assets. One or more intermediate hazard levels and one or more corresponding medium-granularity resource allocations (e.g., medium resource allocations) may be associated with one or more intermediate hazard levels to assets and / or one or more low or medium risks to human workers. Finally, a highest hazard level and corresponding most granularity resource allocation (e.g., most resource-intensive allocations) may be associated with a high risk to valuable assets and / or human workers.

[0095] In some embodiments, the risk of a collision or contact between machines (or a machine and a wall or other structure) may be associated with either a low- or medium-granularity resource allocation based on a risk level calculated based on the state of the system (e.g., distance between the machines, machine speed, machine acceleration, or other relevant parameters). The risk of a collision between a machine and a human may be associated with a medium- or high-granularity resource allocation based on an individual risk calculation based on the state of the system (e.g., human location (inside or outside the area), distance to the machine, reach of an industrial robotic arm, speed and acceleration of the human or any surrounding machines, or other contributing factors). Factors in the distance- and / or location-based risk calculation may include whether a human is within a threshold distance, whether two autonomous vehicles (e.g., AGVs) are within a threshold distance, whether a human and / or machine is in a specific area (e.g., a prohibited area or a particularly dangerous area), and whether a human or machine is located a threshold distance away from its expected location. As discussed throughout, the state of different components of the system (e.g., positioning rate, acceleration, etc.) may be measured by sensors installed (or worn by operators) to monitor different aspects of the system state that are determined to be useful in determining the state of the system and whether the system is functioning properly.

[0096] An example of a logged event includes the internal operations of the AGV 1157. The internal operations may include issuing control commands to steer the body of the AGV 1157 and / or actuating the tires. A logged event related to the AGV 1157 may include sensing a value from an associated motor, sensing a value from a weight sensor associated with the AGV 1157 when the AGV 1157 receives a part 1154, and / or sensing a value from a contact sensor associated with the AGV 1157 to detect a collision. When the AGV 1157 passes a part 1154 to the industrial robotic arm 1151 or receives a part 1154 from the industrial robotic arm 1151 and exits the collaboration area 1158, the curtain sensor 1156 may sense the exit and re-execute the process shown in FIG. 6 to reallocate resources. In some embodiments, the worker 1155 may be detected by sensors associated with the industrial robotic arm 1151 and / or the collaborative area 1158 in addition to the curtain sensor 1156 or in the absence of the curtain sensor 1156.

[0097] In some embodiments, the periodic times for recording log events, such as sensor values, may be synchronized among multiple terminal devices 123 and / or I / O control devices 125 to facilitate comparison of multiple sensor values. For example, in some embodiments, the start times may be synchronized so that if the sampling period is 1 ms and the start time is 100 ms, the log recording times at multiple terminal devices 123 are 100 ms, 101 ms, 102 ms, etc. If some of the terminal devices 123 and / or I / O control devices 125 have a time difference that is shorter than the sampling period or an offset from the sampling period of 100.5 ms, 101.5 ms, 102.5 ms, etc., it may be difficult to compare sensor values ​​with logs at other terminal devices 123. In some embodiments, the log recording times may be shared among terminal devices 123 and / or I / O control devices 125, and terminal devices 123 and / or I / O control devices 125 may adjust their log recording times to align with each other.

[0098] FIG. 12 illustrates a system for transporting parts 1254 from a belt conveyor 1253 to a table 1261 through coordination between two industrial robotic arms 1251 a, 1251 b. While FIG. 12 illustrates an example of coordination between a pair of industrial robotic arms, the concepts described in connection with FIG. 12 may also be applied to other control systems involving other types or numbers of devices and / or humans working together to accomplish a task. For example, another application example may be a case in which a first industrial robotic arm holds a metal plate and a second industrial robotic arm cuts the metal plate with a cutting device attached to the tip of the second industrial robotic arm. As mentioned above, the machine is not limited to an industrial robotic arm, but may also be an AGV and / or other machine. In some embodiments, the robot controllers 1250a and / or 1250b of the industrial robot arms 1251a and / or 1251b may be connected to other devices via a wireless network 126, while the robot controllers 1250a and / or 1250b are connected via the control network 122 of FIG. 12 .

[0099] In some embodiments, the time synchronization master 1259 may execute an application that communicates with the resource control device 120 to enable the resource control device 120 to control communication resources. The time synchronization master 1259 may change the transmission frequency of the Sync message and / or the Follow_Up message in response to an instruction to change the transmission frequency from the resource control device 120 (based on a resource control operation). The instruction may include an instruction to change the transmission frequency and the new transmission frequency to be used. In some embodiments, the instruction may instruct either to increase or decrease the frequency, and the time synchronization master 1259 may determine the actual frequency.

[0100] FIG. 13 is a flow diagram 1300 illustrating a causal analysis method related to a system failure. Flow diagram 1300 is described below in relation to a failure in the system depicted in FIG. 12 or a similar system, including coordination and / or cooperation between machines controlled within the system to achieve a desired result. Accordingly, FIG. 13 assumes a situation in which a desired behavior is not achieved, i.e., a system failure, due to a problem with the control algorithm and / or control program in the system depicted in FIG. 12. For example, the failure could be related to industrial robot arm 1251a failing to pick up part 1254 from belt conveyor 1253, or one of industrial robot arms 1251a and / or 1251b dropping part 1254 en route to table 1261. Another example, when cutting a metal plate, is when the cutting line deviates from the desired line or when processing precision is insufficient.

[0101] In some embodiments, the method is performed by an analysis unit 1262 that performs various analyses, control algorithm modifications, and / or application modifications, simulations and / or emulations based on the logged events. The hardware structure of analysis unit 1262 may, in some embodiments, correspond to the structure of device 200 of FIG. 2, device 300 of FIG. 3, or computing device 2005 of FIG. 20. Analysis unit 1262 may, in some embodiments, be implemented as a simulator, emulator, or analysis tool.

[0102] Logs of fault events analyzed using the method illustrated in flow diagram 1300 may be recorded by robot controller 1250a and robot controller 1250b. The recording of logs related to fault events may be similar to the recording of logs discussed in connection with Figures 6, 8, and 11 above. For example, a log registration unit may record the events in log storage within robot controller 1250a and robot controller 1250b.

[0103] At 1301, the analysis device may obtain information related to the fault event (e.g., information recorded by robot controllers 1250a and / or 1250b and stored in log storage). The information may be obtained from a log acquisition unit, which may obtain the stored information at 1301 and transmit the information via a communication unit to analysis device 1262. In some embodiments, the information obtained at 1301 may include control information and sensor information, as described above in connection with FIGS. 8 and 11.

[0104] At 1302, analysis device 1262 may supplement the information received at 1301 to provide an analysis. Supplementation at 1302 may, in some embodiments, include creating a simulation model of the entities forming the control system (e.g., industrial robot arms 1251 a and 1251 b and / or robot controllers 1250 a and / or 1250 b) and / or creating the control logic and / or application logic used. The simulation model may be a three-dimensional simulated geometric model of industrial robot arms 1251 a and / or 1251 b, belt conveyor 1253, part 1254, table 1261, and / or may simulate the laws of physics, such as gravity, when part 1254 falls.

[0105] In some embodiments, the simulation model may include a network simulation model for simulating communication between the robot controller 1250, the resource control device 120, and / or the analysis device 1262 via the control network 122. The time granularity of the simulation model may vary depending on the purpose of the analysis. For example, a fine-time granular model of the joint motors 1252 may be used, and / or a discrete event simulation model that simulates packet communication may be used. In some embodiments, the control application logic of the joint motors 1252 running on the robot controller 1250b may be provided as an execution model of the simulator or may be converted to an execution model of the simulator. Time synchronization software running on the resource control device 120 and the robot controller 1150 may also be modeled.

[0106] At 1303, the analysis device 1262 may recreate the stored events by simulation or emulation from the collected and supplemented information generated at 1302. For example, a fault event may be simulated and / or emulated using control logic based on a model of the system's components. The recreated results (e.g., simulation results) may be recorded at 1304 to improve the control algorithm, to visualize the simulation results for user review, or if needed for other reasons.

[0107] In some embodiments, simulated events are recorded along with simulator time at 1304. For example, as described below in connection with 1306, when control logic within the robot controller 1150 is updated to configure the simulated environment for the next iteration, the output of control commands from the robot controller 1150 to the joint motors 1152 and the time of the output may be recorded. In some embodiments, sensor values ​​in the simulation may also be recorded along with the time in the simulation. The sensor values ​​may correspond to the reaction of the environment to the updated control actuations of the robot controller 1150. The recorded sensor values ​​may be used to improve the accuracy of the environment model. The time in the simulation may be a unified time or a simulated time based on simulated time synchronization between entities (e.g., the industrial robot arm 1251 a, the industrial robot arm 1251 b, and the resource control device 120) that may include simulated time errors.

[0108] At 1305, the analysis unit 1262 may determine whether an exit condition has been met. The exit condition may be set based on the goal of the analysis. For example, if the goal is observation, the exit condition may be that a certain number of iterations have been performed, whereas if the goal is to resolve detected faults, the exit condition may be to have a simulation that does not experience the fault.

[0109] If the termination condition is not met at 1305, the analyzer 1262 may configure the simulated environment for the next iteration of the simulation and / or emulation at 1306. In some embodiments, configuring the simulated environment at 1306 may include updating the control logic and / or the simulated / emulated environment. In some embodiments, configuring at 1306 may include an improvement process if the target control system does not meet the desired operation, functionality, and / or performance.

[0110] For the industrial robot arm 1251a and the industrial robot arm 1251b that drop the part 1254, multiple considerations and / or measurements may be taken into account for the update at 1306. For example, the update may comprise an updated position of the table 1261 as it approaches the industrial robot arm 1251. In some embodiments, the update may modify the control logic of the robot controller 1250a and / or the robot controller 1250b. In some embodiments, the time synchronization method between the industrial robot arm 1251a and the industrial robot arm 1251b may be updated at 1306 to improve synchronization accuracy and coordination performance between the robot arms 1251a and 1251b.

[0111] After configuring the simulated environment for the next iteration, the analysis device 1262 runs the simulation at 1303 to generate an updated result set, which is recorded at 1304. Based on the updated result set, the analysis device 1262 may again determine at 1305 whether the updated result set satisfies an exit condition. For example, if the detected fault associated with the method shown in flow diagram 1300 is that industrial robotic arm 1251 a and industrial robotic arm 1251 b dropped part 1154, the exit condition may be associated with industrial robotic arm 1251 a and industrial robotic arm 1251 b successfully placing part 1154 on table 1261, or may be associated with an error between the placed position of part 1154 and the target position marked on table 1261 being less than a threshold.

[0112] If the analysis device 1262 determines that the termination condition has been met at 1305, it may export the configuration associated with the termination condition being met (e.g., the last configuration used to update the simulation at 1306), and the process may end. At 1307, the analysis device 1262 exports the configuration to update the components of the control system of FIG. 12 with a configuration that will not fail during the simulation at 1303. Additionally or alternatively, the configuration may be exported so that the control system can test the updated configuration in a physical environment to provide feedback regarding the accuracy of the simulation and / or the need to further refine the configuration with respect to the behavior of the physical environment, as described below with respect to FIG. 14.

[0113] In some embodiments, analysis unit 1262 may select 1307 information to send to the terminal device from the information recorded in 1304. In some embodiments, control commands may be selected because the process is focused on evaluating the effect of the control logic during the simulation. However, because sensor values ​​may not be reproduced in the real environment, for example, the sensor values ​​resulting from the implementation of the updated control logic may not be the same as in the simulated environment, the sensor values ​​from the simulation may be omitted to provide feedback on the accuracy of the simulation and / or the effectiveness of the updated configuration.

[0114] In some embodiments, the analysis device 1262 may export 1307 the selected information to each of the terminal devices associated with the updated configuration (e.g., control commands). Because multiple terminal devices may be associated with the updated configuration, the analysis device 1262 may, in some embodiments, identify a subset of configurations associated with each terminal device and provide each terminal device with the subset of configurations corresponding to that terminal device. For example, with reference to FIG. 12 , an updated configuration (e.g., simulation results) associated with the industrial robot arm 1251a is sent to the industrial robot arm 1251a, and an updated configuration (e.g., simulation results) for the industrial robot arm 1251b is sent to the industrial robot arm 1251b. The control commands associated with the updated configurations may be further adjusted by the analysis device 1262 to reflect a future time at which to execute the control commands at the terminal device(s), or may be associated with a time offset used by the terminal device(s) to determine when to execute the control commands provided by the analysis device 1262.

[0115] In some embodiments, the method illustrated in flow diagram 1300 can be conceptualized as applying a digital twin in the cyberspace of analysis device 1262 (e.g., implemented as a cloud) that simulates and optimizes the behavior of the control system shown in FIG. 12. Thus, optimization of the control system using IoT can be achieved. In some embodiments, the simulation model or digital twin generated by the stored information can be combined with other models, or reproduced or analyzed by deploying the model or twin in a virtual space such as the Metaverse.

[0116] The method illustrated in flow diagram 1300 may be used to validate specific control algorithms based on the overall behavior of entities within a control system. For example, a dispatch algorithm for a fleet of AGVs (e.g., an algorithm for dispatching excess AGVs to open areas) may be validated. Similarly, a dispatch algorithm for taxis, including autonomous vehicles, may be validated using the method of flow diagram 1300.

[0117] Furthermore, as a result of the analysis, the analysis unit 1262 may alert an operator to a discrepancy based on integrating the results of multiple sensors. For example, a discrepancy occurs when a camera sensor monitoring an area records an AGV entering the area, but the AGV itself does not record the AGV entering the area. However, image analysis is applied to the camera sensor detection. In this case, a critical time synchronization error between the camera sensor and the AGV, a failure of the camera sensor and / or the AGV, and / or a malfunction in the camera sensor's image analysis may be suspected. The operator can investigate the cause based on the alert and take appropriate measures to improve the stability of the control system. Integrating the results of multiple sensors may, in some embodiments, include integrating one or more of a set of sensors, such as infrared sensors, radar, LiDAR, or other sensors in a similar manner to detect the presence or passage of an entity, without being limited to camera sensors. In some embodiments, when multiple sensors detect the same event, a majority vote may be applied to detect the faulty sensor. This may provide a more accurate assessment and / or identification of the operational and / or faulty sensors. For example, if an AGV equipped with a camera sensor, a curtain sensor, and a position sensor monitors an AGV entering an area, the sensor associated with the AGV may be presented as faulty if only the sensor on the AGV differs from the other sensors.

[0118] FIG. 14 is a flow diagram 1400 of a method for implementing (“playing back”) an updated configuration determined for a simulated device (or environment) as described for a physical device (or environment) in connection with FIG. 13 . The method of FIG. 14 may be performed by each terminal device or a component associated with each terminal device that receives the updated configuration exported at 1307 of FIG. 13 by the analysis device 1262. At 1401, the terminal device (or a log registration unit of the terminal device) receives the updated configuration from the analysis device (e.g., the updated configuration exported at 1307 of FIG. 13 ). As described above, the updated configuration received by the terminal device at 1401 may include a set of control commands associated with the terminal device based on the simulation. The control commands may be received at the terminal device for implementation to test the accuracy of the simulation and / or the effectiveness of a solution provided by the simulation.

[0119] At 1402, the terminal device (or a log registration unit of the terminal device) records (e.g., in log storage of the terminal device) the updated configuration information received from the analysis device (e.g., via the communication unit). In some aspects, recording the updated configuration information may include correcting a time associated with the updated configuration because the time recorded in the simulation is in the past (e.g., the time of the recorded failure event). Thus, in some aspects, the correction may be adding a specified period of time to cause the terminal device(s) to execute the updated configuration (e.g., control command) in the future. In some aspects, the specified period of time may be a known period provided by an operator, such as 1 hour or 45 minutes (based on the assumption that the method of FIG. 13 is completed and the updated configuration is exported in a short time, such as 40 minutes or 30 minutes). In some aspects, the specified period of time may be a calculable period of time, such as a period that schedules the updated control command a specified number (e.g., 4 or 5) minutes after the updated configuration is received. As mentioned above, in some aspects, the analysis device may provide a time offset to be used by each terminal device.

[0120] 15 is a diagram 500 illustrating an example of a time correction that may be performed as part of recording an updated configuration at 1402, according to some aspects of the present disclosure. The first event received from the analysis device (e.g., logged time 1511) and / or control command (e.g., #1) received from the analysis device at 1401 associated with the industrial robot arm 1251a is originally at 3:03:001 in the received log entry 1501 of FIG. 1510, and the execution time 1521 is shifted to 50:00:001 to pass the delivery time of 45:00 in the updated log entry 1501′ of FIG. 1520. The same difference between 50:00:001 and 3:03:001 (e.g., 46:57) is added to the other logs to maintain the relative timing of each entry. In some aspects, the correction may preserve digits smaller than a specified digit (e.g., digits smaller than seconds) and set future times after the current time to digits larger than the specified digit (e.g., seconds). In some aspects, digits smaller than a specified digit may be ignored. For example, if an actuator executing a control command for an updated log operates in microseconds when time is recorded in nanoseconds, digits corresponding to nanoseconds (e.g., those smaller than microseconds) may be truncated (i.e., numeric values ​​associated with time increments smaller than microseconds may be set to 0). The time correction may be performed by one or more of the analysis device 1262, the log registration unit 435, and / or the log acquisition unit 437. For example, the analysis device 1262 may adjust the time in the recording step in 1304. The log registration unit 435 may adjust the time at which the log registration unit 435 registers an event in the log storage 436. The log acquisition unit 437 may adjust the time at which the log acquisition unit 437 extracts logs from the log storage 436. In some embodiments, the operator may notify one or more of the analysis device 1262, the log registration unit 435, and / or the log acquisition unit 437 of either or both the period to be added and the adjusted time of the first log.

[0121] The terminal device determines that the log time has arrived at 1403 and executes the updated configuration. In some aspects, executing the updated configuration at 1403 may include retrieving a log from log storage and then executing events in the retrieved log. Execution of the updated configuration (e.g., control commands) may be based on a synchronized (or system) time such that different terminal devices or other components of a control system execute commands based on a shared time reference (e.g., industrial robot arm 1251a and industrial robot arm 1251b in FIG. 13 ). During execution of the control commands at 1403, the terminal device (or a sensor of the terminal device), other devices, and / or other sensors may collect and / or record information related to the execution of the updated configuration (e.g., control commands) received from the analysis device.

[0122] After executing the updated configuration, the terminal device determines whether a termination condition has been met at 1404. In some aspects, determining whether a termination condition has been met at 1404 may include determining whether there are any unexecuted log entries (e.g., log entries for future times received from the analysis device) and / or determining whether an instruction has been received from an operator to terminate execution of the unexecuted log entries. If the terminal device determines at 1404 that the termination condition has not been met, the terminal device returns to 1403 and executes the next log entry at the time specified in the next log entry.

[0123] If the terminal device determines that the termination condition is met at 1404, the collected and / or recorded information is provided as feedback to the analysis device at 1405, and the method ends. The analysis device may then use the collected and / or recorded information to update a simulation model associated with the control system (e.g., one of the systems shown in FIGS. 1, 11, or 12), which is used to generate an updated configuration related to the updated configuration and feedback to improve performance of future simulations. It will be appreciated that if a fault event is detected or occurs during the course of the method shown in flow diagram 1400, it may trigger additional execution of the methods of FIGS. 13 and 14.

[0124] In some embodiments, the methods of Figures 13 and 14 are performed in parallel. For example, the terminal device may execute a log (e.g., control commands) associated with the analysis of a first event while the analysis device analyzes a second event. Parallel execution of the optimization process (e.g., the method of Figure 13) in the analysis device 1262 and the playback process (e.g., the method of Figure 14) in the terminal device allows for real-time improvement of the control system behavior during operation.

[0125] During execution of the control commands at 1403, the terminal device (or sensors of the terminal device), other devices, and / or other sensors may collect and / or record information (e.g., control commands) related to the execution of the updated configuration. The collected and / or recorded information may be provided as feedback to the analysis device at 1405. The analysis device may then use the collected and / or recorded information to update a simulation model associated with the control system (e.g., one of the systems shown in FIG. 1 , FIG. 11 , or FIG. 12 ) used to generate the updated configuration and the feedback to improve performance of future simulations.

[0126] Additional system aspects that may be applied to any of the systems shown in FIGS. 1, 11, and 12 are described below. In some embodiments, analysis of the transient response of joint motors may be desired when a robot controller activates an industrial robot from a standstill to grab a part on a conveyor belt. The industrial robot arm may impose speed and / or acceleration limits on the TCP depending on the attributes of the part being handled. For example, if the industrial robot arm is transporting a glass of water, the gripping point on the glass may be constrained, and moving at high speeds may cause the water to spill from the glass. Also, moving a flexible part at high speeds may cause the flexible part to flex and deform. In such cases, a large amount of resources may be allocated to the robot controller to accurately record the event.

[0127] In some embodiments involving stopping operations, including emergency stops of industrial robotic arms and joint motors, a large amount of resources may be allocated to the robot controller due to speed and / or acceleration constraints. In some embodiments, additional resources may be allocated to the device when humans are near the device because of increased safety risks. In some embodiments, additional resources may be allocated to the AGV when it moves above a threshold speed (compared to resource allocation in a low-risk environment) because high speed movement is considered dangerous. Conversely, resource allocation may be reduced when the AGV moves at a slower speed (e.g., below a threshold speed).

[0128] In some embodiments, resource allocation may be determined based on the relative speed of the device, moving objects, and / or environmental objects around the entity to which the resource is allocated (e.g., moving objects on a conveyor belt). In some embodiments, resource allocation may be controlled in response to changes in the surrounding environment.

[0129] In some aspects, the terminal device may be equipped with playback functionality only if the terminal device does not include a sensor.

[0130] FIG. 16 shows diagrams 1610 and 1620 illustrating drone 1601 flying in different factory environments, and diagram 1630 illustrating a set of configurations associated with different areas or conditions. As shown in FIG. 1610, when drone 1601 flies through a narrow corridor surrounded by shelves 1602, resources at a first (fine) granularity may be assigned to the drone to record information at the fine granularity (local control commands, control commands received from a higher level, analytics, equipment, state information such as height above ground, speed, angle, or other state variables). In some embodiments, when the drone flies through a large open space or an area without obstacles 1621, as in FIG. 1620, the resource control device may assign resources at a second (coarse) granularity to drone 1611.

[0131] In some embodiments, the resource control device defines resource allocation (e.g., resource configuration) rules for each area and applies the resource allocation rules to the drones 1601 and / or 1611 (e.g., selects the configured resource allocation) depending on the drones' locations. To determine their locations, the drones 1601 and / or 1611 may be equipped with a localization method such as simultaneous localization and mapping (SLAM) and / or sensors may be installed in each area to detect the drones 1601 and / or 1611. In some embodiments, resource allocation may be determined according to the state of the drones, including the area and / or location information, their relative positions relative to other nearby devices, the presence of humans, and / or transient responses.

[0132] In some embodiments, the resource control device may allocate resources according to sensor results of the surrounding environment sensed by sensors (e.g., cameras and / or distance sensors) associated with the drone. For example, the resource control device may allocate a large amount of resources (e.g., for high-granularity logging) to the drone when the distance to other objects in the environment measured by the distance sensor is less than a threshold distance. In some embodiments, the threshold distance may be based on the speed of the drone, such that the threshold distance is the distance that can be covered in a specific period of time (e.g., 0.1 seconds). For example, the measured distance may be lower than a predefined threshold when the drone 1601 flies through a narrow passage between shelves 1602 on both sides, as shown in FIG. 1610, and in such a case, the resource control device may allocate a large amount of resources. Resource allocation based on the surrounding environment may be applied, in some embodiments, not only to drones but also to AGVs, AMRs, and / or autonomous driving vehicles.

[0133] In some embodiments, the resource control device may allocate resources according to the size of the area sensed by a sensor associated with a mobile object. For example, when a mobile object associated with a camera-equipped sensor approaches an intersection near an obstacle or an area with poor visibility, the risk of a collision with another object from outside the range may increase. In such a case, the resource control device may allocate a large amount of resources to the mobile object. By allocating a large amount of resources, the present invention can appropriately record information about dangerous situations caused by obstacles or the surrounding environment. The recorded log can be used for safety measurement before and after an accident. Therefore, the present invention contributes to the safe operation of a control system.

[0134] In some embodiments, the resource controller may allocate resources according to the scope of impact of an operation. For example, a widespread alarm can alert many workers and provide important notification of a dangerous situation. The resource controller may allocate larger resources to alarms to record safe actions. In some embodiments, the resource controller may control resource allocation according to area. For example, the resource controller may allocate larger resources to areas with higher safety risks, such as intersections with high accident rates or areas of human-robot collaboration.

[0135] As another example, the above-described method may be applied to autonomous vehicle driving in accordance with some aspects of the present disclosure. A vehicle equipped with a wireless communication device and a resource control device may change the allocation of communication resources between the vehicle and the vehicle depending on the situation. For example, the resource control device may allocate a large amount of resources when a human is present near the vehicle while the vehicle is moving. In some aspects, the resource control device may change the resource allocation depending on the distance and / or relative speed of other vehicles traveling near the vehicle. For example, when the vehicle is traveling at high speed on a highway, the resource control device may increase the resource allocation. Even when traveling at a low speed, the resource control device may increase the resource allocation during traffic congestion due to the increased risk of collision with vehicles in front and behind.

[0136] In some embodiments, the resource control device may change the resource allocation in response to changes in the surrounding environment. For example, because wireless communication is difficult inside a tunnel or under an overpass, the resource control device may increase the resource allocation to the vehicle before the vehicle enters the tunnel or under an overpass to improve synchronization accuracy. In some embodiments, the resource control device may change the resource allocation when the vehicle passes through a junction due to a traffic accident or congestion. In some embodiments, the resource control device may change the resource allocation when the vehicle enters a pedestrian zone because the risk of approaching people increases.

[0137] In some embodiments, the present invention is applicable to an electronic control unit (ECU), where a GPS receiver and / or ECU communicating with an external synchronization master is the time synchronization master within the vehicle. That is, the ECU may change communication resources between the time synchronization master and / or its own resources depending on the vehicle's driving conditions. For example, if there is a traffic accident or workers ahead (e.g., road construction), the ECU may allocate a large amount of resources to the ECU associated with sensors and / or actuators installed at the front of the vehicle. In some embodiments, the ECU may allocate a large amount of resources to the ECU associated with sensors and / or actuators installed on a particular side of the vehicle (e.g., left or right) when the vehicle turns toward a particular side (e.g., left or right), when the vehicle changes lanes to a particular side, when another vehicle approaches the vehicle from a particular side, or when another vehicle overtakes the vehicle on a particular side. In some embodiments, the ECU may allocate a large amount of resources to ECUs associated with sensors and / or actuators installed at the rear of the vehicle when a following vehicle approaches the vehicle at increased speed or when a following vehicle is following the vehicle. Recorded events related to autonomous driving may be used to improve autonomous driving algorithms and for safe driving verification. As another example, the above-described method may be applied to improve collision detection and / or prevention in manually driven vehicles in accordance with some embodiments of the present disclosure.

[0138] The method may also be applied to improve the performance of service robots in healthcare applications, according to some embodiments of the present disclosure. For example, in some embodiments, a service robot can assist individuals with limited mobility or abilities in their daily lives by assisting them with routine tasks, such as carrying dishes and / or cleaning. Furthermore, a surgical robot can be controlled by a doctor to assist in performing surgery. A resource control device may control resources according to the state of the robot in such healthcare applications. For example, the state may be classified as one of the following: when the robot performs an action on a person; when the robot does not perform an action on a person but a person is nearby; and when no person is nearby the robot (e.g., cleaning or carrying an item). Each state may be associated with a different amount of resources (e.g., based on a calculated risk), as described in connection with FIGS. 10-15 .

[0139] In some embodiments, the resource control device 120 may change resources depending on the area and / or location. For example, in a hospital waiting room or at home, where contact with people is a greater risk, the resource control device may allocate a large amount of resources to a robot in such areas, but if the robot is in a robot-only space in a hospital or a space where contact with people is limited, the resource control device may allocate fewer resources to the robot. This may provide robot safety verification and control algorithm improvements for service robots in the healthcare field where robots directly interact with people.

[0140] In some aspects, the time resolution at the time of recording may be changed as part of resource control, as described in connection with FIGS. 6 and 13 . The time resolution may be changed depending on the state of the control system (e.g., the state as evaluated in 602 of FIG. 6 ). The time resolution of a recorded log may be changed when the log registration unit registers the log in the log storage and / or when the log registration unit retrieves the log from the log storage. An analysis device or operator may specify the time resolution when requesting log acquisition from the log acquisition unit. When requested, the log acquisition unit may thin out data sampled at a high resolution (e.g., when the event detection time is 1:00:05.123456800 nanoseconds), and the log acquisition unit may provide the data to the requester for accurate analysis. In some aspects, the log acquisition unit may provide data sampled at a low resolution (e.g., when the event detection time is 1:00:05 seconds) to an external entity. This saves storage and / or communication resources when accurate analysis is not required.

[0141] In some aspects, the sampling frequency during recording may be changed in real time as part of resource control, as discussed in connection with FIGS. 6 and 13. The sampling frequency may be changed depending on the state of the control system (e.g., the state as evaluated in 602 of FIG. 6). The sampling frequency of a recorded log may be changed when the log registration unit registers the log in the log storage and / or when the log registration unit retrieves the log from the log storage. For example, a sensor may measure a characteristic (e.g., position, velocity, acceleration, etc.) at a first frequency (or a first rate), and the log registration unit may register a subset of the measurements at a second, lower frequency (or a lower rate). An analysis device or an operator may specify the sampling frequency when requesting log acquisition from the log acquisition unit. The log acquisition unit may thin out frequently sampled data when requested by the log acquisition unit or provide the data upon request. In some aspects, the log acquisition unit may perform data imputation on infrequently sampled data and provide the data to an external entity.

[0142] Examples of controlled resources (e.g., 602 in FIG. 6 ) may, in some aspects, include one or more of communication / transmission frequency, communication bandwidth, frequency bandwidth, time slots, communication paths, sampling frequency, sampling resolution, and storage capacity. When logging with fine time granularity in log storage, the sampling frequency is shortened and / or high-resolution sampling is configured. If the allocated capacity of event storage is insufficient for the associated sampling rate, the allocated capacity may be increased. Recording relationships between humans, machines, and mobile entities with fine time granularity may, in some aspects, rely on improved time synchronization accuracy. Supporting improved time-synchronized communication may, in some aspects, involve increasing the communication frequency, communication bandwidth, frequency bandwidth, and / or time slots when implementing a time-synchronized communication protocol.

[0143] The communication path may be changed based on communication delay and / or synchronization accuracy. For example, a boundary clock (BC) or a transparent clock (TC) defined in IEEE 1588 is a network relay switch that can improve synchronization accuracy. Furthermore, a low-latency route or a fixed-latency route may be preferred in some aspects of time synchronization. Therefore, a route via a BC or a TC, and / or a preferred route, may be selected and assigned in route selection between a time synchronization master and a terminal device. In some aspects, redundant communication routes may be used to reliably communicate time synchronization protocol packets. In such cases, any of the communication frequency, communication bandwidth, frequency bandwidth, time slots, and communication routes may be assigned (e.g., increased) to achieve the target synchronization accuracy. For example, if the target accuracy can be achieved by simply increasing the communication frequency, other resources such as communication bandwidth, frequency bandwidth, and time slots may not need to be changed.

[0144] In some embodiments, an intermediate storage that collects information stored in each log storage of the multiple terminal devices may be deployed in the control system shown in FIG. 1 . In some embodiments, the resource control device may provide the information collection function. In some embodiments, the intermediate storage may remove the time from the collected information and store the collected information in the order of the time it was received. The intermediate storage may be configured to save storage capacity for system analysis when only the order of occurrence is meaningful.

[0145] In some embodiments, the source device, time synchronization unit, log registration unit, and / or log storage of a recorded event may be implemented separately in different devices, thereby introducing communication delays when identifying and recording events. To accurately record events, in some embodiments, the source device and recording device may be connected using a deterministic, real-time network or a network capable of measuring communication latency. TSN and real-time Ethernet can provide real-time, deterministic communication, and the transparent clock defined in IEEE 1588 can measure the residence time of packets within the transparent clock. In some embodiments, when the log registration unit records a received event, it may subtract the communication delay from the capture time (803 in FIG. 8). By subtracting the fixed or measured delay, the event occurrence time can be accurately measured. In some embodiments, if an event is received via a path with an unknown communication delay, the log registration unit may record the event with an indication that the communication delay is unknown.

[0146] In some embodiments, a terminal device may communicate with other devices that do not have time synchronization and / or log storage capabilities and record events at the other devices in the terminal device. Similarly, in some embodiments, the terminal device may communicate with other devices to provide updated configurations and / or instruct the other devices to execute the updated configurations (control commands related to the other devices in the updated configurations), as described in connection with FIGS. 13 and 14 with respect to the terminal device. The recorded logs may include device identification and / or event identification within the device. To accurately record events, low-latency wired communication or short-range wireless communication is preferred for connection with the device. The device may be a fixed device installed nearby, such as a camera or curtain sensor, or a mobile object located in close proximity in time, such as an AGV or AMR. In some configurations, a configuration may obtain the benefits of the present invention by using a terminal device instead of a device, even if the device does not have time synchronization and / or log recording capabilities.

[0147] In some embodiments, the event and the synchronized time may be recorded (e.g., by a log registration unit) and associated with an index indicating the accuracy of the time synchronization. In some embodiments, the time synchronization may increase as the time elapsed since the last time synchronization with another device (e.g., a time synchronization master) increases. Thus, in some embodiments, the previous time synchronization and / or the time elapsed since the previous time synchronization may be used as the index. In some embodiments, the time difference between the time received in the synchronization packet from the time synchronization master and the synchronized time of the device at the time of receiving the synchronization packet may be used as the index. An index generated by leveling the time difference may be used.

[0148] When comparing the time received in a synchronization packet from a time synchronization master with the synchronized time at the time of reception, the communication latency with the synchronization master may, in some embodiments, be added to the received time from the time synchronization master if the communication latency is known or obtainable. In some embodiments, an identification of the time synchronization algorithm and / or a performance index of the time synchronization algorithm may be used, since synchronization accuracy depends on the time synchronization algorithm executed by the time synchronization unit. In some embodiments, the accuracy of a clock device, such as an oscillator used in the device, may be used as an index. Such an index of time synchronization accuracy may be used to calculate the error, deviation, and confidence interval of the real time associated with the logged event.

[0149] In some embodiments where the simulation analysis is sensitive to minute time differences and multiple events may occur within a very short time period, events may be simulated using a simulation of events using different timings for a period based on the recorded index, as described in connection with Figure 13. In some embodiments, recording the log with the index also allows an operator to recognize poor time synchronization accuracy, for example, by receiving a warning when the elapsed time since the last synchronized time exceeds a threshold. Based on this warning, the operator can take measures to improve the stable operation of the present invention.

[0150] In some embodiments, the log acquisition unit may provide information extracted from the log storage in response to a request from an external device, such as an operator, a server, a cloud, a resource control device, and / or a terminal device. If the log acquisition unit is not synchronized with the request due to a communication error, an abnormality in the log acquisition unit, setup, or the like, the log acquisition unit may reject the request. Rejection may occur when the elapsed time since the last time synchronization exceeds a threshold (e.g., indicating a long free-running state). The log acquisition unit may determine whether to respond to the request depending on the state of the control system and the time synchronization accuracy when recording the requested log. For example, if the request log relates to a fine time granularity (e.g., when a machine and a worker approach each other or when a mobile object approaches an intersection), high-precision synchronization is preferable when recording the request log. On the other hand, if the device is operating slowly and / or no events are occurring around the device, high-precision synchronization when recording the request log may not be available or may not be desired. Therefore, low-precision synchronization may be sufficient for the application when recording the request log, and the request log may be provided.

[0151] In some embodiments, an improved control algorithm generated by the analysis process shown in FIG. 13 may be delivered to another device different from the original source device of the analyzed log. Such delivery occurs when a device corresponds to the state of the information stored in the original device. For example, if the control algorithm of an industrial robot arm along a production line is improved, the improved algorithm may be delivered to another robot in the same situation. In some embodiments, if the control algorithm of a drone or autonomous vehicle in a particular area is improved, the improved algorithm may be delivered to another drone or autonomous vehicle passing through the same area. This process may, in some embodiments, include real-time updating of the control algorithm. For example, the analysis device may deliver the improved control algorithm to devices determined to be in the same state (or approaching the same state) as the original device from which the control algorithm was generated after the improvement of the control algorithm shown in FIG. 13.

[0152] In some aspects, the terminal device may respond to a request for resource control from the resource control device. If the terminal device rejects or does not respond to the request, the resource control device may present a warning via one or more of a notification, a GUI, and / or other presentation means, such as a warning lamp or other physical indicator. In some aspects, if the terminal device rejects the request, the terminal device may provide a reason for the rejection. Examples of reasons for the rejection include one or more of resource depletion, lack of control means of the terminal device, and software and / or hardware abnormalities. The reason may be indicated by a predefined error code.

[0153] In some embodiments, the terminal device or I / O control device may change its behavior to improve log recording based on instructions from the I / O resource control unit. For example, if the terminal device or I / O control device is moving at high speed and communication of the time synchronization protocol becomes unstable, the terminal device or I / O control device may slow down or stop. This behavior change improves the accuracy of log recording, allowing the device's operation history to be verified and accurately analyzed, for example, when a moving object approaches an intersection.

[0154] One or more of the resource control unit, communication resource control unit, storage resource control unit, computer resource control unit, and I / O resource control unit may be configured according to a hierarchical structure spanning multiple resource control devices. For example, a resource control unit that controls a resource may be arranged for one area, and an integrated resource control unit that manages a collection of resource control units may be arranged for an integrated area that combines multiple areas. Other classifications may be based on the type of target object and / or time period. Humans may be one category of type-based classification.

[0155] In some aspects, when the resource control device allocates resources to multiple terminal devices, the resource control device may allocate larger resources to a terminal device (or actuator) having a larger number of component devices associated with the sensor (or actuator) or to a terminal device having a sensor (or actuator) covering a larger space. For example, a camera can monitor multiple terminal devices in a monitoring area. A LiDAR sensor can compare monitoring spaces according to its specifications. Additionally or alternatively, a terminal device that provides a time synchronization function and / or a logging function of another device may be identified as being associated with the other device, and the resource control device may allocate resources according to the number of associated devices and / or the sensing space.

[0156] In some embodiments, the present invention is configured to utilize energy efficiently, thereby reducing energy consumption based on a resource control device that allocates resources according to the control state of each component of the system. Energy savings may be based on the control state, including risk levels as described above, or the battery level of battery-powered system components (e.g., components such as AGVs and AMRs).

[0157] FIG. 17 is a flow diagram 1700 illustrating a method according to some embodiments of the present disclosure. In some embodiments, the method is performed by an analysis device (e.g., analysis device 1262 or computing device 2005) that performs various analyses, control algorithm and / or application modifications, simulations, and / or emulations based on logged events. At 1701, the analysis device may identify a state (e.g., a control system state) of multiple synchronized devices at a particular time. In some embodiments, the control system state may relate to one or more of the following: a type of at least one device, a surrounding environment of at least one device, or a change in the relationship of at least one device to at least one other device among the multiple devices. In some embodiments, the control system state may include one or more of the type of at least one other device, the presence of a human, a distance to at least one other device or human, a relative velocity between at least one device and at least one other device or human, or a relative acceleration between at least one device and at least one other device or human. In some embodiments, the state of the control system may be related to one or more dynamic characteristics of the at least one device, the dynamic characteristics including either a velocity or an acceleration of the at least one device. The state of the control system may in some embodiments include an indicator of the at least one device transitioning from a steady state to a dynamic state, or one or more of a position of the at least one device, a shape of the at least one device's surrounding environment, an area of ​​the at least one device's surrounding environment, an area sensed by the at least one device, or an area accessible to the at least one device.

[0158] For example, referring to FIG. 6 , in some aspects, a state of a control system may be determined in 602. The state of the control system determined in 602 may include the state of a set of terminal devices and a communication network providing communication between the control system and the terminal devices (e.g., the state of a set of network relay devices, wireless base stations, I / O control devices, etc.). In some aspects, the state of the terminal device may include an identification of the type of device (e.g., a localized robotic agent, an autonomous object, or other controllable device), a variable characteristic of the terminal device, such as velocity and acceleration, an indication of whether the terminal device is in a dynamic state or a steady state, information about the environment of the terminal device, the location of the terminal device, information about a set of other objects in the vicinity of the terminal device (e.g., object type, distance, relative velocity, relative acceleration, and / or other relevant information). The state of the communication network may include an identification of available paths and / or routes to different terminal devices via the communication network, and resources (e.g., used and available resources) associated with each path and / or route via the communication network. In some aspects, the determination at 602 is made based on a determination and / or detection that a triggering event (eg, an event that triggers resource control) has occurred at 601.

[0159] At 1702, the analysis device may allocate a set of time synchronization resources for transmitting time synchronization information from at least one of the plurality of synchronized devices based on the state of each of the plurality of synchronized devices identified at 1701. In some aspects, the allocated set of time synchronization resources may include one or more of time resources within a set of slots, frequency resources within a frequency range, bandwidth resources, and communication path resources, where the set of time resources is associated with one or more of a wired communication network or a wireless communication network. Such time synchronization resources may, in some aspects, be referred to as time synchronization communication resources. In some aspects, the set of time synchronization resources may include a set of logging parameters for recording sensor data associated with the at least one device. The logging parameters, in some aspects, may relate to one or more of a duration of each data recording event of the plurality of data recording events, a frequency of the data recording events in the plurality of data recording events, a resolution of the sensor data collected during each data recording event of the plurality of data recording events, or a data storage allocation for the sensor data collected during one or more data recording events in the plurality of data recording events. In some embodiments, an existing set of logging parameters prior to the assignment at 1702 may be changed (e.g., updated) based on the state of at least one device or control system due to the assignment at 1702. Such logging parameters may, in some embodiments, be referred to as time-synchronized recording and / or reporting resources and may be associated with events (e.g., data recording events) that an analysis device (e.g., a control system operator) has decided to monitor.

[0160] In some aspects, the set of logging parameters in the set of time synchronization resources may be a first set of logging parameters that instructs at least one device among the plurality of devices to start data recording for a data recording event simultaneously with at least one other device among the plurality of devices starting data recording for the data recording event. In some aspects, the set of time synchronization resources may include a second set of logging parameters for the at least one other device that indicates a frequency of data recording events that is either equal to the frequency of data recording events instructed in the first set of logging parameters or an integer multiple of the frequency of data recording events instructed in the first set of logging parameters. In some aspects, the resources controlled by the resource control device may include a transmission frequency of packets in a time synchronization protocol, an event sampling frequency, an event sampling resolution, and / or an event storage capacity allocation. In some aspects, the resources controlled by the resource control device may include a set of time slots and / or frequency bands in wireless communication, time slots for time-division communication such as TSN, and / or a route selection for synchronization packets in terms of synchronization accuracy and communication latency. For example, with reference to FIGS. 6 and 13, the analysis device may perform resource allocation associated with a resource control operation at 604 or 1306 .

[0161] Finally, at 1703, the analysis device may transmit a control message to at least one device of the plurality of devices indicating the determined allocated set of time synchronization resources for transmitting time synchronization information from the at least one device. The control message transmitted to the at least one device at 1703 may include an indication of either a time synchronization communication resource, a time synchronization recording and / or reporting resource, or a set of control commands. For example, with reference to FIGS. 6 and 13, the analysis device may transmit the control message as part of a resource control operation at 604 or when exporting a configuration at 1307.

[0162] FIG. 18 is a flow diagram 1800 illustrating a method according to some embodiments of the present disclosure. In some embodiments, the method is performed by an analysis device (e.g., analysis device 1262 or computing device 2005) that performs various analyses, control algorithm and / or application modifications, simulations, and / or emulations based on logged events. At 1801, the analysis device may identify a state (e.g., a control system state) of multiple synchronized devices at a particular time. In some embodiments, the control system state may relate to one or more of the following: a type of at least one device, a surrounding environment of at least one device, or a change in the relationship of at least one device to at least one other device among the multiple devices. The control system state, in some embodiments, may include one or more of the type of at least one other device, the presence of a human, a distance to at least one other device or human, a relative velocity between at least one device and at least one other device or human, or a relative acceleration between at least one device and at least one other device or human. In some embodiments, the state of the control system may be related to one or more dynamic characteristics of the at least one device, the dynamic characteristics including either a velocity or an acceleration of the at least one device. The state of the control system may, in some embodiments, include an indication of the at least one device transitioning from a steady state to a dynamic state, or one or more of a position of the at least one device, a shape of the at least one device's surrounding environment, an area of ​​the at least one device's surrounding environment, an area sensed by the at least one device, or an area accessible to the at least one device.

[0163] For example, referring to FIG. 6 , in some aspects, a state of a control system may be determined in 602. The state of the control system determined in 602 may include the state of a set of terminal devices and a communication network providing communication between the control system and the terminal devices (e.g., the state of a set of network relay devices, wireless base stations, I / O control devices, etc.). In some aspects, the state of the terminal device may include an identification of the type of device (e.g., a localized robotic agent, an autonomous object, or other controllable device), a variable characteristic of the terminal device, such as velocity and acceleration, an indication of whether the terminal device is in a dynamic state or a steady state, information about the environment of the terminal device, the location of the terminal device, information about a set of other objects in the vicinity of the terminal device (e.g., object type, distance, relative velocity, relative acceleration, and / or other relevant information). The state of the communication network may include an identification of available paths and / or routes to different terminal devices via the communication network, and resources (e.g., used and available resources) associated with each path and / or route via the communication network. In some aspects, the determination at 602 is made based on a determination and / or detection that a triggering event (eg, an event that triggers resource control) has occurred at 601.

[0164] At 1802, the analysis device may allocate a set of time synchronization resources for transmitting time synchronization information from at least one of the plurality of synchronized devices based on the state of each of the plurality of synchronized devices identified at 1801. In some aspects, the allocated set of time synchronization resources may include one or more of time resources within a set of slots, frequency resources within a frequency range, bandwidth resources, and communication path resources, where the set of time resources is associated with one or more of a wired communication network or a wireless communication network. The set of time synchronization resources may, in some aspects, include a set of logging parameters for recording sensor data associated with the at least one device. In some aspects, the logging parameters may relate to one or more of: a duration of each data recording event of the plurality of data recording events; a frequency of the data recording events in the plurality of data recording events; a resolution of sensor data collected during each data recording event of the plurality of data recording events; or an allocation of data storage for sensor data collected during one or more data recording events in the plurality of data recording events. In some aspects, an existing set of logging parameters prior to the allocation at 1802 may be changed (e.g., updated) based on the state of the at least one device or control system due to the allocation at 1802.

[0165] In some aspects, the set of logging parameters in the set of time synchronization resources may be a first set of logging parameters that instructs at least one device among the plurality of devices to start data recording for a data recording event simultaneously with at least one other device among the plurality of devices starting data recording for the data recording event. The set of time synchronization resources may, in some aspects, include a second set of logging parameters for the at least one other device that indicates a frequency of data recording events that is either equal to the frequency of data recording events instructed in the first set of logging parameters or an integer multiple of the frequency of data recording events instructed in the first set of logging parameters. In some aspects, the resources controlled by the resource control device may include a packet transmission frequency in a time synchronization protocol, an event sampling frequency, an event sampling resolution, and / or an event storage capacity allocation. In some aspects, the resources controlled by the resource control device may include a set of time slots and / or frequency bands in wireless communication, time slots for time-division communication such as TSN, and / or a route selection for synchronization packets in terms of synchronization accuracy and communication latency. For example, with reference to FIGS. 6 and 13, the analysis device may perform resource allocation associated with a resource control operation at 604 or 1306 .

[0166] At 1803, the analysis device may transmit a control message to at least one device of the plurality of devices indicating the determined allocated set of time synchronization resources for transmitting time synchronization information from the at least one device. The control message transmitted to the at least one device at 1803 may include an indication of any of the time synchronization communication resources, the time synchronization recording and / or reporting resources, or the set of control commands. For example, with reference to FIGS. 6 and 13, the analysis device may transmit the control message as part of a resource control operation at 604 or when exporting a configuration at 1307.

[0167] At 1804, the analysis device may send a request for information recorded by the at least one device. In some embodiments, the request for information may be implied in a set of logging parameters in a set of time synchronization resources. For example, with reference to FIGS. 6 and 13, the analysis device may export a configuration at 1307 to update components of the control system (e.g., as part of the resource control operation of 604). Finally, at 1805, the analysis device may receive information recorded by the at least one device based on the logging parameters. In some embodiments, receiving the information recorded by the at least one device may be based on the request sent at 1804. For example, with reference to FIGS. 6 and 13, the analysis device may receive information recorded by the at least one device based on the logging parameters as part of determining a state of the control system at 602 and / or obtaining event information at 1301.

[0168] FIG. 19 is a flow diagram 1900 illustrating a method according to some embodiments of the present disclosure. In some embodiments, the method is performed by an analysis device (e.g., analysis device 1262 or computing device 2005) that performs various analyses, control algorithm and / or application modifications, simulations, and / or emulation based on recorded events. The method illustrated in flow diagram 1900 may, in some embodiments, be performed in parallel or as part of the methods illustrated in flow diagrams 1700 and 1800. At 1901, the analysis device may analyze the behavior of at least one (simulated) device based on at least one of the updated control algorithms of the at least one simulated device and information collected from the at least one device. In some embodiments, the analysis device may simulate one or more components of the control system based on state information identified (e.g., 1701 and / or 1801) and / or received (e.g., 1805) by the analysis device and may generate updated control algorithms based on initial results of the simulation. Based on the state information and the updated control algorithms, one or more subsequent simulations may be performed to generate additional updated control algorithms to improve the results of the simulation. For example, with reference to Figure 13, the analysis device may replay the stored events from the information collected through the simulation or emulation at 1303, the complementary information generated at 1302, the configuration and associated control logic at 1306, and the simulated environment for the next iteration of the simulation and / or emulation.

[0169] At 1902, the analysis device sends updated control algorithm information to the at least one device based on the simulated behavior of the at least one simulated device. The updated control algorithm information may include an updated set of time synchronization resources (e.g., including time synchronization communication resources, time synchronization recording and / or reporting resources, and / or logic associated with a set of control algorithms and / or control commands). For example, referring to FIG. 13 , the analysis device may export a configuration at 1307 to update components of the control system.

[0170] Finally, at 1903, the analysis device transmits the updated control algorithm to at least one other device based on the simulated behavior of the at least one simulated device. In some embodiments, the updated control algorithm may be transmitted to at least one other device based on the at least one other device satisfying a set of conditions associated with the at least one simulated device. As described above, an updated control configuration generated for or based on an initial terminal device may, in some embodiments, be transmitted to additional terminal devices having the same functionality. For example, with reference to FIG. 13, an improved control algorithm generated by the analysis process shown in FIG. 13 may be delivered to another device different from the original source device of the analyzed log.

[0171] As described above, in some embodiments, resource allocation for logging is controlled according to the state of the control system, humans, robots, autonomous mobile objects, and control devices for recording behavior within the control system, so that the actions of multiple entities and interactions between the entities within the control system are accurately recorded. Furthermore, the recorded behavior and / or interactions are verified and optimized through analysis using simulations in a virtual space to improve system operation. This enables safety confirmation and optimization of control system operation, resulting in improved safety, reliability, performance, and operational efficiency.

[0172] 20 illustrates an example computing environment including a computing device suitable for use in some implementations. The computing device 2005 of the computing environment 2000 may include one or more processing units, cores, or processors 2010, memory 2015 (e.g., RAM, ROM, etc.), internal storage 2020 (e.g., magnetic, optical, solid-state storage, and / or organic), and / or an I / O interface 2025, any of which may be coupled to a communication mechanism or bus 2030 for communicating information or may be incorporated into the computing device 2005. The I / O interface 2025 may be configured to receive images from a camera or to provide images to a projector or display, depending on the desired implementation.

[0173] The computing device 2005 may be communicatively coupled to an input / user interface 2035 and an output device / interface 2040. Either or both of the input / user interface 2035 and the output device / interface 2040 may be wired or wireless interfaces and may be detachable. The input / user interface 2035 may include any device, component, sensor, or physical or virtual interface that may be used to provide input (e.g., buttons, touchscreen interface, keyboard, pointing / cursor controller, microphone, camera, Braille, motion sensor, accelerometer, optical reader, etc.). The output device / interface 2040 may include a display, television, monitor, printer, speaker, Braille, etc. In some implementations, the input / user interface 2035 and the output device / interface 2040 are incorporated into the computing device 2005 or are physically coupled to the computing device 2005. In other implementations, other computing devices may function as or provide the functionality of the input / user interface 2035 and output device / interface 2040 of the computing device 2005 .

[0174] Examples of computing devices 2005 may include, but are not limited to, highly mobile devices (e.g., smartphones, devices mounted on vehicles or other machines, devices carried by humans or animals, etc.), mobile devices (e.g., tablets, notebooks, laptops, personal computers, portable televisions, radios, etc.), and devices not designed for mobility (e.g., desktop computers, other computers, information kiosks, televisions, radios with one or more processors embedded and / or coupled thereto, etc.).

[0175] Computing device 2005 may be communicatively coupled (e.g., via I / O interface 2025) to external storage 2045 and to a network 2050 for communicating with any number of networked components, devices, and systems, including one or more computing devices of the same or different configurations. Computing device 2005 or any connected computing device may function, provide services, or be referred to as a server, client, thin server, general-purpose machine, special-purpose machine, or another label.

[0176] I / O interface 2025 may include, but is not limited to, wired and / or wireless interfaces using any communication or I / O protocol or standard (e.g., Ethernet, 802.11x, Universal Serial Bus, WiMax, modem, cellular network protocols, etc.) to communicate information to and / or from at least all connected components, devices, and networks within computing environment 2000. Network 2050 may be any network or combination of networks (e.g., the Internet, a local area network, a wide area network, a telephone network, a cellular network, a satellite network, etc.).

[0177] The computing device 2005 can use and / or communicate using computer-usable and computer-readable media, including transitory and non-transitory media. Transitory media include transmission media (metallic cables, optical fibers), signals, carrier waves, etc. Non-transitory media include magnetic media (e.g., disks, tapes), optical media (e.g., CD-ROMs, digital video disks, Blu-ray disks), solid-state media (RAM, ROM, flash memory, solid-state storage), and other non-volatile storage or memory.

[0178] The computing device 2005 may be used in some exemplary computing environments to implement techniques, methods, applications, processes, or computer-executable instructions. The computer-executable instructions may be obtained from, stored on, or obtained from a transitory medium. The executable instructions may be from one or more of any programming language, scripting language, and machine language (e.g., C, C++, C#, Java, Visual Basic, Python, Perl, JavaScript, etc.).

[0179] The processor(s) 2010 may run under any operating system (OS) (not shown) in a native or virtual environment. One or more applications may be deployed, including a logic unit 2060, an application programming interface (API) unit 2065, an input unit 2070, an output unit 2075, and an inter-unit communication mechanism 2095 through which different units communicate with each other, with the OS, and with other applications (not shown). The described units and elements may vary in design, function, configuration, or implementation and are not limited to the provided description. The processor(s) 2010 may be in the form of a hardware processor, such as a central processing unit (CPU), or may be a combination of hardware and software units.

[0180] In some examples, when information or instructions to execute are received by API unit 2065, the information may be communicated to one or more other units (e.g., logic unit 2060, input unit 2070, output unit 2075). In some cases, logic unit 2060 may be configured to control the flow of information between units and direct the services provided by API unit 2065, input unit 2070, and output unit 2075 in some implementations described above. For example, the flow of one or more processes or implementations may be controlled by logic unit 2060 alone or in conjunction with API unit 2065. Input unit 2070 may be configured to obtain inputs for the calculations described in this implementation, and output unit 2075 may be configured to provide outputs based on the calculations described in this implementation.

[0181] The processor(s) 2010 may be configured to identify states of the plurality of synchronized devices at a particular time. The processor(s) 2010 may be configured to allocate a set of time synchronization resources for transmitting time synchronization information from at least one device of the plurality of devices based on the identified states of each of the plurality of synchronized devices. The processor(s) 2010 may be configured to transmit, to at least one device of the plurality of devices, a control message indicating the determined allocated set of time synchronization resources for transmitting time synchronization information from the at least one device. The processor(s) 2010 may be configured to receive information recorded by the at least one device based on a set of logging parameters for recording sensor data associated with the at least one device. The processor(s) 2010 may be configured to send a request for information recorded by the at least one device. The processor(s) 2010 may be configured to calculate forces and moments for each time step during response measurement based on the second intermediate matrix and the modal characteristics.

[0182] The processor(s) 2010 may also be configured to simulate the behavior of the at least one simulated device based on at least one or more of the updated control algorithm of the at least one simulated device and the information collected from the at least one device. The processor(s) 2010 may also be configured to send updated control algorithm information to the at least one device based on the simulated behavior of the at least one simulated device. The processor(s) 2010 may also be configured to send the updated control algorithm to the at least one other device based on the simulated behavior of the at least one simulated device if the state of the at least one other device satisfies a set of conditions associated with the at least one simulated device.

[0183] Some portions of the detailed descriptions are presented in terms of algorithms and symbolic representations of operations within a computer. These algorithms and symbolic representations are the means used by those skilled in the data processing arts to convey the substance of their innovations to others skilled in the art. An algorithm is a sequence of defined steps leading to a desired end state or result. The implementation involves the manipulation of physical quantities to produce a tangible result.

[0184] Unless otherwise indicated, it will be apparent from this description that throughout this description, discussions using terms such as "processing," "computing," "calculating," "determining," "displaying," and the like may include operations and processes of a computer system or other information processing that manipulate and transform data represented as physical (electronic) quantities in the registers and memory of the computer system into other data similarly represented as physical quantities in the memory and registers of the computer system or other information storage, transmission, or display device.

[0185] Implementations may also relate to apparatuses for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may include one or more general-purpose computers selectively activated or reconfigured by one or more computer programs. Such computer programs may be stored on computer-readable media, such as computer-readable storage media or computer-readable signal media. Computer-readable storage media include, but are not limited to, tangible media such as optical disks, magnetic disks, read-only memory, random-access memory, solid-state devices, drives, and any other type of tangible and non-transitory medium suitable for storing electronic information. Computer-readable signal media include media such as carrier waves. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. A computer program may include a pure software implementation containing instructions for performing the operations of a desired implementation.

[0186] Although various general-purpose systems can be used with the programs and modules according to the embodiments herein, it may prove convenient to construct more specialized apparatus to perform the desired method steps. Moreover, the implementations are not described with reference to any particular programming language. It will be understood that a variety of programming languages ​​can be used to implement the teachings of the implementations described herein. The instructions of the programming language may be executed by one or more processing devices, such as a central processing unit (CPU), processor, or controller.

[0187] As is known in the art, the operations described above may be performed by hardware, software, or a combination of software and hardware. While various aspects of the implementations may be implemented using circuits or logic devices (hardware), other aspects may be implemented using instructions stored on a machine-readable medium (software) that, when executed by a processor, cause the processor to form a method for performing the implementation of the present application. Moreover, while some implementations of the present application may be implemented exclusively in hardware, other implementations may be implemented exclusively in software. Furthermore, the various functions described may be performed in a single unit or across multiple components in any number of ways. When implemented by software, the method may be executed by a processor, such as a general-purpose computer, based on instructions stored on a computer-readable medium. If desired, the instructions may be stored on the medium in a compressed and / or encrypted format.

[0188] Furthermore, other implementations of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the teachings herein. Elements of the various aspects and / or implementations described may be used alone or in any combination. It is intended that the specification and implementations be considered as examples only, with the true scope and spirit of the present application being indicated by the following claims. [Explanation of symbols]

[0189] 120 Resource Control Unit 122 Control Network 123a~123c Terminal devices 125a~125d I / O control devices 200, 300 devices 201 Central Processing Unit (CPU) 202 Communication control integrated circuits (ICs) 203 Physical layer (PHY) 204 memory 205 Non-volatile storage media 206 Bus 301 CPU 302 Communication Control IC 304 memory 305 Non-volatile storage media 306 Bus 307 I / O unit 308a, 308b baseband processing units 309a, 309b Radio Frequency (RF) Processing Unit 310a, 310b antennas 400 devices 430 Communication Unit 431 Time Synchronization Unit 432 Processing Unit 433 Input Unit 434 Output Unit 435 Log Registration Unit 436 Log Storage 437 Log Acquisition Unit 540 Control System State Determination Unit 1150 Robot Controller 1151 Industrial Robot Arm 1152 Joint Motor 1155 workers 1156 Curtain Sensor 1157 AGV 1158 Collaboration Area 1250a, 1250b robot controller 1251a, 1251b Industrial Robot Arm 1253 Belt conveyor 1254 parts 1259 Time Synchronization Master 1261 Table 1601, 1611 Drone 1602 shelf 2000 Computing Environment 2005 Computer Devices 2010 processor 2035 Input / User Interface 2040 Output Device / Interface 2060 Logic Unit 2065 API units 2070 Input Unit 2075 output unit 2095 Inter-unit communication mechanism

Claims

1. identifying a state of a plurality of synchronized devices at a particular time; determining a frequency for transmitting time synchronization information from at least one of the plurality of synchronized devices based on the identified state of each of the plurality of synchronized devices, and allocating a set of time synchronization resources for transmitting the time synchronization information according to the determined frequency; transmitting a control message to the at least one device among the plurality of synchronized devices indicating the allocated set of time synchronization resources determined for transmitting the time synchronization information from the at least one device; The state is the device includes at least one of the type of the device, the speed or acceleration of the device, an indication of whether the device is in a dynamic state or a stationary state, information about the environment of the device, the location of the device, and information about objects in the vicinity of the device; The information of the object is A method including at least one of the type of object, the distance to the device, the relative velocity to the device, and the relative acceleration to the device.

2. 2. The method of claim 1, wherein the set of time synchronization resources includes one or more of time resources within a set of slots, frequency resources in a frequency range, bandwidth resources, or communication route resources, and the set of time synchronization resources is associated with one or more of a wired communication network or a wireless communication network.

3. the step of allocating the set of time synchronization resources further comprises: allocating the set of time synchronization resources based on one or more of a type of at least one other device among the plurality of synchronized devices, a presence of a human, a distance to the at least one other device or the human, a relative velocity between the at least one device and the at least one other device or the human, or a relative acceleration between the at least one device and the at least one other device or the human; 2. The method of claim 1, wherein allocating the set of time synchronization resources is based on one or more of a type of the at least one device, an ambient environment of the at least one device, or a change in a relationship of the at least one device to at least one other device among the plurality of synchronized devices.

4. the step of allocating the set of time synchronization resources further comprises: based on one or more of a shape of an environment of the at least one device, an area of ​​the environment of the at least one device, an area sensed by the at least one device, or an area accessible to the at least one device; 2. The method of claim 1, wherein allocating the set of time synchronization resources is based on one or more of a type of the at least one device, an ambient environment of the at least one device, or a change in a relationship of the at least one device to at least one other device among the plurality of synchronized devices.

5. the step of allocating the set of time synchronization resources further comprises:

2. The method of claim 1, wherein allocating the set of time synchronization resources is based on one or more of a type of the at least one device, an ambient environment of the at least one device, or a change in a relationship of the at least one device to at least one other device among the plurality of synchronized devices.

6. The set of time synchronization resources includes a set of logging parameters for recording sensor data associated with the at least one device, and the method further comprises: The method of claim 1 , comprising receiving information logged by the at least one device based on the set of logging parameters.

7. 7. The method of claim 6, wherein the set of logging parameters relates to one or more of: a duration of each data recording event of a plurality of data recording events; a frequency of the data recording events of the plurality of data recording events; a resolution of sensor data collected during each data recording event of the plurality of data recording events; or a data storage allocation for the sensor data collected during one or more of the data recording events of the plurality of data recording events.

8. 8. The method of claim 7, wherein the set of logging parameters is a first set of logging parameters that indicates that at least one device in the plurality of synchronized devices begins recording data for the data recording event at the same time that at least one other device in the plurality of synchronized devices begins recording data for the data recording event, and a second set of logging parameters for the at least one other device indicates a frequency of the data recording event, the frequency being either equal to the frequency of the data recording event indicated in the first set of logging parameters or an integer multiple of the frequency of the data recording event indicated in the first set of logging parameters.

9. The method of claim 6 , further comprising the step of sending a request for the information recorded by the at least one device, and receiving the information recorded by the at least one device is based on the request.

10. 7. The method of claim 6, wherein the information recorded by the at least one device includes a converted time based on a first time stored by the at least one device and a reference time difference.

11. The method of claim 6 , wherein the information recorded by the at least one device includes an index of time synchronization accuracy of the information recorded by the at least one device.

12. further simulating the behavior of at least one simulated device based on one or more of the updated control algorithm of the at least one simulated device and the information collected from the at least one device; and transmitting updated control algorithm information to the at least one simulated device based on the simulated behavior of the at least one simulated device.

13. 13. The method of claim 12, further comprising transmitting the updated control algorithm to the at least one other device based on the simulated behavior of the at least one simulated device if a state of the at least one other device satisfies a set of conditions associated with the at least one simulated device.

14. Memory and at least one processor coupled to the memory, wherein the at least one processor performs, based at least in part on the information stored in the memory: Identifying the state of multiple synchronized devices at a particular time; determining a frequency for transmitting time synchronization information from at least one of the plurality of synchronized devices based on the identified state of each of the plurality of synchronized devices, and allocating a set of time synchronization resources for transmitting the time synchronization information according to the determined frequency; sending a control message to the at least one device of the plurality of synchronized devices indicating the assigned set of time synchronization resources determined for transmitting the time synchronization information from the at least one device; The state is the device includes at least one of the type of the device, the speed or acceleration of the device, an indication of whether the device is in a dynamic state or a stationary state, information about the environment of the device, the location of the device, and information about objects in the vicinity of the device; The information of the object is The apparatus is configured to include at least one of the type of the object, the distance to the device, the relative velocity to the device, and the relative acceleration to the device.

15. A computer-readable medium having computer-executable code stored thereon, the computer-executable code, when executed by a processor, causing the processor to: Identifying the state of multiple synchronized devices at a particular time; determining a frequency for transmitting time synchronization information from at least one of the plurality of synchronized devices based on the identified state of each of the plurality of synchronized devices, and allocating a set of time synchronization resources for transmitting the time synchronization information in accordance with the determined frequency; causing the at least one device of the plurality of synchronized devices to transmit a control message indicating the assigned set of time synchronization resources determined for transmitting the time synchronization information from the at least one device; The state is the device includes at least one of the type of the device, the speed or acceleration of the device, an indication of whether the device is in a dynamic state or a stationary state, information about the environment of the device, the location of the device, and information about objects in the vicinity of the device; The information of the object is A computer-readable medium configured to include at least one of the type of object, the distance to the device, the relative velocity to the device, and the relative acceleration to the device.

Images