Processing system, extended operating apparatus, and space system

Abstract

To provide a control system capable of improving efficiency of arithmetic processing using an estimation model.SOLUTION: A control system 2 includes an arithmetic unit 11. The arithmetic unit 11 is configured to execute arithmetic processing for controlling an actuator using an estimation model 41, based on the change of state of a space acquired from a sensor 20. The arithmetic unit 11 is allowed to use an extended operating apparatus 80 configured to execute arithmetic processing using the estimation model 41. The arithmetic unit 11 executes a part of the arithmetic processing when the extended operating apparatus 80 is available, and causes the extended operating apparatus 80 to execute the rest of the processing.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a control system, an extended computing device, and a spatial system, and more particularly to a control system, an extended computing device, and a spatial system that perform spatial control. [Background technology]

[0002] Patent Document 1 discloses an inference device that performs inference using an inference model (estimation model). The inference device of Patent Document 1 uses the inference model to infer, for example, whether or not a product has a defect based on image data of an image of the product. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2022 / 190966 Summary of the Invention [Problem to be solved by the invention]

[0004] However, recently, there has been a demand for more efficient calculation processing using estimation models.

[0005] The present disclosure aims to provide a control system, an extended calculation device, and a spatial system that can improve the efficiency of calculation processing using an estimation model. [Means for solving the problem]

[0006] A control system according to an embodiment of the present disclosure includes a calculation unit. The calculation unit is configured to execute a calculation process for controlling an actuator using an estimation model based on a state change of a space acquired from a sensor. The calculation unit can use an extended calculation device configured to execute the calculation process using the estimation model. When the calculation unit can use the extended calculation device, the calculation unit executes a part of the calculation process and causes the extended calculation device to execute the rest of the calculation process.

[0007] According to an aspect of the present disclosure, an extended arithmetic unit is available from the control system. When the extended arithmetic unit is available from the arithmetic unit of the control system, the extended arithmetic unit does not execute the part of the arithmetic processing using the partial model, and executes the rest of the arithmetic processing using a partial model of the neural network from the predetermined intermediate layer to the output layer.

[0008] According to another aspect of the present disclosure, an extended arithmetic device is available from the control system, and when available from the arithmetic unit of the control system, the extended arithmetic device does not execute the part of the arithmetic processing using the partial model, and executes the remainder of the arithmetic processing using a partial model of the neural network from the input layer to the predetermined intermediate layer.

[0009] A spatial system according to one aspect of the present disclosure includes a first control system and a second control system. The first control system and the second control system are the control systems. The calculation unit of the first control system can use the calculation unit of the second control system as the extended calculation device. The calculation unit of the first control system executes the part of the calculation process of the first control system when the calculation unit of the second control system is available. The calculation unit of the second control system executes the rest of the calculation process of the first control system when used by the calculation unit of the first control system. Effect of the Invention

[0010] According to a control system, an extended calculation device, and a spatial system according to one aspect of the present disclosure, it is possible to improve the efficiency of calculation processing using an estimation model. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing the configuration of a spatial system including a control system according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram showing the configuration of the control system of the above embodiment. [Diagram 3] FIG. 3 is a block diagram showing a configuration of an actuator in the control system. [Figure 4] FIG. 4 is a schematic diagram of an estimation model and a partial model in the above control system. [Diagram 5] FIG. 5 is a flowchart showing the operation of the control system. [Figure 6] FIG. 6 is a block diagram showing a configuration of a spatial system according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Below, a control system, an extended computing device, and a spatial system according to aspects of the present disclosure will be described with reference to the drawings.

[0013] (Embodiment 1) (1) Overall structure As shown in Fig. 1, a control system 2 according to the first embodiment is used in a spatial system 1. The spatial system 1 includes a control system 2 and a plurality of (two in Fig. 1) actuators 30. As shown in Figs. 1 and 2, the control system 2 includes a processing unit 10 and one or more (two in Figs. 1 and 2) sensors 20. The control system 2 performs arithmetic processing using an estimation model 41 (see Figs. 2 and 4) based on state changes in the space AR1 detected by the plurality of sensors 20. The control system 2 controls the plurality of actuators 30 provided in the space AR1 based on the results of the arithmetic processing.

[0014] Each of the multiple actuators 30 is a device that exhibits a behavior that brings about a change in the state of the space AR1. In this disclosure, the actuator 30 refers to a device that causes a change in the state of the space AR1, such as a change in airflow, sound, or brightness, and is not limited to a device that performs mechanical motion. In other words, the actuator 30 refers to a device that changes the state of the space AR1 based on the result of arithmetic processing in the control system 2.

[0015] 3, each of the actuators 30 includes an operation unit 31 that acts on the space AR1, a communication unit 32, and a control unit 33. Each of the actuators 30 is, for example, a lighting fixture, an air conditioner, a speaker, a display device, or the like.

[0016] The operation unit 31 changes at least one of, for example, lighting, air conditioning, air flow, sound, and image. Specifically, the operation unit 31 is, for example, a light-emitting unit of a lighting fixture, an indoor unit of an air conditioner, a sound-producing unit of a speaker, a display unit of a display device, etc. The operation unit 31 operates according to a control instruction received from the control system 2 via the communication unit 32.

[0017] The communication unit 32 is a communication interface that enables communication with the control system 2, such as Bluetooth (registered trademark), BLE (Bluetooth Low Energy), wireless LAN, or wired LAN.

[0018] When the control unit 33 receives a control instruction from the control system 2, it controls the operation unit 31 to perform an operation based on the control instruction.

[0019] (2) Control system configuration As shown in FIG. 2, the control system 2 includes a processing unit 10 and one or more sensors 20 (two sensors in FIGS. 1 and 2).

[0020] (2.1) Sensors Each of the multiple sensors 20 detects a change in the state of the space AR1. The change in the state of the space AR1 refers to, for example, a change in the number of people in the space AR1, human activity, room temperature, brightness, etc. The multiple sensors 20 acquire information indicating the activity of people in the space AR1, such as an image acquired from the space AR1, or a sound acquired from the space AR1. Alternatively, the multiple sensors 20 acquire information indicating the state in the space AR1, such as room temperature, brightness, etc.

[0021] The sensor 20 is, for example, a condition detection device such as a camera, a microphone, a brightness sensor, or a temperature sensor.

[0022] (2.2) Processing section The processing unit 10 receives a signal indicating the state of the space AR1 from the sensor 20, and performs calculation processing using an estimation model 41 (see FIG. 4) based on the received signal. The processing unit 10 controls a plurality of actuators 30 provided in the space AR1 based on the result of the calculation processing. For example, the processing unit 10 estimates the number of people entering the space AR1 based on the signal indicating an image of the space AR1 from the sensor 20, and controls the actuator 30, which is an air conditioner, according to the number of people in the space AR1.

[0023] As shown in FIG. 2, the processing unit 10 includes a calculation unit 11, a storage unit 12, a communication unit 13, and a control unit 14.

[0024] The calculation unit 11 is configured to execute a calculation process using an estimation model 41 based on a signal received from the sensor 20. The result of the calculation process is used to control the actuator 30. The content of the calculation process relates to at least one of, for example, human density, human movement, environmental data, and energy consumption. In addition, the calculation process using the estimation model 41 is a calculation process related to at least one of human emotion, human activity level, human sensible temperature, and abnormality inference. For example, when the sensor 20 outputs an image as the state of the space AR1, the calculation process includes one or more of classification, detection, semantic segmentation, and instance.

[0025] As shown in FIG. 4, the estimation model 41 is a neural network model including an input layer 50, an output layer 70, and a plurality of intermediate layers 60 (two intermediate layers 61 and 62 in FIG. 4).

[0026] 2, the calculation unit 11 acquires a signal required for the calculation process from the sensor 20, and acquires an estimation model 41 used for the calculation process from the storage unit 12. Then, the calculation unit 11 inputs the signal acquired from the sensor 20 to an input layer 50 of the estimation model 41, and calculates output data of an output layer 70.

[0027] The calculation unit 11 includes hardware for calculation processing, such as a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

[0028] The calculation unit 11 can use the extended calculation device 80. More specifically, the calculation unit 11 is configured to be connectable to the extended calculation device 80. Here, the extended calculation device 80 refers to a device that can be connected to the control system 2 and has a function of executing calculation processing using the estimation model 41. The extended calculation device 80 includes hardware for calculation processing, such as a CPU, a GPU, an FPGA, or an ASIC. Here, the calculation unit 11 being able to use the extended calculation device 80 refers to a state in which the calculation unit 11 can cause the extended calculation device 80 to execute a part of the calculation processing and obtain the result.

[0029] When the processing unit 11 is able to use the extended arithmetic device 80, it executes a part of the arithmetic processing and makes the extended arithmetic device 80 execute the remainder of the arithmetic processing. When the processing unit 11 is able to use the extended arithmetic device 80, it does not execute the remainder of the arithmetic processing that it would like the extended arithmetic device 80 to execute.

[0030] When the extended arithmetic unit 80 is unavailable, the arithmetic unit 11 performs arithmetic processing using the estimation model 41 based on the signal received from the sensor 20. On the other hand, when the extended arithmetic unit 80 is available, the arithmetic unit 11 performs part of the arithmetic processing based on the signal received from the sensor 20 using a partial model 43 that is a part of the estimation model 41. The partial model 43 is a part of the estimation model 41 from the input layer 50 to a predetermined intermediate layer 62. The extended arithmetic unit 80 performs the rest of the arithmetic processing using a partial model 42 that is a part of the estimation model 41. The partial model 42 is a part of the estimation model 41 from the predetermined intermediate layer 62 to the output layer 70.

[0031] The storage unit 12 is a storage medium that stores an estimation model 41 and partial models 42 and 43 that are parts of the estimation model 41.

[0032] The communication unit 13 is a communication interface that enables communication with the actuator 30 and other control systems 2. The communication unit 13 is, for example, an interface such as Bluetooth, BLE, wireless LAN, or wired LAN.

[0033] The control unit 14 acquires a signal indicating the state of the space AR1 from the sensor 20. The control unit 14 causes the calculation unit 11 to perform calculation processing using the estimation model 41 based on the signal received from the sensor 20. The control unit 14 also transmits an instruction to the actuator 30 based on the calculation result of the calculation unit 11.

[0034] The processing unit 10 and the extended arithmetic unit 80 include a computer system. The computer system is mainly composed of a processor and a memory as hardware. The processor executes a program recorded in the memory of the computer system, thereby realizing the function of the processing unit 10 in the present disclosure. The program may be pre-recorded in the memory of the computer system, may be provided through an electric communication line, or may be provided by being recorded in a non-transitory recording medium such as a memory card, an optical disk, or a hard disk drive that can be read by the computer system. The processor of the computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large-scale integrated circuit (LSI). The integrated circuits such as IC or LSI referred to here are called by different names depending on the degree of integration, and include integrated circuits called system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). Furthermore, FPGAs, which are programmed after the manufacture of LSIs, or logic devices that can reconfigure the connection relationship inside the LSI or reconfigure the circuit partition inside the LSI can also be adopted as processors. The electronic circuits may be integrated in one chip, or may be distributed among multiple chips. The multiple chips may be integrated into one device, or may be distributed across multiple devices. The computer system referred to here includes a microcontroller having one or more processors and one or more memories. Therefore, the microcontroller is also composed of one or more electronic circuits including a semiconductor integrated circuit or a large-scale integrated circuit.

[0035] Furthermore, it is not essential for the processing unit 10 that the multiple functions of the processing unit 10 and the extended arithmetic device 80 are integrated into one housing, and the components of the processing unit 10 may be distributed across multiple housings. For example, the sensor 20 may be provided in a housing different from that of the processing unit 10.

[0036] (3) Operation FIG. 5 is a flowchart showing a series of operations based on detection of a state change in the space AR1 by the sensor 20 in the control system 2 according to the first embodiment.

[0037] The control unit 14 of the control system 2 determines whether or not the calculation unit 11 can use the extended calculation device 80 (step S11).

[0038] When the extended calculation device 80 is available (Yes in step S11), the calculation unit 11 executes a part of the calculation process based on the detection of a state change in the space AR1 by the sensor 20 (step S12). More specifically, the calculation unit 11 executes a calculation process using the partial model 43, which is a part of the calculation process using the estimation model 41. The calculation unit 11 inputs the state change in the space AR1 acquired from the sensor 20 to the input layer of the partial model 43, which corresponds to the input layer 50 of the estimation model 41. Then, the calculation unit 11 calculates intermediate data as data output from the output layer of the partial model 43, which corresponds to the intermediate layer 62 of the estimation model 41.

[0039] The calculation unit 11 then delegates the remainder of the calculation process to the extended calculation device 80 (step S13). More specifically, the calculation unit 11 outputs intermediate data that is the output of the partial model 43, and the partial model 42. The extended calculation device 80 inputs the intermediate data to the input layer of the partial model 42, which corresponds to the intermediate layer 62 of the estimation model 41. Then, the extended calculation device 80 calculates output data as data output from the output layer of the partial model 42, which corresponds to the output layer 70 of the estimation model 41. The calculation unit 11 obtains the output data from the extended calculation device 80 as a calculation result using the estimation model 41.

[0040] Next, the control unit 14 controls the actuator 30 based on the output data (step S14). The control unit 14 controls the actuator 30 based on the state of the space AR1 estimated as the output data. For example, when the control unit 14 estimates that the sensible temperature of the people in the space AR1 is rising, the control unit 14 instructs the actuator 30, which is an air conditioner, to lower the set temperature. Also, for example, when the control unit 14 estimates that the activity level of the people in the space AR1 is decreasing, the control unit 14 instructs the actuator 30, which is a display device, to display an image.

[0041] On the other hand, if the calculation unit 11 cannot use the extended calculation device 80 (No in step S11), the calculation unit 11 executes all of the calculation processes based on the state changes of the space AR1 by the sensor 20 (step S15). The calculation unit 11 inputs the state changes of the space AR1 acquired from the sensor 20 to the input layer 50 of the estimation model 41. Then, the calculation unit 11 calculates output data as data to be output from the output layer 70 of the estimation model 41. The subsequent processes are as in step S14 described above.

[0042] (4) Effects The control system 2 according to the first embodiment includes a calculation unit 11. The calculation unit 11 is configured to execute calculation processing for controlling the actuator 30 using an estimation model 41 based on a state change of the space AR1 acquired from the sensor 20. The calculation unit 11 can use an extended calculation device 80 configured to execute calculation processing using the estimation model 41. When the extended calculation device 80 is available, the calculation unit 11 executes a part of the calculation processing and makes the extended calculation device 80 execute the rest. As a result, when the extended calculation device 80 is available, the calculation unit 11 can share the calculation processing between the calculation unit 11 and the extended calculation device 80. Therefore, the calculation resources of the control system 2 can be used efficiently.

[0043] Moreover, the control system 2 according to the first embodiment can acquire state changes from the sensor 20. The estimation model 41 is a neural network including an input layer 50, an output layer 70, and an intermediate layer 60. When the extended arithmetic device 80 is available, the arithmetic unit 11 performs part of the arithmetic processing using a partial model 43 from the input layer 50 to a predetermined intermediate layer 62 of the neural network. This allows the arithmetic unit 11 and the extended arithmetic device 80 to share the arithmetic processing so as not to affect the calculation result. Furthermore, the arithmetic unit 11 performs the arithmetic processing based on the state change of the space acquired from the sensor 20, so that the arithmetic unit 11 can start the arithmetic processing promptly. Furthermore, the arithmetic unit 11 outputs the result of the arithmetic processing based on the state change of the space acquired from the sensor 20 to the extended arithmetic device 80 and causes the extended arithmetic device 80 to perform the rest of the arithmetic processing. Therefore, the amount of data transferred between the control system 2 and the extended arithmetic device 80 can be reduced.

[0044] Moreover, the extended arithmetic device 80 according to the first embodiment is available from the control system 2. When the extended arithmetic device 80 is available from the arithmetic unit 11 of the control system 2, the extended arithmetic device 80 does not execute a part of the arithmetic processing using the partial model 43, and executes the rest of the arithmetic processing using the partial model 42 from a predetermined intermediate layer 62 to the output layer 70 of the neural network 41. As a result, the extended arithmetic device 80 performs the arithmetic processing using the partial model 42 based on the arithmetic result by the arithmetic unit 11 of the control system 2, thereby calculating the final result of the arithmetic processing using the estimated model 41. Therefore, it is possible to reduce the amount of data transferred between the control system 2 and the extended arithmetic device 80, and to improve the efficiency of the arithmetic processing by the control system 2 and the extended arithmetic device 80.

[0045] (Embodiment 2) (1) Composition When the extended arithmetic device 80 is available, the control system 2 according to the second embodiment divides data indicating a state change of the space AR1 acquired from the sensor 20 into two partial data. The calculation unit 11 performs a calculation using the estimation model 41 based on one of the two partial data, and the extended arithmetic device 80 performs a calculation using the estimation model 41 based on the other of the two partial data.

[0046] In the control system 2 according to the second embodiment, the sensor 20 is a video camera. The sensor 20 outputs a video including a plurality of frames obtained by capturing an image of the space AR1 as a state change of the space AR1.

[0047] The calculation unit 11 performs calculations using the estimation model 41 based on the video data acquired from the sensor 20. The calculation unit 11 divides the video data including a plurality of frames acquired from the sensor 20 into first partial data including one or more frames and second partial data including one or more frames. More specifically, the calculation unit 11 divides the video data into the first partial data including one or more frames that are a part of the plurality of frames, and the second partial data including one or more frames that are the remainder of the plurality of frames. For example, the first partial data includes odd-numbered frames of the video data, and the second partial data includes even-numbered frames of the video data.

[0048] The calculation unit 11 performs a part of the calculation process using the estimation model 41 based on the video data, using the estimation model 41 based on the first partial data. The calculation unit 11 also causes the extended calculation device 80 to perform the remaining part of the calculation process using the estimation model 41 based on the video data, using the estimation model 41 based on the second partial data. That is, the calculation unit 11 performs a part of the calculation process based on the state change of the space AR1 acquired from the sensor 20, using the first partial data. The extended calculation device 80 performs the remaining part of the calculation process based on the state change of the space AR1 acquired from the sensor 20, using the second partial data. The calculation unit 11 generates a result of the calculation process based on the state change of the space AR1 acquired from the sensor 20, based on a part of the result of the calculation process based on the first partial data performed by the calculation unit 11 itself and the remaining result of the calculation process based on the second partial data performed by the extended calculation device 80.

[0049] (2) Effects The control system 2 according to the second embodiment can acquire a state change from the sensor 20. The sensor 20 outputs a video including a plurality of frames as a state change. The arithmetic processing is based on the video. When the arithmetic unit 11 can use the extended arithmetic unit 80, the arithmetic unit 11 divides the video into one or more frames used for a part of the arithmetic processing and one or more frames used for the remainder of the arithmetic processing. This allows the arithmetic unit 11 and the extended arithmetic unit 80 to share the arithmetic processing without affecting the calculation result. In addition, the arithmetic processing performed by the arithmetic unit 11 and the arithmetic processing performed by the extended arithmetic unit 80 are not dependent on each other. Therefore, the control system 2 and the extended arithmetic unit 80 can perform a part and the remainder of the arithmetic processing in parallel, and therefore the arithmetic processing can be performed efficiently.

[0050] (Embodiment 3) (1) Composition In the third embodiment, as shown in FIG. 6, the space system 1 includes a plurality of control systems 2. The space system 1 includes control systems 2a and 2b. The control system 2a performs a calculation process using an estimation model 41 based on state changes in the space AR1 detected by the plurality of sensors 20. The control system 2a controls a plurality of actuators 30 provided in the space AR1 based on the results of the calculation process. The control system 2b performs a calculation process using an estimation model 41 (see FIG. 2) based on state changes in the space AR2 detected by the plurality of sensors 20. The control system 2b controls a plurality of actuators 30 provided in the space AR2 based on the results of the calculation process.

[0051] A control system 2 functions as an extended arithmetic device 80 with respect to another control system 2. Below, a case where a control system 2b functions as the extended arithmetic device 80 with respect to a control system 2a will be described. The control system 2b has the same configuration as the control system 2a. Here, the fact that the control system 2b has the same configuration as the control system 2a means that the calculation unit 11 of the control system 2b performs arithmetic processing using an estimation model 41 based on state changes in the space AR2 detected by the multiple sensors 20.

[0052] When the processing margin of the calculation unit 11 is large, the control unit 14 of the control system 2b notifies the control system 2a via the communication unit 13 that the control system 2b is capable of operating as the extended calculation device 80. The processing margin of the calculation unit 11 is a parameter indicating how much margin there is in the calculation capacity of the calculation unit 11. More specifically, the processing margin of the calculation unit 11 is the proportion of time per unit time during which the calculation unit 11 is not performing calculations.

[0053] When the calculation unit 11 of the control system 2a receives a notification from the control system 2b that the control system 2a can operate as the extended calculation device 80, the calculation unit 11 uses the control system 2b as the extended calculation device 80. The calculation unit 11 of the control system 2a executes a part of the calculation process based on the detection of a state change in the space AR1 by the sensor 20. More specifically, the calculation unit 11 of the control system 2a executes a calculation process using the partial model 43 among the calculation processes using the estimation model 41. The calculation unit 11 of the control system 2a inputs the state change in the space AR1 acquired from the sensor 20 to the input layer of the partial model 43, which corresponds to the input layer 50 of the estimation model 41. Then, the calculation unit 11 of the control system 2a calculates intermediate data as data output from the output layer of the partial model 43, which corresponds to the intermediate layer 62 of the estimation model 41.

[0054] The calculation unit 11 of the control system 2a delegates the remaining calculation processing to the control system 2b, which is the extended calculation device 80. More specifically, the calculation unit 11 of the control system 2a outputs intermediate data, which is the output of the partial model 43, and the partial model 42 to the control system 2b.

[0055] The calculation unit 11 of the control system 2b executes the rest of the calculation process based on the state change detection of the space AR1 using the intermediate data and the partial model 42. At this time, the calculation unit 11 of the control system 2b operates as the calculation unit 11 of the control system 2b and as the extended calculation device 80 of the control system 2a in a time-sharing manner. In other words, the time during which the calculation unit 11 of the control system 2b operates includes a period during which the calculation unit 11 of the control system 2b performs calculation processing and a period during which the calculation unit 11 of the control system 2a performs calculation processing. For example, when the processing margin of the calculation unit 11 of the control system 2b is 60%, the ratio of the time during which the calculation unit 11 of the control system 2b operates as the calculation unit 11 of the control system 2b to the time during which the calculation unit 11 of the control system 2b operates as the extended calculation device 80 of the control system 2a is 4:6. As a result, the calculation unit 11 of the control system 2b can perform calculation processing as the calculation unit 11 of the control system 2b, and then take over the calculation of the calculation unit 11 of the control system 2a using the free time. Therefore, the calculation efficiency can be improved in the entire control systems 2a and 2b.

[0056] The calculation unit 11 of the control system 2b inputs the intermediate data received from the control system 2a to the input layer of the partial model 42, which corresponds to the intermediate layer 62 of the estimation model 41. Then, the calculation unit 11 of the control system 2b calculates output data as data output from the output layer of the partial model 42, which corresponds to the output layer 70 of the estimation model 41. The calculation unit 11 of the control system 2a obtains the output data from the calculation unit 11 of the control system 2b as a calculation result using the estimation model 41.

[0057] The calculation unit 11 of the control system 2b can use the extended calculation device 80. The extended calculation device 80 of the control system 2b may be the control system 2a or the extended calculation device 80 according to the first embodiment.

[0058] (2) Effects The control system 2 (2b) of the third embodiment can operate as the extended arithmetic device 80 of the control system 2 (2a) having the same configuration as the control system 2 (2b). When the control system 2b operates as the extended arithmetic device 80 of the control system 2a, the period during which the arithmetic unit 11 of the control system 2b operates includes a period during which arithmetic processing is performed as the arithmetic unit 11 of the control system 2b and a period during which arithmetic processing is performed as the extended arithmetic device 80 of the control system 2a. This allows the multiple control systems 2 to take over arithmetic processing for each other, thereby making the arithmetic processing more efficient.

[0059] The spatial system 1 of the third embodiment includes a control system 2a and a control system 2b. The calculation unit 11 of the control system 2a can use the calculation unit 11 of the control system 2b as an extended calculation device 80. When the calculation unit 11 of the control system 2a can use the calculation unit 11 of the control system 2b, the calculation unit 11 of the control system 2a executes a part of the calculation processing of the control system 2a. When the calculation unit 11 of the control system 2b is used by the calculation unit 11 of the control system 2a, the calculation unit 11 of the control system 2b executes the rest of the calculation processing of the control system 2a. This allows the multiple control systems 2 included in the spatial system 1 to take over the calculation processing for each other, thereby making the calculation processing more efficient.

[0060] (Embodiment 4) (1) Composition The control system 2 according to the fourth embodiment is capable of communicating with the actuator 30. When the calculation unit 11 of the control system 2 is able to use the extended calculation device 80, the calculation unit 11 executes a part of the calculation process using the estimation model 41, using a partial model 42 from a predetermined intermediate layer 62 to an output layer 70. The extended calculation device 80 executes the rest of the calculation process using the estimation model 41, using a partial model 43 from the input layer 50 to the predetermined intermediate layer 62.

[0061] Before the calculation unit 11 itself performs a part of the calculation process, it delegates the remainder of the calculation process to the extension calculation device 80. More specifically, the calculation unit 11 outputs the state change of the space AR1 acquired from the sensor 20 and the partial model 43. The extension calculation device 80 inputs the state change of the space AR1 acquired from the sensor 20 to the input layer of the partial model 43, which corresponds to the input layer 50 of the estimation model 41. Then, the extension calculation device 80 calculates intermediate data as data output from the output layer of the partial model 43, which corresponds to the intermediate layer 62 of the estimation model 41.

[0062] The calculation unit 11 then executes a part of the calculation process. More specifically, the calculation unit 11 acquires intermediate data which is the remaining result of the calculation process performed by the extended calculation device 80, and inputs the intermediate data to the input layer of the partial model 42 which corresponds to the intermediate layer 62 of the estimation model 41. Then, the calculation unit 11 calculates output data as data to be output from the output layer of the partial model 42 which corresponds to the output layer 70 of the estimation model 41.

[0063] The control unit 14 of the control system 2 controls the actuator 30 based on the result of the calculation process performed by the calculation unit 11 .

[0064] (2) Effects The control system 2 according to the fourth embodiment is capable of communicating with the actuator 30. The estimation model 41 is a neural network including an input layer 50, an output layer 70, and an intermediate layer 60. When the extended arithmetic device 80 is available, the calculation unit 11 executes a part of the arithmetic processing using a partial model 42 from a predetermined intermediate layer 62 to the output layer 70 of the neural network. As a result, regardless of whether the calculation unit 11 can use the extended arithmetic device 80 or not, the calculation unit 11 itself calculates the calculation result of the arithmetic processing using the estimation model 41. Therefore, in the control system 2, when the control unit 14 controls the actuator 30 based on the result of the arithmetic processing using the estimation model 41, it is not necessary to transfer the result of the arithmetic processing from the extended arithmetic device 80 to the control system 2.

[0065] Moreover, the extended arithmetic device 80 according to the fourth embodiment can be used from the control system 2. When the extended arithmetic device 80 is available from the arithmetic unit 11 of the control system 2, it does not perform calculations using the partial models 42, but performs calculations using the partial models 43 from the input layer 50 to a predetermined intermediate layer 62 of the neural network. This allows the extended arithmetic device 80 to calculate intermediate data to be output to the arithmetic unit 11 by performing calculation processing using the partial models 43. This makes it possible to reduce the amount of data transferred between the control system 2 and the extended arithmetic device 80, and to improve the efficiency of calculation processing by the control system 2 and the extended arithmetic device 80.

[0066] (Modification) (1) Composition In a modification of the fourth embodiment, similarly to the third embodiment, the spatial system 1 includes a plurality of control systems 2. Each control system 2 functions as an extended arithmetic device 80 with respect to the other control systems 2. Below, a case where the control system 2b functions as the extended arithmetic device 80 with respect to the control system 2a will be described.

[0067] When the processing margin of the calculation unit 11 is large, the control unit 14 of the control system 2b notifies the control system 2a via the communication unit 13 that it is capable of operating as an extended calculation device 80.

[0068] When the calculation unit 11 of the control system 2a receives a notification from the control system 2b that the control system 2a can operate as the extended calculation device 80, the calculation unit 11 uses the control system 2b as the extended calculation device 80. The calculation unit 11 of the control system 2a executes a part of the calculation process based on the detection of a state change of the space AR1 by the sensor 20. More specifically, the calculation unit 11 causes the control system 2b operating as the extended calculation device 80 to execute a calculation process using the partial model 43 among the calculation processes using the estimation model 41. The calculation unit 11 of the control system 2b inputs the state change of the space AR1 acquired from the sensor 20 to the input layer of the partial model 43 corresponding to the input layer 50 of the estimation model 41. Then, the calculation unit 11 of the control system 2b calculates intermediate data as data output from the output layer of the partial model 43 corresponding to the intermediate layer 62 of the estimation model 41.

[0069] The calculation unit 11 of the control system 2a uses the intermediate data and the partial model 42 to execute the remaining part of the calculation process.

[0070] (2) Effects The spatial system 1 of the modified example of the fourth embodiment includes a control system 2a and a control system 2b. The calculation unit 11 of the control system 2a can use the calculation unit 11 of the control system 2b as an extended calculation device 80. When the calculation unit 11 of the control system 2a is available, the calculation unit 11 of the control system 2a executes a part of the calculation processing of the control system 2a. When the calculation unit 11 of the control system 2b is used by the calculation unit 11 of the control system 2a, the calculation unit 11 of the control system 2b executes the rest of the calculation processing of the control system 2a. This allows the multiple control systems 2 included in the spatial system 1 to take over the calculation processing for each other, thereby making the calculation processing more efficient.

[0071] (Other Modifications of the Embodiment) (1) In the first to fourth embodiments and the modified examples, the control system 2 includes two sensors 20, but the number of sensors 20 may be one or three or more. Moreover, the control system 2 may not include the sensor 20, but may be configured to be able to communicate with the sensor 20.

[0072] (2) In embodiments 1 to 4 and the modified examples, the control system 2 delegates part of the calculation processing to the extended calculation device 80, but the control system 2 may divide the calculation processing into three or more parts and delegate two or more of them to two or more extended calculation devices 80.

[0073] (3) In the modified examples of the third and fourth embodiments, the control system 2 capable of operating as the extended arithmetic device 80 includes the sensor 20, but the control system 2 capable of operating as the extended arithmetic device 80 may not include the sensor 20. In addition, the control system 2 according to the fourth embodiment includes the sensor 20, but may not include the sensor 20.

[0074] (4) In the third embodiment and the modified example of the fourth embodiment, the estimation model 41 is divided by the predetermined intermediate layer 62, but the intermediate layer 60 of the estimation model 41 may have three or more intermediate layers. In addition, the predetermined intermediate layer may be the intermediate layer 61, or when the intermediate layer 60 includes three or more intermediate layers, it may be any one of them.

[0075] (Aspect) The control system (2) according to the first aspect includes a calculation unit (11). The calculation unit (11) is configured to execute calculation processing for controlling an actuator (30) using an estimation model (41) based on a state change of a space (AR1) acquired from a sensor (20). The calculation unit (11) can use an extended calculation device (80) configured to execute calculation processing using the estimation model (41). When the calculation unit (11) can use the extended calculation device (80), the calculation unit (11) executes a part of the calculation processing and has the extended calculation device (80) execute the rest of the calculation processing.

[0076] According to the control system (2) of the above aspect, when the calculation unit (11) is able to use the extended calculation device (80), the calculation processing can be shared between the calculation unit (11) and the extended calculation device (80). This allows efficient use of the calculation resources of the control system (2).

[0077] The control system (2) according to the second aspect is the first aspect and is capable of acquiring a state change from the sensor (20). The estimation model (41) is a neural network including an input layer (50), an output layer (70), and at least one intermediate layer (62). When the extended calculation device (80) is available, the calculation unit (11) executes a part of the calculation process using a partial model (43) of the neural network from the input layer (50) to a predetermined intermediate layer (62).

[0078] According to the control system (2) of the above aspect, the calculation unit (11) and the extended calculation device (80) can share the calculation processing so as not to affect the calculation result. Furthermore, since the calculation unit (11) performs the calculation processing based on the state change of the space acquired from the sensor (20), the calculation unit (11) can promptly start the calculation processing. Furthermore, the calculation unit (11) outputs the result of the calculation processing based on the state change of the space acquired from the sensor (20) to the extended calculation device (80) to cause the extended calculation device (80) to perform the rest of the calculation processing. Therefore, the amount of data transferred between the control system (2) and the extended calculation device (80) can be reduced.

[0079] The extended arithmetic device (80) according to the third embodiment can be used by the control system (2) according to the second embodiment. When the extended arithmetic device (80) is used by the calculation unit (11) of the control system (2), the extended arithmetic device (80) does not execute a part of the arithmetic processing using the partial model (43), and executes the rest of the arithmetic processing using the partial model (42) from a predetermined intermediate layer (62) to an output layer (70) of the neural network.

[0080] According to the extended arithmetic device 80 of the above aspect, it is possible to calculate a final result of the arithmetic processing using the estimation model 41 by performing arithmetic processing using the partial model 42 based on the calculation result by the calculation unit 11 of the control system 2. Therefore, it is possible to reduce the amount of data transferred between the control system 2 and the extended arithmetic device 80, and to improve the efficiency of the arithmetic processing by the control system 2 and the extended arithmetic device 80.

[0081] A control system (2) according to a fourth aspect is capable of acquiring a state change from a sensor in the first aspect. The sensor (20) outputs a video including a plurality of frames as the state change. The arithmetic processing is arithmetic processing based on the video. When the extended arithmetic device (80) is available, the arithmetic unit (11) divides the video into one or more frames used for part of the arithmetic processing and one or more frames used for the remainder of the arithmetic processing.

[0082] According to the control system (2) of the above aspect, the calculation unit (11) and the extended calculation device (80) can share the calculation processing without affecting the calculation result. Furthermore, there is no mutual dependency between the part of the calculation processing performed by the calculation unit (11) and the rest of the calculation processing performed by the extended calculation device (80). Therefore, the control system (2) and the extended calculation device (80) can perform the part and the rest of the calculation processing in parallel, and therefore the calculation processing can be performed efficiently.

[0083] The control system (2) according to the fifth aspect is the first, second or fourth aspect, in which the calculation unit (11) can operate as an extended calculation device (80) of a control system (2a) having the same configuration as a first control system (2b) which is the control system (2). When the calculation unit (11) of the control system (2b) operates as the extended calculation device (80) of the second control system (2a), the period during which the calculation unit (11) operates includes a first period during which the calculation unit (11) performs calculation processing as the calculation unit (11) of the first control system (2b) and a second period during which the calculation unit (11) performs calculation processing as the extended calculation device (80) of the second control system (2a).

[0084] According to the control system (2) of the above aspect, the multiple control systems (2) can take over the arithmetic processing of each other, thereby making it possible to improve the efficiency of the arithmetic processing.

[0085] The control system (2) according to the sixth aspect is capable of communicating with the actuator (30) in the first aspect. The estimation model (41) includes an input layer (50), an output layer (70), and at least one intermediate layer (62). When the extended calculation device (80) is available, the calculation unit (11) executes a part of the calculation process using a partial model of the neural network from a predetermined intermediate layer (62) to the output layer (70).

[0086] According to the control system (2) of the above aspect, when controlling the actuator (30) based on the result of the calculation processing using the estimation model (41), there is no need to transfer the result of the calculation processing from the extended calculation device (80) to the control system (2).

[0087] The extended arithmetic device (80) according to the seventh aspect is available to the control system (2) according to the sixth aspect. When the extended arithmetic device (80) is available to the calculation unit (11) of the control system (2), it does not execute a part of the arithmetic processing using the partial model (42), and executes the rest of the arithmetic processing using the partial model (43) from the input layer (50) to a predetermined intermediate layer (62) of the neural network.

[0088] According to the extended arithmetic device (80) of the above aspect, the extended arithmetic device (80) can calculate intermediate data to be output to the calculation unit (11) by performing arithmetic processing using the partial model (43). Therefore, it is possible to reduce the amount of data transferred between the control system (2) and the extended arithmetic device (80), and to improve the efficiency of arithmetic processing by the control system (2) and the extended arithmetic device (80).

[0089] The spatial system (1) according to the eighth aspect includes a first control system (2a) and a second control system (2b). The calculation unit (11) of the first control system (2a) can use the calculation unit (11) of the second control system (2b) as an extended calculation device (80). When the calculation unit (11) of the first control system (2a) is able to use the calculation unit (11) of the second control system (2b), the calculation unit (11) of the first control system (2a) executes a part of the calculation processing of the first control system (2a). When the calculation unit (11) of the second control system (2b) is used by the calculation unit (11) of the first control system (2a), the calculation unit (11) of the second control system (2b) executes the rest of the calculation processing of the first control system (2a).

[0090] According to the spatial system (1) of the above aspect, the calculation processing can be shared among the multiple control systems (2), so that the calculation processing can be made more efficient. [Explanation of symbols]

[0091] 1. Spatial Systems 2. Control System 2a Second control system, first control system 2b First control system, second control system 11 Arithmetic section 20 Sensors 30 Actuator 41 Estimation model 42 Partial Model 43 Partial Model 50 Input Layer 62 Middle Class 70 Output layer 80 Extended Arithmetic Unit AR1 space

Claims

1. a calculation unit configured to execute a calculation process for controlling the actuator using an estimation model based on a state change of the space acquired from the sensor; Equipped with The calculation unit is an extended computing device is available that is configured to perform the computation using the estimation model; When the extended arithmetic unit is available, a part of the arithmetic processing is executed, and the remaining part is executed by the extended arithmetic unit. Control system.

2. The state change can be obtained from the sensor; the estimation model is a neural network including an input layer, an output layer, and at least one hidden layer; the calculation unit executes the part of the calculation process by using a partial model of the neural network from the input layer to a predetermined intermediate layer when the extended calculation device is available. The control system of claim 1 .

3. Utilizable from a control system according to claim 2, when the partial model is available from the computing unit of the control system, the part of the computational process is not executed using the partial model, and the rest of the computational process is executed using a partial model of the neural network from the predetermined intermediate layer to the output layer. Extended Arithmetic Unit.

4. The state change can be obtained from the sensor; the sensor outputs a video including a plurality of frames as the state change; the arithmetic processing is arithmetic processing based on the moving image, the computing unit, when the extended computing device is available, divides the video into one or more frames used for the part of the computing process and one or more frames used for the remainder of the computing process; The control system of claim 1 .

5. The calculation unit is operable as the extended calculation device of a second control system having the same configuration as a first control system that is the control system, When the calculation unit operates as the extended calculation device of the second control system, a period during which the calculation unit operates includes a first period during which the calculation unit executes the calculation process as the calculation unit of the first control system, and a second period during which the calculation unit executes the calculation process as the extended calculation device of the second control system. The control system of claim 1 .

6. capable of communicating with the actuator; the estimation model is a neural network including an input layer, an output layer, and at least one hidden layer; the calculation unit executes the part of the calculation process by using a partial model of the neural network from a predetermined intermediate layer to the output layer when the extended calculation device is available. The control system of claim 1 .

7. 7. A control system according to claim 6, when the partial model is available from the computing unit of the control system, the part of the computational process is not executed using the partial model, and the rest of the computational process is executed using a partial model of the neural network from the input layer to the predetermined intermediate layer. Extended Arithmetic Unit.

8. A control system comprising a first control system and a second control system, the first control system and the second control system being the control system according to claim 1; The calculation unit of the first control system is The calculation unit of the second control system can be used as the extended calculation device, When the computing unit of the second control system is available, the part of the computing process of the first control system is executed; The computing unit of the second control system, when utilized by the computing unit of the first control system, executes the remainder of the computing process of the first control system. Spatial system.

Images