Control System

Abstract

To provide a control system that can improve real-time performance by calculating a margin time from the sensing time of an infrastructure sensor, a wireless communication time, and a reaction time of a control target, and controlling the execution order of applications and communications.SOLUTION: A control system 1 includes: mobile body detection units 11 and 12 that transmit information on a mobile body; and a flight control system 13 that receives the information on the mobile body and transmits control information to a control processing unit 152 of the mobile body based on the information on the mobile body. The flight control system 13 includes: a margin time calculation unit 135 that calculates a first margin time based on a reaction time of the mobile body and an estimated communication control necessary time; a processing management unit 136 that determines execution order of the control information; an application unit 137 that calculates the control information based on the information on the mobile body and the execution order determined by the processing management unit 136; and a transmission control unit 138 that calculates a second margin time based on the reaction time of the mobile body and an actual communication control necessary time, and transmits the control information to the control processing unit 152 according to the second margin time.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a technology for improving the real-time performance of a control system including wireless communication. [Background technology]

[0002] Control systems are becoming more autonomous in order to provide transportation and transport services for limited areas. Examples include vehicle-based transport systems within factory grounds, warehouse management systems using automated forklifts, AGVs (Automated Guided Vehicles), and AGVs with robotic arms, robot taxis and self-driving buses for specific areas, and self-driving trucks in mines.

[0003] In the fields where these services are provided, mobile objects that autonomously recognize, judge, and control themselves (autonomous mobile objects), mobile objects operated by humans (manned mobile objects), and people such as pedestrians and workers coexist and perform their respective tasks.

[0004] Because the sensing range of moving objects is limited, there is a demand for technology that can monitor the blind spots of moving objects using sensors (infrastructure sensors) installed on the road or within facilities, and that can notify moving objects and people (their devices) connected via wireless communication of recognition information, or instruct moving objects and people to act.

[0005] Patent Document 1 describes a technology in which infrastructure sensors recognize other moving objects or people hiding in blind spots of a moving object, and share or notify the relevant moving object of information. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-215785 A Summary of the Invention [Problem to be solved by the invention]

[0007] The technology described in Patent Document 1 is expected to improve safety for moving objects and people.

[0008] However, because wireless communication conditions vary depending on the location, such as the shadows of buildings and vehicles, communication times vary depending on the location of the moving object or person. Therefore, when sharing recognition information or issuing instructions to multiple moving objects or people, it takes time to communicate with moving objects or people with poor radio wave conditions, which may result in delays in sharing recognition information or issuing instructions to other moving objects or people.

[0009] Furthermore, the technology described in Patent Document 1 does not take into consideration situations in which there are different types of moving objects or moving objects and people in the field. The time it takes to share recognition information, react after receiving a control instruction, and finish an action (control reaction time) depends on the moving object and the person.

[0010] For example, when a stop command is sent to a person and a moving object that is different from the person, and the control reaction time required for the braking distance is taken into consideration, if the moving object requires a longer control reaction time than the person, if a collision warning is issued to both at the same time, the person will take longer to notice the warning than the moving object, but the final control reaction time will be shorter because the action completion time until the object is completely stopped is shorter.

[0011] In other words, in an area with a blind spot, if there is a risk of collision between a person with poor communication conditions and a mobile object with good communication conditions, there is a problem that sharing information in the order of the mobile object and the person will increase the risk of collision more than if the opposite were to happen.

[0012] In this way, when the operating environment of the control system is complex, it is difficult to set priorities in advance for the order in which processes and communications are to be executed.

[0013] The present invention has been made to solve the above-mentioned problems, and aims to provide a control system that calculates a slack time from the sensing time of an infrastructure sensor, the wireless communication time, and the reaction time of the controlled object, and controls the execution order of applications and communications based on the slack time, thereby improving real-time performance. [Means for solving the problem]

[0014] The control system of the present invention for solving the above problems is configured as follows.

[0015] The control system includes a mobile object detection unit that detects a mobile object and transmits information about the mobile object, and a control system that receives the information about the mobile object transmitted by the mobile object detection unit and transmits control information to a control processing unit of the mobile object based on the received information about the mobile object, the control system including a margin time calculation unit that calculates a first margin time based on an estimated communication control required time including a reaction time of the mobile object, and a margin time calculation unit that calculates a first margin time based on an estimated communication control required time including a reaction time of the mobile object, the first margin time being calculated in ascending order of the first margin time. NiA The application unit calculates the control information. Real The apparatus comprises a processing management unit that determines a row order, an application unit that calculates the control information based on the information of the mobile body and the execution order determined by the processing management unit, and a transmission control unit that calculates a second margin time based on an actual required time for communication control including a reaction time of the mobile body, and transmits the control information to the control processing unit in the order of shortest second margin time. Effect of the Invention

[0016] According to the control system of the present invention, a slack time is calculated from the sensing time of the infrastructure sensor, the wireless communication time, and the reaction time of the controlled object, and based on that, the execution order of applications and communications is controlled, thereby providing a control system that can improve real-time performance. [Brief description of the drawings]

[0017] [Figure 1] 1 is an overall schematic configuration diagram of a control system according to a first embodiment. [Diagram 2] 1 is an internal configuration diagram of a control system according to a first embodiment. [Diagram 3] 4 is a process flow of a recognition processing unit according to the first embodiment. [Figure 4] 4 is a process flow of a receiving unit according to the first embodiment. [Diagram 5] 11 is a process flow of an input elapsed time calculation unit according to the first embodiment. [Figure 6] 11 is a process flow of a communication performance measuring server processing unit according to the first embodiment. [Figure 7] 4 is a process flow of a control time estimation unit according to the first embodiment. [Figure 8] 1 is a process flow of a slack time calculation unit according to the first embodiment. [Figure 9] FIG. 4 is a diagram for explaining calculation of a slack time according to the first embodiment. [Figure 10] 4 is a process flow of a process management unit according to the first embodiment. [Figure 11] 4 is a process flow of an application unit according to the first embodiment. [Figure 12] 4 is a process flow of a transmission control unit according to the first embodiment. [Figure 13] 11 is a process flow of a communication performance measurement client processing unit according to the first embodiment. [Figure 14] 4 is a process flow of a control processing unit according to the first embodiment. [Figure 15] FIG. 11 is a diagram illustrating an example of thread priorities according to the first embodiment. [Figure 16] FIG. 4 is a diagram showing an example of object information according to the first embodiment. [Figure 17] FIG. 11 is a diagram illustrating an example of a control target information table according to the first embodiment. [Figure 18] FIG. 11 is an explanatory diagram of an example of execution timing (FIFO) of a control system, which is an example different from the first embodiment. [Figure 19] FIG. 2 is an explanatory diagram of an example of execution timing of the control system according to the first embodiment. [Figure 20] FIG. 11 is an explanatory diagram of another example of the execution timing (FIFO) of the control system, which is different from the first embodiment. [Figure 21] FIG. 11 is an explanatory diagram of another example of the execution timing of the control system according to the first embodiment. [Figure 22] FIG. 11 is an internal configuration diagram of a management system of a control system according to a second embodiment. [Diagram 23]13 is a process flow of a control time estimation unit according to the second embodiment. [Figure 24] 13 is a process flow of an application unit according to the second embodiment. [Diagram 25] FIG. 11 is a diagram showing an example of object information according to the second embodiment. [Figure 26] FIG. 11 is a diagram showing another example of object information according to the second embodiment. [Figure 27] FIG. 11 is a diagram illustrating an example of a control target information table according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] The control system of the present invention calculates a margin of time from the sensing time of the moving object detection unit, the wireless communication time, and the reaction time of the controlled object, and based on that, controls the execution order of applications and communications, thereby providing recognition information to the controlled object in real time or notifying it of action instructions.

[0019] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0020] In the embodiment, an infrastructure sensor is described as an example of the moving body detection unit, but the present invention is not limited to the infrastructure sensor and can be applied to any detection device that detects a moving body. EXAMPLES

[0021] Example 1 FIG. 1 is a schematic diagram of an entire configuration of a control system according to a first embodiment of the present invention.

[0022] In FIG. 1, a control system 1 is a system that controls an infrastructure sensor 11, an infrastructure sensor 12, a control system 13, wireless devices 14, an automobile 15, a bus 16, a pedestrian 17 carrying a wireless communication device 17TR, a pedestrian 18, and a traffic light 19.

[0023] FIG. 1 shows an explanatory portion 1A indicating that a pedestrian 17, a pedestrian 18, a traffic light 19, a car 15, and a bus 16 each have a different margin time.

[0024] The infrastructure sensor 11 and the infrastructure sensor 12 communicate with each other by wired communication or wireless communication. The wireless device 14 wirelessly communicates with the automobile 15, the bus 16, the pedestrian 17, and the traffic light 19. If the pedestrian 18 carries a communication device, the wireless device 14 wirelessly communicates with the wireless communication device of the pedestrian 18.

[0025] FIG. 2 is a diagram showing the internal configuration of the control system 13 according to the first embodiment, and is a diagram showing the process flow of each unit.

[0026] 2, a recognition processing unit 111 of the infrastructure sensors 11 and 12 analyzes the sensing content and transmits detected object information 13A to a receiving unit 131 of a control system 13. The times of the control system 13 and the infrastructure sensors 11 and 12 are synchronized by a time synchronization unit 112A of the control system 13 and a time synchronization unit 112B of the infrastructure sensors 11 and 12.

[0027] The receiving unit 131 transmits the object information 13A to the input elapsed time calculation unit 132, the control time estimation unit 134, and the application unit 137. The communication performance measuring server processing unit 133 transmits time information to the communication performance measuring client processing unit 151, and upon receiving the time information, the communication performance measuring client processing unit 151 returns the time information to the communication performance measuring server processing unit 133. The communication performance measuring server processing unit 133 calculates an estimated communication time based on the received time information and transmits it to the control time estimation unit 134.

[0028] The input elapsed time calculation unit 132 calculates a sensing elapsed time from the time information included in the object information 13A, and transmits it to the slack time calculation unit 135. The control time estimation unit 134 calculates an estimated control time from the object information 13A and the estimated communication time, and transmits it to the slack time calculation unit 135.

[0029] The slack time calculation unit 135 calculates the slack time from the sensing elapsed time and the estimated control time, and transmits it to the process management unit 136 .

[0030] The process management unit 136 starts the application unit 137 based on the margin time. The application unit 137 notifies the transmission control unit 138 of the control object information, the transmission data, and the margin time. The transmission control unit 138 transmits the transmission data according to the margin time to the control processing unit 152 of the control objects 15, 16, 17, and 19 according to the control object information.

[0031] The operation flow according to the first embodiment will be described in detail below.

[0032] Fig. 3 shows a processing flow of the recognition processing unit 111. Each step in Fig. 3 will be described below.

[0033] (Figure 3: Step 1111) The recognition processing unit 111 identifies an object from the sensing information of the infrastructure sensors 11 and 12, assigns a time stamp to the object, and transmits the information to the control system 13.

[0034] Fig. 4 shows a processing flow of the receiving unit 131. Each step in Fig. 4 will be described below.

[0035] (Figure 4: Step 1311) The receiver 131 acquires information from the infrastructure sensors 11 and 12, and transmits it to the application unit 137, the input elapsed time calculation unit 132, and the control time estimation unit . Fig. 5 shows a process flow of the input elapsed time calculation unit 132. Each step in Fig. 5 will be described below.

[0036] (Figure 5: Step 1321) The input elapsed time calculation unit 132 compares the time stamp information added to the received information with the reference time of the control system 13 to calculate the sensing elapsed time (T IS +T UL The result is sent to the slack time calculation unit 135.

[0037] Fig. 6 shows a processing flow of the communication performance measuring server processing unit 133. Each step in Fig. 6 will be described below.

[0038] (Figure 6: Step 1331) The communication performance measuring server processing unit 133 transmits the time information to each of the control targets 15, 16, 17 and 19.

[0039] (Figure 6: Step 1332) The communication performance measuring server processing unit 133 waits until it receives the time information sent from the controlled objects 15 , 16 , 17 and 19 .

[0040] (Figure 6: Step 1333) The communication performance measuring server processing unit 133 estimates the communication time with the controlled objects from the time information received from the controlled objects 15, 16, 17 and 19 and the current time, and updates the estimated communication time in the controlled object information table 13B.

[0041] Fig. 7 shows a process flow of the control time estimation unit 134. Each step in Fig. 7 will be described below.

[0042] (Figure 7: Step 1341) The control time estimation unit 134 refers to the control target information table 13B based on the object ID of the object information, acquires the reaction time and the estimated communication time of the control target, and calculates the estimated control time. The calculation result is transmitted to the slack time calculation unit 135.

[0043] The reaction time of the control target does not need to refer to the control target information table 13 B. For example, it may be estimated based on information on whether the control target is looking at the device or not, whether the control target is facing the direction of the moving object, or the age of the person, from the object information.

[0044] In other words, the moving body includes a pedestrian, the control processing unit 152 is a wireless communication device 17TR ​​carried by the pedestrian 17, and the control time estimation unit 134 can estimate the reaction time of the pedestrian 17 based on whether or not the pedestrian 17 is looking at the wireless communication device 17TR, or whether the pedestrian is facing in the direction of the moving body such as a car 15 or a bus 16, or on information about the age of the pedestrian 17.

[0045] Fig. 8 shows a process flow of the slack time calculation unit 135. Each step in Fig. 8 will be described below.

[0046] (Figure 8: Step 1351) The margin time calculation unit 135 calculates the margin time T1 from the sensing elapsed time and the estimated control time. Laxity In the first embodiment, the total latency (T E2EL ), E SA is treated as a fixed time.

[0047] The following equation (1) is the margin time T1 Laxity The following formula (2) is the calculation formula for the margin time T2 Laxity This is the calculation formula. T1 Laxity =T E2EL -(T IS +T UL +E SA +E DL +E VA ) ···(1) T2 Laxity =T E2EL -(T IS +T UL +T SA +E DL +E VA ) ···(2)

[0048] In the above formula (1), the margin time (T1 Laxity ) is the total latency (T E2EL ) to the sensing elapsed time (T IS +T UL ) and estimated application execution time (E SA ) and estimated communication time (E DL ) and the control reaction time (E VA ) can be calculated from the difference

[0049] In the above formula (2), the margin time (T2 Laxity ) is the total latency (T E2EL ) to the sensing elapsed time (T IS +T UL ) and actual application execution time (TSA ) and estimated communication time (E DL ) and the control reaction time (E VA ) can be calculated from the difference

[0050] Overall latency (T E2EL ) is the time from sensing to completion of a control instruction to a moving object or person (the time from when the moving object detection unit receives information about the moving object to when the moving object ends its action). Also, the overall latency is the time taken into consideration for the moving objects 15, 16, 17, and 19 to stop, or the reaction time from notifying the person 18 of an instruction or warning by image, sound, or vibration, to starting an action, and ending the action.

[0051] For example, this information may include the total time it takes for a moving object to be stopped by remote control (steering control), or the total time it takes for a person to be stopped by displaying, sounding, or vibrating a device carried by the person.

[0052] That is, the control information transmitted to the control processing unit 152 of the moving object is information on stopping or steering control of the vehicle, or information on images, sounds, and vibrations that give instructions or warnings to people.

[0053] FIG. 9 is a diagram illustrating the above formulas (1) and (2).

[0054] In FIG. 9, T IS is the sensing time (detection time) for the infrastructure sensors 11 and 12 to detect the control targets 15, 16, 17, and 19, and T UL is the uplink time of the sensor signal (detection signal) from the infrastructure sensors 11 and 12. E SA is the estimated application execution time of the application part 137, E DL is the estimated communication time between the control system 13 and the control processing unit 152 of the controlled object, E VA is the reaction time of the controlled object. The estimated communication time E DL and the controlled reaction time E VA The sum of these is the estimated control time.

[0055] In addition, the estimated control time (Estimated communication time E DL + Controlled object reaction time E VA ) and the estimated application execution time E SA and sensing time T IS and uplink time T UL The sum of and is the estimated required time for communication control.

[0056] In addition, the estimated control time (Estimated communication time E DL + Controlled object reaction time E VA ) and the actual application execution time T SA and sensing time T IS and the uplink time T UL The sum of and is the actual required communication control time.

[0057] Actual application execution time T SA is the time from when the application starts to when it finishes. Estimated application execution time E SA is the actual application execution time T SA The first margin time T1 is set to a longer time. Laxity is the time from when the sensor signal is uplinked to when the application is executed, and the second margin time T2 Laxity is the time from when the application ends to when communication starts.

[0058] Fig. 10 shows a process flow of the process management unit 136. Each step in Fig. 10 will be described below.

[0059] (Figure 10: Step 1361) The processing management unit 136 has a margin time T1 Laxity Based on this, the execution order of the application unit 137 is determined (the processing execution order of the control information is determined). For example, Laxity The execution order of the application unit 137 is determined so that the calculation of the control information is executed in the order of the shortest margin time T1. LaxityIf the value is negative, the process may not be executed. If the value is negative, the process result is often meaningless, so the stability of the entire control system is improved by canceling the process and prioritizing the execution of other processes.

[0060] Fig. 11 shows a processing flow of the application unit 137. Each step in Fig. 11 will be described below.

[0061] (Figure 11: Step 1371) The application unit 137 executes a control application based on the object information.

[0062] (Figure 11: Step 1372) The application section 137 receives the control object, the transmission data, and the margin time T1 Laxity is transmitted to the transmission control unit 138.

[0063] Fig. 12 shows a processing flow of the transmission control unit 138. Each step in Fig. 12 will be described below.

[0064] (Figure 12: Step 1381) The transmission control unit 138 calculates a margin time T2 from the sensing elapsed time, the estimated control time, and the actual application execution time. Laxity Calculate.

[0065] (Figure 12: Step 1382) The transmission control unit 138 determines the margin time T2 Laxity The transmission control unit 138 determines the order of transmission of the transmission data according to the margin time T2 Laxity The control information is transmitted to the control processing unit 152 in the order of the shortest margin time T2. Laxity If the value is negative, the transmitted data may be discarded. If the value is negative, the transmitted data is often meaningless, so the stability of the entire control system can be improved by canceling the transmission of the data and giving priority to other transmitted data.

[0066] Alternatively, measures to shorten the communication time may be implemented. For example, instead of retransmission-type communication using ACK, a method of continuously transmitting the same transmission packet in advance or a method of using a different communication path may be used.

[0067] (Figure 12: Step 1383) The transmission control unit 138 transmits data with a high priority. For example, the margin time T2 Laxity Sends short transmission data.

[0068] Fig. 13 shows a processing flow of the communication performance measurement client processing unit 151. Each step in Fig. 13 will be described below.

[0069] (Figure 13: Step 1511) When the communication performance measurement client processing unit 151 receives time information from the control system 13, the communication performance measurement client processing unit 151 performs the communication performance measurement of the control system 13 without changing the time information. server A reply is sent to the processing unit 133.

[0070] Fig. 14 shows a processing flow of the control processing unit 152. Each step in Fig. 14 will be described below.

[0071] (Figure 14: Step 1521) The control processing unit 152 executes control processing based on the transmission data from the transmission processing unit 138. For example, if the controlled object is an automobile 15 or a bus 16, it executes processing to stop the vehicle from traveling. If the controlled object is a pedestrian 17, it indirectly controls the behavior by vibrating or warning the device the pedestrian is carrying and displaying a stop command on the screen.

[0072] When the controlled object is a traffic light 19, the color of the traffic light is changed to a given color (for example, red).

[0073] FIG. 15 is a diagram showing an example of thread priorities in the control system 13.

[0074] In FIG. 15, a communication performance measurement thread 1330 executes a communication performance measurement server processing unit 133 .

[0075] The reception processing thread 1310 executes the reception unit 131 .

[0076] The slack time calculation thread 1350 executes the input elapsed time calculation unit 132 , the control time estimation unit 134 , the slack time calculation unit 135 , and the process management unit 136 .

[0077] The application thread 1370 executes the application portion 137 .

[0078] The transmission processing thread 1380 executes the transmission control unit 138 .

[0079] FIG. 16 is a diagram showing an example of the object information 13A.

[0080] 16, object information 13A is composed of an object type, an object ID, and a timestamp given by the infrastructure sensors 11 and 12. Fig. 16 shows an example in which the sensed information as the object type is vehicle A. Also, an example in which the object ID is 1 and the timestamp is 12:34:000 milliseconds is shown.

[0081] FIG. 17 is a diagram showing an example of the control target information table 13B.

[0082] In FIG. 17, the control object information table 13B is made up of an object ID, an item, a reaction time, an IP address, and an estimated communication time.

[0083] FIG. 18 is a diagram showing an example of the execution timing of a control system when the execution order of application unit 137 is determined based on the commonly used FIFO (First In First Out) without using any slack time, unlike in the first embodiment, with the passage of time indicated horizontally.

[0084] The example shown in FIG. 18 is an example in which two CPUs, a first CPU and a second CPU, are used.

[0085] 18, a first CPU executes a reception processing thread 1310 and a margin calculation thread 1350 in the order in which two pieces of object information are received, and a second CPU executes an application thread A 13701 (long margin time) and an application thread B 13702 (short margin time). A first CPU executes a transmission processing thread 1380 in the order in which two pieces of object information are received.

[0086] Then, for the object information received earlier, transmission data 13D1 (long margin time) is transmitted, and control process thread 1421 (vehicle A) is executed. Also, for the object information received later, transmission data 13D2 (short margin time) is transmitted, and control process thread 1422 (pedestrian B) is executed.

[0087] In the example shown in FIG. 18, the control system 13 executes the application unit 137 in the order in which the instructions are received, so that the instruction to the vehicle A is given first, and the instruction to the pedestrian B, who has a longer communication time and reaction time, is delayed.

[0088] The example shown in FIG. 19 is an example in the case of the first embodiment of the present invention, and is a diagram showing an example of the execution timing of the control system executed based on the slack time.

[0089] The example shown in FIG. 19 is also an example using two CPUs, a first CPU and a second CPU, like the example shown in FIG.

[0090] In FIG. 19, a first CPU executes a reception processing thread 1310 and a margin time calculation thread 1350 in the order in which two pieces of object information are received, and a second CPU executes an application thread A 13701 (long margin time) and an application thread B 13702 (short margin time).

[0091] The execution of the application thread A 13701 (long margin time) and the application thread B 13702 (short margin time) is adjusted so that the processing of the later received object information precedes the processing of the earlier received object information.

[0092] In the transmission processing thread 1380, the first CPU processes, of the two pieces of object information, the object information with the shorter margin time first, and the object information with the longer margin time second.

[0093] Then, for object information with a long margin time, transmission data 13D1 (long margin time) is transmitted, and control processing thread 1421 (vehicle A) is executed. Also, for object information with a short margin time, transmission data 13D2 (short margin time) is transmitted, and control processing thread 1422 (pedestrian B) is executed prior to control processing thread 1421 (vehicle A).

[0094] 19, the control system 13 gives priority to the execution of the application thread B 13702 having a shorter margin of time. This enables the control processing thread 1422 of the pedestrian B to be executed earlier than the control processing thread 1421 of the vehicle A (vehicle A).

[0095] FIG. 20 is a diagram showing another example of the execution timing of the control system when the execution order of the application unit 137 is determined based on the commonly used FIFO (First In First Out) without using any slack time, unlike in the first embodiment, with the passage of time indicated horizontally.

[0096] The example shown in FIG. 20 is an example using two CPUs, a first CPU and a second CPU, similar to the example shown in FIG.

[0097] 20, a first CPU executes a reception processing thread 1310 and a margin calculation thread 1350 in the order in which the two pieces of object information are received, and a second CPU executes an application thread A 13701 (long margin time) and an application thread B 13702 (short margin time). A first CPU executes a transmission processing thread 1380 in the order in which the two pieces of object information are received.

[0098] Then, for the object information received earlier, transmission data 13D1 (long margin time) is transmitted, and control process thread 1421 (vehicle A) is executed. Also, for the object information received later, transmission data 13D2 (short margin time) is transmitted, and control process thread 1422 (pedestrian B) is executed.

[0099] In the example shown in FIG. 20, the control system 13 executes the application unit 137 in the order in which the instructions are received, so that the instruction to the vehicle A is given first, and the instruction to the pedestrian B, who has a longer communication time and reaction time, is delayed.

[0100] The example shown in FIG. 21 is another example in the case of the first embodiment of the present invention, and is a diagram showing an example of the execution timing of the control system executed based on the slack time.

[0101] The example shown in FIG. 21 is also an example using two CPUs, a first CPU and a second CPU, like the example shown in FIG.

[0102] In FIG. 21, a first CPU executes a reception processing thread 1310 and a margin time calculation thread 1350 in the order in which two pieces of object information are received, and a second CPU executes an application thread A 13701 (long margin time) and an application thread B 13702 (short margin time).

[0103] In the transmission processing thread 1380, the first CPU processes the two pieces of object information in the order in which they are received.

[0104] Then, for the object information with a long margin time received earlier, transmission data 13D1 (long margin time) is delayed, and for the object information with a short margin time received later, transmission data 13D2 (short margin time) is transmitted before transmission data 13D1 (long margin time). As a result, control processing thread 1422 (pedestrian B) is executed before control processing thread 1421 (vehicle A).

[0105] In the first embodiment of the present invention shown in Fig. 21, the control system 13 gives priority to executing the application thread B 13702 having a shorter margin of time. This makes it possible to execute the control processing thread 1422 of pedestrian B earlier than the control processing thread 1421 of vehicle A (vehicle A).

[0106] As described above, according to the first embodiment of the present invention, the order of applications and transmission data is determined based on the margin time taking into consideration the wireless communication time with a specific control target and the reaction time of the control target. Therefore, even if the wireless communication time with a moving object or a person fluctuates, the order can be dynamically updated, thereby improving real-time performance.

[0107] In other words, a control system can be provided that calculates a slack time from the sensing time of the infrastructure sensor, the wireless communication time, and the response time of the controlled object, and based on that, controls the execution order of applications and communications, thereby improving real-time performance.

[0108] According to the first embodiment of the present invention, it is possible to improve the real-time nature of information sharing and control instructions among autonomous moving bodies, manned moving bodies, and people in the field.

[0109] In the first embodiment, the reaction time is fixed, but the present invention is not limited to this. For example, the reaction time may be changed according to the situation, such as shortening the reaction time when a pedestrian is looking at the screen of the device.

[0110] In addition, the situation where the change occurs may be due to the moving speed of the controlled object or the sensing distance of the infrastructure sensor. For example, if the application is a remote emergency stop control, the faster the moving speed, the longer the braking distance until the object stops, so the longer the reaction time in anticipation of this.

[0111] In the first embodiment, the overall latency is a fixed value, but is not limited to this. For example, it may vary depending on the moving speed or sensing distance of the controlled object. Specifically, the faster the moving speed is and the shorter the sensing distance is, the shorter the overall latency may be.

[0112] According to the first embodiment, application processing with a short margin of time is given priority for execution, and therefore the present invention can be applied to different types of applications whose priority level changes depending on the situation.

[0113] Furthermore, according to the first embodiment, the transmission process of the transmission data with a short margin time is executed with priority, so that the present embodiment can be applied to different types of data communications in which the priority level changes depending on the situation.

[0114] Moreover, according to the first embodiment, by estimating the wireless communication time with the control target, it is possible to flexibly deal with a situation in which communication with a specific control target is poor.

[0115] Furthermore, according to the first embodiment, processing is performed based on the reaction times of moving objects and people, so that the control target can be flexibly controlled even in various situations.

[0116] Furthermore, according to the first embodiment, when the slack time becomes negative, the stability of the control system can be improved by halting application processing or discarding transmission packets.

[0117] In the first embodiment, when the slack time becomes negative, a measure to shorten the communication time may be implemented. For example, instead of retransmission type communication by ACK, a method of continuously transmitting the same transmission packet in advance or a method of using a different communication path can be used to improve real-time performance.

[0118] Example 2 Next, a second embodiment of the present invention will be described.

[0119] The second embodiment will be described with reference to the drawings, focusing on the differences from the first embodiment.

[0120] FIG. 22 is a diagram showing the internal configuration of the control system 13 according to the second embodiment and a process flow of each unit.

[0121] In the second embodiment, object information 13C and object information 13D are transmitted from the infrastructure sensors 11 and 12 to the receiving unit 131. The application unit 139 determines the presence or absence of the control targets 15, 16, 17, and 19 based on the input object information 13C and object information 13D, and transmits the determined result to the input elapsed time calculation unit 132 and the control time estimation unit 130. The control time estimation unit 130 updates the reaction time and the estimated behavior end time stored in the control target information table 13E based on the object information 13C and object information 13D.

[0122] The operation flow according to the second embodiment will be described in detail below.

[0123] Fig. 23 is a diagram showing a processing flow of the control time estimation unit 130. Each step in Fig. 23 will be described below.

[0124] (Figure 23: Step 1301) The control time estimation unit 130 updates the reaction time and the estimated behavior end time stored in the control object information table 13E based on the information (speed and state) of the control object held by the object information 13C and the object information 13D. For example, the faster the moving speed of the control object becomes, the longer the estimated behavior end time becomes.

[0125] For example, when the state of the controlled object changes from not looking at the information terminal to looking at it, the reaction time The reaction time is shortened. In addition, the reaction time is not limited to the above. For example, the reaction time may be shortened when the person to be controlled is gazing at the moving object, such as when the person's body is facing the direction of the moving object. Furthermore, the reaction time may be changed based on the person's age.

[0126] (Figure 23: Step 1302) The control time estimation unit 130 refers to the control target information table 13E based on the object ID of the object information 13C or the object information 13D, acquires the reaction time of the control target and the estimated communication time, and calculates the estimated control time. The control time estimation unit 130 transmits the calculation result to the slack time calculation unit 135.

[0127] Fig. 24 is a diagram showing a processing flow of the application unit 139. Each step in Fig. 24 will be described below.

[0128] (Figure 24: Step 1391) The application unit 139 determines whether or not control is necessary based on the object information 13C and 13D, and if control is necessary, determines which object the control should be performed on.

[0129] (Figure 24: Step 1392) In step 1392, if control is required, the process proceeds to step 1393, and if control is not required, the process ends.

[0130] (Figure 24: Step 1393) The application unit 139 transmits to the transmission control unit 138 control target information for the control target determined to require control, the transmission data, and the margin time.

[0131] FIG. 25 is a diagram showing an example of the object information 13C.

[0132] 25, object information 13C is composed of an object type, an object ID, a speed, and a time stamp given by the infrastructure sensors 11 and 12. Compared to object information 13A (FIG. 16) in the first embodiment, speed information is added to object information 13C.

[0133] FIG. 26 is a diagram showing an example of the object information 13D.

[0134] 26, object information 13D is composed of an object type, an object ID, a state, and a time stamp given by the infrastructure sensors 11 and 12. Compared to object information 13A (FIG. 16) in the first embodiment, state information is added to object information 13D.

[0135] FIG. 27 is a diagram showing an example of the control target information table 13E.

[0136] 27, the control object information table 13E includes an object ID, an item, a reaction time, an estimated behavior end time, an IP address, and an estimated communication time. Compared to the control information table 13B (FIG. 17) in the first embodiment, the control object information table 13E includes an estimated behavior end time.

[0137] According to the second embodiment, in addition to being able to obtain the same effects as those of the first embodiment, it is possible to provide a control system capable of dealing with a variety of situations by changing the estimated behavior end time according to the movement speed of the control target.

[0138] According to the second embodiment, accurate control is possible by changing the reaction time of the controlled object depending on the state of the object. [Explanation of symbols]

[0139] 1···Control system, 11, 12··Infrastructure sensor, 13···Control system, 13A, 13C, 13D···Object information, 13B, 13E··Control target information table, 14···Wireless equipment, 15···Automobile, 16···Bus, 17, 18···Pedestrian, 17TR···Wireless communication equipment, 19···Traffic signal, 111···Recognition processing unit, 112A, 112B···Time synchronization unit, 130, 134···Control time estimation unit, 131···Reception unit, 132···Input elapsed time calculation unit, 133···Communication performance measurement server processing unit, 135···Margin of error calculation unit, 136···Processing management unit, 137, 139···Application unit, 138···Transmission control unit, 151···Communication performance measurement client processing unit, 152···Control processing unit

Claims

1. A moving object detection unit that detects a moving object and transmits information about the moving object; a control system that receives the information about the moving object transmitted by the moving object detection unit, and transmits control information to a control processing unit of the moving object based on the received information about the moving object; Equipped with The control system includes: a margin time calculation unit that calculates a first margin time based on an estimated communication control required time including a reaction time of the moving object; a process management unit that determines an execution order such that the application units calculate the control information in ascending order of the first margin times; an application unit that calculates the control information based on the information of the moving object and the execution order determined by the process management unit; a transmission control unit that calculates a second margin time based on an actual required communication control time including a reaction time of the moving object, and transmits the control information to the control processing unit in order of the shortest second margin time; A control system comprising:

2. 2. The control system of claim 1, the first margin time is calculated by subtracting an estimated required communication control time including a reaction time of the moving object from an overall latency, which is a time from when the moving object detection unit receives the information of the moving object to when the moving object moves and stops moving, A control system according to claim 1, wherein the second margin time is calculated by subtracting an actual required time for communication control, including a reaction time of the mobile object, from the total latency.

3. 3. The control system of claim 2, the estimated required communication control time is a value obtained by adding together an estimated communication time between the control system and the control processing unit, a reaction time of the moving object, an estimated application execution time of the application unit, a time for the moving object detection unit to detect the moving object, and an uplink time of a detection signal for detecting the moving object, A control system characterized in that the actual communication control required time is a value obtained by adding together the estimated communication time between the control system and the control processing unit, the reaction time of the moving object, the actual application execution time of the application unit, the time for the moving object detection unit to detect the moving object, and the uplink time of the detection signal detecting the moving object.

4. 4. The control system of claim 3, The control system further comprises a control time estimation unit that estimates the estimated communication time and the reaction time of the moving object.

5. 4. The control system of claim 3, A control system comprising: a control unit that stops calculation of the control information of the application unit when the first margin time or the second margin time becomes negative.

6. 4. The control system of claim 3, A control system characterized in that the moving object includes a pedestrian, and the control processing unit is a wireless communication device carried by the pedestrian.

7. 4. The control system of claim 3, The control system according to claim 1, wherein the processing management unit stops calculation of the control information having a long first margin time and causes the application unit to calculate the control information having a short first margin time.

8. 4. The control system of claim 3, The control system according to claim 1, wherein the transmission control unit stops transmission of the control information having a long second margin time and executes transmission of the control information having a short second margin time.

9. 5. The control system of claim 4, The control system is characterized in that the moving body includes a pedestrian, the control processing unit is a wireless communication device carried by the pedestrian, and the control time estimation unit estimates the reaction time of the pedestrian based on information on whether the pedestrian is looking at the wireless communication device, whether the pedestrian is facing in the direction of the moving body, or the pedestrian's age.

10. 2. The control system of claim 1, A control system characterized in that the control information transmitted to the control processing unit of the moving body is information regarding stopping or steering control of the vehicle, or information regarding images, sounds, or vibrations that give instructions or warnings to people.

Images