Integrated systems, integration methods, integrated programs

Abstract

We provide an integrated system that suppresses integration errors in data transmitted from multiple sensors. [Solution] The processor in an integration system for integrating message MS transmitted to a communication network from multiple sensors whose sensing fields overlap is configured to perform the following actions during a repeatedly started queue period PQ: enqueue multiple message MS from each sensor into memory; identify the oldest message MS with the most recent enqueue timing among the message MS with the most recent enqueue timing for each sensor during the queue period PQ as the reference message MSb from the reference sensor; and output integrated data Di by integrating message MS from integration candidate sensors other than the reference sensor, whose enqueue timings appear around the same time as the reference message MSb during the queue period PQ, into the reference message MSb.

Description

Technical Field

[0001] The present disclosure relates to an integration technique for integrating transmission data from a plurality of sensors.

Background Art

[0002] In the integration technique disclosed in Patent Document 1, packet data transmitted from a plurality of sensors having a common detection target to a communication network are integrated with each other.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the integration technique disclosed in Patent Document 1, a set of packet data to be integrated is determined by analyzing the position and time as feature amounts from the packet data of each sensor. However, depending on the communication status of the communication network, there is a concern that integration errors may occur because even packet data that should not be integrated due to a deviation in the transmission time are determined to match simply by comparing the acquisition times.

[0005] An object of the present disclosure is to provide an integration system that suppresses integration errors between transmission data from a plurality of sensors. Another object of the present disclosure is to provide an integration method that suppresses integration errors between transmission data from a plurality of sensors. Still another object of the present disclosure is to provide an integration program that suppresses integration errors between transmission data from a plurality of sensors.

Means for Solving the Problems

[0006] The following describes the technical means of solving the problem described in this disclosure. Note that the claims and the reference numerals in parentheses in this section indicate the correspondence with the specific means described in the embodiments detailed later, and do not limit the technical scope of this disclosure.

[0007] The first aspect of this disclosure is, An integrated system having a processor (12) for integrating messages (MS) transmitted to a communication network (NW) from multiple sensors (3) whose sensing fields overlap, The processor is During a repeatedly initiated queue period (PQ), multiple messages from each sensor are enqueued. During the queue period, among the messages with the most recent enqueue timing for each sensor, the message with the oldest enqueue timing is identified as the reference message (MSb) from the reference sensor (3b), and The system is configured to output integrated data (Di) which is created by integrating messages from integrated candidate sensors (3c), which are sensors other than the reference sensor, into the reference message, specifically messages whose enqueue timing occurs before or after the reference message during the queue period.

[0008] A second aspect of this disclosure is, A method for integrating messages (MS) transmitted to a communication network (NW) from multiple sensors (3) whose sensing fields overlap, which is performed by a processor (12), During a repeatedly initiated queue period (PQ), multiple messages from each sensor are enqueued. During the queue period, among the messages with the most recent enqueue timing for each sensor, the message with the oldest enqueue timing is identified as the reference message (MSb) from the reference sensor (3b), and This includes outputting integrated data (Di) which is a compilation of messages from integrated candidate sensors (3c), which are sensors other than the reference sensor, where the enqueue timing of messages appears before or after the reference message during the queue period, and these messages are integrated into the reference message.

[0009] A third aspect of this disclosure is: An integration program that stores messages (MS) transmitted to a communication network (NW) from multiple sensors (3) whose sensing fields overlap, in a storage medium (10), and includes instructions for causing a processor (12) to perform the integration, During a repeatedly initiated queue period (PQ), multiple messages from each sensor are enqueued. During the queue period, among the messages with the most recent enqueue timing for each sensor, the message with the oldest enqueue timing is identified as the reference message (MSb) from the reference sensor (3b), and The command includes instructions to output integrated data (Di) which is created by integrating messages from integrated candidate sensors (3c), which are sensors other than the reference sensor, into the reference message, specifically messages whose enqueue timing occurs before or after the reference message during the queue period.

[0010] According to these first to third embodiments, messages transmitted to the communication network from each sensor whose sensing field of view overlaps are enqueued multiple times in a storage medium during a repeatedly initiated queue period. Therefore, among the messages with the most recent enqueue timing for each sensor during the queue period, the message with the oldest enqueue timing is identified as the reference message from the reference sensor. During such a queue period, messages from candidate integration sensors other than the reference sensor whose enqueue timings occur around the same time as the reference message can be narrowed down to be appropriate integration targets with the reference message. Thus, it becomes possible to output integrated data with improved integration accuracy by suppressing integration errors.

Brief Description of the Drawings

[0011] [Figure 1] It is a block diagram showing the overall configuration of one embodiment. [Figure 2] It is a block diagram showing the functional configuration of an integrated system according to one embodiment. [Figure 3] It is a time chart for explaining an example of transmission of messages from a plurality of sensors according to one embodiment. [Figure 4] It is a flowchart showing an integrated flow according to one embodiment. [Figure 5] It is a time chart for explaining an integrated flow according to one embodiment. [Figure 6] It is a time chart for explaining an integrated flow according to one embodiment. [Figure 7] It is a time chart for explaining an integrated flow according to one embodiment. [Figure 8] It is a time chart for explaining an integrated flow according to one embodiment.

Modes for Carrying Out the Invention

[0012] Hereinafter, an embodiment of the present disclosure will be described based on the drawings.

[0013] An integrated system 1 according to an embodiment shown in FIGS. 1 and 2 integrates messages MS transmitted from a plurality (three in this embodiment) of sensors 3 to a communication network NW. Therefore, in each sensor 3 for which the integrated system 1 targets the integration of the message MS, at least a part of the sensing fields that are the ranges of sensing respectively need to overlap with each other.

[0014] As each sensor 3, for example, sensing devices such as an optical scanning sensor (so-called LiDAR), a camera sensor, and a radar sensor are assumed. All of the sensors 3 may be of the same type, may all be of different types, or may be at least one each of different types. The installation locations of the sensors 3 may be stationary structures within a common facility (such as a data center, etc.), may be common vehicles to each other, or at least one sensor may be assigned to such stationary structures and vehicles respectively.

[0015] For each sensor 3, regarding the target sensed within each sensing field of view, at least packet data representing position information is transmitted and output to the communication network NW as an individual message MS. Further, in FIGS. 1 to 3, each sensor 3 formally distinguished by attaching ordinal numbers (the first to the third in this embodiment) repeatedly transmits and outputs each message MS with a transmission period PS of different lengths. In order to realize such data transmission of the message MS, for example, each sensor 3 may be configured to include a recognition processing circuit for recognizing a target from the sensing data of the sensing device and a communication processing circuit for incorporating at least the position information of the recognized target into the message MS and outputting it. Note that at least position information regarding the target may be incorporated into the message MS from each sensor 3. Therefore, in the message MS from each sensor 3, at least one type among, for example, speed information, acceleration information, angle information, attitude information, skeleton information, etc. regarding the target may be incorporated in association with the position information.

[0016] As shown in FIGS. 1 and 2, the integrated system 1 is installed in a facility (such as a data center, etc.) or a vehicle together with the communication system 5. The integrated system 1 is connected to the communication network NW through the communication system 5. The communication network NW is constructed from a wired communication line and / or a wireless communication line according to the installation locations of the integrated system 1 and each sensor 3.

[0017] As shown in Figure 1, the integrated system 1 is comprised of at least one dedicated computer. The dedicated computer comprising the integrated system 1 has at least one memory 10 and one processor 12 that cooperate to integrate the messages MS transmitted from each sensor 3 to the communication network NW. The memory 10 is at least one type of non-transitory tangible storage medium, such as semiconductor memory, magnetic media, and optical media, which non-temporarily stores programs and data that can be read by the computer. The processor 12 includes at least one type as a core, such as a CPU, GPU, and RISC-CPU.

[0018] In the integrated system 1, the processor 12 executes multiple instructions included in the integration program stored in the memory 10 to integrate the messages MS transmitted from each sensor 3 to the communication network NW. This allows the integrated system 1 to construct multiple functional blocks for integrating the messages MS from each sensor 3. The multiple functional blocks constructed in the integrated system 1 include an enqueue block 100, a specific block 110, and an integration block 120, as shown in Figure 2.

[0019] The construction of these blocks 100, 110, and 120 enables the integration method for combining messages MS from each sensor 3, which is performed according to the integration flow shown in Figure 4. The integration flow is executed at predetermined control cycles during the startup of the integration system 1. In the integration flow, each "S" represents multiple steps executed by multiple instructions included in the management program.

[0020] In S10, the enqueue block 100 enqueues multiple messages MS from each sensor 3 during the current queue period PQ (hereinafter simply referred to as the current queue period PQ), which is one of the queue periods PQ that are repeatedly started for each control cycle of the integrated flow. At this time, the messages MS are enqueued into storage queues, which are reserved as buffers for each sensor 3 in the memory 10, so that they can be stored for each sensor 3 throughout the current queue period PQ. Note that on the time axis shown in Figure 5, the time at the left end of the rectangular frame represents the oldest time when the queue period PQ started, while the time at the right end of the rectangular frame represents the latest time when the queue period PQ ended. Therefore, in the following explanation, the enqueue timing refers to the time when the messages MS were enqueued into the storage queue on the time axis in Figure 5.

[0021] In S10, the enqueue block 100 specifically sets the queue period PQ such that it is longer than the longest period PSl shown in Figure 3 among the transmission periods PS of the messages MS from each sensor 3. In particular, in this embodiment, it is desirable that the queue period PQ be adjusted to be longer than the control period for repeating the integrated flow, so that the queue period PQ for the current flow execution starts in parallel with the queue period PQ for the previous flow execution.

[0022] As shown in Figure 4, in S20 following S10, the specific block 110 identifies the reference message MSb for the current queue period PQ. At this time, as shown in Figure 5, in the current queue period PQ, first, the message MSn with the most recent enqueue timing is extracted for each sensor 3. Then, in the current queue period PQ, the message MSn with the oldest enqueue timing among the most recent message MSns for each sensor 3 is identified as the reference message MSb from the reference sensor 3b, as shown in Figure 5.

[0023] As shown in Figure 4, in S30 following S20, the integration block 120 generates and outputs integrated data Di, which integrates the message MS from other sensors 3 with the reference message MSb from the reference sensor 3b. At this time, as shown enclosed in a rectangular frame in Figure 6, during the current queue period PQ, first, the message MS from the integration candidate sensor 3c, which is a sensor 3 other than the reference sensor 3b, is extracted. Then, during the current queue period PQ, among the message MS from the integration candidate sensor 3c, the message MSc, whose enqueue timing appears around the same time as the reference message MSb, is identified as the target for integration with the reference message MSb, as shown in Figure 6.

[0024] Specifically, the integration block 120 in S30 sets the integration period PI so that it surrounds the enqueue timing of the reference message MSb. At this time, the integration period PI is adjusted to be shorter than the queue period PQ. At the same time, the integration period PI is adjusted to ensure a period of half its value (i.e., PI / 2) before and after the reference message MSb. Furthermore, the integration period PI is adjusted to be shorter than the shortest transmission period PSs of the message MS from each sensor 3, as shown in Figure 3.

[0025] Within the integration period PI of S30, which has been adjusted in this way, an enqueue timing will occur for messages MSc from at least one of the integration candidate sensors 3c, as shown in Figure 7, in addition to the timing shown in Figure 6. In other words, not all messages MS from integration candidate sensors 3c other than the reference sensor 3b will occur within the integration period PI, as shown in Figure 7. Therefore, the integration block 120 in S30 will select the messages MSc from the integration candidate sensors 3c for integration with the reference message MSb, limited to those messages MSc for which an enqueue timing has occurred within the integration period PI.

[0026] In S30, during message integration, the integration block 120 monitors and extracts the time difference Δt that appears in the enqueue timing of the integration target message MSc from the integration candidate sensor 3c, as shown in Figure 8, based on the enqueue timing of the reference message MSb. Then, in S30, the integration block 120 corrects the target position information contained in the integration target message MSc from the integration candidate sensor 3c according to the extracted time difference Δt. At this time, the target position information may be corrected according to the time difference Δt by using the velocity information incorporated into the sensor message MS of sensor 3, which is capable of sensing or recognizing the velocity information of the target. Alternatively, the target position information may be corrected according to the time difference Δt by using a Kalman filter that follows a kinematics model. In particular, when using a Kalman filter, it is preferable to use the sensor message MS acquired in the current queue period PQ, which is allocated for the length required for filtering, and / or the sensor message MS acquired in the queue period PQ from previous flow executions.

[0027] In S30, the integration block 120 generates integrated data Di as output data by integrating at least one integration target message MSc, whose position information has been corrected according to the time difference Δt, into a reference message MSb. At this time, it is desirable that the relative positional relationship between the corrected integration target message MSc and the reference message MSb is calibrated to 1:1 according to the characteristic parameters of each corresponding sensor 3. This makes it possible to generate integrated data Di in a data structure that associates the position information in the corrected integration target message MSc with the position information in the reference message MSb.

[0028] The integrated data Di output by S30 described above will be used at least for the location type in the facility or vehicle where the integrated system 1 is installed, for example, for high-precision target recognition processing, target position estimation processing, and control processing that takes into account interaction with the target. Note that once S30 is completed with the output of the integrated data Di, this execution of the integrated flow is finished.

[0029] (Effects and Benefits) The effects and advantages of this embodiment, as described above, will be explained below.

[0030] According to this embodiment, messages MS transmitted from each sensor 3 whose sensing fields of view overlap to the communication network NW are enqueued in memory 10 multiple times during the repeatedly started queue period PQ. Therefore, among the messages MS (MSn in this embodiment) whose enqueuing timing is most recent for each sensor 3 during the queue period PQ, the message MS with the oldest enqueuing timing is identified as the reference message MSb from the reference sensor 3b. In such a queue period PQ, among the messages MS from integration candidate sensors 3c other than the reference sensor 3b, messages MS (MSc in this embodiment) whose enqueuing timing appears around the same time as the reference message MSb can be narrowed down as appropriate integration targets with the reference message MSb. Thus, it becomes possible to output integrated data Di with improved integration accuracy by suppressing integration errors.

[0031] According to this embodiment, within an integration period PI that is shorter than the queue period PQ, with the enqueue timing of the reference message MSb sandwiched before and after, the message MS (MSc in this embodiment) from the integration candidate sensor 3c where the enqueue timing appears is integrated into the reference message MSb. As a result, even if a message MS is from the integration candidate sensor 3c, messages MS that would cause integration errors due to a long interval between enqueue timings can be excluded from integration with the reference message MSb. Therefore, the effect of suppressing integration errors can be enhanced.

[0032] According to this embodiment, within an integration period PI shorter than the shortest transmission period PSs of the messages MS from each sensor 3, the message MS from the integration candidate sensor 3c (MSc in this embodiment) where the enqueue timing appears is integrated into the reference message MSb. This makes it possible to avoid a situation where multiple messages MS from the same integration candidate sensor 3c are integrated into the reference message MSb. Therefore, it is possible to ensure the reliability of the effect of suppressing integration errors.

[0033] According to this embodiment, the position information contained in the message MS to be integrated (MSc in this embodiment) from the candidate integration sensor 3c is corrected according to the time difference Δt that occurs in the enqueue timing of the message MS to be integrated (MSc in this embodiment) from the candidate integration sensor 3c, based on the enqueue timing of the reference message MSb. By integrating the message MS from the candidate integration sensor 3c, whose position information has been corrected in this way, with the reference message MSb, it becomes possible to output integrated data Di that has improved not only the integration accuracy but also the position accuracy.

[0034] (Other embodiments) Although one embodiment has been described above, this disclosure is not to be construed as being limited to the embodiment described herein, and can be applied to various embodiments without departing from the gist of this disclosure.

[0035] In the modified example, the dedicated computer constituting the integrated system 1 may have at least one of the digital circuit and the analog circuit as a processor. Here, the digital circuit is at least one of the following, for example, ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), SOC (System on a Chip), PGA (Programmable Gate Array), and CPLD (Complex Programmable Logic Device). Such a digital circuit may also have memory for storing programs.

[0036] In the modified example, the transmission period PS of the message MS from each sensor 3 may all be the same length. In the modified example, the transmission period PS of the message MS from each sensor 3 may be at least one of different lengths, and the above-described embodiment is one example of this. In the modified example, the number of sensors 3 that the integrated system 1 integrates the message MS with may, of course, be other than the three as in the above-described embodiment. [Explanation of symbols]

[0037] 1: Integrated system, 3: Sensor, 3b: Reference sensor, 3c: Candidate integrated sensor, 10: Memory, 12: Processor, Di: Integrated data, MS: Message, MSb: Reference message, NW: Communication network, PI: Integration period, PQ: Queue period, PS: Transmission cycle, PSs: Shortest cycle, Δt: Time difference

Claims

1. An integrated system having a processor (12) for integrating messages (MS) transmitted to a communication network (NW) from multiple sensors (3) whose sensing fields overlap, The aforementioned processor, In a repeatedly initiated queue period (PQ), multiple messages from each of the sensors are enqueued. During the aforementioned queue period, among the messages with the most recent enqueue timing for each of the aforementioned sensors, the message with the oldest enqueue timing is identified as the reference message (MSb) from the reference sensor (3b). An integration system configured to output integrated data (Di) which is an integrated candidate sensor (3c) other than the reference sensor, in which messages whose enqueue timing occurs before or after the reference message during the queue period are integrated into the reference message.

2. The output of the aforementioned integrated data is: The integration system according to claim 1, which includes integrating the message from the integration candidate sensor whose enqueue timing appears into the reference message within an integration period (PI) that is shorter than the queue period, with the enqueue timing of the reference message sandwiched before and after it.

3. The output of the aforementioned integrated data is: The integration system according to claim 2, further comprising integrating the messages from the integration candidate sensors in which the enqueue timing occurs within the integration period, which is shorter than the shortest transmission period (PSs) of the messages from each of the sensors, into the reference message.

4. The output of the aforementioned integrated data is: Based on the enqueue timing of the reference message, the position information included in the message from the integrated candidate sensor is corrected according to the time difference (Δt) that occurs in the enqueue timing of the message from the integrated candidate sensor, The integration system according to claim 1, comprising integrating the message from the integrated candidate sensor, whose positional information has been corrected, into the reference message.

5. A method for integrating messages (MS) transmitted to a communication network (NW) from multiple sensors (3) whose sensing fields overlap, which is performed by a processor (12), In a repeatedly initiated queue period (PQ), multiple messages from each of the sensors are enqueued. During the aforementioned queue period, among the messages with the most recent enqueue timing for each of the aforementioned sensors, the message with the oldest enqueue timing is identified as the reference message (MSb) from the reference sensor (3b). An integration method that includes outputting integrated data (Di) obtained by integrating messages from an integration candidate sensor (3c), which is a sensor other than the reference sensor, into the reference message, where the enqueue timing of such messages occurs before or after the reference message during the queue period.

6. An integration program that stores messages (MS) transmitted to a communication network (NW) from multiple sensors (3) whose sensing fields overlap, in a storage medium (10) for the purpose of integrating such messages, and includes instructions for causing a processor (12) to perform the integration, In a repeatedly initiated queue period (PQ), multiple messages from each of the sensors are enqueued. During the aforementioned queue period, among the messages with the most recent enqueue timing for each of the aforementioned sensors, the message with the oldest enqueue timing is identified as the reference message (MSb) from the reference sensor (3b). An integration program that includes the instruction to output integrated data (Di) obtained by integrating messages from an integration candidate sensor (3c), which is a sensor other than the reference sensor, that have an enqueue timing that occurs before or after the reference message during the queue period, with respect to the reference message.

Images