Communication device, communication method, and program

Abstract

To make the association of time stamp and packets appropriate.SOLUTION: A communication device according to one aspect includes: request means for requesting transmission of a packet; transfer means for transferring the packet based on the packet transmission request; transmission means for transmitting the packet transferred by the transfer means and outputting a time stamp based on the transmission of the packet; and control means for controlling an output of a tag associating the packet and the time stamp based on predetermined information readable from an interface between the transfer means and the transmission means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a communication device, a communication method, and a program. [Background technology]

[0002] A technology that generates virtual viewpoint content using images from multiple viewpoints obtained by synchronously capturing images from multiple cameras installed at different positions has been attracting attention. By using virtual viewpoint content, viewers can, for example, watch highlight scenes of a soccer or basketball game from various angles, which gives viewers a more realistic feeling than ordinary images.

[0003] One of the network-based synchronization methods for achieving synchronized imaging is PTP (Precision Time Protocol). In PTP, a master device that generates accurate time sends PTP packets with that time information attached, and slave devices receive the PTP packets to achieve time synchronization.

[0004] One method of connecting multiple communication devices that make up a network is called a daisy chain, in which each communication device is connected in series. Each communication device has multiple connectors, and a daisy chain connection can be established by connecting each connector to another communication device.

[0005] In a communication network in which communication devices are connected in a daisy chain, various communication packets are transferred, such as PTP packets for time synchronization between communication devices or between a master device and a communication device, TCP / IP packets for controlling communication devices, and image / video packets captured by a camera.

[0006] These communication packets are transferred based on the transmission order control of the DMAC (Direct Memory Access Controller). At this time, the requester requests the DMAC to transfer the communication packet, which is then transferred to the MAC (Media Access Control) / PHY (Physical Layer). If the MAC / PHY supports hardware timestamp output, it can output the timestamp of the communication packet with extremely high synchronization accuracy of less than 1 microsecond. When outputting a hardware timestamp, it is common to set identification information called a tag in the PHY / MAC, which associates the communication packet with the timestamp to be output.

[0007] Patent Document 1 discloses a method for transferring communication packets using a DMAC. In the method disclosed in Patent Document 1, when multiple requesters that transmit data make transmission requests to the DMAC, arbitration of data transfer is performed by prioritizing the data according to the data size. If the size of the data to be transferred is small, it is transferred with priority over data with a large size. If no small data transfers are being performed, transfers of large data are transferred with priority. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-242718 Summary of the Invention [Problem to be solved by the invention]

[0009] However, in the DMA transfer disclosed in Patent Document 1, arbitration can cause the order of data transferred by the DMAC to differ from the order in which the requester requested transmission, and the data may not be transferred from the DMAC in the order in which the requester requested transmission. In this case, the requester does not know the order in which the data will be transferred, so it cannot set an appropriate tag in the PHY / MAC, and it is not possible to associate the timestamp output from the PHY / MAC with the communication packet requested to be transmitted by the requester. The problem to be solved by the present invention is to properly associate timestamps with packets. [Means for solving the problem]

[0010] A communication device according to one embodiment includes a request means for making a request to transmit a packet, a forwarding means for forwarding the packet based on the request to transmit the packet, a transmitting means for transmitting the packet forwarded by the forwarding means and outputting a timestamp based on the transmission of the packet, and a control means for controlling the output of a tag that associates the packet with the timestamp based on specified information that can be read from an interface between the forwarding means and the transmitting means. [Effects of the Invention]

[0011] According to one aspect of the present invention, it is possible to properly associate timestamps with packets. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of the configuration of a synchronized imaging system according to a first embodiment. [Figure 2] FIG. 1 is a block diagram showing an example of the configuration of a camera adapter according to the first embodiment. [Figure 3] FIG. 3 is a diagram showing PTP time synchronization processing according to the first embodiment. [Figure 4] 5 is a flowchart showing a packet transmission request process according to the first embodiment. [Figure 5]5 is a flowchart showing a packet transmission process according to the first embodiment. [Figure 6] 5 is a flowchart showing a transmission timestamp control process according to the first embodiment. [Figure 7] 4 is a flowchart showing PTP time synchronization processing according to the first embodiment. [Figure 8] 10 is a flowchart showing a packet transmission request process according to the second embodiment. [Figure 9] 11 is a flowchart showing a packet transmission request process according to the third embodiment. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of a camera adapter according to a fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the present invention, and not all combinations of features described in the embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). The technical scope of the present invention is determined by the claims and is not limited by the individual embodiments below.

[0014] First Embodiment FIG. 1 is a block diagram showing an example of the configuration of a synchronized imaging system according to the first embodiment. In FIG. 1, the synchronized imaging system 100 can synchronize imaging from multiple viewpoints using multiple cameras installed at different locations in, for example, a gymnasium, stadium, or concert hall, and generate a virtual viewpoint image using the images from the multiple viewpoints obtained through the synchronized imaging. By generating a virtual viewpoint image, the synchronized imaging system 100 allows a user to view, for example, highlight scenes of a soccer or basketball game from various angles, or to view gymnastics, martial arts, or the like from a desired viewpoint. The synchronized imaging system 100 uses PTP to achieve synchronized imaging. The synchronized imaging system 100 performs PTP time synchronization processing based on the timestamps of communication packets. The synchronized imaging system 100 can process both still images and video. In this embodiment, unless otherwise specified, the term "image" refers to both video and still images.

[0015] The synchronized imaging system 100 includes a time server 101, a hub 102, a control terminal 103, an image computing server 104, a user terminal 105, and sensor systems 110a to 110z. The hub 102 and the sensor systems 110a to 110z are connected via daisy chains 106a to 106z.

[0016] The number of sensor systems 110a to 110z is not limited and may be any number. Furthermore, the sensor systems 110a to 110z do not necessarily have to have the same configuration, and for example, the sensor systems 110a to 110z may be different models.

[0017] The time server 101 has a function of distributing time information, and distributes time information synchronized with a Global Navigation Satellite System (GNSS) 107 to each of the sensor systems 110a to 110z. Note that the time server 101 may distribute the local time generated by the time server 101 as a master time to each of the sensor systems 110a to 110z.

[0018] The hub 102 distributes PTP packets with time information added that are distributed by the time server 101 to each destination, and distributes image data that each of the sensor systems 110a to 110z transmits to the image computing server 104 to each destination.

[0019] The control terminal 103 manages the operating status of each block included in the synchronized imaging system 100 via a network, and performs parameter setting control, etc. Here, the network may be GbE (Gigabit Ethernet) or 10 GbE conforming to the IEEE standard, which is Ethernet (registered trademark), or may be a combination of interconnect Infiniband, industrial Ethernet, etc. The network is not limited to these, and other types of networks may also be used.

[0020] Each of the sensor systems 110a to 110z includes a camera 111a to 111z and a camera adapter 112a to 112z. The sensor systems 110a to 110z are not necessarily limited to this configuration. For example, they may include an audio device such as a microphone, a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), or a camera platform for controlling the camera orientation. Furthermore, each of the sensor systems 110a to 110z may be configured with one camera adapter and multiple cameras, or one camera and multiple camera adapters.

[0021] The sensor systems 110a to 110z are connected to one another by a daisy chain. The daisy chain connection reduces the number of connecting cables and saves power for wiring work when the volume of image data increases due to higher resolutions of captured images such as 4K or 8K and higher frame rates. However, the connection may also be a star-type network configuration in which each sensor system 110a to 110z transmits and receives data between the sensor systems 110a to 110z via the hub 102.

[0022] Although FIG. 1 illustrates an example in which all of the sensor systems 110a-110z are daisy-chained, this is not limiting. For example, the sensor systems 110a-110z may be divided into several groups, and the sensor systems 110a-110z may be daisy-chained for each group. This configuration is particularly effective in stadiums. For example, a stadium may have multiple floors, and the sensor systems 110a-110z may be installed on each floor. In this case, input to the image computing server 104 can be performed for each floor or for each half circumference of the stadium. This also simplifies installation and increases the flexibility of the system, even in locations where it is difficult to wire all of the sensor systems 110a-110z in a single daisy chain.

[0023] Each of the cameras 111a to 111z captures an image of a subject from a different direction. In the following description, the cameras 111a to 111z are assumed to have the same configuration, but the cameras may have different performance or models.

[0024] Each of the camera adapters 112a to 112z is connected to a corresponding one of the cameras 111a to 111z, and controls each of the cameras 111, acquires captured images, provides synchronization signals, sets the time, etc. Each of the camera adapters 112a to 112z also performs image processing on the captured image data captured by each of the cameras 111a to 111z.

[0025] For example, each of the camera adapters 112a-112z generates a virtual viewpoint image based on the foreground and background images captured from multiple viewpoints by outputting a foreground image and a background image from the imaging data captured by each of the cameras 111a-111z. Note that some of the camera adapters 112a-112z may output a foreground image separated from the captured image, but may not output a background image. Each of the camera adapters 112a-112z then transmits the image-processed data to the image computing server 104 via the hub 102 connected by a daisy chain.

[0026] At this time, each of the camera adapters 112a to 112z operates as a PTP slave and synchronizes time with the time server 101, which is a GM (Grand Master) device. Each of the camera adapters 112a to 112z also operates as a PTP master and operates as a BC (Boundary Clock) device that synchronizes time with other sensor systems 110a to 110z.

[0027] Alternatively, each camera adapter 112a to 112z may operate as a TC (Transparent Clock) device that performs time correction on PTP packets from a GM device and transfers them to other sensor systems 110a to 110z. Alternatively, each camera adapter 112a to 112z may operate as an OC (Ordinary Clock) device that only performs time synchronization using PTP packets from a GM device and PTP packets transferred from other sensor systems 110a to 110z.

[0028] Alternatively, each camera adapter 112a-112z may perform time synchronization using PTP packets from the GM device and PTP packets transferred from other sensor systems 110a-110z, and may perform time correction on the PTP packets. In this case, each camera adapter 112a-112z may operate as a TC+OC device that transfers the time-corrected PTP packets to other sensor systems 110a-110z. Then, each camera adapter 112a-112z can provide imaging timing (control clock) to each camera 111a-111z using the synchronized time and reference signal.

[0029] The image computing server 104 processes data acquired from each of the sensor systems 110a to 110z. For example, the image computing server 104 reconstructs the imaging data acquired from each of the sensor systems 110a to 110z, converts the data format, and then stores the data according to the identifier, data type, and frame number of each of the cameras 111a to 111z. The image computing server 104 then receives a designation of a viewpoint from the control terminal 103, reads out the corresponding imaging data from the stored information based on the designated viewpoint, and performs rendering processing to generate a virtual viewpoint image. Note that at least some of the functions of the image computing server 104 may be provided by the control terminal 103, each of the sensor systems 110a to 110z, or the user terminal 105.

[0030] The user terminal 105 receives the rendered image from the image computing server 104 and provides an image of a viewpoint according to the specification of the user operating the user terminal 105. Note that the virtual viewpoint content may be generated by the image computing server 104, or may be generated by the control terminal 103 or the user terminal 105.

[0031] FIG. 2 is a block diagram showing an example of the configuration of the camera adapter according to the first embodiment. For the software-implemented functions of the functional blocks shown in FIG. 2, a program for providing the function of each functional block is stored in a memory such as a read-only memory (ROM). The program is then loaded into a random access memory (RAM) and executed by a central processing unit (CPU). For the hardware-implemented functions, a dedicated circuit may be automatically generated on an FPGA from a program for implementing the function of each functional block using, for example, a predetermined compiler. FPGA stands for Field Programmable Gate Array. Alternatively, a gate array circuit may be formed in a similar manner to an FPGA to implement the function as hardware. Alternatively, the function may be implemented using an application-specific integrated circuit (ASIC). Note that the functional block configuration shown in FIG. 2 is merely an example; multiple functional blocks may constitute a single functional block, or one functional block may be divided into blocks that perform multiple functions.

[0032] 2, sensor system 110 includes camera 111 and camera adapter 112. Note that sensor system 110 can be used as each of sensor systems 110a to 110z in FIG. 1. Camera 111 can be used as each of cameras 111a to 111z in FIG. 1. Camera adapter 112 can be used as each of camera adapters 112a to 112z in FIG. 1.

[0033] The camera adapter 112 includes a CPU 200, a video packet processing unit 201, a first memory unit 202, a second memory unit 203, a DMA control unit 204, a PHY / MAC 205, and a timestamp control unit 210. The CPU 200, the video packet processing unit 201, the first memory unit 202, the second memory unit 203, the DMA control unit 204, and the timestamp control unit 210 are interconnected via a system bus 206. The camera adapter 112 is also connected to the camera 111 via an image transmission cable 209.

[0034] The CPU 200 executes application software at the application layer defined by the OSI (Open Systems Interconnection) reference model and controls the entire camera adapter 112. At this time, the CPU 200 stores data, programs, and the like required to execute the application software in the first storage unit 202. The CPU 200 then reads and writes data from and to the first storage unit 202 via the system bus 206 as needed. Specifically, the CPU 200 performs PTP stack processing, generates descriptors for DMA transfer control using the DMA control unit 204, and generates tags to be set in the PHY / MAC 205. The tag is identification information that associates a communication packet requested to be transmitted by the DMA control unit 204 with a timestamp output by the PHY / MAC 205. The CPU 200 may be a GPU (Graphics Processing Unit).

[0035] Video packet processor 201 receives image data captured by camera 111 via image transmission cable 209 and packetizes the image data. Then, video packet processor 201 generates a descriptor for DMA transfer control using DMA controller 204. Note that the functions of video packet processor 201 may be realized by dedicated hardware such as an ASIC, by a CPU, or by hardware using an FPGA or the like. Camera adapter 112 may include a CPU separate from CPU 200 in order to realize the functions of video packet processor 201 by the CPU.

[0036] The first memory unit 202 is a storage area used by the CPU 200. The first memory unit 202 can be configured with semiconductor memory such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). The internal area of ​​the first memory unit 202 is divided into an area for storing programs and an application buffer area used by application software, and stores descriptors and tags for packet transfer. The descriptor is information used by the DMA control unit 204 for packet transfer. The descriptor includes information such as address information on the source side where data is transferred, address information on the destination side, and the transfer size of the data transfer.

[0037] The second storage unit 203 is a storage area used by the video packet processing unit 201. The second storage unit 203 can be configured with a semiconductor memory such as a DRAM or an SRAM. The internal area of ​​the second storage unit 203 stores descriptors for transmitting image packets.

[0038] The first storage unit 202 and the second storage unit 203 may be used exclusively by the CPU 200 and the video packet processing unit 201, respectively, but are not limited to such use. The first storage unit 202 and the second storage unit 203 may be combined into one storage unit and shared by the CPU 200 and the video packet processing unit 201.

[0039] The DMA control unit 204 executes a DMA transfer in response to a packet transmission request from the CPU 200 or the video packet processing unit 201. At this time, the DMA control unit 204 executes the DMA transfer in accordance with the contents of the descriptor generated by the CPU 200 or the video packet processing unit 201. Here, the DMA control unit 204 has multiple transmission request ports that receive DMA transfer requests, and performs priority control for each transmission request port. At this time, the packet transmission order is changed for each transmission request port. Note that within one transmission request port, the packet transmission order is not changed. Then, after completing the DMA transfer, the DMA control unit 204 notifies the CPU 200 or the video packet processing unit 201 of a completion interrupt 207 corresponding to the descriptor used.

[0040] The PHY / MAC 205 is an interface with a network 208 such as a LAN (Local Area Network). The PHY / MAC 205 is responsible for communication control of the PHY layer (physical layer) and MAC layer (data link layer) in communication. The PHY / MAC 205 outputs a transmission / reception timestamp and a tag for a communication packet transmitted or received between the PHY / MAC 205 and the network 208. The timestamp is time information indicating when the PHY / MAC 205 transmitted or received a packet. The timestamp is used for calculations such as time correction for PTP protocol processing.

[0041] The timestamp control unit 210 controls the output of tags that associate communication packets with timestamps based on predetermined information readable from the interface between the DMA control unit 204 and the PHY / MAC 205. For example, the timestamp control unit 210 can read the predetermined information based on snooping of the interface between the DMA control unit 204 and the PHY / MAC 205. The predetermined information includes information that a requester issuing a packet transmission request requests to be transmitted to a transmission request port of the DMA control unit 204, the packet type of the packet transmitted by the DMA control unit 204, and tags embedded in the header and data fields of the transmitted packet. At this time, the CPU 200 or the video packet processing unit 201 can operate as a requester. The timestamp control unit 210 also inputs tags that associate packets with timestamps to the PHY / MAC 205 in accordance with the order of the packets output from the DMA control unit 204.

[0042] The timestamp control unit 210 includes a register unit 211, a tag storage unit #N212, a selection processing unit 213, a TS&TAG temporary storage unit 214, a distribution processing unit 215, and a TS&TAG storage unit #N216.

[0043] A tag is set in the register unit 211 by the CPU 200. At this time, the tag is set in the register unit 211 before or simultaneously with a DMA transfer request by the CPU 200. The CPU 200 then reads the timestamp and tag output by the PHY / MAC 205 via the register unit 211. Note that the video packet processing unit 201 may set the tag and read the timestamp and tag, just like the CPU 200. In the following description, it is assumed that the CPU 200 sets the tag and reads the timestamp and tag.

[0044] The TAG storage unit #N212 stores tags written by the CPU 200 via the register unit 211. The TAG storage unit #N212 can be configured with a plurality of FIFO (First Input First Output) memories, and the number of TAG storage units #N212 provided is equal to the number of transmission request ports of the DMA control unit 204. The TAG storage unit #N212 can be used as a plurality of queues for storing tags generated by a requester that requests the transmission of a communication packet. In this case, each TAG storage unit #N212 is associated with a transmission request port.

[0045] The selection processing unit 213 controls the output of tags that associate communication packets with timestamps, based on predetermined information that can be read from an interface used when transferring communication packets. The selection processing unit 213 can read the predetermined information by snooping the bus interface between the DMA control unit 204 and the PHY / MAC 205. The predetermined information includes information about the requester that made the transmission request, information about the transmission request port of the DMA control unit 204 from which the transmission packet is being sent, the packet type of the transmission packet, and the tag attached to the transmission packet. The selection processing unit 213 then selects a TAG storage unit #N 212 based on the predetermined information, and sends the tag stored in the selected TAG storage unit #N 212 to the PHY / MAC 205.

[0046] The TS&TAG temporary storage unit 214 temporarily stores the timestamp and tag output by the PHY / MAC 205. The TS&TAG temporary storage unit 214 can be configured as a FIFO memory. When the TS&TAG temporary storage unit 214 stores the timestamp and tag, it notifies the allocation processing unit 215 that the information has been stored.

[0047] The distribution processor 215 distributes the timestamps and tags stored in the TS&TAG temporary storage unit 214 to the TS&TAG storage unit #N 216. For example, the distribution processor 215 distributes the timestamps and tags based on each requester, each transmission request port to which the requester has sent a transmission request to the DMA control unit 204, each type of packet transferred by the DMA control unit 204, or a combination of these. At this time, the selection processor 213 notifies the distribution processor 215 of selection information that selects the TAG storage unit #N 212 based on predetermined information, and the distribution processor 215 can perform distribution processing based on the selection information. Alternatively, the CPU 200 or the video packet processor 201 includes the distribution condition information in the tag and sets it in the register unit 211. The distribution processor 215 may then read the distribution condition information from the tag stored in the TS&TAG temporary storage unit 214 and perform distribution processing.

[0048] The TS&TAG storage unit #N216 stores the timestamps and tags allocated by the allocation processing unit 215. The TS&TAG storage unit #N216 can be configured with a plurality of FIFO memories. The timestamps and tags stored in the TS&TAG storage unit #N216 are read by the CPU 200 via the register unit 211. The TS&TAG storage unit #N216 can be used as a plurality of queues that store the timestamps and tags output from the PHY / MAC 205.

[0049] Fig. 3 is a diagram showing PTP time synchronization processing according to the first embodiment. Fig. 3 shows an example of a transmission and reception sequence of synchronization packets between a GMC (Grand Master Clock) device and a PTP slave device in PTP time synchronization. In this case, the time server 101 in Fig. 1 operates as a GMC device, and the sensor system 110 in Fig. 2 operates as a PTP slave device.

[0050] 3, the time server 101 transmits a synchronization packet Sync at regular intervals to the sensor system 110. When the time server 101 transmits the synchronization packet Sync, it stores a transmission timestamp t1 at that time. When the sensor system 110 receives the synchronization packet Sync, it stores a reception timestamp t2 at that time.

[0051] Next, in step S302, the time server 101 places the transmission timestamp t1, which was used to transmit the synchronization packet Sync, on a synchronization packet Follow_Up and transmits it to the sensor system 110. When the sensor system 110 receives the synchronization packet Follow_Up, it retains the transmission timestamp t1 placed on the synchronization packet Follow_Up. Note that FIG. 3 shows a method in which the time server 101 places the transmission timestamp t1 of the synchronization packet Sync on the synchronization packet Follow_Up and transmits it. In addition to this method, the time server 101 may place the transmission timestamp t1 of the synchronization packet Sync on the synchronization packet Sync itself and transmit it, rather than on the synchronization packet Follow_Up.

[0052] Next, in step S303, the sensor system 110 transmits a synchronization packet Delay_Request to the time server 101. When the sensor system 110 transmits the synchronization packet Delay_Request, it holds a transmission timestamp t3 of the Delay_Request at that time. When the time server 101 receives the synchronization packet Delay_Request, it holds a reception timestamp t4 of the Delay_Request at that time.

[0053] Next, in step S304, the time server 101 transmits a synchronization packet Delay_Response to the sensor system 110. At this time, the time server 101 transmits the reception timestamp t4 of the synchronization packet Delay_Request in the synchronization packet Delay_Response to the sensor system 110. When the sensor system 110 receives the synchronization packet Delay_Response, it holds the reception timestamp t4 contained in the synchronization packet Delay_Response.

[0054] The sensor system 110 calculates the average transmission delay time Td from the exchange of synchronization packets Sync, Follow_Up, Delay_Request, and Delay_Response in steps S301 to S304, and corrects the time within the sensor system 110. The average transmission delay time Td can be given by the following equation. Td={(t2-t1)+(t4-t3)} / 2

[0055] Specifically, the sensor system 110 transmits a synchronization packet Delay_Request to the time server 101. Then, the timestamp control unit 210 in Fig. 2 stores a transmission timestamp t3 in the PTP time synchronization processing sequence in a predetermined TS&TAG storage unit #N216. Then, the CPU 200 executes the PTP time synchronization processing using the transmission timestamp t3.

[0056] The following describes the sending of tags to the PHY / MAC 205 in FIG. 2, the allocation process of timestamps and tags, and the PTP time synchronization process.

[0057] Fig. 4 is a flowchart showing packet transmission request processing according to the first embodiment. In the first embodiment, an example is taken in which CPU 200 and video packet processing unit 201 in Fig. 2 issue transmission requests to DMA control unit 204, and each use one transmission request port of DMA control unit 204. Note that steps S402 to S406 represent processing by CPU 200, and steps S412 to S416 represent processing by video packet processing unit 201.

[0058] In step S401 of FIG. 4, this process starts after the sensor system 110 of FIG. 2 is started. Next, in step S402, the CPU 200 determines whether there is a packet to transmit. The packet to transmit is, for example, a TCP / IP packet requested by an application and used for control between the control terminal 103 and the sensor system 110. The packet to transmit may also be a PTP packet for time synchronization between the time server 101 and the sensor system 110. If there is a packet to transmit, the CPU 200 proceeds to step S403, and if there is no packet to transmit, the CPU 200 returns to step S402.

[0059] In step S403, CPU 200 prepares for packet transmission. Specifically, CPU 200 stores transmission data in first memory unit 202 and generates a descriptor and tag for packet transfer by DMA control unit 204. When CPU 200 has completed preparation for packet transmission, it proceeds to step S404.

[0060] In step S404, CPU 200 sets the tag generated in step S403 in register unit 211. Specifically, CPU 200 writes the tag in register unit 211 corresponding to the request port of DMA control unit 204 used by CPU 200. When CPU 200 completes writing the tag, it proceeds to step S405.

[0061] In step S405, the CPU 200 requests packet transmission from the DMA control unit 204. When the CPU 200 completes the packet transmission request, the process proceeds to step S406.

[0062] In step S406, the CPU 200 determines whether to continue the series of processes in steps S402 to S405. At this time, if a factor that causes the communication state with the external device to end has occurred, the CPU 200 proceeds to step S407, and if a factor that causes the communication state with the external device to end has not occurred, the CPU 200 returns to step S402.

[0063] In step S412, the video packet processing unit 201 determines whether there are any packets to transmit. For example, when the control terminal 103 requests the sensor system 110 to capture an image using the camera 111, the video packet processing unit 201 processes the captured image data and generates and transmits a video packet. If there are any packets to transmit, the video packet processing unit 201 proceeds to step S413, and if there are no packets to transmit, the video packet processing unit 201 returns to step S412.

[0064] In step S413, video packet processor 201 prepares for packet transmission. Specifically, video packet processor 201 stores transmission data in second memory unit 203 and generates a descriptor and tag for packet transfer by DMA control unit 204. Note that video packet processor 201 may have a separate memory unit inside and generate descriptors and tags in that memory unit. When video packet processor 201 has completed preparation for packet transmission, it proceeds to step S414.

[0065] In step S414, video packet processing unit 201 sets the tag generated in step S413 in register unit 211. Specifically, video packet processing unit 201 writes the tag in register unit 211 corresponding to the request port of DMA control unit 204 used by video packet processing unit 201. When video packet processing unit 201 completes writing the tag, it proceeds to step S415.

[0066] In step S415, the video packet processor 201 requests packet transmission from the DMA controller 204. When the video packet processor 201 completes the packet transmission request, the process proceeds to step S416.

[0067] In step S416, video packet processor 201 determines whether to continue the series of processes from step S412 to S415. At this time, if a factor that causes communication with the external device to be terminated has occurred, video packet processor 201 proceeds to step S407, and if a factor that causes communication with the external device to be terminated has not occurred, video packet processor 201 returns to step S412.

[0068] When the CPU 200 and the video packet processor 201 proceed to step S407, they end the process. At this time, the CPU 200 and the video packet processor 201 do not issue a packet transmission request to the DMA controller 204, and the sensor system 110 goes into a non-communication state with external devices.

[0069] FIG. 5 is a flowchart showing a packet transmission process according to the first embodiment. In step S501 of FIG. 5, this process starts after the sensor system 110 of FIG. 2 is started. Next, in step S502, the DMA control unit 204 determines whether or not a packet transmission request has been received in step S405 or step S415 of Fig. 4. If a transmission request has been received, the DMA control unit 204 proceeds to step S503, and if a transmission request has not been received, the DMA control unit 204 returns to step S502.

[0070] In step S503, the DMA control unit 204 performs priority control of packet transmission. At this time, the DMA control unit 204 can be provided with dedicated transmission request ports assigned to the CPU 200 and the video packet processing unit 201, respectively. The CPU 200 and the video packet processing unit 201 then issue transmission requests to their respective dedicated transmission request ports. Here, the DMA control unit 204 determines the order of packet transmission in accordance with predetermined priority control rules.

[0071] For example, assume that a priority control rule is set such that even if CPU 200 issues a transmission request before video packet processing unit 201 does, video packets from video packet processing unit 201 are transmitted with the highest priority. In this case, DMA control unit 204 transmits the video packets requested for transmission by video packet processing unit 201 before the packets requested for transmission by CPU 200. Alternatively, the CPU 200 or the video packet processing unit 201 may use different transmission request ports depending on the packet type. For example, a priority control rule may be set so that PTP packets used for time synchronization are transmitted with priority, and video packets or TCP / IP packets used for control are transmitted with lower priority. These priority control rules are merely examples, and the priority control rules are not limited to these. After determining the packet transmission order, the DMA control unit 204 proceeds to step S504.

[0072] In step S504, the DMA control unit 204 transmits packets according to the descriptor in the packet transmission order determined in step S503 to the PHY / MAC 205. When the DMA control unit 204 completes packet transmission, the process proceeds to step S505.

[0073] In step S505, the PHY / MAC 205 receives the packet sent by the DMA control unit 204, and also receives the tag sent by the timestamp control unit 210 in Fig. 2. The tag sent from the timestamp control unit 210 to the PHY / MAC 205 is the tag selected by the selection processing unit 213 based on predetermined information. Upon receiving the packet and tag, the PHY / MAC 205 proceeds to step S506.

[0074] In step S506, the PHY / MAC 205 transmits the packet received from the DMA control unit 204 to the network 208. Then, the PHY / MAC 205 transmits a timestamp and a tag to the timestamp control unit 210 based on the timing of packet transmission completion. After transmitting the packet to the network 208 and the timestamp and a tag to the timestamp control unit 210, the PHY / MAC 205 proceeds to step S507.

[0075] In step S507, the PHY / MAC 205 determines whether to continue the series of processes from step S502 to S506. If a factor that causes communication with the external device to end has occurred, the PHY / MAC 205 proceeds to step S508, and if a factor that causes communication with the external device to end has not occurred, the PHY / MAC 205 returns to step S502.

[0076] When the process proceeds to step S508, the PHY / MAC 205 ends the process. At this time, the CPU 200 and the video packet processor 201 do not issue a packet transmission request to the DMA controller 204, and the sensor system 110 is in a non-communication state with external devices.

[0077] FIG. 6 is a flowchart showing the transmission timestamp control process according to the first embodiment. In step S601 in FIG. 6, the process starts after the sensor system 110 in FIG. 2 is started. Next, in step S602, the timestamp control unit 210 determines whether the CPU 200 or the video packet processing unit 201 wrote a tag to the register unit 211 in step S402 or step S412 of Fig. 4. If a tag has been written to the register unit 211, the timestamp control unit 210 proceeds to step S603, and if a tag has not been written to the register unit 211, the timestamp control unit 210 returns to step S602.

[0078] In step S603, register unit 211 stores the tag written by CPU 200 or video packet processing unit 201 in TAG storage unit #N212. At this time, register unit 211 stores the tag in TAG storage unit #N212 for each transmission request port held by DMA control unit 204. For example, register unit 211 stores the tag written by CPU 200 in TAG storage unit 212 dedicated to CPU 200, and stores the tag written by video packet processing unit 201 in TAG storage unit 212 dedicated to video packet processing unit 201. Here, an example is taken in which CPU 200 and video packet processing unit 201 each use one transmission request port, but CPU 200 and video packet processing unit 201 may use two or more transmission request ports. After register unit 211 writes the tag in TAG storage unit #N212, it transitions to step S604.

[0079] In step S604, the timestamp control unit 210 determines whether to continue the series of processes in steps S602 and S603. If a factor that causes communication with the external device to end has occurred, the timestamp control unit 210 proceeds to step S605, and if a factor that causes communication with the external device to end has not occurred, the timestamp control unit 210 returns to step S602.

[0080] Furthermore, in step S612, the selection processing unit 213 determines whether or not it has detected the packet transmitted by the DMA control unit 204 to the PHY / MAC 205 in step S504 of Fig. 5. For example, the selection processing unit 213 can detect packet transmission when the TVALID signal in AXI4-STREAM is asserted. However, the method for detecting packet transmission is not limited to the method using the AXI4-STREAM interface, and other methods may also be used. If the selection processing unit 213 detects a packet, it proceeds to step S613.

[0081] In step S613, the selection processing unit 213 reads predetermined information by snooping the bus interface between the DMA control unit 204 and the PHY / MAC 205. The predetermined information includes information about the requester that made the transmission request, information about which transmission request port in the DMA control unit 204 the transmission packet is being sent from, the packet type of the transmission packet, or a tag added to the transmission packet. After reading the predetermined information, the selection processing unit 213 proceeds to step S614.

[0082] As a method of reading the specified information, for example, the DMA control unit 204 transmits the transmission request port information on the TUSER, TDEST signal, etc. of AXI4-STREAM, and the selection processing unit 213 performs snooping processing to read the transmission request port information.

[0083] Alternatively, the selection processing unit 213 may read the EtherType field by snooping the MAC header of the packet transmitted by the DMA control unit 204. In this case, the selection processing unit 213 can determine the type of the transmitted packet from the value stored in the EtherType field.

[0084] Alternatively, the CPU 200 or the video packet processing unit 201 may embed tags in the header area or data area of ​​the transmission packets. Then, the selection processing unit 213 may snoop the communication interface between the DMA control unit 204 and the PHY / MAC 205 and read the tags embedded in the header area or data area of ​​the transmission packets. Here, by embedding tags in the header area or data area of ​​the transmission packets, the order of the tags is changed in accordance with a change in the order of the packets output from the DMA control unit 204. Therefore, even if the order of the packets output from the MA control unit 204 is changed, the selection processing unit 213 can read tags that associate packets with timestamps in accordance with the order of the packets output from the DMA control unit 204.

[0085] Alternatively, a dedicated interface for transmitting the predetermined information may be provided in the DMA control unit 204. In this case, the DMA control unit 204 transmits the predetermined information directly to the selection processing unit 213 via the dedicated interface, so that the selection processing unit 213 can acquire the predetermined information.

[0086] In step S614, the selection processing unit 213 selects TAG storage unit #N212 based on the predetermined information read in step S613. For example, the selection processing unit 213 determines whether the packet was sent by the CPU 200 or the video packet processing unit 201 based on the predetermined information read in step S613, and selects TAG storage unit #N212. Alternatively, the selection processing unit 213 may obtain packet type information from EtherType, determine whether the packet was sent by the CPU 200 or the video packet processing unit 201, and select TAG storage unit #N212. When the selection processing unit 213 has completed the selection of TAG storage unit #N212, the process proceeds to step S615.

[0087] In step S615, the selection processing unit 213 transmits the tag stored in the TAG storage unit #N212 determined in step S614 to the PHY / MAC 205. If no tag is stored in the TAG storage unit #N212 determined in step S614, the selection processing unit 213 transmits an arbitrary tag to the PHY / MAC 205, instead of the tag set by the CPU 200 or the video packet processing unit 201. After transmitting the tag to the PHY / MAC 205, the selection processing unit 213 proceeds to step S616.

[0088] In step S616, the selection processing unit 213 determines whether to continue the series of processes from step S612 to S615. If a factor that causes the communication state with the external device to end has occurred, the selection processing unit 213 proceeds to step S605, and if a factor that causes the communication state with the external device to end has not occurred, the selection processing unit 213 proceeds to step S612.

[0089] In step S622, the TS&TAG temporary storage unit 214 temporarily stores the timestamp and tag output by the PHY / MAC 205. Then, the TS&TAG temporary storage unit 214 notifies the allocation processing unit 215 of the completion of storing the timestamp and tag. When the TS&TAG temporary storage unit 214 has completed storing the timestamp and tag, the process proceeds to step S623, and if the storage of the timestamp and tag has not been completed, the process returns to step S622.

[0090] In step S623, the distribution processing unit 215 receives a notification that the timestamps and tags have been stored from the TS&TAG temporary storage unit 214. Then, the distribution processing unit 215 distributes the timestamps and tags stored in the TS&TAG temporary storage unit 214 to the TS&TAG storage unit #N216 based on predetermined information.

[0091] Specifically, the sorting processor 215 receives a notification from the TS&TAG temporary storage unit 214 that the timestamp and tag have been stored, and receives selection information indicating which TAG storage unit #N 212 the selection processor 213 has selected based on predetermined information. The sorting processor 215 then performs sorting based on this information. Alternatively, the CPU 200 and the video packet processor 201 may include sorting condition information in a tag and set it in the register unit 211. The sorting processor 215 may then read the sorting condition information from the tag stored in the TS&TAG temporary storage unit 214 and perform sorting. When the sorting processor 215 completes sorting the timestamp and tag, the process proceeds to step S624.

[0092] In step S624, the distribution processor 215 determines whether to continue the series of processes in steps S622 to S623. If a factor that causes communication with the external device to be terminated has occurred, the distribution processor 215 proceeds to step S605, and if a factor that causes communication with the external device to be terminated has not occurred, the distribution processor 215 returns to step S622.

[0093] When the process proceeds to step S605, the timestamp control unit 210 ends the process. At this time, the CPU 200 and the video packet processing unit 201 do not issue a packet transmission request to the DMA control unit 204, and the sensor system 110 is in a non-communication state with external devices.

[0094] 7 is a flowchart showing the PTP time synchronization process according to the first embodiment. In the PTP time synchronization process, the CPU 200 uses the transmission timestamp of the synchronization packet Delay_Request allocated by the timestamp control unit 210. Then, the CPU 200 performs the PTP time synchronization process between the time server 101 and the sensor system 110.

[0095] In step S701 of FIG. 7, when it is determined that the time server 101 of the synchronized imaging system 100 in FIG. 1 is the PTP master and the sensor system 110 in FIG. 2 is the PTP slave, this process starts.

[0096] Next, in step S702, the sensor system 110 determines whether or not it has received synchronization packets Sync and Follow_Up from the time server 101. If it has received synchronization packets Sync and Follow_Up, the sensor system 110 proceeds to steps S703 and S704, and if it has not received synchronization packets Sync and Follow_Up, it returns to step S702. Here, the transmission and reception processing of the synchronization packets Sync and Follow_Up and the PTP time synchronization processing are performed by the CPU 200 or the video packet processing unit 201 of the sensor system 101. In the following description, it is assumed that the processing of the synchronization packets is performed by the CPU 200.

[0097] In step S703, the CPU 200 acquires the timestamps t1 and t2 of the synchronization packets Sync and Follow_Up. After acquiring the timestamps t1 and t2, the CPU 200 proceeds to step S708.

[0098] In step S704, the CPU 200 transmits a synchronization packet Delay_Request to the time server 101 and acquires a timestamp t3. Specifically, the CPU 200 requests the DMA control unit 204 to transmit a packet and sets the tag of the synchronization packet Delay_Request in the register unit 211 of the timestamp control unit 210. The timestamp control unit 210 then transmits the tag to the PHY / MAC 205. The PHY / MAC 205 then outputs the timestamp and tag at the timing when it completes transmission of the synchronization packet Delay_Request. The timestamp control unit 210 then stores the timestamp and tag in the TS&TAG storage unit #N 216, making them readable from the register unit 211. The DMA control unit 204 then notifies the CPU 200 of the completion of transmission by completing transmission of the synchronization packet Delay_Request. Upon receiving a transmission completion notification from the DMA control unit 204, the CPU 200 verifies that the tag generated when requesting packet transmission matches the tag read from the register unit 211. Then, the CPU 200 acquires the timestamp with the matching tag as the timestamp t3 of the synchronization packet Delay_Request. When the transmission of the synchronization packet Delay_Request is completed, the CPU 200 proceeds to step S705.

[0099] In step S705, the CPU 200 determines whether or not a synchronization packet Delay_Response has been received from the time server 101. If the synchronization packet Delay_Response has been received, the CPU 200 proceeds to step S707, and if the synchronization packet Delay_Response has not been received, the CPU 200 proceeds to step S706.

[0100] In step S706, the CPU 200 determines whether a certain time has passed without receiving a synchronization packet Delay_Response from the time server 101. If the certain time has passed, the CPU 200 returns to step S702, and if the certain time has not passed, the CPU 200 returns to step S705.

[0101] In step S707, the CPU 200 acquires timestamp t4 from the synchronization packet Delay_Response. At this time, the PHY / MAC 205 receives the synchronization packet Delay_Response transmitted from the time server 101, and the DMA control unit 204 DMA-transfers the packet to a predetermined storage unit. The CPU 200 then acquires timestamp t4, the time at which the time server 101 received the synchronization packet Delay_Request, from the synchronization packet Delay_Response. After acquiring timestamp t4, the CPU 200 proceeds to step S708.

[0102] In step S708, CPU 200 uses the acquired timestamps t1 to t4 to calculate the time difference between the time of time server 101 and the time of sensor system 110, and determines the amount of correction. Once CPU 200 determines the amount of time correction, the process proceeds to step S709.

[0103] In step S709, CPU 200 corrects the time of sensor system 110 based on the correction amount determined in step S708. Upon completion of this correction process, sensor system 110 can synchronize its time with that of time server 101. Upon completion of the time correction, CPU 200 proceeds to step S710.

[0104] Here, if the amount of time correction performed at one time is large, it may cause instability in the operation of the entire sensor system 110. Therefore, the CPU 200 may perform the time correction gradually over a certain period of time, or may limit the amount of correction performed at one time.

[0105] In step S710, the sensor system 110 determines whether to continue operating. At this time, the control terminal 103 in FIG. 1 may instruct all of the sensor systems 110a to 110z to stop the synchronization process, or may instruct only the specified sensor systems 110a to 110z to stop the synchronization process. If the sensor system 110 continues operating, it returns to step S702; if it continues operating, it proceeds to step S711 and ends this process.

[0106] As described above, according to the first embodiment, the DMA control unit 204 may transfer communication packets to the PHY / MAC 205 in an order different from the order in which the requester requested the packets to be transmitted. Here, the timestamp control unit 210 acquires predetermined information that can be read from the interface between the DMA control unit 204 and the PHY / MAC 205. Then, based on the predetermined information, the timestamp control unit 210 controls the output of tags that associate communication packets with timestamps. This allows the requester to associate the timestamps output from the PHY / MAC 205 with the communication packets that the requester requested to be transmitted.

[0107] Second Embodiment 2 issues a transmission request to the DMA control unit 204, and uses two or more transmission request ports of the DMA control unit 204. Note that the DMA control unit 204, PHY / MAC 205, and timestamp control unit 210 operate in the same manner as in the first embodiment, and therefore a description thereof will be omitted.

[0108] Fig. 8 is a flowchart showing packet transmission request processing according to the second embodiment. Fig. 8 shows an example in which the CPU 200 issues a transmission request for each packet type to the DMA control unit 204. In the second embodiment, the packet types are TCP / IP packets, PTP packets, and other packets, but this is just an example and the packet types are not limited to these.

[0109] In step S801 of FIG. 8, after the sensor system 110 is started, this process starts. Next, in step S802, the CPU 200 determines whether there is a packet to transmit. The packet to transmit is, for example, a TCP / IP packet requested by an application and used for control between the control terminal 103 and the sensor system 110. The packet to transmit may also be a PTP packet for time synchronization between the time server 101 and the sensor system 110. If there is a packet to transmit, the CPU 200 proceeds to step S803, and if there is no packet to transmit, the CPU 200 returns to step S802.

[0110] In step S803, CPU 200 determines whether the packet transmitted in step S802 is a TCP / IP packet. If the packet transmitted is a TCP / IP packet, CPU 200 proceeds to step S804, and if the packet transmitted is not a TCP / IP packet, CPU 200 proceeds to step S813.

[0111] In step S804, the CPU 200 prepares for packet transmission. Specifically, the CPU 200 stores transmission data in the first storage unit 202 and generates a descriptor and a tag for packet transfer by the DMA control unit 204. When the CPU 200 has completed preparation for packet transmission, the process proceeds to step S805.

[0112] In step S805, the CPU 200 sets the tag generated in step S803 in the register unit 211. Specifically, the CPU 200 writes the tag in the register unit 211 corresponding to the transmission request port of the DMA control unit 204 used by the CPU 200. For example, the CPU 200 uses three transmission request ports for transmitting TCP / IP packets, PTP packets, and other packets. The CPU 200 then writes each tag in the register unit 211 corresponding to each transmission request port. After completing the tag writing, the CPU 200 proceeds to step S806.

[0113] In step S806, CPU 200 requests packet transmission from DMA control unit 204. Specifically, CPU 200 sends transmission requests to transmission request ports of DMA control unit 204 that correspond to TCP / IP packets, PTP packets, and other packets. When CPU 200 completes the transmission requests, it proceeds to step S807.

[0114] In step S813, the CPU 200 determines whether the packet transmitted in step S802 is a PTP packet. If the packet to be transmitted is a PTP packet, the CPU 200 proceeds to step S814, and if the packet to be transmitted is not a PTP packet, the CPU 200 proceeds to step S824.

[0115] The processing in steps S814 to S816 corresponds to the processing in steps S804 to S806. However, in steps S814 to S816, a PTP packet transmission process is carried out. When the CPU 200 completes the processing in step S816, the process proceeds to step S807.

[0116] The processing in steps S824 to S826 corresponds to the processing in steps S804 to S806. However, in steps S824 to S826, transmission processing is performed for transmission packets other than TCP / IP packets and PTP packets. Upon completing the processing in step S826, the CPU 200 proceeds to step S807.

[0117] In step S807, CPU 200 determines whether to continue the series of processes in steps S802 to S806, S813 to S816, and S824 to S826. At this time, if a factor that causes communication with the external device to end has occurred, CPU 200 proceeds to step S808, and if a factor that causes communication with the external device to end has not occurred, CPU 200 returns to step S802.

[0118] When the CPU 200 proceeds to step S808, it ends the process. At this time, the CPU 200 does not issue a packet transmission request to the DMA control unit 204, and the sensor system 110 is in a non-communication state with the external device.

[0119] As described above, according to the second embodiment, the DMA control unit 204 transfers communication packets to the PHY / MAC 205 in an order different from the order in which the requester made the transmission requests. Even in this case, the requester can use two or more transmission request ports of the DMA control unit 204, and can associate the timestamps output from the PHY / MAC 205 with the communication packets requested to be transmitted by the requester.

[0120] <Third embodiment> 2 sends a transmission request to the DMA control unit 204, and the CPU 200 and the video packet processing unit 201 each use two or more transmission request ports of the DMA control unit 204. Note that the DMA control unit 204, PHY / MAC 205, and timestamp control unit 210 operate in the same manner as in the first embodiment, and therefore a description thereof will be omitted.

[0121] Fig. 9 is a flowchart showing a packet transmission request process according to the third embodiment. Note that Fig. 9 shows an example in which the CPU 200 and the video packet processing unit 201 issue a transmission request to the DMA control unit 204 for each packet type.

[0122] In step S901 of Fig. 9, this process starts after starting up the sensor system 110. Note that the processes of steps S802 to S807, S813 to S816, and S824 to S826 of Fig. 9 are the same as those of Fig. 8, and therefore description thereof will be omitted. In step S902, the video packet processing unit 201 determines whether there are any packets to transmit. For example, when the control terminal 103 requests the sensor system 110 to capture an image using the camera 111, the video packet processing unit 201 processes the captured image data to generate and transmit a video packet. The video packet processing unit 201 may also process data other than video packets to generate and transmit a packet. If there are any packets to transmit, the video packet processing unit 201 proceeds to step S903, and if there are no packets to transmit, the video packet processing unit 201 returns to step S902.

[0123] In step S903, the video packet processing unit 201 determines whether the packet to be transmitted in step S902 is a video packet. If the packet to be transmitted is a video packet, the video packet processing unit 201 proceeds to step S904, and if the packet to be transmitted is not a video packet, the video packet processing unit 201 proceeds to step S914.

[0124] In step S904, video packet processing unit 201 prepares for packet transmission. Specifically, video packet processing unit 201 stores transmission data in second storage unit 203 and generates a descriptor and a tag for packet transfer by DMA control unit 204. Alternatively, video packet processing unit 201 may have a separate storage unit inside and generate the descriptor and the tag in that storage unit. When video packet processing unit 201 has completed preparation for packet transmission, it proceeds to step S905.

[0125] In step S905, video packet processing unit 201 sets the tag generated in step S904 in register unit 211. Specifically, video packet processing unit 201 writes the tag in register unit 211 corresponding to the request port of DMA control unit 204 used by video packet processing unit 201. For example, video packet processing unit 201 uses two transmission request ports to transmit video packets or other packets. Then, video packet processing unit 201 writes each tag in register unit 211 corresponding to each transmission request port. When video packet processing unit 201 has completed writing the tag, it proceeds to step S906.

[0126] In step S906, the video packet processing unit 201 requests packet transmission from the DMA control unit 204. Then, the video packet processing unit 201 issues a transmission request to the transmission request ports of the DMA control unit 204 that correspond to the video packets and other communication packets. When the video packet processing unit 201 completes the transmission requests, the process proceeds to step S907.

[0127] The processing of steps S914 to S916 corresponds to the processing of steps S904 to S906. However, in steps S914 to S916, the video packet processing unit 201 performs processing for transmitting communication packets other than video packets. Upon completing the processing of step S916, the video packet processing unit 201 proceeds to step S907.

[0128] In step S907, the video packet processor 201 determines whether to continue the series of processes in steps S902 to S906 and S914 to S916. At this time, if a factor that causes the communication state with the external device to end has occurred, the video packet processor 201 proceeds to step S908, and if a factor that causes the communication state with the external device to end has not occurred, the video packet processor 201 returns to step S902.

[0129] When the process proceeds to step S908, the video packet processor 201 ends the process. At this time, the video packet processor 201 does not issue a packet transmission request to the DMA controller 204, and the sensor system 110 is in a non-communication state with external devices.

[0130] As described above, according to the third embodiment, the DMA control unit 204 transfers communication packets to the PHY / MAC 205 in an order different from the order in which each of the multiple requesters made a transmission request. Even in this case, each of the multiple requesters can use two or more transmission request ports of the DMA control unit 204, and it is possible to associate the timestamps output from the PHY / MAC 205 with the communication packets requested to be transmitted by the requesters.

[0131] <Fourth embodiment> FIG. 10 is a block diagram showing an example of the configuration of a camera adapter according to the fourth embodiment. 10, the sensor system 110′ includes a camera adapter 112′ instead of the camera adapter 112 in FIG. 2. The camera adapter 112′ includes a FIFO 220 in addition to the camera adapter 112 in FIG. 2. The FIFO 220 is provided between the DMA control unit 204 and the PHY / MAC 205. Here, when the timestamp control unit 210 inputs a tag to the PHY / MAC 205 simultaneously with or before the communication packet, the DMA control unit 204 stores the communication packet in the FIFO 220 and then inputs the communication packet to the PHY / MAC 205.

[0132] As described above, according to the fourth embodiment, by providing a FIFO 220 between the DMA control unit 204 and the PHY / MAC 205, it is possible to adjust the timing from reading the specified information to adding a tag to a transmission packet.

[0133] <Other embodiments> The present invention may be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium. One or more functions of the above-described embodiments may also be realized by a process in which one or more processors in a computer of the system or device read and execute the program. Alternatively, one or more functions may also be realized by a circuit (e.g., an FPGA or an ASIC) that realizes the functions. [Explanation of symbols]

[0134] 100 synchronized imaging system, 101 time server, 102 hub, 103 control terminal, 104 image computing server, 105 user terminal, 107 GNSS, 110 sensor system

Claims

1. request means for making a packet transmission request; a forwarding means for forwarding the packet based on a transmission request for the packet; a transmitting means for transmitting the packet transferred by the transferring means and outputting a timestamp based on the transmission of the packet; a control means for controlling output of a tag that associates the packet with the timestamp based on predetermined information that can be read from an interface between the transfer means and the transmission means; A communication device comprising:

2. 2. The communication device according to claim 1, wherein the control means reads the predetermined information based on snooping of an interface between the transfer means and the transmission means.

3. The control means inputs tags that associate the packets with the timestamps to the transmission means in accordance with the order of the packets transferred by the transfer means.

3. The communication device according to claim 1 or 2.

4. 4. A communication device according to claim 1, wherein the predetermined information includes at least one of information requested for transmission at a transmission request port of the transfer means, a packet type of a packet transferred by the transfer means, and a tag embedded in a header area and a data area of ​​the packet.

5. The control means a plurality of first queues for storing tags generated by the request means; a selection processing means for selecting one first queue from the plurality of first queues based on the predetermined information and outputting the tag stored in the selected first queue to the transmission means; 5. The communication device according to claim 1, further comprising:

6. The control means a plurality of second queues for storing timestamps and tags output from the transmitting means; For each of the requesting means, for each of the transmission request ports to which the requesting means has made a transmission request to the transferring means, The above-mentioned method is performed based on the type of packet transferred by the transfer means or a combination thereof. a distribution processing means for distributing the timestamp and the tag to the second queue; 6. The communication device according to claim 1, further comprising:

7. a first-in, first-out storage means provided between the transfer means and the transmission means; 7. The communication device according to claim 1, wherein when the control means inputs the tag to the transmission means at the same time as the packet or before the packet, the transfer means stores the packet in the memory means before inputting it to the transmission means.

8. 8. The communication device according to claim 1, wherein the requesting means is compatible with PTP (Precision Time Protocol).

9. making a request to transmit a packet; forwarding the packet based on a request to transmit the packet; transmitting the forwarded packet and outputting a timestamp based on the transmission of the packet; a step of controlling output of a tag that associates the packet with the timestamp based on predetermined information that can be read from an interface used when transferring the packet; A communication method comprising:

10. For causing a computer to operate as a communication device according to any one of claims 1 to 8 program.

Images