Information processing device, communication system, control method and program for information processing device

Abstract

The objective is to provide an information processing device, a communication system, a control method for the information processing device, and a program that, with a simple configuration, can represent the data acquisition timing of a sensor periodically driven on one side using a time managed on the other side. [Solution] The image processing device 100 includes an information generation means (communication unit 102) that generates reception time information relating to the reception time based on the timing of packet reception, each time a packet is received; an information storage means (control unit 101) that sequentially stores the reception time information each time it is generated by the information generation means; and a packet information assignment means (control unit 101) that assigns the oldest reception time information among the reception time information stored in the information storage means to the packet each time a packet is received.

Description

Technical Field

[0001] 6This invention relates to an information processing apparatus, a communication system, a control method for an information processing apparatus, and a program.

Background Art

[0002] 7Conventionally, a communication system having a plurality of communication devices (communication equipment) and in which these communication devices are connected to each other so as to be capable of wireless communication is known. In this communication system, synchronization is taken between the communication devices. For example, Patent Document 1 discloses a wireless communication system including a master device and a plurality of slave devices communicably connected to the master device via wireless communication. In the wireless communication system described in Patent Document 1, the timing at which each slave device drives a sensor is synchronized according to an instruction from the master device. Further, Patent Document 2 discloses a wireless synchronization system including a first device and a second device communicably connected to the first device via wireless communication. In the wireless synchronization system described in Patent Document 2, the first device and the second device each have a device that operates according to a synchronization signal.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In communication systems, for example, there is a need to express the data acquisition timing of a sensor periodically driven by one of two communication devices using a time managed by the other device. However, neither the system described in Patent Document 1 nor the system described in Patent Document 2 can meet this requirement. In other words, there is a problem in that the data acquisition timing of a sensor periodically driven by one side cannot be expressed using a time managed by the other side. Furthermore, meeting the above requirement may lead to a complex system configuration.

[0005] The present invention has been made in view of the above-mentioned problems. The present invention aims to provide an information processing device, a communication system, a control method for the information processing device, and a program that, with a simple configuration, can express the data acquisition timing of a sensor periodically driven on one side using a time managed on the other side. [Means for solving the problem]

[0006] To achieve the above objective, the present invention provides an information processing device that is communicably connected to a transmitting device having a transmitting means for periodically transmitting packets, and has a receiving means for receiving packets in synchronization with the transmission cycle of the packets transmitted from the transmitting means, wherein the transmitting device has a data generation means for periodically generating data, a data information assignment means for assigning transmission time information relating to the transmission time based on the transmission timing of the packets transmitted from the transmitting means to the data, and a packet generation means for generating a first packet to be transmitted by the transmitting means, which includes the data to which the transmission time information has been assigned, and the information processing device is characterized by having an information generation means for generating reception time information relating to the reception time based on the reception timing of the first packet received by the receiving means, an information storage means for sequentially storing the reception time information each time the reception time information is generated by the information generation means, and a packet information assignment means for assigning the oldest reception time information among the reception time information stored in the information storage means to the first packet each time the first packet is received by the receiving means. [Effects of the Invention]

[0007] According to the present invention, with a simple configuration, the data acquisition timing of a sensor that is periodically driven on one side can be represented by a time managed on the other side. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example of the hardware configuration of a communication system according to the first embodiment. [Figure 2] This sequence diagram shows the processes executed between the communication unit of the image processing device and the communication unit of the controller when BLE communication (BLE connection) is established. [Figure 3] This diagram illustrates the synchronization timing of BLE communication processing performed in the controller's communication unit. [Figure 4]This is a timing chart showing the execution timing of processes performed by the image processing device and controller. [Figure 5] This flowchart shows the processes executed by the controller. [Figure 6] This is a flowchart showing the processes performed by the image processing device. [Figure 7] This is a timing chart showing the execution timing of processes performed by the image processing apparatus and controller according to the second embodiment. [Modes for carrying out the invention]

[0009] The embodiments of the present invention will be described in detail below with reference to the drawings. However, the configurations described in the following embodiments are merely illustrative, and the scope of the present invention is not limited to the configurations described in each embodiment. For example, each part constituting the present invention can be replaced with any configuration that can perform a similar function. In addition, any configuration may be added. Furthermore, any two or more configurations (features) from each embodiment can be combined.

[0010] <First Embodiment> The first embodiment will be described below with reference to Figures 1 to 6. Figure 1 is a block diagram showing an example of the hardware configuration of a communication system according to the first embodiment. As shown in Figure 1, the communication system 1000 has an image processing device (information processing device) 100 and a controller (transmitter) 200, which are connected to each other wirelessly. The controller 200 is an operating device for operating the image processing device 100. When a user operation is performed via the controller 200, the image processing device 100 performs processing according to the user operation. In this embodiment, the image processing device 100 is a head-mounted display (HMD) that is detachably attached to the user's head. Also, in this embodiment, the user who operates the controller 200 and the user who wears the image processing device 100 are the same user. That is, the user operates the controller 200 while wearing the image processing device 100 on their head. The image processing device 100 can display a composite image which is a composite of an image captured from the area in front of the user and content such as CG (computer graphics) in a form corresponding to the user's posture relative to the image processing device 100. The image processing device 100 is not limited to a head-mounted display; for example, it may be a handheld display that can be held and used by the user.

[0011] As shown in Figure 1, the image processing device 100 includes a control unit 101, a communication unit 102, an inertial detection unit 103, an imaging unit 104, and an image display unit 105, which are connected to each other so as to be able to communicate with one another. The control unit 101 includes, for example, one or more processors (not shown), such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage unit (not shown) for storing various programs, etc. The processor controls the entire image processing device 100 by executing programs. The various programs include, for example, programs that cause the processor (computer) to execute each process (control method of the information processing device) described later. The storage unit includes, for example, a non-volatile memory such as Flash ROM for storing control programs executed by the processor, and RAM used as a working area necessary for executing the control programs. The control unit 101 also has a timer (not shown) for measuring time. The communication unit 102 transmits and receives wireless packets (packets) with the controller 200, that is, it functions as a transmitting means for sending wireless packets to the controller 200 and as a receiving means for receiving wireless packets from the controller 200. The transmission of wireless packets is performed periodically. The communication unit 102 is also capable of communication compliant with the Bluetooth® low energy standard (hereinafter sometimes referred to as "BLE" or "BLE communication"). This enables communication with low power consumption, for example. The communication method of the communication unit 102 is not limited to BLE communication. The communication unit 102 consists of, for example, an antenna, a modulation / demodulation circuit, a processor for media access control, and a timer for timing control.

[0012] The inertial detection unit 103 is composed of, for example, an IMU (Inertial Measurement Unit). The inertial detection unit 103 detects inertial information related to the inertia generated in the image processing device 100. The inertial information is not particularly limited and may include, for example, the amount of movement of the image processing device 100 when a user moves in real space while wearing the image processing device 100 on their head, or data such as the rotation angle when the user changes their posture in real space. The inertial detection unit 103 detects the inertial information periodically, i.e., at predetermined intervals, according to instructions from the control unit 101. This inertial information is stored in a memory (not shown) built into the inertial detection unit 103. The inertial detection unit 103 can also receive a synchronization signal (hereinafter sometimes referred to as "FSYNC") from the control unit 101. When the inertial detection unit 103 receives a synchronization signal, it measures the time from the timing of receiving the synchronization signal to the detection of the inertial information (hereinafter sometimes referred to as "FSYNC time"). The FSYNC time is stored in the memory built into the inertia detection unit 103. The inertia detection unit 103 outputs the inertia information and FSYNC time stored in the memory according to instructions from the control unit 101. After this output, the inertia detection unit 103 erases the inertia information and FSYNC time stored in the memory.

[0013] The imaging unit 104 is composed of, for example, multiple cameras. The imaging unit 104 is driven according to instructions from the control unit 101 to capture images of a subject at predetermined intervals, obtain captured images, and periodically output the captured images to the control unit 101. The imaging unit 104 also outputs signals synchronized with the imaging timing to the control unit 101. The image display unit 105 is composed of, for example, a liquid crystal panel or an organic EL panel. The image display unit 105 displays images according to instructions from the control unit 101. The control unit 101 generates an image that can be displayed on the image display unit 105 based on, for example, the captured images from the imaging unit 104 and the data received from the controller 200, and transmits it to the image display unit 105. This allows the image to be displayed on the image display unit 105.

[0014] As shown in Figure 1, the controller 200 includes a control unit 201, a communication unit 202, an inertia detection unit (data generation means) 203, and a button operation unit 204, which are connected to each other so as to be able to communicate with one another. The control unit 201, like the control unit 101, includes one or more processors (not shown), such as a CPU or MPU, and a storage unit (not shown) for storing various programs. The processor controls the entire controller 2000 by executing programs. The storage unit includes, for example, a non-volatile memory such as Flash ROM for storing control programs executed by the processor, and RAM used as a working area necessary for executing the control programs. The control unit 201 also has a timer (not shown) for measuring time. The communication unit 202 transmits and receives wireless packets with the image processing device 100, that is, it has the function of a transmitting means for sending wireless packets to the image processing device 100 and the function of a receiving means for receiving wireless packets from the image processing device 100. The reception of wireless packets is synchronized with the transmission cycle of packets transmitted from the image processing device 100. Furthermore, the communication unit 202, like the communication unit 102, is capable of BLE-compliant communication. However, the communication method in the communication unit 202 is not limited to BLE communication. The communication unit 202 consists of, for example, an antenna, a modulation / demodulation circuit, a processor that performs media access control, and a timer that controls timing. The inertia detection unit 203, like the inertia detection unit 103, consists of, for example, an IMU. The inertia detection unit 203 periodically detects (generates) inertia information (data) related to the inertia that occurs when the controller 200 is operated. The inertia information is not particularly limited and could include, for example, the amount of movement of the controller 200 when a user moves their arm in real space while holding the controller 200, or the rotation angle when the arm is rotated in real space. The button operation unit 204 consists of, for example, a pressable button or a touch sensor. The button operation unit 204 outputs operation information such as whether a button is pressed or whether a touch is detected by the touch sensor to the control unit 201.

[0015] Figure 2 is a sequence diagram showing the processing performed by the communication unit of the image processing device and the communication unit of the controller when BLE communication (BLE connection) is established. In this embodiment of BLE communication, the communication unit 102 of the image processing device 100 functions as the central unit, and the communication unit 202 of the controller 200 functions as the peripheral unit. As shown in Figure 2, when BLE communication is established, the communication unit 102 and the communication unit 202 repeat a connection event at each connection interval. The connection interval can be set to any natural number multiple of 1.25 msec, for example, within the range of 7.5 msec to 4 sec. This connection interval is determined by the communication unit 102, which functions as the central unit. The connection interval determined by the communication unit 102 is then notified to the communication unit 202, which functions as the peripheral unit. As a result, the communication unit 102 and the communication unit 202 can communicate with each other at the same period, that is, at the same timing. Furthermore, in a connection event, the central communication unit 102 sends a packet to the peripheral communication unit 202 (see TX102 in Figure 2). This allows communication unit 202 to receive the packet from communication unit 102 (see RX202 in Figure 2). Then, communication unit 202 sends a response packet to communication unit 102 for the packet it received from communication unit 102 (see TX202 in Figure 2). This allows communication unit 102 to receive the response packet from communication unit 202 (see RX102 in Figure 2).

[0016] Figure 3 is a diagram illustrating the synchronization timing of BLE communication processing performed by the controller's communication unit. This Figure 3 is a graph showing the change in current consumption over time in the communication unit 202, which functions as a peripheral. The connection interval here is set to 15 msec. As shown in Figure 3, the communication unit 202 repeatedly cycles between a sleep state and a wake state, i.e., a wake state, once BLE communication is established with the communication unit 102. This cycle also occurs in the communication unit 102. A connection event occurs while the communication unit 202 is in the wake state, and the receive process (RX202) and transmit process (TX202) are executed. As shown in Figure 3, power consumption increases at the timing when each process is executed. When the connection event is completed, the communication unit 202 sets the wake time on the timer based on the time of the current connection event and the connection interval, so that it enters the wake state a predetermined time before the next connection event starts, and then transitions to the sleep state. After transitioning to the sleep state, when time has elapsed and it is time to wake up, the communication unit 202 transitions to the wake-up state via a timer interrupt. After transitioning to the wake-up state, the communication unit 202 outputs a control signal to the control unit 201 indicating the wake-up timing of the communication unit 202 at, for example, times t_w1, t_w2, and t_w3. The communication unit 102, which functions as the central unit, operates in the same manner as the communication unit 202. Specifically, the communication unit 102 outputs a control signal to the control unit 101 indicating the wake-up timing of the communication unit 102 at times t_w1, t_w2, and t_w3.

[0017] FIG. 4 is a timing chart showing the execution timing of processes executed by the image processing apparatus and the controller. FIG. 5 is a flowchart showing the processes executed by the controller. FIG. 6 is a flowchart showing the processes executed by the image processing apparatus. Referring to FIGS. 4 and 5, the processes executed by the controller 200 will be described, and referring to FIGS. 4 and 6, the processes executed by the image processing apparatus 100 will be described. First, the processes executed by the controller 200 will be described. The program based on the flowchart shown in FIG. 5 starts when the controller 200 is connected to the image processing apparatus 100 by BLE communication. As shown in FIG. 5, in step S501, the control unit 201 of the controller 200 starts the operation of the inertial detection unit 203. Thereby, the inertial detection unit 203 detects and acquires inertial information at a predetermined period (data generation step). In the present embodiment, the inertial detection unit 203 acquires inertial information every 5 msec, which is a divisor of the connection interval of 15 msec. As shown in the timing chart of the IMU in FIG. 4, the acquisition timings of the inertial information are time t_i1, time t_i2, and time t_i3.

[0018] In step S502, the control unit 201 determines whether or not it has received a control signal indicating the wake-up timing from the communication unit 202. As a result of the determination in step S502, if the control unit 201 determines that it has received the control signal, the process proceeds to step S503. On the other hand, as a result of the determination in step S502, if the control unit 201 determines that it has not received the control signal, the process waits at step S502. As shown in the timing chart of BLE_Wake in FIG. 4, the reception timings of the control signal are time t_w1, time t_w2, time t_w3, and time t_w4.

[0019] In step S503, the control unit 201 outputs a synchronization signal (FSYNC) indicating the same timing as the BLE_Wake. As shown in the timing chart of IMU_FSYNC in FIG. 4, the synchronization signal is output at the same timing as time t_w1, time t_w2, time t_w3, and time t_w4. When the inertial detection unit 203 receives the synchronization signal from the control unit 201, it measures (generates) the time (hereinafter referred to as "FSYNC time") from the reception of the synchronization signal to the detection of inertial information. As the FSYNC time, for example, the time from the time when the synchronization signal (the first synchronization signal) of the IMU_FSYNC at the same timing as time t_w1 is emitted to time t_i2 (= t_fsync1) is measured. Therefore, the FSYNC time becomes transmission time information regarding the transmission time based on the wake-up timing in the communication unit 202. Specifically, the FSYNC time is the difference between the wake-up timing (packet transmission timing) in the communication unit 202 and the acquisition timing of inertial information. Then, the control unit 201 assigns the FSYNC time to the inertial information acquired in step S501 (data information assignment step). Thus, in this embodiment, the control unit 201 also functions as data information assignment means for assigning the FSYNC time. Note that in the controller 200, the part that functions as data information assignment means may be provided separately from the control unit 201. Also, in this embodiment, a synchronization signal is output from the control unit 201 to the inertial detection unit 203, but this is not limiting. For example, a synchronization signal may be output from the communication unit 202 to the inertial detection unit 203.

[0020] In step S504, the control unit 201 starts the timer so that the timer ends after time t_delay. Time t_delay will be described later.

[0021] In step S505, the control unit 201 determines whether the timer started in step S504 has finished, that is, whether time has expired. If the control unit 201 determines that time has expired as a result of the determination in step S505, the process proceeds to step S506. On the other hand, if the control unit 201 determines that time has not expired as a result of the determination in step S505, the process remains in a waiting state at step S505.

[0022] In step S506, the control unit 201 acquires the inertial information to which the FSYNC time was assigned in step S503. The number of inertial information entries acquired in step S506 is the same as the number of inertial information entries stored in the inertial detection unit 203 up to the previous step S506.

[0023] In step S507, the control unit 201 transmits the inertia information generated in step S506 to the communication unit 202. As shown in the data transmission timing chart in Figure 4, the transmission timings for the inertia information are time t_a1, t_a2, t_a3, and t_a4. Before transmitting the inertia information, button information may be obtained from the button operation unit 204 and added to the inertia information. When the communication unit 202 receives the inertia information from the control unit 201, it generates one packet (first packet) that includes the inertia information and can be transmitted to the image processing device 100 (packet generation step). Thus, in this embodiment, the communication unit 202 also functions as a packet generation means for generating packets. In the controller 200, the part that functions as a packet generation means may be provided separately from the communication unit 202. After setting the packet in the communication queue, the communication unit 202 transitions to a sleep state. The packet set in the communication queue is transmitted to the communication unit 102 of the image processing device 100 in the next connection event. After step S507 is executed, the process returns to step S502 and the subsequent steps are executed in order. The control unit 201 terminates the process shown in the flowchart in Figure 5, for example, if the power to the controller 200 is turned off or if BLE communication with the image processing device 100 is disconnected.

[0024] In this embodiment, the control signal indicating the wake-up timing of the communication unit 202 is generated every 15 msec, which is the period of the BLE connection interval, and the operating period of the inertia detection unit 203 is 5 msec. Therefore, in most cases, the number of inertia information items acquired in step S506 will be three. The period of the BLE connection interval is generated based on the internal clock of the communication unit 202. The operating period of the inertia detection unit 203 is generated based on the internal clock of the inertia detection unit 203. The internal clock of the communication unit 202 and the internal clock of the inertia detection unit 203 operate independently of each other. Therefore, due to the difference between the two clocks, the number of inertia information items acquired in step S506 may be two or four. In step S507, the communication unit 202 receives 2 to 4 pieces of inertia information and sets these pieces of inertia information together into a single packet in the communication queue. In this way, the communication unit 202 can include multiple pieces of data of the same type generated by the inertia detection unit 203 into a single packet.

[0025] Here, we will explain the time t_delay. In this embodiment, it is necessary to generate a control signal that indicates the wake-up timing of the communication unit 202. Therefore, the communication unit 202 needs to ensure a period of sleep between the completion of a connection event and the next connection event. The time t_delay is set to satisfy the following equation (1).

[0026]

number

[0027] In equation (1), "t_I" is the connection interval, "tS" is the minimum period during which the communication unit 202 can enter a sleep state, "t106" is the processing time for step S506, and "t107" is the processing time for step S507. Specifically, "t106" is a fixed value determined according to the processing time required for the control unit 201 to acquire inertia information from the inertia detection unit 203. "t107" is a fixed value determined according to the time required for the control unit 201 to send a packet to the communication unit 202 and the time required for the communication unit 202 to generate the packet and set it in the communication queue.

[0028] Next, the processing performed by the image processing device 100 will be described with reference to Figures 4 and 6. The program based on the flowchart shown in Figure 6 is started when the image processing device 100 is connected to the controller 200 by BLE communication. As shown in Figure 6, in step S601, the control unit 101 of the image processing device 100 determines whether or not it has received a control signal indicating the wake-up timing from the communication unit 102. If the control unit 101 determines that it has received a control signal as a result of the determination in step S601, the process proceeds to step S602. On the other hand, if the control unit 101 determines that it has not received a control signal as a result of the determination in step S601, the process proceeds to step S603. As shown in the BLE_Wake timing chart in Figure 4, the timings for receiving the control signal are times t_w1, t_w2, t_w3, and t_w4.

[0029] In step S602, the control unit 101 obtains the time at which it received the control signal from the communication unit 102 and stores it in the RAM of the control unit 101 as time information relating to that time. As shown in the timing chart of reception time storage and BLE data reception in Figure 4, the reception timings of the time information are time t_c1, time t_c2, time t_c3, and time t_c4. The time information relating to the time of reception of the control signal is also reception time information relating to the reception time based on the reception timing of the packet from the communication unit 202 received by the communication unit 102. This time information is generated each time a packet is received by the communication unit 102 (information generation step). Each time reception time information is generated, the reception time information is sequentially stored in the RAM of the control unit 101 (information storage step). Thus, in this embodiment, the communication unit 102 also functions as an information generation means for generating reception time information. In the image processing device 100, the part that functions as an information generation means may be provided separately from the communication unit 102. Also, the RAM of the control unit 101 functions as an information storage means for storing reception time information. In the image processing device 100, the part that functions as an information storage means may be provided separately from the communication unit 102. Furthermore, the time information may be information relating to the time since the control unit 101 was started, or information relating to the time since the signal was input from the imaging unit 104. Also, if the signal from the imaging unit 104 is used as a reference, the amount of deviation from the imaging time performed by the imaging unit 104 can be recorded.

[0030] In step S603, the control unit 101 determines whether the communication unit 102 has received the packet (BLE data) sent to the image processing device 100 in step S507. If the control unit 101 determines that the packet has been received as a result of the determination in step S603, the process proceeds to step S604. On the other hand, if the control unit 101 determines that the packet has not been received as a result of the determination in step S603, the process returns to step S601 and the subsequent steps are executed in order.

[0031] In step S604, the control unit 101 acquires the packet received by the communication unit 102.

[0032] In step S605, the control unit 101 determines whether the packet determined to have been received in step S603 is the first packet (see time d0 in Figure 4) since the start of the program based on the flowchart shown in Figure 6. If the control unit 101 determines, as a result of the determination in step S605, that it is the first packet, the process returns to step S601 and the subsequent steps are executed in order. On the other hand, if the control unit 101 determines, as a result of the determination in step S605, that it is not the first packet, the process proceeds to step S606.

[0033] In step S606, the control unit 101 stores the packet acquired in step S604 in its RAM. As shown in the timing chart for receiving time storage and BLE data reception in Figure 4, the packet storage timings are times d1, d2, d3, and d4. Due to the processing in steps S605 and S606, the first packet received after starting the program based on the flowchart shown in Figure 6 is discarded without being processed, while subsequent packets are stored in the RAM of the control unit 101 and processed in subsequent steps.

[0034] In step S607, the control unit 101 adds the time information saved in step S602 to the packet saved in step S606 (packet information addition step). Thus, in this embodiment, the control unit 101 also functions as a packet information addition means that adds time information to a packet each time a packet is received. Note that in the image processing device 100, the part that functions as a packet information addition means may be provided separately from the control unit 101. Here, depending on the BLE communication state, there may be multiple time information saved in step S602. In this case, the oldest time information among the multiple time information is added to the packet. As shown in the timing chart of reception time saving and BLE data reception in Figure 4, for example, a packet saved at time d1 is added with time information received at time t_c1. Similarly, a packet saved at time d2 is added with time information received at time t_c2, and a packet saved at time d3 is added with time information received at time t_c3.

[0035] In step S608, each time the oldest reception time information is assigned in step S607, the control unit 101 erases (deletes) the reception time information from the RAM of the control unit 101. Thus, in this embodiment, the control unit 101 also functions as an erasure means for erasing reception time information. Note that in the image processing device 100, the part that functions as an erasure means may be provided separately from the control unit 101.

[0036] In step S609, the control unit 101 performs processing based on the packet obtained in step S607. For example, the control unit 101 performs processing using the associated time information (t_cN), the inertia information of the inertia detection unit 203 contained in the packet, and the FSYNC time (t_fsyncN) (where "N" is a natural number). In this way, the control unit 101 also functions as a processing execution means that performs processing based on packets. Note that in the image processing device 100, the part that functions as a processing execution means may be provided separately from the control unit 101. After step S609 is executed, the process returns to step S601 and the subsequent steps are executed in order. Note that the control unit 101 terminates the processing shown in the flowchart in Figure 6 if, for example, the power to the image processing device 100 is turned off or BLE communication with the controller 200 is disconnected.

[0037] For example, the time t_i2 at which the inertia detection unit 203 of the controller 200 detects inertia information can be expressed using time t_c1 and time t_fsync1 as shown in equation (2) below.

[0038]

number

[0039] Here, the inertia detection unit 203 operates with a period of 5 msec, so time t_i1 can be expressed as shown in equation (3) below, and time t_i3 can be expressed as shown in equation (4) below.

[0040]

number

[0041] The times t_i1 to t_i3, when the inertial detection unit 203 of the controller 200 detects inertial information, are the data acquisition timings of the inertial detection unit 203, which is periodically driven on the controller 200 side (one side). Equations (2) to (4) allow times t_i1 to t_i3 to be expressed in terms of time managed on the image processing device 100 side (the other side). The control unit 101 can acquire the time-dependent change in the attitude of the controller 200 at the time on the image processing device 100 by using times t_i1 to t_i3 and the inertial information from the inertial detection unit 203. Furthermore, if the time information of the image processing device 100 is based on the signal from the imaging unit 104, the attitude of the controller 200 can be acquired based on the operation timing of the imaging unit 104.

[0042] Through the control described above (hereinafter referred to as "timing control"), when a user of the image processing device 100 operates it using the controller 200, it becomes possible to achieve operation that matches the user's intention, thereby improving the user experience. For example, consider a case where an image of a car is displayed on the image display unit 105 of the image processing device 100, and the user projects a ray from the controller 200. In this case, on the image display unit 105, the ray will appear to be projected toward the car. The user intends to project the ray toward the car's license plate, but without timing control, the ray may end up being projected toward the car's headlights. Therefore, by using timing control, that is, by representing the time on the controller 200 with the time managed by the image processing device 100 as described above, the ray will be projected toward the car's license plate as intended by the user.

[0043] <Second Embodiment> The second embodiment will be described below with reference to Figure 7, focusing on the differences from the previously described embodiment, and similar matters will be omitted. The first embodiment is an embodiment in which wireless communication is successful in each connection event of BLE communication, but wireless communication may fail due to, for example, aging of the image processing device 100 or controller 200, interference from other wireless communication, etc. This embodiment will describe the case in which wireless communication fails. Figure 7 is a timing chart showing the execution timing of the processes performed by the image processing device and controller according to the second embodiment. As shown in Figure 7, in the BLE communication connection events CE0 and CE1, wireless communication is successful, and packets are saved at times d0 and d1. On the other hand, in the connection event CE2, wireless communication fails. In the subsequent connection event CE3, wireless communication is successful, packets that could not be saved in connection event CE2 are saved at time d2, and packets that should have been saved in connection event CE2 are saved at time d3.

[0044] When this situation is applied to the flowchart shown in Figure 6, in step S606, if the control unit 101 of the image processing device 100 receives a packet from the communication unit 102 that could not be saved in connection event CE2, it saves the packet to the RAM of the control unit 101.

[0045] In step S607, the control unit 101 associates time t_c2, which is the oldest time information, with the packet saved in step S606, since time t_c2 and time t_c3 are stored in RAM as time information.

[0046] In step S608, the control unit 101 erases time t_c2 from its RAM. Subsequently, if the control unit 101 receives a packet that should be saved in connection event CE2, in step S607, only time t_c3 is saved as time information in the control unit 101's RAM, and time t_c3 is associated with the packet. In this way, even if wireless communication fails, it is possible to accurately associate the packet received afterward with the time information.

[0047] While preferred embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of its gist. The present invention provides a program that implements one or more functions of the embodiments described above to a system or device via a network or storage medium. It can also be implemented by one or more general-purpose processors (ASICs) of the computer of the system or device reading and executing the program. Furthermore, the present invention can also be implemented by a dedicated processor (e.g., an ASIC or FPGA) that implements one or more functions. Moreover, the present invention can also be implemented by a combination of a general-purpose processor and a dedicated processor. Here, "processor" refers to a processor in a broad sense and includes both general-purpose processors and dedicated processors. Furthermore, the process of implementing the present invention may be executed by only one processor, or by the cooperation of multiple processors located in physically separate locations.

[0048] Each embodiment of the disclosure includes the following configurations, methods, and programs. (Configuration 1) An information processing device having a transmitting means that periodically transmits packets and a receiving means that receives the packets in synchronization with the transmission cycle of the packets transmitted from the transmitting means, The transmitting device is A data generation means that periodically generates data, A data information assignment means that assigns transmission time information relating to the transmission time based on the transmission timing of the packet transmitted from the transmission means to the aforementioned data, The system includes a packet generation means that generates a first packet to be transmitted by the transmission means, which includes the data to which the transmission time information has been added, The aforementioned information processing device is Information generation means for generating reception time information relating to the reception time based on the reception timing of the first packet received by the receiving means, for each timing at which the first packet is received by the receiving means, Each time the reception time information is generated by the information generation means, an information storage means sequentially stores the reception time information, An information processing apparatus characterized by having a packet information assignment means that, each time the receiving means receives the first packet, assigns to the first packet the oldest reception time information among the reception time information stored in the information storage means. (Configuration 2) The information processing apparatus according to Configuration 1, characterized in that each time the oldest reception time information is assigned by the packet information assignment means, the information processing apparatus has an erasure means for erasing the reception time information from the information storage means. (Configuration 3) The receiving means repeatedly enters a sleep state and a wake state after waking from the sleep state. The information processing apparatus according to configuration 1 or 2, characterized in that the information generation means generates the reception time information based on the timing at which the receiving means returns from the sleep state to the wake state. (Configuration 4) An information processing device according to any one of Configurations 1 to 3, characterized by having a processing execution means for performing processing based on the first packet. (Configuration 5) A communication system comprising a transmitting device having transmitting means for periodically transmitting packets, and an information processing device having receiving means for receiving packets in synchronization with the transmission cycle of the packets transmitted from the transmitting means, The transmitting device is A data generation means that periodically generates data, A data information assignment means that assigns transmission time information relating to the transmission time based on the transmission timing of the packet transmitted from the transmission means to the aforementioned data, The system includes a packet generation means that generates a first packet to be transmitted by the transmission means, which includes the data to which the transmission time information has been added, The aforementioned information processing device is Information generation means for generating reception time information relating to the reception time based on the reception timing of the first packet received by the receiving means, for each timing at which the first packet is received by the receiving means, Each time the reception time information is generated by the information generation means, an information storage means sequentially stores the reception time information, A communication system characterized by having a packet information assignment means that, each time the first packet is received by the receiving means, assigns to the first packet the oldest reception time information among the reception time information stored in the information storage means. (Configuration 6) The communication system according to Configuration 5, characterized in that the data generation means generates inertial information relating to the inertia that occurs when the transmitting device is operated, as the data. (Configuration 7) The transmitting means repeatedly enters a sleep state and a recovery state after recovering from the sleep state. The communication system according to configuration 5 or 6, characterized in that the data information assignment means generates the transmission time information based on the timing when the transmission means returns from the sleep state to the wake state, and assigns the transmission time information. (Configuration 8) The communication system according to any one of Configurations 5 to 7, characterized in that when multiple identical data are generated by the data generation means, the packet generation means includes the multiple data in a single packet. (Configuration 9) The communication system according to any one of Configurations 5 to 8, wherein the transmitting means and the receiving means are each capable of communication in accordance with the Bluetooth low energy standard. (Configuration 10) The communication system according to any one of Configurations 5 to 9, characterized in that the transmitting device is a controller for operating the information processing device, and the information processing device is a head-mounted display operated by the controller and capable of displaying an image. (Method 1) A method for controlling an information processing device having a receiving means for receiving packets, which is communicatively connected to a transmitting device having a transmitting means for periodically transmitting packets, and which is synchronized with the transmission period of the packets transmitted from the transmitting means, The transmitting device is A data generation means that periodically generates data, A data information assignment means that assigns transmission time information relating to the transmission time based on the transmission timing of the packet transmitted from the transmission means to the aforementioned data, The system includes a packet generation means that generates a first packet to be transmitted by the transmission means, which includes the data to which the transmission time information has been added. The control method for the information processing device is as follows: An information generation step of generating reception time information relating to the reception time based on the reception timing of the first packet received by the receiving means, for each timing at which the first packet is received by the receiving means; Each time the reception time information is generated in the information generation step, an information storage step is performed to sequentially store the reception time information. A control method for an information processing device, characterized by comprising: a packet information assignment step, which, each time the first packet is received by the receiving means, assigns to the first packet the oldest reception time information among the reception time information stored in the information storage step. (Program 1) A program characterized by causing a computer to execute the control method described in Method 1. [Explanation of Symbols]

[0049] 100 Image Processing Devices 101 Control Unit 102 Communications Department 200 controllers 201 Control Unit 202 Communications Department 203 Inertial detection unit 1000 Communication Systems

Claims

1. An information processing device having a transmitting means that periodically transmits packets and a receiving means that receives the packets in synchronization with the transmission cycle of the packets transmitted from the transmitting means, The transmitting device is A data generation means that periodically generates data, A data information assignment means that assigns transmission time information relating to the transmission time based on the transmission timing of the packet transmitted from the transmission means to the aforementioned data, The packet generation means includes the data to which the transmission time information is attached, and generates a first packet to be transmitted by the transmission means, The aforementioned information processing device is Information generation means for generating reception time information relating to the reception time based on the reception timing of the first packet received by the receiving means, for each timing when the first packet is received by the receiving means, Each time the reception time information is generated by the information generation means, an information storage means sequentially stores the reception time information, An information processing apparatus characterized by having a packet information assignment means that, each time the receiving means receives the first packet, assigns to the first packet the oldest reception time information among the reception time information stored in the information storage means.

2. The information processing apparatus according to claim 1, further comprising an erasure means for deleting the reception time information from the information storage means each time the oldest reception time information is assigned by the packet information assignment means.

3. The receiving means repeatedly cycles between a sleep state and a wake state after waking from the sleep state. The information processing apparatus according to claim 1, characterized in that the information generation means generates the reception time information based on the timing at which the receiving means returns from the sleep state to the wake state.

4. The information processing apparatus according to claim 1, characterized by having a processing execution means for performing processing based on the first packet.

5. A communication system comprising: a transmitting device having transmitting means for periodically transmitting packets; and an information processing device having receiving means for receiving packets in synchronization with the transmission cycle of the packets transmitted from the transmitting means, The transmitting device is A data generation means that periodically generates data, A data information assignment means that assigns transmission time information relating to the transmission time based on the transmission timing of the packet transmitted from the transmission means to the aforementioned data, The packet generation means includes the data to which the transmission time information is attached, and generates a first packet to be transmitted by the transmission means, The aforementioned information processing device is Information generation means for generating reception time information relating to the reception time based on the reception timing of the first packet received by the receiving means, for each timing when the first packet is received by the receiving means, Each time the reception time information is generated by the information generation means, an information storage means sequentially stores the reception time information, A communication system characterized by having a packet information assignment means that, each time the first packet is received by the receiving means, assigns to the first packet the oldest reception time information among the reception time information stored in the information storage means.

6. The communication system according to claim 5, characterized in that the data generation means generates inertial information relating to the inertia generated when the transmitting device is operated as the data.

7. The transmitting means repeatedly cycles between a sleep state and a wake state after waking from the sleep state. The communication system according to claim 5, characterized in that the data information assigning means generates the transmission time information and assigns the transmission time information based on the timing when the transmission means returns from the sleep state to the wake state.

8. The communication system according to claim 5, characterized in that, when the packet generation means generates multiple identical data, the multiple data are included in a single packet.

9. The communication system according to claim 5, characterized in that the transmitting means and the receiving means are each capable of communication in accordance with the Bluetooth low energy standard.

10. The communication system according to claim 5, characterized in that the transmitting device is a controller for operating the information processing device, and the information processing device is a head-mounted display that is operated by the controller and capable of displaying an image.

11. A method for controlling an information processing device having a receiving means for receiving packets, which is communicatively connected to a transmitting device having a transmitting means for periodically transmitting packets, and which is synchronized with the transmission cycle of the packets transmitted from the transmitting means, The transmitting device is A data generation means that periodically generates data, A data information assignment means that assigns transmission time information relating to the transmission time based on the transmission timing of the packet transmitted from the transmission means to the aforementioned data, The system includes a packet generation means that generates a first packet to be transmitted by the transmission means, which includes the data to which the transmission time information has been added. The control method for the information processing device is as follows: Information generation step of generating reception time information relating to the reception time based on the reception timing of the first packet received by the receiving means, for each timing when the first packet is received by the receiving means; Each time the reception time information is generated in the information generation step, an information storage step is performed to sequentially store the reception time information. A control method for an information processing device, characterized by comprising: a packet information assignment step, which, each time the first packet is received by the receiving means, assigns to the first packet the oldest reception time information among the reception time information stored in the information storage step.

12. A program characterized by causing a computer to execute the control method described in claim 11.

Images