Image communication device

Abstract

To provide a technique capable of reducing deterioration in visibility of scroll display while allowing a transmitted image to be displayed on a display device without causing a large delay.SOLUTION: An image communication device includes: a first generation unit that generates reference image data that indicates a reference image corresponding to a first captured image, and has a data amount smaller than that of first captured image data, based on first captured image data generated by a camera and indicating the first captured image; a first transmitting unit that transmits the reference image data; a receiving unit that receives operation information indicating an operation for scrolling the reference image in a display region smaller than the reference image; a determination unit that determines, based on the operation information, an image of a region of interest to be cut out from a second captured image generated by the camera and indicated by second captured image data; a second generation unit that generates display image data indicating the image of the region of interest by cutting out the image of the region of interest from the second captured image using the second captured image data; and a second transmitting unit that transmits the display image data.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an image communication device and a display control device. [Background technology]

[0002] Patent Document 1 discloses an image processing device that transmits a part of a captured image generated by a camera to a display device as a transmitted image. This image processing device can reduce the amount of image data to be transmitted compared to a configuration that transmits the entire captured image, and enables the image to be displayed on the display device without significant delay in display. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-145612 A Summary of the Invention [Problem to be solved by the invention]

[0004] A method is conceivable in which the position of the transmitted image in the captured image is changed by scrolling the transmitted image in the display area of ​​the display device that received the transmitted image. However, with this method, if the scrolled image is not larger than the display area, there is a high possibility that the scrolled image will not be displayed in at least a part of the display area as a result of the scrolling, and the visibility of the scroll display will be reduced.

[0005] An object of the present invention is to provide an image communication technique that enables a transmitted image to be displayed on a display device without causing a significant delay, while minimizing the deterioration of scroll visibility. [Means for solving the problem]

[0006] An image communication device according to one embodiment of the present invention includes a first generation unit that generates reference image data indicating a reference image corresponding to a first captured image based on first captured image data indicating a first captured image, the reference image data having a data amount smaller than that of the first captured image data; a first transmission unit that transmits the reference image data; a receiving unit that receives operation information indicating an operation to scroll the reference image indicated by the reference image data in a display area smaller than the reference image; a determination unit that determines an image of an attention area to be cut out from a second captured image indicated by second captured image data generated by the camera based on the operation information; a second generation unit that generates display image data indicating the image of the attention area by cutting out the image of the attention area from the second captured image using the second captured image data; and a second transmission unit that transmits the display image data. Effect of the Invention

[0007] According to one aspect of the present invention, it is possible to display a transmitted image on a display device without causing a significant delay, while minimizing the deterioration of visibility of the scroll display. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 illustrates a communication system CS. [Diagram 2] 1A and 1B are diagrams showing an example of a captured image b, a reference area f1, and an attention area f2. [Diagram 3] FIG. 13 is a diagram for explaining reference image data c1 and target image data c2. [Figure 4] FIG. 13 is a diagram showing an example of changing the position of the attention area f2. [Diagram 5] FIG. 1 illustrates an example of an image communication device 10. [Figure 6] FIG. 2 is a diagram illustrating an example of a communication server 2. [Figure 7] FIG. 2 is a diagram illustrating an example of a terminal device 3. [Figure 8] FIG. 2 is a diagram for explaining the operation of the image communication device 10. [Figure 9] FIG. 13 is a diagram showing an example of transmission of reference image data c1 and target image data c2. [Figure 10] FIG. [Figure 11] FIG. 13 is a diagram showing an example of changing the size of a reference image d1. [Figure 12] FIG. 13 is a diagram for explaining a fourth modified example. [Figure 13] FIG. 13 is a diagram showing an example of an additional display area 3c. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A: First embodiment A1: Communication System CS FIG. 1 is a diagram showing a communication system CS. The communication system CS includes a mobile object 1, a communication server 2, and a terminal device 3. The mobile object 1 and the terminal device 3 communicate with the communication server 2 via a network NW. The mobile object 1 and the terminal device 3 may each communicate directly with the communication server 2 without going through the network NW. The network NW is a wired network or a wireless network. The network NW may include a wired network and a wireless network.

[0010] The moving object 1 is a remotely controlled vehicle. The remotely controlled vehicle is also called a robot vehicle. The remotely controlled vehicle is used for delivering goods. The remotely controlled vehicle may be used for observing the surroundings of the remotely controlled vehicle, instead of delivering goods. The moving object 1 is not limited to a remotely controlled vehicle, and may be, for example, a wheelchair, a drone, or a vehicle directly operated by an elderly person. The moving object 1 includes a camera 4 and an image communication device 10.

[0011] The camera 4 is a 360-degree camera. The camera 4 is not limited to a 360-degree camera, and may be a camera having an angle of view less than 360 degrees (for example, a 180-degree camera). The camera 4 may be built into the image communication device 10. The camera 4 generates captured image data a by capturing an image. The captured image data a indicates a captured image b. The captured image b represents the surroundings of the moving object 1. The camera 4 repeatedly captures an image, thereby repeatedly generating the captured image data a.

[0012] The image communication device 10 communicates with each of the camera 4 and the communication server 2. The image communication device 10 receives captured image data a from the camera 4. The image communication device 10 generates transmission image data c based on the captured image data a. The image communication device 10 transmits the transmission image data c to the communication server 2.

[0013] The transmission image data c has a data amount smaller than that of the captured image data a. The transmission image data c shows an image of a partial area of ​​the captured image b. An example of the partial area of ​​the captured image b is the reference area f1 in the captured image b. Another example of the partial area of ​​the captured image b is the attention area f2 in the captured image b.

[0014] 2 is a diagram showing an example of a reference region f1, an attention region f2, and a captured image b. The captured image b is larger than both the reference region f1 and the attention region f2. The reference region f1 is larger than the attention region f2. The reference region f1 includes the attention region f2.

[0015] 2 further shows a reference image d1 and an image of interest d2. The reference image d1 is an image of a reference region f1. The image of interest d2 is an image of an area of ​​interest f2. The reference image d1 and the image of interest d2 are each a part of the captured image b and correspond to the captured image b.

[0016] 1 generates reference image data c1 as transmission image data c showing a reference image d1. The image communication device 10 generates attention image data c2 as transmission image data c showing an attention image d2. The attention image data c2 is an example of display image data.

[0017] The captured image data a used to generate the reference image data c1 is different from the captured image data a used to generate the target image data c2. The image communication device 10 transmits the target image data c2 at a timing different from the timing of transmitting the reference image data c1.

[0018] The communication server 2 communicates with each of the image communication device 10 and the terminal device 3. The communication server 2 receives reference image data c1 and image data of interest c2 from the image communication device 10. The communication server 2 transmits the reference image data c1 and image data of interest c2 to the terminal device 3.

[0019] The terminal device 3 is a smartphone. The terminal device 3 is not limited to a smartphone, and may be, for example, a personal computer or a tablet. The terminal device 3 is used by a user U1. The terminal device 3 has a display area 3a. The terminal device 3 receives reference image data c1 and attention image data c2 from the communication server 2. The terminal device 3 displays a reference image d1 in the display area 3a based on the reference image data c1. The reference image d1 is used to adjust the position of the attention area f2 in the captured image b. The terminal device 3 displays an attention image d2 in the display area 3a based on the attention image data c2. The attention image d2 is used, for example, by the user U1 to check the situation around the moving object 1.

[0020] The terminal device 3 receives a scroll operation from the user U1. The scroll operation is an operation for scrolling the reference image d1 in the display area 3a. The terminal device 3 transmits operation information e indicating the scroll operation to the communication server 2. The operation information e indicates a scroll direction g1 and a scroll amount g2 (amount of movement of the reference image d1). The scroll amount g2 is indicated in units of pixels in the reference image d1. For example, the scroll amount is indicated as 5 pixels.

[0021] The communication server 2 receives operation information e from the terminal device 3. The communication server 2 transmits the operation information e to the image communication device 10. The image communication device 10 adjusts the position of the attention area f2 in the captured image b based on the operation information e.

[0022] A2: Reference image data c1 and target image data c2 Fig. 3 is a diagram for explaining reference image data c1 and target image data c2. Fig. 3 shows timings t1 and t2. Timing t1 is a timing that is freely determined. Timing t2 is a timing that is later than timing t1.

[0023] The camera 4 generates captured image data a1 as the captured image data a at a timing t1. The captured image data a1 is an example of the first captured image data. The captured image data a1 shows a captured image b1 as the captured image b. The captured image b1 is an example of the first captured image.

[0024] At timing t2, the camera 4 generates captured image data a2 as the captured image data a. The captured image data a2 is an example of the second captured image data. The captured image data a2 shows a captured image b2 as the captured image b. The captured image b2 is an example of the second captured image.

[0025] The camera 4 may or may not further generate captured image data a at each of the timings before the timing t1, between the timings t1 and t2, and after the timing t2.

[0026] The image communication device 10 generates reference image data c1 indicating a reference image d1 based on the captured image data a1. The reference image d1 is displayed in the display area 3a. The reference image d1 is larger than the display area 3a. Therefore, the reference image d1 does not fit in the display area 3a. Therefore, only a part of the reference image d1 is displayed in the display area 3a. When the reference image d1 is scrolled in the display area 3a, a part of the reference image d1 that was not displayed in the display area 3a at the start of scrolling is displayed in the display area 3a.

[0027] The image communication device 10 generates image data c2 of interest that indicates an image d2 of interest based on the captured image data a2. The image d2 of interest is displayed in the display area 3a.

[0028] The size of the target image d2 is equal to the size of the display area 3a, so that the target image d2 fits within the display area 3a.

[0029] The position of the attention area f2 in the captured image b2 is changed in response to scrolling of the reference image d1 in the display area 3a.

[0030] FIG. 4 is a diagram showing an example of changing the position of the attention area f2 in response to scrolling of the reference image d1 in the display area 3a.

[0031] The position of the attention area f2 in the captured image b2 is moved by the scroll amount g2 in a direction g3 opposite to the scroll direction g1. For example, when the reference image d1 is scrolled downward in the display area 3a by the scroll amount g2, the position of the attention area f2 in the captured image b2 is moved upward by the scroll amount g2.

[0032] A3: Image communication device 10 5 is a diagram showing an example of an image communication device 10. The image communication device 10 includes an input device 11, an output device 12, communication devices 13 and 14, a storage device 15, and a processing device 16. At least one of the input device 11 and the output device 12 may be omitted.

[0033] The input device 11 includes a touch panel. The input device 11 may include a plurality of operation keys in addition to the touch panel. The input device 11 may include a plurality of operation keys without including a touch panel. The input device 11 accepts operations performed by the user U2.

[0034] The output device 12 includes a display. The output device 12 displays various information. The output device 12 may include a touch panel. When the output device 12 includes a touch panel, it does not have to include a display. A touch panel may be used as both the input device 11 and the output device 12.

[0035] The communication device 13 communicates with the camera 4. The communication device 13 wirelessly communicates with the camera 4, for example, by Bluetooth (registered trademark). Bluetooth is an example of short-range wireless communication. The communication device 13 may also communicate with the camera 4 by wire. The communication device 13 receives captured image data a from the camera 4.

[0036] The communication device 14 communicates with the communication server 2. The communication device 14 communicates with the communication server 2 via the network NW. The communication device 14 may communicate with the communication server 2 without via the network NW.

[0037] The storage device 15 is a recording medium readable by the processing device 16. The storage device 15 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), and an Electrically Erasable Programmable Read Only Memory (EEPROM). The volatile memory is, for example, a Random Access Memory (RAM). The storage device 15 stores a program PG1.

[0038] The processing device 16 includes one or more central processing units (CPUs). The one or more CPUs are an example of one or more processors. Each of the processor and the CPU is an example of a computer.

[0039] The processing device 16 reads the program PG1 from the storage device 15. The processing device 16 executes the program PG1 to function as a generating unit 161, a transmitting unit 162, a receiving unit 163, a determining unit 164, a generating unit 165, a transmitting unit 166, and an operation control unit 167. The generating unit 161 may include the generating unit 165. The transmitting unit 162 may include the transmitting unit 166. The generating unit 161 may be configured by a circuit such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). At least one of the transmitting unit 162, the receiving unit 163, the determining unit 164, the generating unit 165, the transmitting unit 166, and the operation control unit 167 may be configured by a circuit such as a DSP, an ASIC, a PLD, or an FPGA.

[0040] The generating unit 161 is an example of a first generating unit. The generating unit 161 acquires captured image data a1. The generating unit 161 generates reference image data c1 based on the captured image data a1. For example, the generating unit 161 generates the reference image data c1 by cutting out a reference image d1 from a captured image b1 using the captured image data a1. The reference image data c1 has a data amount smaller than the data amount of the captured image data a1.

[0041] The transmission unit 162 is an example of a first transmission unit. The transmission unit 162 acquires the reference image data c1 from the generation unit 161. The transmission unit 162 transmits the reference image data c1. For example, the transmission unit 162 transmits the reference image data c1 from the communication device 14 to the communication server 2.

[0042] The receiving unit 163 receives various information. For example, the receiving unit 163 receives operation information e from the communication server 2 via the communication device 14. The operation information e indicates an operation of scrolling a reference image d1 indicated by reference image data c1 in the display area 3a of the terminal device 3. The operation information e indicates a scroll direction g1 in the reference image d1 and a scroll amount g2 in the reference image d1 (amount of movement of the reference image d1).

[0043] The determining unit 164 acquires the operation information e from the receiving unit 163. The determining unit 164 determines an image of the attention area f2 (attention image d2) cut out from the captured image b2 indicated by the captured image data a2, based on the operation information e.

[0044] The generating unit 165 is an example of a second generating unit. The generating unit 165 acquires captured image data a2. The generating unit 165 generates attention image data c2 based on the captured image data a2. For example, the generating unit 165 generates attention image data c2 indicating the attention image d2 by cutting out the attention image d2 from the captured image b2 using the captured image data a2. The attention image data c2 has a data amount smaller than the data amount of the captured image data a2.

[0045] The transmission unit 166 is an example of a second transmission unit. The transmission unit 166 acquires the attention image data c2 from the generation unit 165. The transmission unit 166 transmits the attention image data c2. For example, the transmission unit 166 transmits the attention image data c2 from the communication device 14 to the communication server 2.

[0046] The operation control unit 167 controls various operations. For example, the operation control unit 167 controls the operation of the camera 4.

[0047] A4: Communication server 2 6 is a diagram showing an example of the communication server 2. The communication server 2 is an example of a display control device. The communication server 2 includes a communication device 21, a storage device 22, and a processing device 23.

[0048] The communication device 21 communicates with each of the image communication device 10 and the terminal device 3 via the network NW. The communication device 21 may communicate with each of the image communication device 10 and the terminal device 3 without via the network NW.

[0049] The storage device 22 is a recording medium that can be read by the processing device 23. The storage device 22 includes, for example, a non-volatile memory and a volatile memory. The storage device 22 stores the program PG2.

[0050] The processing device 23 includes one or more CPUs. The processing device 23 reads the program PG2 from the storage device 22. The processing device 23 executes the program PG2 to function as a receiving unit 231, a display control unit 232, and an operation control unit 233. At least one of the receiving unit 231, the display control unit 232, and the operation control unit 233 may be configured by a circuit such as a DSP, an ASIC, a PLD, or an FPGA.

[0051] The receiving unit 231 is an example of a data receiving unit. The receiving unit 231 receives reference image data c1 and attention image data c2 from the image communication device 10 via the communication device 21. The receiving unit 231 receives operation information e from the terminal device 3 via the communication device 21.

[0052] The display control unit 232 transmits reference image data c1 from the communication device 21 to the terminal device 3, thereby causing a reference image d1 to be displayed in the display area 3a of the terminal device 3. The display control unit 232 transmits attention image data c2 from the communication device 21 to the terminal device 3, thereby causing an attention image d2 to be displayed in the display area 3a of the terminal device 3.

[0053] The operation control unit 233 controls various operations. For example, the operation control unit 233 acquires operation information e from the receiving unit 231. The operation control unit 233 transmits the operation information e from the communication device 21 to the image communication device 10.

[0054] The operation control unit 233 transmits first completion information h1 to the image communication device 10 every time the display control unit 232 transmits reference image data c1 to the terminal device 3. The first completion information h1 is information indicating the completion of transmission of the reference image data c1 from the communication server 2 to the terminal device 3. The first completion information h1 is received by the receiving unit 163 of the image communication device 10.

[0055] The operation control unit 233 transmits second completion information h2 to the image communication device 10 every time the display control unit 232 transmits the attention image data c2 to the terminal device 3. The second completion information h2 is information indicating the completion of the transmission of the attention image data c2 from the communication server 2 to the terminal device 3. The second completion information h2 is received by the receiving unit 163 of the image communication device 10.

[0056] The first completion information h1 and the second completion information h2 are used by the image communication device 10 to determine the status of transmission of the reference image data c1 and the target image data c2 from the communication server 2 to the terminal device 3.

[0057] A5: Terminal device 3 7 is a diagram showing an example of the terminal device 3. The terminal device 3 includes an input device 31, a display device 32, a communication device 33, a storage device 34, and a processing device 35.

[0058] The input device 31 includes a touch panel. The input device 31 may include a plurality of operation keys in addition to the touch panel. The input device 31 may include a plurality of operation keys without including a touch panel. The input device 31 accepts an operation performed by the user U1. The operation performed by the user U1 is, for example, a scroll operation.

[0059] The display device 32 displays various information. The display device 32 may include a touch panel. The touch panel may be used as both the input device 31 and the display device 32.

[0060] The communication device 33 communicates with the communication server 2. The communication device 33 communicates with the communication server 2 via the network NW. The communication device 33 may communicate with the communication server 2 without via the network NW.

[0061] The storage device 34 is a recording medium that can be read by the processing device 35. The storage device 34 includes, for example, a non-volatile memory and a volatile memory. The storage device 34 stores the program PG3.

[0062] The processing device 35 includes one or more CPUs. The processing device 35 reads the program PG3 from the storage device 34. The processing device 35 executes the program PG3 to function as an operation control unit 351. The operation control unit 351 may be configured by a circuit such as a DSP, an ASIC, a PLD, or an FPGA.

[0063] The operation control unit 351 controls various operations. For example, the operation control unit 351 receives reference image data c1 and target image data c2 from the communication server 2 via the communication device 33. The operation control unit 351 displays a reference image d1 in the display area 3a of the display device 32 based on the reference image data c1. The operation control unit 351 displays a target image d2 in the display area 3a of the display device 32 based on the target image data c2.

[0064] The operation control unit 351 receives a scroll operation for scrolling the reference image d1 in the display area 3a from the user U1 via the input device 31. The scroll operation is, for example, a swipe operation on the reference image d1 displayed in the display area 3a. The scroll operation is not limited to a swipe operation, and may be, for example, an operation on an operation button for scrolling. The operation control unit 351 scrolls the reference image d1 in the display area 3a in response to the scroll operation. Furthermore, the operation control unit 351 transmits operation information e indicating the scroll operation from the communication device 33 to the communication server 2.

[0065] A6: Description of operation FIG. 8 is a diagram for explaining the operation of the image communication apparatus 10. As shown in FIG.

[0066] In step S101, the operation control unit 167 sets an initial value of the reference region f1 and an initial value of the attention region f2. For example, the initial values ​​of the reference region f1 and the attention region f2 are default values. The initial values ​​of the reference region f1 and the attention region f2 are not limited to default values ​​and may be set by, for example, the user U1 or the user U2.

[0067] Next, in step S102, the operation control unit 167 determines whether the image data of interest c2 has been transmitted n (numeric value n) times since the previous transmission of the reference image data c1, where n is an integer equal to or greater than 1.

[0068] For example, the operation control unit 167 determines whether the target image data c2 has been transmitted n times after the previous transmission of the reference image data c1, based on the reception history of the first completion information h1 and the second completion information h2.

[0069] The first completion information h1 is information indicating the completion of the transmission of the reference image data c1 from the communication server 2 to the terminal device 3. The first completion information h1 is transmitted from the communication server 2 to the image communication device 10 every time the display control unit 232 transmits the reference image data c1 to the terminal device 3.

[0070] The second completion information h2 is information indicating the completion of the transmission of the reference image data c1 from the communication server 2 to the terminal device 3. The second completion information h2 is transmitted from the communication server 2 to the image communication device 10 every time the display control unit 232 transmits the attention image data c2 to the terminal device 3.

[0071] The reception history of the first completion information h1 and the second completion information h2 means the history of reception of the first completion information h1 and the second completion information h2 executed by the receiving unit 163, respectively.

[0072] The operation control unit 167 may determine whether the attention image data c2 has been transmitted n times after the previous transmission of the reference image data c1 based on the transmission history of the reference image data c1 and the attention image data c2. The transmission history of the reference image data c1 and the attention image data c2 includes the history of the transmission of the reference image data c1 executed by the transmission unit 162 and the history of the transmission of the attention image data c2 executed by the transmission unit 166. In this case, the communication of the first completion information h1 and the second completion information h2 can be omitted.

[0073] If the operation control unit 167 determines in step S102 that the target image data c2 has been transmitted n times since the previous transmission of the reference image data c1 (step S102: YES), the generation unit 161 generates the reference image data c1 in step S103.

[0074] For example, the generating unit 161 first acquires the latest captured image data a generated by the camera 4 as captured image data a1. Next, the generating unit 161 uses the captured image data a1 to generate reference image data c1 by cutting out an image of a reference region f1 from a captured image b1 indicated by the captured image data a1 as a reference image d1.

[0075] Subsequently, in step S104, the transmission unit 162 transmits the reference image data c1 to the communication server 2.

[0076] For example, the transmission unit 162 first acquires the latest reference image data c1 generated by the generation unit 161. Then, the transmission unit 162 transmits the latest reference image data c1 from the communication device 14 to the communication server 2.

[0077] In the communication server 2, the receiving unit 231 receives the latest reference image data c1. Then, the display control unit 232 acquires the latest reference image data c1 from the receiving unit 231. Then, the display control unit 232 transmits the latest reference image data c1 from the communication device 21 to the terminal device 3, thereby causing the reference image d1 to be displayed in the display area 3a of the terminal device 3.

[0078] If the operation control unit 167 determines in step S102 that the target image data c2 has not been transmitted n times since the previous transmission of the reference image data c1 (step S102: NO), the generation unit 165 generates the target image data c2 in step S105.

[0079] For example, the generating unit 165 first acquires the latest captured image data a generated by the camera 4 as captured image data a2. Next, the generating unit 165 uses the captured image data a2 to generate target image data c2 by cutting out an image of a target region f2 from a captured image b2 indicated by the captured image data a2 as a target image d2.

[0080] Next, in step S106, the transmission unit 166 transmits the target image data c2 to the communication server 2.

[0081] For example, the transmission unit 166 first acquires the latest image data of interest c2 generated by the generation unit 165. Then, the transmission unit 166 transmits the latest image data of interest c2 from the communication device 14 to the communication server 2.

[0082] In the communication server 2, the receiving unit 231 receives the latest attention image data c2. Then, the display control unit 232 acquires the latest attention image data c2 from the receiving unit 231. Then, the display control unit 232 transmits the latest attention image data c2 from the communication device 21 to the terminal device 3, thereby causing the attention image d2 to be displayed in the display area 3a of the terminal device 3.

[0083] Next, in step S107, the operation control unit 167 determines whether the receiving unit 163 has received the operation information e.

[0084] When the operation control unit 351 receives a scroll operation for scrolling the reference image d1 in the display area 3a of the terminal device 3 from the user U1 via the input device 31, the operation information e is transmitted from the terminal device 3 to the image communication device 10 via the communication server 2. The operation information e indicates an operation for scrolling the reference image d1 in the display area 3a of the terminal device 3. The operation information e indicates a scroll direction g1 in the reference image d1 and a scroll amount g2 in the reference image d1 (amount of movement of the reference image d1).

[0085] When the operation control unit 351 receives a touch operation on the image of interest d2 from the user U1 via the input device 31 while the image of interest d2 is being displayed in the display area a3, it changes the image displayed in the display area a3 from the image of interest d2 to the reference image d1.

[0086] For example, when the operation control unit 351 receives a touch operation on the image of interest d2 from the user U1 via the input device 31, the operation control unit 351 first identifies the latest reference image data c1 from the reference image data c1 received in the past. Next, the operation control unit 351 changes the image displayed in the display area a3 from the image of interest d2 to the reference image d1 indicated by the identified latest reference image data c1. The operation control unit 351 may obtain the latest reference image data c1 from the communication server 2.

[0087] Thereafter, when the operation control unit 351 receives a scroll operation for scrolling the reference image d1 from the user U1 via the input device 31, the operation control unit 351 transmits operation information e to the communication server 2. Upon receiving the operation information e, the communication server 2 transmits the operation information e to the image communication device 10.

[0088] When the operation control unit 167 determines in step S107 that the receiving unit 163 has received the operation information e (step S107: YES), in step S108 the determining unit 164 updates the position of the attention area f2 based on the operation information e.

[0089] For example, the determination unit 164 first acquires the operation information e from the receiving unit 163. Next, the determination unit 164 updates the position of the attention area f2 in the captured image b2 by moving the position of the attention area f2 in the direction opposite to the scroll direction g1 indicated by the operation information e by the scroll amount g2 indicated by the operation information e. By updating the position of the attention area f2, the determination unit 164 determines an image of the attention area f2 (attention image d2) cut out from the captured image b2 indicated by the captured image data a2 based on the operation information e.

[0090] Next, in step S109, the generating unit 161 adjusts the numerical value n used in step S102. The adjustment of the numerical value n means adjusting the generation timing of the newly generated reference image data c1 in a situation where the reference image data c1 is newly generated after the receiving unit 163 receives the operation information e.

[0091] The generation unit 161 first identifies a reception frequency i indicating a frequency at which the reception unit 163 receives the operation information e, based on the reception of the operation information e by the reception unit 163. The generation unit 161 adjusts the numerical value n, based on the reception frequency i of the operation information e.

[0092] For example, the generation unit 161 decreases the numerical value n as the reception frequency i of the operation information e increases. Also, the generation unit 161 increases the numerical value n as the reception frequency i of the operation information e decreases. Therefore, the transmission frequency of the reference image data c1 is proportional to the reception frequency i of the operation information e.

[0093] The generating unit 161 may set the numerical value n to a numerical value n1 when the reception frequency i of the operation information e exceeds a threshold value j, and may set the numerical value n to a numerical value n2 when the reception frequency i of the operation information e does not exceed the threshold value j. Here, the numerical value n1 and the numerical value n2 are each an integer equal to or greater than 1. The numerical value n1 is smaller than the numerical value n2.

[0094] When step S109 is completed, or when the operation control unit 167 determines in step S107 that the receiving unit 163 has not received the operation information e (step S107: NO), the operation control unit 167 executes step S110.

[0095] In step S110, the operation control unit 167 determines whether the input device 11 has accepted an operation to end the operation.

[0096] If the operation control unit 167 determines in step S110 that the input device 11 has not accepted an operation to end the operation (step S110: NO), the process returns to step S102. If the operation control unit 167 determines in step S110 that the input device 11 has accepted an operation to end the operation (step S110: YES), the process ends.

[0097] A7: Example of sending reference image data c1 and target image data c2 Fig. 9 is a diagram showing an example of transmission of reference image data c1 and target image data c2. As shown in Fig. 9, while two pieces of reference image data c1 are transmitted, n pieces of target image data c2 are transmitted.

[0098] A8: Summary of the first embodiment According to the first embodiment, the generating unit 161 generates reference image data c1 based on the captured image data a1. The reference image data c1 indicates the reference image d1. The data amount of the reference image data c1 is smaller than the data amount of the captured image data a1. The transmitting unit 162 transmits the reference image data c1. The receiving unit 163 receives the operation information e. The operation information e indicates an operation of scrolling the reference image d1 in the display area 3a smaller than the reference image d1. The determining unit 164 determines an image of the attention area f2 cut out from the captured image b2 indicated by the captured image data a2 based on the operation information e. The generating unit 165 generates attention image data (display image data) c2 indicating the image of the attention area f2 by cutting out the image of the attention area f2 from the captured image b2 using the captured image data a2. The transmitting unit 166 transmits the attention image data c2.

[0099] The amount of data of the reference image data c1 is smaller than that of the captured image data a1. The image data of interest c2 is generated by cutting out an image of the attention area f2 from the captured image b2 indicated by the captured image data a2. Therefore, the amount of data of the attention image data c2 is smaller than that of the captured image data a2. Therefore, compared to a configuration in which the captured image data a1 and the captured image data a2 are transmitted, the amount of data of the transmitted image data can be reduced, and the delay in displaying an image based on the image data can be reduced.

[0100] Furthermore, the display area 3a in which the reference image d1 is scrolled is smaller than the reference image d1. Therefore, compared to a configuration in which the display area 3a in which the reference image d1 is scrolled is equal to or larger than the size of the reference image d1, the possibility that an area in the display area 3a in which the reference image d1 is not displayed during scrolling is reduced, and deterioration in visibility of scrolling can be limited.

[0101] FIG. 10 is a diagram showing a comparative example in which the size of the reference image d1 is equal to the size of the display area 3a. As shown in FIG. 10, when the reference image d1 having the same size as the display area 3a is scrolled in the display area 3a, an area 3b (area 3b indicated by diagonal lines) in which the reference image d1 is not displayed is always generated in the display area 3a. In the first embodiment, the display area 3a is smaller than the reference image d1. That is, the reference image d1 is larger than the display area 3a. Therefore, the possibility that an area 3b in which the reference image d1 is not displayed is generated in the display area 3a during scrolling is lower than in the comparative example shown in FIG. 10. Therefore, the deterioration of the visibility of the scroll can be reduced.

[0102] Moreover, the reference image d1 is an image of a partial area of ​​the captured image b1. The generating unit 161 generates the reference image data c1 by cutting out the reference image d1 from the captured image b1 using the captured image data a1. Therefore, the reference image data c1 having a smaller data amount than the captured image data a1 can be easily generated without degrading the image quality of the reference image.

[0103] B: Variation The following are modified aspects of the above-described embodiment. Two or more aspects selected from the following modified aspects may be combined as appropriate within the scope of not being mutually contradictory.

[0104] B1: First modified example In the first embodiment, the generation unit 161 may adjust the size of the reference image d1 based on the operation information e. Adjusting the size of the reference image d1 means adjusting the size of the reference region f1.

[0105] The generating unit 161 repeatedly generates reference image data c1. In a situation where the generating unit 161 newly generates reference image data c1 after the receiving unit 163 receives operation information e, the generating unit 161 adjusts the size of the reference image d1 indicated by the newly generated reference image data c1 based on the operation information e.

[0106] For example, the generation unit 161 makes the size of the reference image d1 proportional to the scroll amount g2 indicated by the operation information e. In this case, the larger the scroll amount g2 is, the larger the reference image d1 is.

[0107] The generation unit 161 may adjust the size of the reference image d1 based on the scroll direction g1 indicated by the operation information e. For example, the generation unit 161 enlarges the reference image d1 in the direction opposite to the scroll direction g1 indicated by the operation information e. As an example, when the scroll direction is downward, the generation unit 161 enlarges the upper end d1a of the reference image d1 in the upward direction as shown in FIG.

[0108] The operation information e indicates a scroll operation performed by the user U1. Therefore, according to the first modified example, the size of the reference image d1 can be adjusted in accordance with the scroll operation performed by the user U1.

[0109] B2: Second variant In the first embodiment and the first modified example, the generation unit 161 may adjust the size of the reference image d1 based on the captured image data a1.

[0110] For example, the generating unit 161 first performs a first image recognition process on the captured image data a1 to identify an object shown in the captured image data a1. The first image recognition process is a process for identifying an object shown in the captured image data a1 by analyzing the captured image data a1. Next, the generating unit 161 adjusts the size of the reference image d1 based on the object shown in the captured image data a1.

[0111] As an example, when the captured image data a1 indicates a moving object, the generation unit 161 enlarges the reference image d1. When the captured image data a1 indicates the beginning of a slope, the generation unit 161 may enlarge the reference image d1. For example, when the captured image data a1 indicates the beginning of a downhill slope, the generation unit 161 enlarges the bottom end of the reference image d1 in the downward direction. When the captured image data a1 indicates a corner, the generation unit 161 may enlarge the reference image d1. For example, when the captured image data a1 indicates a corner, the generation unit 161 enlarges the right end of the reference image d1 in the right direction and the left end of the reference image d1 in the left direction.

[0112] According to the second modification, the generating unit 161 adjusts the size of the reference image d1 based on the captured image data a1. For this reason, for example, when the captured image data a1 shows an object to be noted (for example, a moving object, the start of a slope, or a corner), the reference image d1 can be enlarged. This increases the possibility that the reference image d1 shows an object to be noted, and the object to be noted can be easily shown in the image of interest d2 by scrolling the reference image d1. This makes it easier for the user U1 of the terminal device 3 to confirm an object to be noted around the moving object 1.

[0113] B3: Third variant In the first embodiment and the first to second modified examples, the generation unit 161 may adjust the size of the reference image d1 based on the topography around the camera 4 that is indicated by the map information corresponding to the position of the camera 4.

[0114] For example, the generation unit 161 acquires camera position information indicating the position of the camera 4. The camera position information is generated by the camera 4 using, for example, a Global Positioning System (GPS). Note that the camera position information may be generated by the image communication device 10 using a GPS or the moving object 1 using a GPS.

[0115] Next, the generation unit 161 acquires map information corresponding to the position of the camera 4 indicated by the camera position information from a map server that manages map information. The map information corresponding to the position of the camera 4 is map information indicating a map centered on the position of the camera 4. Note that the map information corresponding to the position of the camera 4 may be map information indicating a map including the position of the camera 4.

[0116] Next, the generation unit 161 adjusts the size of the reference image d1 based on the topography around the camera 4 that is indicated by the acquired map information.

[0117] As an example, when the map information indicates the start of a slope, the generation unit 161 enlarges the reference image d1. For example, when the map information indicates the start of a downhill slope, the generation unit 161 enlarges the bottom end of the reference image d1 in the downward direction. When the map information indicates a corner, the generation unit 161 may enlarge the reference image d1. For example, when the map information indicates a corner, the generation unit 161 enlarges the right end of the reference image d1 in the right direction and the left end of the reference image d1 in the left direction.

[0118] According to the third modification, the generation unit 161 adjusts the size of the reference image d1 based on the terrain around the camera 4, which is indicated by the map information corresponding to the position of the camera 4. For this reason, even in a situation where the terrain around the camera 4 is difficult to distinguish from the captured image data a1, for example, the size of the reference image d1 can be adjusted based on the terrain around the camera 4. This makes it easier for the user U1 of the terminal device 3 to confirm, for example, an object that is worthy of attention, around the moving object 1.

[0119] B4: Fourth variant FIG. 12 is a diagram for explaining the fourth modified example. In the first embodiment and the first to third modified examples, when the captured image data a1 indicates a detection object m, the generation unit 161 may further generate additional image data c3 using the captured image data a1. The detection object m is, for example, a moving object. The detection object m is not limited to a moving object, and may be, for example, a slope or a corner. The additional image data c3 indicates an additional image d3 obtained by lowering the resolution of the captured image b1 indicated by the captured image data a1. The additional image data c3 indicates the detection object m.

[0120] For example, the generation unit 161 first executes a second image recognition process on the captured image data a1 to determine whether the captured image data a1 indicates the detection object m. The second image recognition process is a process for determining whether the captured image data a1 indicates the detection object m by analyzing the captured image data a1.

[0121] When the generating unit 161 determines that the captured image data a1 indicates the detection object m, the generating unit 161 uses the captured image data a1 to generate additional image data c3 in addition to the reference image data c1.

[0122] The transmitting unit 162 further transmits the additional image data c3 to the communication server 2 in addition to the reference image data c1.

[0123] The receiving unit 231 of the communication server 2 receives the additional image data c3 in addition to the reference image data c1. The display control unit 232 of the communication server 2 acquires the additional image data c3 in addition to the reference image data c1 from the receiving unit 231. The display control unit 232 transmits the additional image data c3 in addition to the reference image data c1 to the terminal device 3.

[0124] The operation control unit 351 of the terminal device 3 receives the additional image data c3 in addition to the reference image data c1. Next, the operation control unit 351 causes the display device 32 to display the reference image d1 indicated by the reference image data c1 in the display area 3a, and also causes the display device 32 to display the additional image d3 indicated by the additional image data c3 in the additional display area 3c. When the reference image d1 displayed in the display area 3a is touched or clicked, the operation control unit 351 may cause the display device 32 to display the additional image d3 indicated by the additional image data c3 in the additional display area 3c.

[0125] The additional display area 3c is an area that is set on the display device 32 together with the display area 3a. Therefore, the terminal device 3 displays an additional image (an image obtained by lowering the resolution of the captured image b1) d3 in parallel with the display of the reference image d1. The additional display area 3c is smaller than the additional image d3. Therefore, only a part of the additional image d3 is displayed in the additional display area 3c. The size of the additional display area 3c is equal to the size of the display area 3a. The additional display area 3c may be larger or smaller than the display area 3a.

[0126] When the user U1 performs an operation to scroll the additional image d3 in the additional display area 3c, the operation control unit 351 receives an additional scroll operation to scroll the additional image d3 in the additional display area 3c from the user U1 via the input device 31. The additional scroll operation is, for example, a swipe operation on the additional image d3 displayed in the additional display area 3c. The additional scroll operation is not limited to a swipe operation, and may be, for example, an operation on an operation button for scrolling. The operation control unit 351 scrolls the additional image d3 in the additional display area 3c in response to the additional scroll operation. Furthermore, the operation control unit 351 transmits additional operation information indicating the additional scroll operation from the communication device 33 to the communication server 2. The additional operation information indicates the direction of scrolling and the scroll amount (the amount of movement of the additional image d3). The scroll amount indicated by the additional operation information is indicated in units of pixels in the additional image d3. For example, the scroll amount is indicated as 10 pixels.

[0127] The receiving unit 231 of the communication server 2 receives the additional operation information. The display control unit 232 of the communication server 2 acquires the additional operation information from the receiving unit 231. The display control unit 232 transmits the additional operation information to the image communication device 10.

[0128] The receiving unit 163 of the image communication device 10 receives the additional operation information. The determining unit 164 of the image communication device 10 acquires the additional operation information from the receiving unit 163. The determining unit 164 determines an image of the attention area f2 (attention image d2) cut out from the captured image b2 indicated by the captured image data a2 based on the additional operation information. The method of determining the image of the attention area f2 (attention image d2) based on the additional operation information is similar to the method of determining the image of the attention area f2 based on the operation information e.

[0129] According to the fourth modified example, when the captured image data a1 indicates a detection object m, a part of the additional image d3 representing the detection object m is displayed in the additional display area 3c of the terminal device 3. Therefore, the user U1 of the terminal device 3 can recognize that the additional image d3 represents the detection object m by adding the display of the additional image d3. Therefore, the user U1 of the terminal device 3 can fit the detection object m into the attention area f2 by performing an additional scroll operation.

[0130] 13, the size of the additional display area 3c may be equal to the size of the additional image d3. In this case, the additional image d3 (an image obtained by lowering the resolution of the captured image b1) is entirely displayed in the additional display area 3c. Thus, the user U1 of the terminal device 3 can recognize the position of the detection object m by visually checking the additional image d3.

[0131] When the size of the additional display area 3c is equal to the size of the additional image d3, the additional display area 3c displays an image k with a frame having the same size as the display area 3a superimposed on the additional image d3.

[0132] A user U1 of the terminal device 3 moves the frame image k in the additional display area 3c. The operation control unit 351 of the terminal device 3 generates additional operation information based on the movement of the frame image k. In this case, the scroll direction indicated by the additional operation information is the opposite direction to the movement direction of the frame image k. The scroll amount indicated by the additional operation information is indicated in units of pixels in the additional image d3. For example, the scroll amount is indicated as seven pixels.

[0133] The transmission unit 162 may adjust the transmission frequency of the additional image data c3 depending on the type of the detection object m. The transmission unit 162 identifies the type of the detection object m by executing a third image recognition process on the captured image data a1 showing the detection object m. The third image recognition process is a process for identifying the type of the detection object m shown in the captured image data a1 by analyzing the captured image data a1.

[0134] For example, if a moving object is more important than a slope or a corner, the transmitting unit 162 operates as follows. If the captured image data a1 indicates a moving object as the detection target m, the transmitting unit 162 sets the transmission frequency of the additional image data c3 indicating the moving object to be the same as the transmission frequency of the reference image data c1. If the captured image data a1 indicates a slope or a corner as the detection target m, the transmitting unit 162 sets the transmission frequency of the additional image data c3 indicating the slope or corner to be lower than the transmission frequency of the reference image data c1. For example, the transmitting unit 162 transmits the additional image data c3 indicating the slope or corner once every two or three times the reference image data c1 is transmitted.

[0135] Therefore, the frequency with which an additional image showing a detection object m having a high importance is displayed on the terminal device 3 can be made higher than the frequency with which an additional image showing a detection object m having a low importance is displayed on the terminal device 3.

[0136] B5: Fifth variant In the fourth modified example, when the captured image data a1 indicates a detection object m, the generating section 161 may further generate position data indicating the position of the detection object m in the captured image b1 indicated by the captured image data a1.

[0137] In this case, the transmission unit 162 further transmits the position data to the communication server 2 in addition to the additional image data c3.

[0138] The receiving unit 231 of the communication server 2 receives the position data in addition to the additional image data c3. The display control unit 232 of the communication server 2 acquires the position data in addition to the additional image data c3 from the receiving unit 231. The display control unit 232 uses the additional image data c3 and the position data to display on the display device 32 of the terminal device 3 an additional image in which a mark indicating the position where the detection object m exists is added to the additional image d3. The mark is a figure (e.g., a circle or a polygon) surrounding the position where the detection object m exists. The mark is not limited to a figure surrounding the position where the detection object m exists, and may be, for example, an arrow pointing to the position where the detection object m exists. The mark may have a specific color (e.g., red or purple) or may flash. The mark is also an example of highlighting the detection object m.

[0139] For example, the display control unit 232 transmits the additional image data c3 and the position data to the terminal device 3, thereby causing the additional image to be displayed on the display device 32 of the terminal device 3. As an example, when the operation control unit 351 of the terminal device 3 receives the position data in addition to the additional image data c3, it generates additional image data indicating the additional image using the additional image data c3 and the position data. Next, the operation control unit 351 uses the additional image data to cause the display device 32 to display the additional image in the additional display area 3c.

[0140] The display control unit 232 of the communication server 2 may generate additional image data using the additional image data c3 and the position data. In this case, the display control unit 232 transmits the additional image data to the terminal device 3, thereby causing the additional image to be displayed on the display device 32 of the terminal device 3. As an example, upon receiving the additional image data, the operation control unit 351 of the terminal device 3 causes the display device 32 to display the additional image in the additional display area 3c by using the additional image data.

[0141] According to the fifth modification, the user U1 of the terminal device 3 can easily recognize the position of the detection object m by visually checking the mark shown in the additional image. Therefore, the user U1 of the terminal device 3 can fit the detection object m into the additional display area b3 or the frame image k without getting lost. Therefore, the time required to fit the detection object m into the additional display area b3 or the frame image k can be shortened, and the time required to obtain the attention image d2 representing the detection object m can be shortened.

[0142] B6: 6th variant In the fourth and fifth modified examples, the reference image data c1 may be omitted.

[0143] According to the sixth modified example, the additional image d3 obtained by lowering the resolution of the captured image b1 is another example of the reference image. The additional image data c3 is another example of the reference image data. In this case, the reference image data (additional image data c3) having a smaller amount of data than the captured image data a1 can be easily generated without omitting the object represented by the captured image b1.

[0144] B7: 7th variant In the first embodiment and the first to fifth modified examples, the generation unit 161 may blur an image of an object that is a first distance or more away from the camera 4 in the reference image d1 to reduce the resolution of the image of the object. In this case, the amount of data in the reference image data c1 can be further reduced. Note that the distance from the camera 4 to the object is measured by, for example, a distance sensor. This example is particularly effective when the user U1 of the terminal device 3 is interested in an object that is less than the first distance away from the camera 4.

[0145] In the first embodiment and the first to sixth modified examples, the generating unit 165 may blur the image of an object that is at a distance from the camera 4 that is equal to or greater than the second distance in the image of interest d2 to reduce the resolution of the image of the object. In this case, the amount of data of the image data of interest c2 can be further reduced. The second distance may be the same as or different from the first distance. This example is particularly effective when the user U1 of the terminal device 3 is interested in an object that is at a distance less than the second distance from the camera 4.

[0146] In the fourth to sixth modified examples, the generation unit 161 may blur the image of an object that is a third distance or more away from the camera 4 in the additional image d3 to reduce the resolution of the image of the object. In this case, the amount of data in the additional image data c3 can be further reduced. The third distance may be the same as the first distance or the second distance, or may be different from both the first distance and the second distance. This example is particularly effective when the user U1 of the terminal device 3 is interested in an object that is less than the third distance away from the camera 4.

[0147] B8: 8th variant In the first embodiment and the first to seventh modified examples, the operation control unit 351 of the terminal device 3 may display six shortcut keys on the display device 32 for instantly checking the situation in six directions, namely up, down, left, right, front, back, and forth, of the camera 4. The six shortcut keys correspond one-to-one to the six directions, namely up, down, left, right, front, back, and forth, of the camera 4. Note that the number of shortcut keys is not limited to six, and may be one or more, and each shortcut key corresponds to one of the directions starting from the camera 4.

[0148] When any of the six shortcut keys (for example, the shortcut key corresponding to the upward direction of the camera 4) is operated, the operation control unit 351 transmits shortcut key information indicating the direction of the operated shortcut key to the communication server 2.

[0149] The receiving unit 231 of the communication server 2 receives the shortcut key information. The display control unit 232 of the communication server 2 acquires the shortcut key information from the receiving unit 231. The display control unit 232 transmits the shortcut key information to the image communication device 10.

[0150] The receiving unit 163 of the image communication device 10 receives the shortcut key information. The determining unit 164 of the image communication device 10 acquires the shortcut key information from the receiving unit 163. The determining unit 164 determines an image of the attention area f2 (attention image d2) cut out from the captured image b2 indicated by the captured image data a2 based on the shortcut key information. For example, the determining unit 164 determines an image in the direction indicated by the shortcut key information from the captured image b2 (for example, an image in the upward direction of the camera 4) as the attention image d2. Thereafter, the operation is executed in the same manner as in the first embodiment.

[0151] According to the eighth modification, in addition to changing the position of the attention area f2 by scrolling the reference image d1, it is also possible to change the position of the attention area f2 by using a shortcut key. Therefore, an image in a direction associated with the shortcut key can be easily set as the attention image d2.

[0152] B9: 9th variant In the first embodiment and the first to eighth modifications, in a configuration in which a 360-degree camera is used as the camera 4, the operation control unit 351 of the terminal device 3 may operate as follows.

[0153] When a portion of the attention image d2 located on one side (e.g., the left side) of the display area 3a is dragged to the side opposite to the one side of the display area 3a (e.g., the right side), the operation control unit 351 of the terminal device 3 generates inversion information. The inversion information is information indicating that the position of the attention area f2 in the captured image b2 is to be changed to a position shifted by 180 degrees from the current position of the attention area f2. The operation control unit 351 transmits the inversion information to the communication server 2.

[0154] The receiving unit 231 of the communication server 2 receives the inversion information. The display control unit 232 of the communication server 2 acquires the inversion information from the receiving unit 231. The display control unit 232 transmits the inversion information to the image communication device 10.

[0155] The receiving unit 163 of the image communication device 10 receives the inversion information. The determining unit 164 of the image communication device 10 acquires the inversion information from the receiving unit 163. The determining unit 164 determines an image of the attention area f2 (attention image d2) cut out from the captured image b2 indicated by the captured image data a2 based on the inversion information. For example, the determining unit 164 changes the position of the attention area f2 in the captured image b2 to a position shifted 180 degrees from the current position of the attention area f2. Thereafter, the operation is executed in the same manner as in the first embodiment.

[0156] According to the ninth modified example, the operation of shifting the position of the attention area f2 by 180 degrees can be simplified and the operation time can be shortened, compared to a configuration in which the position of the attention area f2 is shifted by 180 degrees by scrolling.

[0157] B10: 10th variant In the first embodiment and the first to ninth modified examples, in a configuration in which a 360-degree camera is used as the camera 4, the image of interest d2 can be selected from an image of 360 degrees around the camera 4. Therefore, the range in which the image of interest d2 can be selected is wide, and the user U1 of the terminal device 3 can easily check the situation around the moving object 1.

[0158] B11: 11th variant In the first embodiment and the first to tenth modifications, the camera 4 and the image communication device 10 may be mounted on a stationary object instead of the moving object 1. The stationary object is, for example, an apparatus fixed in a forest for observing animals, or an apparatus fixed for observing the weather.

[0159] C:Other (1) In each of the first embodiment and the first to eleventh modifications, the storage devices 15, 22, and 34 may include a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory device (e.g., a card, a stick, a key drive), a CD-ROM (Compact Disc-ROM), a register, a removable disk, a hard disk, a floppy (registered trademark) disk, a magnetic strip, a database, a server, or other suitable storage medium. The program may also be transmitted from a network via a telecommunications line.

[0160] (2) Each of the first embodiment and the first to eleventh variants may be applied to a system using LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-Wide Band), Bluetooth (registered trademark), or other appropriate system, and / or a next-generation system extended based on these.

[0161] (3) The information and the like described in each of the first embodiment and the first to eleventh modifications may be represented using any of a variety of different technologies. For example, data, information, and the like that may be mentioned throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields, magnetic particles, optical fields, photons, or any combination thereof. Note that terms described in this specification and / or terms necessary for understanding this specification may be replaced with terms having the same or similar meanings.

[0162] (4) In each of the first embodiment and the first to eleventh modifications, the input and output information, etc. may be stored in a specific location (for example, a memory) or may be managed by a table. The input and output information, etc. may be overwritten, updated, or added. The output information, etc. may be deleted. The input information, etc. may be transmitted to another device.

[0163] (5) In each of the first embodiment and the first to eleventh variants, the determination may be made based on a value represented by one bit (0 or 1), based on a Boolean value (true or false), or based on a comparison of numerical values ​​(e.g., comparison with a predetermined value).

[0164] (6) The order of the process procedures, sequences, or flow charts illustrated in the first embodiment and the first to eleventh modifications may be changed as long as there is no inconsistency. For example, the methods described herein present elements of various steps in an exemplary order, and are not limited to the particular order presented.

[0165] (7) Each function illustrated in Figure 5, Figure 6, or Figure 7 may be realized by any combination of hardware and software. Furthermore, each function may be realized by a single device, or may be realized by two or more devices configured separately from each other.

[0166] (8) The programs exemplified in each of the first embodiment and the first to eleventh modifications should be broadly construed to mean instructions, instruction sets, code, code segments, program code, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, or functions, regardless of whether they are called software, firmware, middleware, microcode, or hardware description language, or by other names. In addition, software, instructions, etc. may be transmitted and received via a transmission medium. For example, when software is transmitted from a website, a server, or other remote source using wired technologies such as coaxial cable, fiber optic cable, twisted pair, and digital subscriber line (DSL), and / or wireless technologies such as infrared, radio, and microwave, these wired and / or wireless technologies are included within the definition of a transmission medium.

[0167] (9) In each of the first embodiment and the first to eleventh modifications, the terms "system" and "network" are used interchangeably.

[0168] (10) In each of the first embodiment and the first to eleventh modifications, the terminal device 3 may be a mobile station, which may also be referred to by those skilled in the art as a subscriber station, a mobile unit, a subscriber unit, a wireless unit, a remote unit, a mobile device, a wireless device, a wireless communication device, a remote device, a mobile subscriber station, an access terminal, a mobile terminal, a wireless terminal, a remote terminal, a handset, a user agent, a mobile client, a client, or some other appropriate terminology.

[0169] (11) In each of the first embodiment and the first to eleventh modifications, the term "based on" does not mean "based only on," unless otherwise specified. In other words, the term "based on" means both "based only on" and "based at least on."

[0170] (12) As used herein, any reference to an element using designations such as "first" and "second" does not generally limit the quantity or order of those elements. These designations may be used herein as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must precede the second element in some way.

[0171] (13) In the first embodiment and each of the first to eleventh modifications, to the extent that the terms "including," "comprising," and variations thereof are used in this specification or claims, these terms are intended to be inclusive, similar to the term "comprises." Furthermore, the term "or" used in this specification or claims is not intended to be an exclusive logical OR.

[0172] (14) Throughout this application, where articles have been added by translation, such as a, an and the in English, these articles include the plural unless the context clearly indicates otherwise.

[0173] (15) In this specification, the term "apparatus" may be replaced with other terms such as circuit, device, or unit.

[0174] (16) It is clear to those skilled in the art that the present invention is not limited to the embodiments described in this specification. The present invention can be implemented in modified and altered forms without departing from the spirit and scope of the present invention as defined by the claims. Therefore, the description in this specification is for illustrative purposes only and does not have any limiting meaning on the present invention. In addition, multiple aspects selected from the aspects exemplified in this specification may be combined.

[0175] D: Aspects understood from the above-mentioned embodiments or modifications The following aspects can be understood from at least one of the above-described embodiments or modified examples.

[0176] D1: First mode The image communication device according to the first aspect includes a first generating section, a first transmitting section, a receiving section, a determining section, a second generating section, and a second transmitting section. The first generating section generates reference image data indicating a reference image corresponding to a first captured image based on first captured image data indicating a first captured image, the reference image data having a data amount smaller than that of the first captured image data. The first transmitting section transmits the reference image data. The receiving section receives operation information indicating an operation of scrolling the reference image indicated by the reference image data in a display area smaller than the reference image. The determining section determines an image of an attention area to be cut out from a second captured image indicated by second captured image data generated by the camera based on the operation information. The second generating section generates display image data indicating an image of the attention area by cutting out an image of the attention area from the second captured image using the second captured image data. The second transmitting section transmits the display image data.

[0177] According to this aspect, it is possible to display the transmitted image on the display device without causing a significant delay, while minimizing the decrease in visibility of the scroll display.

[0178] D2: Second mode In an example of the first aspect (second aspect), the reference image is an image of a partial area of ​​the first captured image. The first generation unit generates the reference image data by cutting out the reference image from the first captured image using the first captured image data. According to this aspect, by cutting out the reference image from the first captured image using the first captured image data, it is possible to easily generate reference image data having a smaller amount of data than the first captured image data.

[0179] D3: Third mode In an example of the second aspect (third aspect), the first generating unit repeatedly generates the reference image data, and in a situation where the first generating unit newly generates the reference image data after the receiving unit receives the operation information, the first generating unit adjusts a size of the reference image indicated by the newly generated reference image data based on the operation information. According to this aspect, the size of the reference image can be adjusted in response to scrolling of the reference image.

[0180] D4: Fourth mode In an example of the second or third aspect (fourth aspect), the first generation unit adjusts the size of the reference image based on the first captured image data. According to this aspect, for example, the size of the reference image can be adjusted according to an object indicated by the first captured image data.

[0181] D5: Fifth mode In any one of the second to fourth aspects (fifth aspect), the first generation unit adjusts the size of the reference image based on the terrain around the camera indicated by map information corresponding to the position of the camera. According to this aspect, even in a situation where the terrain around the camera is difficult to distinguish from the first captured image data, the size of the reference image can be adjusted based on the terrain around the camera.

[0182] D6: Sixth mode In an example of the second aspect (sixth aspect), the first generating unit repeatedly generates the reference image data. The first transmitting unit transmits the reference image data every time the reference image data is generated. In a situation where the reference image data is newly generated after the receiving unit receives the operation information, the first generating unit adjusts a generation timing of the newly generated reference image data based on a frequency with which the receiving unit receives the operation information.

[0183] Receiving operation information less frequently means that the reference image is scrolled less frequently. Receiving operation information more frequently means that the reference image is scrolled more frequently. According to this aspect, the first generating unit adjusts the generation timing of newly generated reference image data based on the frequency with which the receiving unit receives operation information. Therefore, for example, when the reference image is scrolled less frequently, the frequency with which the reference image data is generated can be reduced. When the reference image is scrolled more frequently, the frequency with which the reference image data is generated can be increased. Therefore, the frequency with which the reference image data is generated can be adjusted as necessary.

[0184] D7: Seventh mode In any one of the first to sixth aspects (seventh aspect), when the first captured image data indicates a detection target, the first generating unit further generates additional image data indicating an additional image obtained by lowering the resolution of the first captured image using the first captured image data. The first transmitting unit further transmits the additional image data. The receiving unit further receives additional operation information indicating an operation of scrolling the additional image indicated by the additional image data in a display area smaller than the reference image. The determining unit determines an image of the attention area based on the additional operation information. According to this aspect, when the first captured image data indicates a detection target, a part of the additional image indicating the detection target is displayed in the display area. Therefore, a user looking at the display area can recognize that the additional image indicates the detection target by adding the display of the additional image. Therefore, the user can fit the detection target into the attention area by scrolling the additional image obtained by lowering the resolution of the first captured image.

[0185] D8: Eighth mode In an example of the seventh aspect (eighth aspect), when the first captured image data indicates a detection object, the first generating unit further generates position data indicating a position of the detection object in the first captured image. The first transmitting unit further transmits the position data. According to this aspect, the position data can be used to identify the position of the detection object in the first captured image.

[0186] D9: 9th mode A display control device according to a ninth aspect includes a data receiving unit that receives the additional image data and the position data from the image communication device according to the eighth aspect, and a display control unit that uses the additional image data and the position data to cause a display device to display an image in which a mark indicating the position where the detection object exists is added to the additional image. According to this aspect, it is possible to display an image in which a mark indicating the position where the detection object exists is added to the additional image.

[0187] D10: 10th mode In an example of the first aspect (tenth aspect), the reference image is an image obtained by lowering the resolution of the first captured image. The first generation unit generates the reference image data by using the first captured image data. According to this aspect, reference image data having a smaller amount of data than the first captured image data can be easily generated by lowering the resolution of the first captured image. [Explanation of symbols]

[0188] 1...mobile object, 2...communication server, 3...terminal device, 4...camera, 10...image communication device, 11...input device, 12...output device, 13...communication device, 14...communication device, 15...storage device, 16...processing device, 21...communication device, 22...storage device, 23...processing device, 31...input device, 32...display device, 33...communication device, 34...storage device, 35...processing device, 161...generation unit, 162...transmission unit, 163...reception unit, 164...determination unit, 165...generation unit, 166...transmission unit, 167...operation control unit, 231...reception unit, 232...display control unit, 233...operation control unit, 351...operation control unit, CS...communication system.

Claims

1. a first generation unit that generates, based on first captured image data generated by a camera and indicating a first captured image, reference image data indicating a reference image corresponding to the first captured image, the reference image data having a data amount smaller than a data amount of the first captured image data; a first transmission unit that transmits the reference image data; a receiving unit for receiving operation information indicating an operation of scrolling the reference image indicated by the reference image data in a display area smaller than the reference image; a determination unit that determines, based on the operation information, an image of an attention area to be cut out from a second captured image indicated by second captured image data generated by the camera; a second generation unit that generates display image data indicating an image of the attention area by cutting out an image of the attention area from the second captured image using the second captured image data; A second transmission unit that transmits the display image data; An image communication device comprising:

2. the reference image is an image of a partial area of ​​the first captured image, the first generation unit generates the reference image data by extracting the reference image from the first captured image using the first captured image data; 2. The image communication apparatus according to claim 1.

3. The first generation unit repeatedly generates the reference image data, In a situation where the reference image data is newly generated after the receiving unit receives the operation information, the first generating unit adjusts a size of a reference image indicated by the newly generated reference image data based on the operation information.

3. The image communication apparatus according to claim 2.

4. The first generation unit adjusts a size of the reference image based on the first captured image data.

4. The image communication apparatus according to claim 2 or 3.

5. The first generation unit adjusts the size of the reference image based on a topography around the camera indicated by map information corresponding to the position of the camera.

5. The image communication apparatus according to claim 2.

6. The first generation unit repeatedly generates the reference image data, The first transmission unit transmits the reference image data every time the reference image data is generated; In a situation where the reference image data is newly generated after the receiving unit receives the operation information, the first generating unit adjusts a generation timing of the newly generated reference image data based on a frequency at which the receiving unit receives the operation information.

3. The image communication apparatus according to claim 2.

7. the first generation unit further generates, when the first captured image data indicates a detection object, additional image data indicating an additional image obtained by lowering a resolution of the first captured image, using the first captured image data; The first transmission unit further transmits the additional image data, the receiving unit further receives additional operation information indicating an operation of scrolling the additional image indicated by the additional image data in a display area smaller than the reference image; The determination unit determines an image of the attention area based on the additional operation information. The image communication apparatus according to claim 1 .

8. When the first captured image data indicates a detection object, the first generation unit further generates position data indicating a position of the detection object in the first captured image; The first transmission unit further transmits the position data.

8. The image communication apparatus according to claim 7.

9. the reference image is an image obtained by lowering the resolution of the first captured image, The first generation unit generates the reference image data by using the first captured image data.

2. The image communication apparatus according to claim 1.

Images