IMAGE TRANSMISSION SYSTEM, IMAGE TRANSMISSION METHOD, AND PROGRAM

Abstract

To appropriately set a data transmission timing in a system that synthesizes a plurality of imaging data which are continuously acquired into one image data and transmits the imaging data to a remote location.SOLUTION: An image transmission system comprises: a plurality of cameras; and a communication device. The communication device is configured to perform processing for sequentially acquiring imaging data from the plurality of cameras, respectively, processing for generating composite image data which is a composition of the last imaging data acquired by the plurality of cameras upon each acquisition of imaging data from a main camera, which is one of the plurality of cameras, and processing for transmitting the composite image data to a predetermined terminal.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to a technique for synthesizing a plurality of image data and transmitting the synthesized data to a remote location. [Background technology]

[0002] Japanese Patent Application Publication No. 2011-221686 discloses a technology for wirelessly transmitting images of the vehicle's surroundings taken by an on-board camera to a center outside the vehicle. The images transmitted consist of multiple images in chronological order. The frequency of image transmission is changed based on the level of difficulty of safe driving, and images are transmitted more frequently when the level of difficulty is high. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2008-027097 A [Patent Document 2] JP 2011-221686 A Summary of the Invention [Problem to be solved by the invention]

[0004] Consider an image transmission system that synthesizes multiple imaging data acquired by multiple cameras to generate one synthetic image data and transmits it to a remote location. Each camera continuously captures images at a predetermined cycle and generates a captured image. The image transmission system sequentially acquires each of the continuously generated imaging data, synthesizes the multiple imaging data, and transmits it to a remote location. At this time, some imaging data may not be acquired due to processing delays, etc. For this reason, it is necessary to set the timing of data transmission appropriately.

[0005] If the image transmission system is a system that transmits each piece of imaging data to a remote location individually without synthesizing the imaging data, the timing of data transmission does not pose a problem. In other words, in this case, if any imaging data is not acquired, the imaging data is not transmitted, and even if any imaging data is not acquired, the transmission of other imaging data that is not not acquired is not affected. However, in a system that synthesizes multiple imaging data and then transmits them to a remote location, such as the image transmission system according to this embodiment, not transmitting one piece of synthetic image data means that all imaging data included in the synthetic image data is not transmitted. Therefore, it is necessary to appropriately set the timing of data transmission.

[0006] One object of the present disclosure is to provide a technology that can appropriately set the timing of data transmission in a system in which multiple pieces of continuously acquired imaging data are synthesized into one image data and then transmitted to a remote location. [Means for solving the problem]

[0007] The first aspect relates to an image transmission system including a plurality of cameras and a communication device. The communication device A process of sequentially acquiring image data from each of a plurality of cameras; A process of generating composite image data by combining the last captured image data of each of the plurality of cameras, each time captured image data is obtained from a main camera that is one of the plurality of cameras; A process of transmitting the composite image data to a predetermined terminal; The apparatus is configured to execute the following steps:

[0008] The second aspect relates to an image transmission method. The image transmission method is as follows: Sequentially acquiring imaging data from each of the plurality of cameras; generating composite image data by combining the imaging data last acquired from each of the plurality of cameras, each time imaging data is acquired from a main camera that is one of the plurality of cameras; Transmitting the composite image data to a predetermined terminal; Includes.

[0009] A third aspect relates to a program executed by a computer. The program causes a computer to execute the image transmission method according to the second aspect. Effect of the Invention

[0010] According to the present disclosure, in a system in which a plurality of pieces of continuously acquired imaging data are synthesized into one piece of image data and then transmitted to a remote location, it is possible to appropriately set the timing of data transmission. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram for explaining an overview of image transmission according to an embodiment of the present disclosure. [Diagram 2] 1 is a block diagram showing an example of the configuration of an image transmission system; [Diagram 3] FIG. 2 is a diagram for explaining an example of a camera and synthetic image data. [Figure 4] FIG. 2 is a block diagram showing a configuration example of a terminal. [Diagram 5] 2 is a block diagram showing an example of a functional configuration of an image transmission system and a terminal. FIG. [Figure 6] 13 is a flowchart illustrating an example of a process performed in a queuing unit. [Figure 7] 4 is a time chart showing transmission timing according to an embodiment of the present disclosure. [Figure 8] 11 is a time chart showing transmission timing in a comparative example. [Figure 9] 11 is a time chart showing transmission timing in a comparative example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0013] 1. Overview 1 is a diagram for explaining an outline of a process related to image transmission performed by an image transmission system 100 according to the present embodiment. The image transmission system 100 is configured to be able to communicate with a predetermined terminal 200 via a network, and transmits image data to the terminal 200. The data transmitted from the image transmission system 100 to the terminal 200 includes composite image data 42z. The composite image data 42z is image data obtained by combining a plurality of pieces of imaging data 41x, 41y. The plurality of pieces of imaging data 41x, 41y are image data obtained by different cameras successively capturing images of the surroundings at predetermined intervals, and are sequentially acquired by the image transmission system 100.

[0014] Hereinafter, the reference symbols 41x and 41y are used to indicate the imaging data when specifying one piece, but the reference symbol 41 is used to refer to any imaging data or to refer collectively. The reference symbol 42z is used to indicate the composite image data when specifying one piece, but the reference symbol 42 is used to refer to any composite image data or to refer collectively.

[0015] The imaging data 41 may be acquired, for example, by an on-board camera mounted on a vehicle.

[0016] The composite image data 42 transmitted from the image transmission system 100 to the terminal 200 is displayed on the display 24 of the terminal 200. When the multiple pieces of imaging data 41 are acquired by an on-board camera, the terminal 200 is exemplified by a remote terminal for remotely supporting the vehicle. In this case, for example, it is assumed that an operator providing remote support for the vehicle monitors the composite image data 42 displayed on the display 24 and provides remote support for the vehicle through the remote terminal 200.

[0017] The composite image data 42 is generated by synthesizing a plurality of pieces of imaging data 41 in accordance with the display area of ​​the display 24. The generation of the composite image data 42 is performed, for example, by PIP (picture-in-picture) processing. In the PIP processing, the plurality of pieces of imaging data 41 are synthesized so that a portion or all of one piece of image data is superimposed on a portion of an area of ​​the other image data. For example, in the example of FIG. 1, imaging data 41x is used as a base image, and imaging data 41y is superimposed on a portion of the imaging data 41x to generate composite image data 42z.

[0018] The number of pieces of imaging data 41 used to generate the composite image data 42 is not particularly limited as long as it is more than one. Similarly, when the number of pieces of imaging data 41 is three or more, the composite image data 42 may be generated by PIP processing.

[0019] In this embodiment, the multiple pieces of imaging data 41 are not transmitted individually to the terminal 200, but are synthesized by the image transmission system 100, and the synthesized image data 42 is transmitted to the terminal 200. By generating the multiple pieces of imaging data 41 as synthesized image data 42 and then transmitting it to the terminal 200, the amount of data to be transmitted can be reduced, and the network load can be reduced. Furthermore, when the synthesized image data 42 is generated by PIP processing, the imaging data 41 to be superimposed when generating the synthesized image data 42 can be compressed, or the superimposed portion of the multiple pieces of imaging data 41 can be deleted. Therefore, the amount of data transmitted from the image transmission system 100 to the terminal 200 can be further reduced.

[0020] The multiple pieces of imaging data 41 are continuously acquired by different cameras. Therefore, the acquired imaging data 41 must be continuously transmitted to the terminal 200. If each piece of imaging data 41 is transmitted to the terminal 200 separately, the image transmission system 100 may transmit the imaging data 41 to the terminal 200 each time it is acquired by the camera. However, if multiple pieces of imaging data 41 are synthesized and then transmitted to the terminal 200, all of the imaging data 41 are simultaneously transmitted to the terminal 200 as synthesized image data 42. Therefore, it is necessary to appropriately set the timing for synthesizing and transmitting the imaging data 41. The timing for synthesizing and transmitting the imaging data 41 by the image transmission system 100 according to this embodiment will be described below.

[0021] 2. Configuration Example First, configuration examples of the image transmission system 100 and the terminal 200 will be described with reference to Fig. 2 to Fig. 4. Fig. 2 is a block diagram showing a configuration example of the image transmission system 100. The image transmission system 100 includes a communication device 11 and a plurality of cameras 14x, 14y. The communication device 11 and the cameras 14x, 14y are connected via a predetermined network. Hereinafter, the reference symbols for the cameras will be 14x and 14y when specifying one camera, but 14 will be used when referring to any camera or when referring to the cameras collectively.

[0022] The communication device 11 is a device that transmits data to the terminal 200. The communication device 11 performs wireless communication with the terminal 200. The communication with the terminal 200 may be performed via a base station on a network, or may be performed directly. Examples of communication standards for this wireless communication include 4G, LTE, and 5G communication standards.

[0023] The communication device 11 includes at least one processor 12 (hereinafter simply referred to as the processor 12) and at least one memory 13 (hereinafter simply referred to as the memory 13). The processor 12 includes a CPU (Central Processing Unit). The memory 13 is a volatile memory such as a DDR memory, and expands various programs used by the processor 12 and temporarily stores various data. The various data include imaging data 41, composite image data 42, layout information, information about the main camera described later, and the like.

[0024] The layout information is information about the layout of the imaging data 41 for generating the composite image data 42 from a plurality of imaging data 41. The layout information may be stored in advance in the memory 13, or may be acquired by communication from the terminal 200 before the generation of the composite image data 42.

[0025] The processor 12 performs various types of data processing by executing a predetermined data processing program stored in the memory 13. The data processing performed by the processor 12 includes a process of acquiring the imaging data 41, a process of generating the composite image data 42, a process of transmitting data to the terminal 200, and the like.

[0026] The camera 14 is a camera for capturing an image of the surroundings and acquiring imaging data 41. The number of cameras 14 is not particularly limited as long as there is more than one. For example, two cameras 14x and 14y are shown in FIG. 2. Each of the multiple cameras 14 transmits imaging data 41 obtained by capturing an image to the communication device 11.

[0027] The camera 14 may be an on-board camera. FIG. 3 shows an example of the cameras 14a, 14b, 14c, 14d, 14f, 14g, 14i, and 14j (cameras 14) as on-board cameras and the composite image data 42e (composite image data 42). Each area shown by a dashed line represents a range captured by each camera 14. In the example of FIG. 3, the composite image data 42e is generated by combining the image data 41a, 41b, 41c, and 41d obtained by the multiple on-board cameras 14a, 14b, 14c, and 14d, respectively. Of these, the camera 14a is a front camera that captures the front of the vehicle VE. The camera 14b is a rear camera that captures the rear of the vehicle VE. The camera 14c is a left side camera that captures the left side of the vehicle VE. The camera 14d is a right side camera that captures the right side of the vehicle VE. The composite image data 42e is generated by PIP processing. Here, parts of the imaging data 41b, 41c, and 41d are superimposed on the imaging data 41a acquired from the camera 14a. The size and arrangement of each imaging data 41 when generating the composite image data 42e are determined based on the layout information.

[0028] Hereinafter, when the composite image data 42 is generated by PIP processing, the camera 14 that acquires the base image data 41 is referred to as the base camera. In the example of Fig. 3, the camera 41a that acquires the image data 41a is the base camera. The layout information may include information about the base camera.

[0029] The image transmission system 100 may generate a plurality of synthetic image data 42. For example, in addition to the synthetic image data 42e, synthetic image data 42h may be generated from the imaging data 41f and 41g acquired from the cameras 14f and 14g, and synthetic image data 42k may be generated from the imaging data 41i and 41j acquired from the cameras 14i and 14j.

[0030] 3, when the camera 14 is an on-board camera, the communication device 11 may be connected to various sensors mounted on the vehicle VE via a predetermined network. Examples of the various sensors include vehicle state sensors such as a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, and a gear position sensor. The communication device 11 can acquire information about the state of the vehicle, such as the speed, acceleration, yaw rate, steering angle, and gear position, by communicating with these sensors.

[0031] 4 is a block diagram showing a configuration example of the terminal 200. The terminal 200 includes a data processing device 21, a display 24, an input device 25, and a communication device 26. The display 24, the input device 25, and the communication device 26 are connected to the data processing device 21 via a predetermined network.

[0032] The data processing device 21 is a computer that performs data processing based on various data held by the terminal 200 and various data acquired by the terminal 200. The data processing device 21 includes at least one processor 22 (hereinafter simply referred to as the processor 22) and at least one memory 23 (hereinafter simply referred to as the memory 23). The processor 22 performs various types of data processing by executing a predetermined data processing program stored in the memory 23. The processor 22 includes a CPU. The memory 23 is a volatile memory such as a DDR memory, and expands various programs used by the processor 22 and temporarily stores various types of data.

[0033] The display 24 displays the composite image data 42 received from the image transmission system 100 to the user of the terminal 200. When the camera 14 is an in-vehicle camera, the terminal 200 may be a terminal for providing remote support for the vehicle VE. In this case, the user of the terminal 200 is assumed to be an operator providing remote support, and the composite image data 42 displayed on the display 24 is monitored by the operator.

[0034] The remote assistance by the operator also includes remote driving. In remote driving, the operator performs driving operations of the vehicle VE, including steering, acceleration, deceleration, etc., with reference to the composite image data 42 displayed on the display 24.

[0035] The input device 25 is operated by a user of the terminal 200. The input device 25 includes, for example, an input unit such as a mouse, a keyboard, a button, a switch, etc., which are operated by the user. When the terminal 200 is a terminal for remotely supporting the vehicle VE, the input device 25 is operated by an operator. When the operator remotely drives the vehicle VE, the input device 25 may include an input device for driving. Examples of the input device for driving include a steering wheel, a shift lever, an accelerator pedal, and a brake pedal. In addition to these, the user of the terminal 200 may input an instruction to the input device 25 for changing the settings of the main camera, which will be described later.

[0036] The communication device 26 performs wireless communication at least with the image transmission system 100. The communication device 26 can receive data from the image transmission system 100 via wireless communication. The communication device 26 can also generate instruction information based on an operation input to the input device 25 by a user of the terminal 200, and transmit the instruction information to the image transmission system 100. When the terminal 200 performs remote assistance or remote driving of the vehicle VE, the instruction information includes information related to the remote assistance or remote driving. The instruction information may also include an instruction related to settings of the main camera, which will be described later.

[0037] 3. Data processing flow An example of functional blocks and a data processing flow of the image transmission system 100 and the terminal 200 will be described with reference to FIG.

[0038] The image transmission system 100 includes a queuing unit 101, a synthesis unit 102, an encoding unit 103, and a communication unit 104. These functional blocks are realized by the processor 12 executing a program stored in the memory 13.

[0039] In the flow of data processing performed by the image transmission system 100, first, various data are acquired in the queuing unit 101. The various data includes imaging data 41 obtained by each camera 14 capturing an image of the surroundings. Each camera 14 sequentially acquires the imaging data 41, and sequentially transmits the acquired imaging data 41 to the queuing unit 101. The queuing unit 101 sequentially acquires the imaging data 41 transmitted from each camera 14.

[0040] When the processor 12 determines that it is transmission timing, the multiple pieces of imaging data 41 are sent from the queuing unit 101 to the synthesis unit 102. Among the multiple pieces of imaging data 41, those that generate the same synthetic image data 42 are sent to the synthesis unit 102 at the same timing. The transmission timing will be described later.

[0041] In the synthesis unit 102, the multiple pieces of imaging data 41 are synthesized to generate synthetic image data 42. When generating the synthetic image data 42, the position and size of each piece of imaging data 41 are specified based on the layout data. The synthetic image data 42 synthesized in the synthesis unit 102 is sent to the encoding unit 103. When the synthesis unit 102 receives the imaging data 41 from the queuing unit 101, the synthetic image data 42 is generated in sequence, and the generated synthetic image data 42 is sent to the encoding unit 103 in sequence.

[0042] The encoding unit 103 performs encoding processing on the composite image data 42. During encoding processing, the composite image data 42 may be compressed. When the encoding unit 103 receives the composite image data 42 from the composition unit 102, encoding processing is performed sequentially, and the composite image data 42 after encoding processing is sequentially sent to the communication unit 104. The communication unit 104 communicates with the terminal 200, and the composite image data 42 received from the encoding unit 103 is sequentially sent to the terminal 200.

[0043] The terminal 200 includes a communication unit 201 and a display unit 202. These functional blocks are realized by the processor 22 executing a program stored in the memory 23.

[0044] The communication unit 201 communicates with the image transmission system 100, and sequentially receives the composite image data 42 transmitted from the image transmission system 100. The received composite image data 42 is sent to the display unit 202 sequentially.

[0045] The display unit 202 decodes the acquired composite image data 42 and sequentially displays it on the display 24. If the composite image data 42 is compressed, the data is expanded.

[0046] In the functional units subsequent to the combining unit 102, data is processed sequentially when it is input, and the data generated as a result of the processing is sent sequentially to the next functional unit. In other words, the timing of transmission of the imaging data 41 from the queuing unit 101 to the combining unit 102 determines the timing of composition and transmission of the imaging data 41. The transmission timing is determined by the processing performed in the queuing unit 101.

[0047] 4. Processing flow example 6 is a flowchart showing an example of processing performed in queuing unit 101. The processing shown in FIG.

[0048] A series of processes starts when imaging data 41 is transmitted from any of the cameras 14 to the image transmission system 100. In step S111, the processor 12 acquires the imaging data 41 transmitted from the camera 14. The acquired imaging data 41 is held by the queuing unit 101 until the transmission timing arrives. In addition, if the previous imaging data 41 acquired from the same camera 14 is already held by the queuing unit 101, it is updated to the newly acquired imaging data 41.

[0049] In step S112, the processor 12 determines whether or not main image data is held in the queuing unit 101. If main image data is held (step S112; Yes), the process proceeds to step S113. If main image data is not held (step S112; No), the process ends.

[0050] The main image data is the imaging data from which the user of the terminal 200 is considered to have obtained the most information, among the imaging data 41 used to generate the composite image data 42. The main image data will be described later.

[0051] In step S113, the processor 12 transmits all of the imaging data 41 used to generate one composite image data 42 from the queuing unit 101 to the synthesis unit 102. That is, the timing at which the process of step S113 is performed is the transmission timing. The imaging data 41 last acquired from each camera 14 is transmitted to the synthesis unit 102. The transmitted imaging data 41 is then used to generate the composite image data 42 in the synthesis unit 102, and the composite image data 42 is transmitted to the terminal 200.

[0052] In step S114, the processor 12 deletes all the main image data held in the queuing unit 101. When the main image data is deleted, the process ends.

[0053] As shown in the series of processes, in this embodiment, the timing when the main image data is acquired is set as the transmission timing. Hereinafter, the camera 14 that acquires the main image data will be referred to as the main camera. The main image data and the main camera will be described in detail.

[0054] The amount of information that the user of the terminal 200 obtains from the imaging data 41 included in the composite image data 42 is not necessarily the same for all imaging data 41. For example, consider a case where the camera 14 is an in-vehicle camera mounted on the vehicle VE and the user of the terminal 200 is a remote support operator as in the example of FIG. 3. When the vehicle VE is traveling forward, it is considered that the remote support operator usually obtains the most information from the front camera 14a. In this way, the main camera is set so that there is one for one composite image data 42, taking into consideration that the amount of information that the user obtains from each imaging data 41 differs. By setting one main camera, the main image data is also determined to be any one of the imaging data 41 used to generate one composite image data 42.

[0055] Examples of a method for setting the main camera will be described. In a first example, the main camera is fixed to a specific camera determined in advance. Any camera may be determined as the main camera, but when the composite image data 42 is generated by PIP processing, it is preferable to set the base camera as the main camera. In this case, the base imaging data 41 becomes the main image data. Since the information most required by the user of the terminal 200 is often obtained from the base camera, it is appropriate to set it as the main camera.

[0056] In the second example, the main camera is changed according to the situation. For example, when the camera 14 is an in-vehicle camera of the vehicle VE, the camera 14 from which the user of the terminal 200 obtains a particularly large amount of information may differ depending on the situation, such as the front camera 14a when the vehicle VE is traveling forward, and the rear camera 14b when the vehicle VE is traveling backward. Therefore, in the second example, the main camera is changed according to the situation. In this case, for example, it is assumed that a camera facing the traveling direction of the vehicle VE is set as the main camera. For example, it is assumed that the front camera 14a is set as the main camera when the vehicle VE is traveling forward, the rear camera 14b is set as the main camera when the vehicle VE is traveling backward, and the left side camera 14c is set as the main camera when the vehicle VE is turning left. The processor 12 can determine the traveling direction of the vehicle VE by acquiring information from, for example, a gear position sensor, a speed sensor, and an acceleration sensor of the vehicle VE, and set the main camera based on the determined result.

[0057] In the third example as well, the main camera can be changed to any camera 14. However, in the third example, the main camera is set by the user of terminal 200. In this case, for example, the user inputs an instruction to input device 25 as to which camera 14 to set as the main camera, and the instruction result is transmitted from terminal 200 to image transmission system 100. Image transmission system 100 sets camera 14 selected by the user as the main camera based on the instruction result.

[0058] The fourth example is a combination of the second and third examples. That is, the main camera may be set to be changeable depending on the situation of the vehicle VE, and may be changed in response to a user instruction when the user instructs it.

[0059] 5. Time Chart Fig. 7 is a time chart showing the transmission timing in this embodiment. In the following description, it is assumed that the composite image data 42 is generated from the imaging data 41a, 41b, 41c, and 41d acquired by the cameras 14a, 14b, 14c, and 14d, respectively. The main camera is set to the camera 14a. The main image data is the imaging data 41a.

[0060] As shown in Fig. 7, imaging data 41 is transmitted from each camera 14 at a predetermined cycle, and is acquired sequentially by the queuing unit 101. The cycle at which the imaging data 41 is acquired may be the same for all cameras 14, or may be different for each camera 14. The timing at which each imaging data 41 is acquired may be simultaneous, or may differ for each camera 14. In the example of Fig. 7, the cycle at which the imaging data 41 is acquired is the same for the four cameras 14. In Fig. 7, the reference numeral for each imaging data 41 is given a number indicating the cycle.

[0061] In this embodiment, the transmission timing is determined based on whether or not imaging data 41a, which is the main image data, has been acquired. Here, the imaging data 41 transmitted from each camera 14 may not be acquired due to a delay in processing by the processor 12. For example, imaging data 41b-5, 41c-5, and 41d-5 that should have been acquired in the fifth period have not been acquired for cameras 14b, 14c, and 14d. In this embodiment, even if acquisition of all other imaging data 41 is not completed, the time T5 at which the main image data 41a-6 is acquired is determined to be the transmission timing.

[0062] When an acquisition failure occurs in any of the imaging data 41, the previous data, i.e., the most recently acquired imaging data 41, is used to generate the composite image data 42. For example, at time T5, the previous imaging data 41b-4, 41c-4, and 41d-4 are used instead of the imaging data 41b-5, 41c-5, and 41d-5 where an acquisition failure occurs to generate the composite image data 42-5.

[0063] Furthermore, for the main camera 14a, the imaging data 41a-4 that should have been acquired in the fourth cycle is not acquired. In this embodiment, in such a case, the synthetic image data 42 is not generated until the imaging data 41a of the main camera 14a is acquired.

[0064] At this time, each piece of imaging data 41 at the transmission timing is updated to the latest imaging data 41 that has been acquired and then sent to the synthesis unit 102. From the time when the imaging data 41b-3, 41c-3, and 41d-3 are acquired until time T4, which is determined as the transmission timing, new imaging data 41b-4, 14c-4, and 14d-4 are acquired from the same cameras 14b, 14c, and 14d. Therefore, the imaging data 41b-3, 41c-3, and 41d-3 are not sent to the synthesis unit 102, and instead, the updated imaging data 41b-4, 14c-4, and 14d-4 are sent to the synthesis unit 102.

[0065] 6. Comparative Example As comparative examples, two typical methods for determining the transmission timing are shown below. Figure 8 is a time chart showing the transmission timing in the first comparative example.

[0066] 8, the transmission timing is the timing at which new imaging data 41 is acquired from all of the cameras 14. For example, at time T1, the latest imaging data 41a-1, 41b-1, 41c-1, and 41d-1 are acquired, and the processor 12 determines that it is the transmission timing.

[0067] After time T1, the imaging data 41b-2 that should have been acquired in the second period is not acquired for the camera 14b, so the next transmission timing is determined to be time T2, when new imaging data 41b-3 is acquired from the camera 14b.

[0068] In particular, the imaging data 41a-2 acquired from the camera 14a is updated to the newly acquired imaging data 41a-3 in the next cycle. That is, the imaging data 41a-2 acquired from the camera 14a in the second cycle is not used for generating the composite image data 42, and is not transmitted to the terminal 200. Therefore, for the user of the terminal 200, the transmission of the main image data is delayed by one cycle.

[0069] Furthermore, there are missing acquisitions of imaging data 41b-5, 41c-5, and 41d-5 in the fifth cycle, and there is missing acquisition of imaging data 41d-6 in the sixth cycle. Therefore, no synthetic image data 42 is generated until time T4 when the latest imaging data 41 is acquired from all of the cameras 14.

[0070] In this case as well, the imaging data 41a-6 acquired from the main camera is updated to the newly acquired imaging data 41a-7. In other words, the imaging data 41a-6 is not used to generate the composite image data 42, and is not transmitted to the terminal 200. For the user of the terminal 200, the transmission of the main image data is delayed by one period.

[0071] 8, if a failure occurs in acquiring any of the imaging data 41 other than the main image data, the transmission of the main image data will be delayed. For the user of the terminal 200, the main image data is the most important data, so it is undesirable for a delay in the transmission of the main image data to occur.

[0072] In contrast, in this embodiment, when main image data is acquired, regardless of whether other captured image data 41 has been acquired, composite image data 42 is generated and transmitted to terminal 200. This makes it possible to eliminate delays in the main image data when it is transmitted to terminal 200. Depending on the user of terminal 200, it becomes possible to obtain important information without missing it.

[0073] Next, a time chart in a second comparative example is shown in Fig. 9. In the second comparative example, the transmission timing is set at a predetermined cycle. In the example of Fig. 9, the predetermined cycle is set to be equal to the cycle at which the imaging data 41 is acquired.

[0074] In this example, first, at time T1, the imaging data 41a-1, 41b-1, 41c-1, and 41d-1 are combined to generate composite image data 42-1. Next, at time T2, when a predetermined period has elapsed since time T1, the next composite image data 42-2 is generated. Furthermore, at time T3, when a predetermined period has elapsed since time T2, the next composite image data 42-3 is generated. The same applies to times T4 and after.

[0075] In some cases, such as the imaging data 41a-4, the main image data may not be acquired in some cycles. In this case, the latest imaging data 41a that has been acquired, that is, the imaging data 41a-3 that is the previous data, is used to generate the composite image data 42.

[0076] In the second comparative example, even if an acquisition omission occurs in any of the captured image data 41, the composite image data 42 is generated after a predetermined period has elapsed, so there is no delay in the main image data as in the first comparative example. However, if an acquisition omission occurs in the main image data, the previous data is used to generate the composite image data 42. Therefore, the composite image data 42 displayed on the display 24 is data in which only the main image data is delayed by one period, and the quality of the composite image data 42 when focusing on the main image data is reduced.

[0077] In contrast, in the present embodiment, if a loss of main image data occurs, the synthetic image data 42 is not generated. Therefore, the main image data used to generate the synthetic image data 42 is not the previous data, and the quality of the synthetic image data 42 when focusing on the main image data is guaranteed.

[0078] 7. Summary As described above, the image transmission system 100 according to the present embodiment synthesizes a plurality of pieces of imaging data 41 acquired from a plurality of cameras 14, and transmits the synthesized image data 42 to a predetermined terminal 200. In determining the transmission timing for synthesizing and transmitting the imaging data 41, a main camera is set, and the timing when the main image data is acquired from the main camera is set as the transmission timing. As long as the main image data is acquired, the synthesized image data 42 is generated without waiting for the latest imaging data 41 to be acquired from the other cameras 14. This allows information that the user particularly needs to be transmitted to the predetermined terminal 200 without omission. Conversely, if the main image data is not acquired, no transmission is performed. This allows the quality of the main image data included in the synthesized image data 42 to be guaranteed. In addition, the frequency of transmission of unimportant data can be reduced, which also leads to a reduction in the load on the network. [Explanation of symbols]

[0079] 11. Communications equipment 12 processors 13. Memory 14 Camera 21 Data processing device 22 processors 23 Memory 24 Display 25 Input Devices 26 Communication Equipment 41 Imaging data 42 Synthetic Image Data 100 Image Transmission System 101 Queuing Department 102 Synthesis section 103 Encoding section 104 Communications Department 200 terminals 201 Communications Department 202 Display section VE vehicle

Claims

1. Multiple cameras and A communication device; Equipped with The communication device includes: A process of sequentially acquiring imaging data from each of the plurality of cameras; A process of setting a main camera as one of the plurality of cameras; a process of generating composite image data by combining the imaging data last acquired from each of the plurality of cameras when the imaging data is acquired from the main camera, and not generating the composite image data when the imaging data is not acquired from the main camera even if new imaging data is acquired from a camera other than the main camera among the plurality of cameras; A process of transmitting the composite image data to a predetermined terminal; configured to run Image transmission system.

2. 2. The image transmission system according to claim 1, The process of generating the composite image data includes: and synthesizing the image data of each of the plurality of cameras other than the base camera so that a part or all of the image data of the base camera of the plurality of cameras is superimposed on a partial area of ​​the image data of the base camera. Image transmission system.

3. 3. The image transmission system according to claim 2, The main camera is the base camera. Image transmission system.

4. 3. The image transmission system according to claim 1, The communication device includes: receiving an input for selecting one of the plurality of cameras; A process of setting the camera selected by the input as a main camera; is configured to further Image transmission system.

5. 3. The image transmission system according to claim 1, The communication device is mounted on a vehicle, The plurality of cameras are a plurality of vehicle-mounted cameras mounted on the vehicle. Image transmission system.

6. Sequentially acquiring imaging data from each of the plurality of cameras; Setting a main camera as one of the plurality of cameras; When the imaging data is acquired from the main camera, composite image data is generated by combining the imaging data last acquired from each of the plurality of cameras, and when the imaging data is not acquired from the main camera, the composite image data is not generated even if new imaging data is acquired from a camera other than the main camera among the plurality of cameras. transmitting the composite image data to a predetermined terminal; Includes Image transmission method.

7. A program which, when executed by a computer, causes the computer to execute the image transmission method according to claim 6.

8. The image transmission system according to claim 2, The main camera can be set to a camera different from the base camera. Image transmission system.

Images