Image transmitting device and image display system

Abstract

To restrain transmission delay of encoded image data.SOLUTION: An image transmitting device includes one or more encoding units 11a to 11d for receiving frame image data output by a plurality of imaging devices 200a to 200d at each predetermined frame time to perform encoding processing, an encoding control unit 12 for controlling the encoding processing, and a connection interface unit 15 for providing a connection interface 30 for connecting to an image receiving device. The one or more encoding units notify the encoding control unit of a code amount of the encoded image data. The encoding control unit determines and notifies a target code amount of the encoded image data to be output by the encoding processing to be performed from the next time onwards on the basis of the code amount notified from the one or more encoding units such that the data transfer speed at the connection interface is equal to or lower than a predetermined target data transfer speed. The one or more encoding units perform the encoding processing on the basis of the target code amount notified from the encoding control unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an image transmitting device and an image display system. [Background technology]

[0002] Conventionally, image display systems have been used in which images such as moving images and still images obtained by an imaging device are encoded and transmitted from an image transmitting device, the image receiving device decodes the received image data, and the image display device displays the decoded data based on the decoded data (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-127120 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the image display system as described above, there is a risk that delays will occur in data transmission between the image transmitting device and the image receiving device due to fluctuations in the amount of captured image data, resulting in problems with image display such as images not being displayed smoothly, etc. For this reason, a technology capable of suppressing transmission delays of encoded image data is desired. [Means for solving the problem]

[0005] As one embodiment of the present disclosure, there is provided an image transmitting device that transmits encoded image data to an image receiving device. The image transmitting device (10) includes one or more encoding units (11a to 11d) that input frame image data output by a plurality of imaging devices (200a to 200d) at predetermined frame times, execute encoding processing, and output the encoded image data, an encoding control unit (12) that controls the encoding processing by the one or more encoding units, and a connection interface unit (15) that connects to the image receiving device and provides a connection interface (30) for transmitting the encoded image data. The one or more encoding units notify the encoding control unit of the code amount of the encoded image data. The encoding control unit determines a target code amount of the encoded image data to be output by the encoding processing executed next time or later based on the code amount notified from the one or more encoding units so that the data transfer rate in the connection interface is equal to or lower than a predetermined target data transfer rate, and notifies the one or more encoding units of the target code amount. The one or more encoding units execute the encoding processing based on the target code amount notified from the encoding control unit.

[0006] According to this form of image transmission device, the encoding control unit determines a target code amount of the encoded image data to be output by the encoding process to be executed next time or later, based on the code amount notified by one or more encoding units, so that the data transfer speed at the connection interface is equal to or lower than a predetermined target data transfer speed, and notifies the target code amount to one or more encoding units.The one or more encoding units then execute the encoding process based on the target code amount notified by the encoding control unit, thereby suppressing large fluctuations in the data transfer speed at the connection interface and suppressing transmission delays of the encoded image data. [Brief description of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a schematic configuration of an image display system including an image transmission device according to an embodiment of the present disclosure. [Diagram 2]10A to 10C are explanatory diagrams showing an example of transition of the code amount output from each encoding unit as a result of the encoding control process. [Diagram 3] 13 is a flowchart showing a procedure for an encoding control process. [Figure 4] FIG. 11 is an explanatory diagram showing an example of a method for determining a target code amount. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] A. First embodiment: A1. Equipment configuration: 1 performs predetermined image processing on moving images obtained by four imaging devices 200a, 200b, 200c, and 200d, and displays the moving images (frame images) after image processing on two display devices 300a and 300b. The image display system 100 is mounted on, for example, a vehicle, and displays images of the vehicle's surroundings obtained by the four imaging devices 200a to 200d and an overhead image obtained based on the surrounding images on two display devices 300a and 300b provided on an instrument panel. The image display system 100 may be mounted on any type of moving body other than a vehicle, or may be used without being mounted on a moving body.

[0009] In this embodiment, the moving image obtained by the four imaging devices 200a to 200d is composed of a plurality of frame images obtained at a predetermined frame time. In the image display system 100, the moving image obtained by the four imaging devices 200a to 200d is compressed and encoded according to a predetermined moving image compression standard and transferred. In this embodiment, the above-mentioned "predetermined moving image compression standard" is MPEG (Moving Picture Experts Group)-2, which defines a plurality of pictures including an I-picture and a P-picture. An I-picture is an image that can be encoded and decoded based on image data output in a single frame time. In contrast, a P-picture is an image that can be encoded and decoded based on image data input in a plurality of frame times. Specifically, a P-picture is encoded and decoded by using a plurality of image data of a temporally previous frame image. In general, the data amount of a P-picture is about 20% of the data amount of an I-picture, and compression efficiency is high. An I-picture corresponds to the "first image" in this disclosure. Also, a P-picture corresponds to the "second image" in this disclosure. In this embodiment, the four imaging devices 200a to 200d perform imaging processes in synchronization with each other. Therefore, the frame timings of the four imaging devices 200a to 200d are substantially the same. Note that the above-mentioned "predetermined video compression standard" is not limited to MPEG-2, and may be any other standard such as H.264.

[0010] The image display system 100 includes an image transmitting device 10 and an image receiving device 20. The image transmitting device 10 and the image receiving device 20 are connected to each other via a serial interface 30. In this embodiment, the serial interface 30 is an LVDS (Low Voltage Differential Signaling) interface. Note that the connection interface that connects the image transmitting device 10 and the image receiving device 20 is not limited to an LVDS interface, and may be any other type of serial interface such as Ethernet (registered trademark).

[0011] The image transmitting device 10 encodes frame images output from the four imaging devices 200a to 200d, and transmits the encoded image data obtained to the image receiving device 20. In this embodiment, the image transmitting device 10 is configured as an SoC (System on Chip). Note that the image transmitting device 10 is not limited to an SoC, and may be configured as a computer device including a CPU, a ROM, a RAM, a storage device, an I / O unit, a housing, and the like. The image transmitting device 10 includes four encoding units 11a, 11b, 11c, and 11d, an encoding control unit 12, a multiplexing unit 13, a transmission buffer 14, and a serial interface unit 15.

[0012] Each of the four encoding units 11a to 11d is connected to one of the four imaging devices 200a to 200d in a one-to-one relationship. The four encoding units 11a to 11d are connected to the encoding control unit 12. The four encoding units 11a to 11d input frame image data output from the corresponding imaging devices 200a to 200d for each frame time and execute encoding processing. The four encoding units 11a to 11d transmit data obtained as a result of the encoding processing (hereinafter referred to as "encoded image data") to the encoding control unit 12. At this time, the four encoding units 11a to 11d transmit the encoded image data to the encoding control unit 12 and notify the encoding control unit 12 of the code amount and picture type of the encoded image data. The picture type is information indicating which image type of image data, I picture or P picture, the encoded image data to be transmitted is. The picture type corresponds to "image type information" in this disclosure.

[0013] The encoding control unit 12 controls the encoding processes performed by the four encoding units 11a to 11d. Specifically, the encoding control unit 12 controls the encoding processes by executing an encoding control process described below, so that the data transfer rate in the serial interface 30 is set to a target data transfer rate or less in advance.

[0014] The multiplexing unit 13 multiplexes and outputs the encoded image data output from each of the encoding units 11a to 11d. The transmission buffer 14 functions as a buffer that absorbs the difference between the generation speed of the multiplexed encoded image data and the output speed of the multiplexed encoded image data from the serial interface 30. The serial interface unit 15 is made up of a group of interfaces for transmitting and receiving data via the serial interface 30.

[0015] The image receiving device 20 receives the multiplexed encoded image data from the image transmitting device 10, and displays a video on the two display devices 300a and 300b based on the received data. In this embodiment, the image receiving device 20 is configured as an SoC, similar to the image transmitting device 10. Note that the image receiving device 20 may be configured as the above-mentioned computer device instead of an SoC.

[0016] The image receiving device 20 includes a serial interface unit 21, a receiving buffer 22, a demultiplexing unit 23, four decoding units 24a, 24b, 24c, and 24d, an image processing unit 25, and two display control units 26a and 26b.

[0017] The serial interface unit 21, like the above-mentioned serial interface unit 15, is made up of a group of interfaces for transmitting and receiving data via a serial interface 30. The receiving buffer 22 functions as a buffer that absorbs the difference between the receiving speed of the multiplexed encoded image data and the demultiplexing speed by the demultiplexing unit 23. The demultiplexing unit 23 demultiplexes the multiplexed encoded image data, and outputs it to four decoding units 24a to 24b.

[0018] Each of the four decoding units 24a to 24d performs a decoding process on the input encoded image data. The image processing unit 25 receives the decoded image data output from the four decoding units 24a to 24d and performs a predetermined image processing on the image data. In this embodiment, the "predetermined image processing" includes a process of integrating frame images obtained by the four imaging devices 200a to 200d to generate one image, that is, a so-called "stitching process." Note that the image processing unit 25 may perform any type of image processing, such as edge enhancement processing, brightness or contrast adjustment processing, in addition to the above-mentioned stitching process or instead of the stitching process.

[0019] The two display control units 26a, 26b are each connected to one of the two display devices 300a, 300b in a one-to-one relationship. The two display control units 26a, 26b cause the corresponding display devices 300a, 300b to display images based on image data after image processing. The two display devices 300a, 300b display images based on image data received from the corresponding display control units 26a, 26b.

[0020] A2. Encoding control process: Before describing the procedure of the encoding control process shown in FIG. 3, the amount of code output from each of the encoding units 11a to 11d as a result of execution of the encoding control process will be described with reference to FIG.

[0021] In FIG. 2, the horizontal axis indicates time, and the vertical axis indicates the code amount output from the encoding units 11a to 11d. In FIG. 2, the same hatching is applied to the code amounts output from the same encoding units among the four encoding units 11a to 11d. At time t1, the encoding unit 11a generates image data of an I picture as encoded image data, and the data amount (code amount) is the code amount I1. At time t2, the encoding unit 11b generates image data of a P picture, and the code amount is the code amount P2. At time t3, the encoding unit 11c generates image data of a P picture, and the code amount is the code amount P3. At time t4, the encoding unit 11d generates image data of a P picture, and the code amount is the code amount P4. The code amounts P2 to P4 output from the three encoding units 11b to 11d are different from each other. This is because the imaging contents are different from each other. The code amount I1 is greater than the three code amounts P2 to P4.

[0022] At time t5, a predetermined time after time t1, the encoding unit 11a generates image data of a P picture, and the amount of codes is the amount of codes P5. At time t6, a predetermined frame time after time t2, the encoding unit 11b generates image data of an I picture, and the amount of codes is the amount of codes I6. At time t7, a predetermined frame time after time t3, the encoding unit 11c generates image data of a P picture, as in the previous time, and the amount of codes is the amount of codes P7. At time t8, a predetermined frame time after time t4, the encoding unit 11d generates image data of a P picture, as in the previous time, and the amount of codes is the amount of codes P8.

[0023] At time t9, a predetermined frame time after time t5, the encoding unit 11a generates image data of a P picture in the same manner as the previous time, and the amount of codes is the amount of codes P9. At time t10, a predetermined frame time after time t6, the encoding unit 11b generates image data of a P picture in this case, and the amount of codes is the amount of codes P10. At time t11, a predetermined frame time after time t7, the encoding unit 11c generates image data of an I picture in this case, and the amount of codes is the amount of codes I11. At time t12, a predetermined frame time after time t8, the encoding unit 11d generates image data of a P picture in the same manner as the previous time, and the amount of codes is the amount of codes P12.

[0024] At time t13, a predetermined frame time after time t9, the encoding unit 11a generates image data of a P picture, as in the previous time, and the code amount is the code amount P13. At time t14, a predetermined frame time after time t10, the encoding unit 11b generates image data of a P picture, as in the previous time, and the code amount is the code amount P14. At time t15, a predetermined frame time after time t11, the encoding unit 11c generates image data of a P picture, and the code amount is the code amount P15. At time t16, a predetermined frame time after time t12, the encoding unit 11d generates image data of an I picture, and the code amount is the code amount I16.

[0025] As described above, in this embodiment, one of the four encoders 11a to 11d is controlled to generate image data of an I-picture, and the other three encoders are controlled to generate image data of a P-picture, in each frame time. At this time, the encoder that generates the image data of the I-picture is selected in order from among the four encoders 11a to 11d. Also, the total amounts of codes generated in each frame time are controlled to be approximately equal to each other.

[0026] The encoding control process shown in Fig. 3 is executed when the power of the image transmitting device 10 is turned on. As shown in Fig. 3, in step S105, each of the encoding units 11a to 11d determines whether or not the next frame time has arrived. Hereinafter, the word "step" will be omitted. If the next frame time has not arrived (S105: NO), S105 is executed again. That is, each of the encoding units 11a to 11d waits until the next frame time arrives.

[0027] When it is determined that the next frame time has arrived (S105: YES), each of the encoding units 11a to 11d performs encoding processing on the image data input from the imaging devices 200a to 200d based on the target code amount and image type notified from the encoding control unit 12 (S110). For example, in the example of FIG. 2, at time t1, the encoding unit 11a performs encoding processing so as to output image data of an I-picture as encoded image data. At this time, each of the encoding units 11a to 11d adjusts the quality of each block included in the image so that the code amount to be output approaches the target code amount. That is, each of the encoding units 11a to 11d can increase the data amount by setting the quality of the block to high quality, and on the other hand, can reduce the data amount by setting the quality of the block to low quality, and can further adjust the code amount by adjusting the ratio between the number of high quality blocks and the number of low quality blocks. The quality of the block can be controlled by a well-known method of adjusting a quantization parameter (QP), for example.

[0028] In S115, each of the encoding units 11a to 11d transmits encoded image data, code amount, and picture type to the encoding control unit 12. For example, in the example of Fig. 2, the encoding unit 11a transmits encoded image data of an I-picture generated at time t1, the code amount I1, and data indicating "I-picture" to the encoding control unit 12.

[0029] In S120, the encoding control unit 12 determines the type of image to be output by each of the encoding units 11a to 11d in the next encoding process, i.e., I-picture or P-picture. As described above, only one of the four encoding units 11a to 11d outputs encoded image data of an I-picture in a certain frame time, and the one encoding unit is determined in order among the four encoding units 11a to 11d. That is, the encoding control unit 12 determines, in a predetermined order, the encoding unit that outputs encoded image data of an I-picture in the next encoding process, and determines the other three encoding units to output a P-picture in the next encoding process.

[0030] In S125, the encoding control unit 12 determines the target code amount for each of the encoding units 11a to 11d in the next encoding process to be executed. A specific method for determining the target code amount will be described with reference to FIG.

[0031] In Fig. 4, the horizontal axis indicates time, and the vertical axis indicates the integrated value of the output code amount (total code amount). In Fig. 4, a straight line L1 indicates the transition of the integrated value of the amount of data that can be transferred in the serial interface 30 (hereinafter, also referred to as "integrated amount of transferable data"). Note that Fig. 4 corresponds to Fig. 2. At time t1, a code amount I1 is generated, and at time t2, the code amount I1 and the code amount P2 generated at time t2 are added together to obtain the integrated code amount.

[0032] When the code amount I1 actually output by the encoding unit 11a is specified at time t1, the code amount I1 is transmitted to the encoding control unit 12 at the above-mentioned S115. At this time, the encoding control unit 12 determines the target code amount for the encoding process to be executed next time (at time t5) by the encoding unit 11a at S125. Specifically, the encoding control unit 12 first adds up the code amount I1 (actual value) and the target code amount Pa previously notified to the three encoding units 11b, 11c, and 11d to obtain a provisional integrated value Ds1. Note that this integrated value Ds1 is different from the actual integrated value Ds0 at time t4. This is because the target code amount is merely a target value and may differ from the code amount obtained when the encoding is actually performed. Next, the encoding control unit 12 specifies the transferable data integrated amount Ds2 at time t9, when the encoding process after the next one is scheduled to be executed by the encoding unit 11a, based on the time until time t9 and the line L1. Next, the encoding control unit 12 subtracts the accumulated value Ds1 from the transferable data accumulated amount Ds2 to calculate a differential data amount ΔDs, which is the amount of data that can be transferred from time t5 to time t9. This differential data amount ΔDs is approximately equal to the amount of data that can be transmitted within the next frame time. Next, the encoding control unit 12 prorates the differential data amount ΔDs at a predetermined ratio to obtain a target code amount for the encoding unit that is scheduled to generate the encoded image data of the I picture and a target code amount for the encoding unit that is scheduled to generate the encoded image data of the P picture. Specifically, the ratio of the code amounts of the I picture and the P picture is specified as a predetermined ratio (for example, 1:5). Then, the differential data amount ΔDs is apportioned using the ratio (1:3) between the number of encoding units scheduled to generate the encoded image data of the I picture and the number of encoding units scheduled to generate the encoded image data of the P picture to obtain the target code amount Ib of the encoding units scheduled to generate the encoded image data of the I picture and the target code amount Pb of the encoding units scheduled to generate the encoded image data of the P picture. Here, for the encoding unit 11a that generated the encoded image data of the I picture at time t1, "P picture" is determined as the image type of the encoded image data to be generated at time t5. Therefore, the target code amount Pb is determined as 1 / 8 of the differential data amount ΔDs as the target code amount of the encoding unit 11a at time t5.Similarly, the target code amount of the encoding section 11b is determined at time t2, the target code amount of the encoding section 11c is determined at time t3, and the target code amount of the encoding section 11d is determined at time t4.

[0033] As shown in FIG. 3, after the completion of the above-mentioned S125, the encoding control unit 12 notifies each of the encoding units 11a to 11d of the image type determined in S120 and the target code amount determined in S125 (S130). At this time, S130 is executed sequentially for the encoding units whose target code amounts have been determined in S125. That is, in the example of FIG. 4, when the target code amount Pb for time t5 is specified at time t1, the encoding unit 11a is notified of the P picture as the image type and the target code amount Pb. After the completion of S130, the process returns to S105. Therefore, when S110 is executed next time, the encoding process is executed based on the image type and the target code amount notified in the current S130.

[0034] As a result of the above-mentioned encoding control process, as shown in Fig. 2, the encoding process is performed in each of the encoding units 11a to 11d, and as shown in Fig. 4, the integrated value of the code amount output from each of the encoding units 11a to 11d is substantially within the line L1 at any timing. Therefore, it is possible to suppress the occurrence of excessive delay in the transmission of encoded image data in the serial interface 30. Therefore, it is possible to suppress the occurrence of problems such as images not being displayed smoothly on the display devices 300a and 300b due to such excessive transmission delay. Furthermore, in a configuration in which the image data after image processing in the image receiving device 20 is used for vehicle control such as braking, acceleration, and steering in addition to display on the imaging devices 200a and 200b, it is possible to suppress the occurrence of delay in vehicle control due to transmission delay of encoded image data.

[0035] According to the image transmitting device 10 of the first embodiment described above, the encoding control unit 12 determines the target code amount of the encoded image data to be output by the encoding process to be executed next time or later, based on the code amount notified by each of the encoding units 11a to 11d, so that the data transfer speed in the serial interface 30 is equal to or lower than a predetermined target data transfer speed (straight line L1), and notifies each of the encoding units 11a to 11d of the target code amount. Each of the encoding units 11a to 11d executes the encoding process based on the target code amount notified by the encoding control unit 12. Therefore, it is possible to suppress large fluctuations in the data transfer speed in the serial interface 30, and to suppress transmission delays of the encoded image data.

[0036] Furthermore, the encoding control unit 12 determines the image type of the encoded image data to be output by the encoding process to be executed next or later based on image type information (picture type) notified from each of the encoding units 11a to 11d, and notifies each of the encoding units 11a to 11d of the determined image type. Each of the encoding units 11a to 11d then executes the encoding process based on the target code amount and image type notified from the encoding control unit 12. Therefore, even in a configuration in which images are encoded and transmitted using an encoding method that mixes I-pictures (first images) and P-pictures (second images), which have different data amounts, large fluctuations in the data transfer rate in the serial interface 30 can be suppressed.

[0037] Furthermore, the image type is determined so that one of the four encoding units 11a to 11d, except for the encoding unit that notified that the image type is an I-picture, will generate an I-picture in the encoding process to be executed next or later. This makes it possible to prevent a particular encoding unit from generating I-pictures preferentially out of I-pictures and P-pictures, and to prevent image quality bias from occurring in images decoded between the encoding units 11a to 11d.

[0038] Furthermore, when the encoding control unit 12 is notified of the code amount from one of the four encoding units 11a to 11d, it adds the notified code amount for that encoding unit and the previously notified target code amount for the other encoding units to obtain an accumulated value (total code amount) Ds1 and an accumulated amount (straight line L1) of transmittable data that can be specified based on the data transfer speed in the serial interface 30.Therefore, the target code amount for the encoding unit that notified the code amount can be determined in a short time compared to a configuration in which the next target code amount for each of the encoding units 11a to 11d is determined based on the code amounts notified from all of the encoding units 11a to 11d.

[0039] In addition, the encoding control unit 12 apportions the amount of data (differential data amount ΔDs) that can be transmitted at the data transfer speed in the serial interface 30 during the period from the time when data transmission of the code amount is completed (time t5) to the start time of the next frame time (time t9) based on the ratio of the data amount between I pictures and P pictures and the number of encoding units 11a to 11d, determines the target code amount for each of the encoding units 11a to 11d, and notifies the encoding units 11a to 11d of this.Therefore, the target code amount for each of the encoding units 11a to 11d can be accurately determined, which can suppress large fluctuations in the data transfer speed in the serial interface 30.

[0040] Furthermore, according to the image display system 100 of the above embodiment, since it includes the image transmitting device 10 and the image receiving device 20, the frame images output by the four imaging devices 200a to 200d can be displayed on the display devices 300a and 300b after image processing. Also, since the image receiving device 20 has the image processing unit 25, when it is desired to change or update (upgrade) the contents of the image processing, it is not necessary to change the configuration of the image transmitting device 10.

[0041] In addition, the image processing performed by the image processing unit 25 includes a stitching process that integrates multiple frame image data output by the four imaging devices 200a to 200d to generate a single image, so that a single image can be generated based on the frame images output by the four imaging devices 200a to 200d.

[0042] B. Other embodiments: (B1) In the above embodiment, the encoding process executed in the encoding units 11a to 11d is a processing method (video compression standard) for generating encoded image data of a first image and a second image having different data amounts, but the present disclosure is not limited to this. A processing method for generating encoded image data of only a single type of image may also be used. In such a configuration, the transmission and reception of image type information (picture type) between each of the encoding units 11a to 11d and the encoding control unit 12 may be omitted.

[0043] (B2) In the above embodiment, one of the four encoding units 11a to 11d is controlled to generate image data of an I picture, and the other three encoding units are controlled to generate image data of a P picture in each frame time, but the present disclosure is not limited to this. When the actual code amount transmitted from each encoding unit 11a to 11d is very small compared to the target code amount, the picture type to be output may be determined to be "I picture" for two or more encoding units among the four encoding units 11a to 11d in the encoding process in the next frame time. Conversely, the picture type to be output may be determined to be "P picture" for all encoding units 11a to 11d.

[0044] (B3) In the above embodiment, the timing when S125 is executed is the timing when each of the encoding units 11a to 11d executes the encoding process, identifies the actual code amount, and transmits it to the encoding control unit 12, but the present disclosure is not limited to this. For example, S125 may be executed for each of the four encoding units 11a to 11d by waiting until time t4 shown in FIG. 4, and the target code amount may be determined. In such a configuration, the next target code amount may be determined using the code amounts actually generated in the four encoding units 11a to 11d. With such a configuration, it is possible to more accurately determine the target code amount for each of the encoding units 11a to 11d, which allows the total code amount in the four encoding units 11a to 11d to fall below the line L1.

[0045] (B4) In S125, after identifying the accumulated value Ds1 at time t5, the accumulated amount of data that can be transferred through the serial interface 30 at time t5 (line L1) may be identified, and any value below line L1 may be determined as the next target code amount for the encoding unit 11a.

[0046] (B5) In the above embodiment, the four imaging devices 200a to 200d perform imaging processing in synchronization with each other, but the present disclosure is not limited to this. The imaging devices 200a to 200d may perform imaging processing asynchronously with each other and output frame images to the encoding units 11a to 11d. In such a configuration, the imaging devices 200a to 200d do not perform imaging in synchronization, and as a result, the encoding units 11a to 11d do not perform encoding in synchronization with each other, so that it may take a long time for all the encoding units 11a to 11d to notify the encoding control unit 12 of the actual code amount. However, according to the image transmission device 10 of the above embodiment, when the code amount is notified from a certain encoding unit, the next target code amount for that encoding unit can be determined early, and the target code amount can be determined before the scheduled time of the next encoding in that encoding unit.

[0047] (B6) The image display system 100 and the image transmission device 10 of the above embodiment are merely examples and can be modified in various ways. For example, the present disclosure is not limited to the image display system 100, but can be realized as the image transmission device 10 alone. Also, for example, the number of the encoding units 11a to 11d is not limited to four, and may be any number of one or two or more. Also, a parallel interface may be used instead of the serial interface 30. In a configuration using a parallel interface in which a plurality of channels are defined, the multiplexing unit 13 and the demultiplexing unit 23 can be omitted by previously associating each channel with each of the encoding units 11a to 11d. Also, for example, when determining the target code amount Pb shown in FIG. 4, the differential data amount ΔDs, which is the difference between the transferable data accumulated amount Ds2 and the accumulated value Ds1, is apportioned, but a value obtained by subtracting a predetermined margin from the differential data amount ΔDs may be apportioned. According to this configuration, it is possible to prevent the accumulated value of the actual code amount from exceeding the transferable data accumulated amount (line L1) with a higher probability. Also, for example, the encoding units 11a to 11d may encode still images in addition to moving images. For example, JPEG (Joint Photographic Experts Group) image data may be generated as encoded image data by the encoding process.

[0048] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features in each embodiment corresponding to the technical features in the form described in the Summary of the Invention column can be appropriately replaced or combined in order to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if the technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0049] 11a to 11d... encoding units, 12... encoding control unit, 15... serial interface unit (connection interface unit), 30... serial interface unit (connection interface), 200a to 200d... imaging device

Claims

1. An image transmitting device that transmits encoded image data to an image receiving device, comprising: one or more encoding units (11a to 11d) that input frame image data output by a plurality of imaging devices (200a to 200d) at predetermined frame times, execute encoding processing, and output the encoded image data; an encoding control unit (12) that controls the encoding process by the one or more encoding units; a connection interface unit (15) for providing a connection interface (30) for connecting to the image receiving device and transmitting the encoded image data; Equipped with the one or more encoding units notify the encoding control unit of a code amount of the encoded image data; the encoding control unit determines a target code amount of the encoded image data to be output by the encoding process to be executed next or later based on the code amount notified from the one or more encoding units so that the data transfer rate in the connection interface is equal to or lower than a predetermined target data transfer rate, and notifies the one or more encoding units of the target code amount; the one or more encoding units execute the encoding process based on the target code amount notified by the encoding control unit. Image transmission device.

2. 2. The image transmitting device according to claim 1, The one or more encoding units, by the encoding process, the encoded image data representing a first image decodable based on the encoded image data output during a single frame time; a second image that can be decoded based on the encoded image data inputted for a plurality of the frame times, the second image having a smaller amount of data than the first image; the one or more encoding units notify the encoding control unit of image type information indicating which image type of the encoded image data to be transmitted is the image data of the first image or the second image; the encoding control unit determines the image type of the encoded image data to be output by the encoding process to be executed next or later based on the image type information notified from the one or more encoding units so that the data transfer rate is equal to or lower than the target data transfer rate, and notifies the one or more encoding units of the determined image type; The one or more encoding units execute the encoding process based on the target code amount and the image type notified by the encoding control unit.

3. 3. The image transmitting device according to claim 2, a plurality of the encoding units provided corresponding to the plurality of image capture devices; An image transmitting device that determines the image type so that one of the multiple encoding units, other than the encoding unit that has notified the first image as the image type, generates the first image in the encoding process that is executed next time or later.

4. 3. The image transmitting device according to claim 2, a plurality of the encoding units provided corresponding to the plurality of image capture devices; An image transmitting device in which, when the encoding control unit is notified of the code amount from one of the multiple encoding units, it adds the notified code amount for that encoding unit and adds the previously notified target code amount for the other encoding units to obtain a total code amount, and determines the next target code amount for each of the encoding units based on the accumulated amount of transmittable data that can be specified based on the data transfer speed in the connection interface.

5. 5. The image transmitting device according to claim 4, an encoding control unit that prorates the amount of data that can be transmitted at the data transfer rate during the period from the completion of data transmission of the total amount of code to the start of the next frame time based on the ratio of the data amounts of the first image and the second image and the number of the encoding units, determines the target amount of code for each of the encoding units, and notifies the encoding unit of the target amount of code.

6. 1. An image display system, comprising: An image transmitting device according to any one of claims 1 to 5, The image receiving device, A receiving side interface unit (21) that provides the connection interface; a decoding unit (24a to 24d) for decoding the encoded image data received via the receiving side interface unit to obtain decoded image data; an image processing unit (25) that executes a predetermined image processing on the decoded image data; a display control unit (26a, 26b) for causing a display device (300a, 300b) to display an image based on the decoded image data after the image processing has been performed; The image receiving device having An image display system (100) comprising:

7. 7. The image display system according to claim 6, An image display system, wherein the image processing includes a stitching process that integrates the frame image data output by the multiple imaging devices to generate a single image.

Images