Image output device, image acquisition device, image output method, and image acquisition method

Abstract

This image output device (10) uses USB to output an image to which an endpoint is assigned, and comprises: an acquisition unit (for example, an image processing unit (11)) that acquires respective images from a plurality of imaging devices (30-32) via an interface different from USB; and a control unit (for example, a USB controller (15)) that executes control for outputting the respective images from the plurality of imaging devices (30-32) to a plurality of endpoints (EP0, EP2). The control unit outputs, to one endpoint (EP0) among the plurality of endpoints (EP0, EP2), a joined image (for example, an image (v5)) obtained by joining two or more images from two or more imaging devices among the plurality of imaging devices (30-32).

Description

Image output apparatus, image acquisition apparatus, image output method, and image acquisition method

[0001] The present disclosure relates to an image output apparatus, an image acquisition apparatus, an image output method, and an image acquisition method.

[0002] There has been known an apparatus that transfers an image captured by an imaging apparatus such as a digital camera to a predetermined apparatus. For example, Patent Document 1 discloses an imaging apparatus that communicates with an information processing apparatus via a general-purpose interface such as USB (Universal Serial Bus).

[0003] Japanese Unexamined Patent Publication No. 2024-72123

[0004] Incidentally, when transferring an image, it is desired to improve transfer efficiency.

[0005] Therefore, the present disclosure provides an image output apparatus, an image acquisition apparatus, an image output method, and an image acquisition method that can improve transfer efficiency when transferring an image.

[0006] An image output apparatus according to an aspect of the present disclosure is an image output apparatus that outputs an image to which an endpoint is assigned by using USB (Universal Serial Bus), the image output apparatus including: an acquisition unit that acquires an image from each of a plurality of imaging apparatuses via an interface different from the USB; and a control unit that executes control to output the image from each of the plurality of imaging apparatuses to a plurality of endpoints, wherein the control unit outputs a combined image obtained by combining two or more images from two or more imaging apparatuses among the plurality of imaging apparatuses to one endpoint among the plurality of endpoints.

[0007] An image acquisition apparatus according to an aspect of the present disclosure is an image acquisition apparatus that acquires an image to which an endpoint is assigned by using USB, the image acquisition apparatus including: an acquisition unit that acquires a combined image obtained by combining two or more images from at least two or more imaging apparatuses among a plurality of imaging apparatuses via one endpoint; and a processing unit that divides the combined image acquired via the one endpoint into the two or more images before combination.

[0008] An image output method according to one aspect of the present disclosure is an image output method executed by an image output device that outputs an image assigned to an endpoint using USB, wherein the device acquires an image from each of a plurality of imaging devices via an interface different from USB, and executes control to output the images from each of the plurality of imaging devices to a plurality of endpoints, and in the execution of the control, outputs a combined image obtained by combining two or more images from two or more of the plurality of imaging devices to one of the plurality of endpoints.

[0009] An image acquisition method according to one aspect of the present disclosure is an image acquisition method performed by an image acquisition device that acquires images assigned to an endpoint using USB, wherein a combined image obtained by combining two or more images from at least two or more imaging devices among a plurality of imaging devices is acquired via one endpoint, and the combined image acquired via one endpoint is divided into the two or more images before combining.

[0010] According to one aspect of this disclosure, it is possible to realize an image output device, etc., that can improve the transfer efficiency when transferring images.

[0011] Figure 1 is a diagram showing the configuration of an image transfer system according to an embodiment. Figure 2 is a block diagram showing the functional configuration of an image output device according to an embodiment. Figure 3 is a block diagram showing the functional configuration of an image acquisition device according to an embodiment. Figure 4 is a flowchart showing the operation of an image transfer system according to an embodiment. Figure 5A is a diagram schematically showing a first example of image merging according to an embodiment. Figure 5B is a diagram schematically showing a second example of image merging according to an embodiment. Figure 5C is a diagram schematically showing a third example of image merging according to an embodiment. Figure 5D is a diagram schematically showing a fourth example of image merging according to an embodiment. Figure 5E is a diagram schematically showing a fifth example of image merging according to an embodiment. Figure 5F is a diagram schematically showing a sixth example of image merging according to an embodiment. Figure 6 is a diagram showing transfer setting information stored in the built-in memory according to an embodiment. Figure 7A is a diagram for explaining the data reading order according to a conventional example. Figure 7B is a diagram for explaining an example of the data reading order according to an embodiment. Figure 7C is a diagram for explaining another example of the data reading order according to an embodiment. Figure 8 is a diagram for explaining an example of image output according to an embodiment. Figure 9 is a diagram illustrating another example of image output according to the embodiment. Figure 10A is a block diagram showing the functional configuration of a conventional image output device. Figure 10B is a diagram illustrating image output according to the conventional example. Figure 10C is a diagram showing transfer setting information stored in the built-in memory according to the conventional example.

[0012] (Background to this Disclosure) Before describing the embodiments of this disclosure, the background to this disclosure will be explained with reference to Figures 10A to 10C. Figure 10A is a block diagram showing the functional configuration of an image output device 100 according to a conventional example. Figure 10B is a diagram for explaining the image output according to a conventional example. Figure 10C is a diagram showing the transfer setting information stored in the built-in memory 105a ("memory" shown in Figure 10A) according to a conventional example.

[0013] The image output device 100 shown in Figure 10A is an image output device that outputs images using UVC (USB Video Class), which is standardized by USB (Universal Serial Bus). Figure 10A shows an example in which the image output device 100 outputs images v0 to v2 acquired from three imaging devices 30 to 32 to three endpoints EP0 to EP2.

[0014] UVC is a standard specification for USB connection between imaging devices such as cameras and devices that handle images (later-stage devices in this specification). An endpoint is a communication endpoint in USB communication. Although the USB cable connecting the image output device 100 and the image acquisition device (see Figure 1, etc.) is physically a common cable, the image acquisition device has multiple endpoints, so the image output device 100 recognizes each of these multiple endpoints as an independent communication path (virtual communication path). In this way, independent communication (e.g., data transfer) can be performed for each endpoint. Figure 10A shows an example in which the image acquisition device (or the device on which the image acquisition device is installed) has at least three endpoints. Note that each endpoint may be configured to include a buffer.

[0015] The image output device 100 comprises an image processing unit 101, an output control unit 102, a transmission control unit 103, a storage unit 104, and a USB controller 105.

[0016] The image processing unit 101 performs predetermined image processing on the image. The output control unit 102 controls which image to output from among multiple images. The transmission control unit 103 stores the image from the output control unit 102 in the storage unit 104.

[0017] The storage unit 104 is a storage device that stores images from the transmission control unit 103. The USB controller 105 reads the images stored in the storage unit 104 and outputs the read images to the endpoint corresponding to the image.

[0018] In conventional systems, the output control unit 102 and the transmission control unit 103 store images v0 from the imaging device 30, image v1 from the imaging device 31, and image v2 from the imaging device 32 as separate images in the storage unit 104. The USB controller 105 reads images v0, v1, and v2 from the storage unit 104 and outputs them to different endpoints based on the transfer setting information stored in the built-in memory 105a. In other words, the USB controller 105 transfers images v0, v1, and v2 via different endpoints. Images v0, v1, and v2 are the transfer data transferred by the image output device 100.

[0019] Specifically, image v0 from imaging device 30 is output to endpoint EP0, image v1 from imaging device 31 is output to endpoint EP1, and image v2 from imaging device 32 is output to endpoint EP2. In this way, conventionally, there is a one-to-one correspondence between imaging devices and endpoints, meaning that each imaging device is assigned a different endpoint.

[0020] As described above, conventionally, when transmitting (transferring) images from multiple imaging devices 30 to 32 via USB using UVC, it is necessary to assign each image to a different endpoint in advance before transmission. This is because, according to the USB standard, only one type of image can be transmitted from a single endpoint.

[0021] In this case, as shown in Figure 10B, when the USB controller 105 switches the endpoint of the data being transferred, a period occurs during which no data is being transferred (the data transfer period shown in Figure 10B). In other words, transfer loss (transmission loss) occurs when switching endpoints. Furthermore, as the number of endpoints used increases, the number of endpoint switches increases, resulting in even greater transfer loss. Thus, conventional methods have the problem of inefficient data transfer and a decrease in data transfer speed.

[0022] Therefore, the inventors of this application diligently studied image output devices and the like that can improve the transfer efficiency when transferring images, and have devised the image output devices and the like shown below.

[0023] Furthermore, because the USB standard sets an upper limit on the number of endpoints (for example, 16 in each direction), there is a limit to the number of imaging devices 30 to 32 that can be connected to the image output device 100.

[0024] Furthermore, the USB controller 105 outputs images to the endpoints using transfer setting information, including image transfer settings, stored in the internal memory 105a. Transfer setting information is configured for each endpoint. In the example in Figure 10A, three endpoints are used, so as shown in Figure 10C, three sets of transfer setting information are stored in the internal memory 105a. Specifically, the internal memory 105a stores transfer setting information for endpoints 0 to 2 (endpoints EP0 to EP2). Thus, it is necessary to write transfer setting information to the internal memory 105a for each endpoint, which increases the amount of memory used.

[0025] The inventors of this application are also diligently considering ways to further improve these issues.

[0026] The embodiments and other details will be described below with reference to the drawings.

[0027] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, components, arrangement positions of components, connection configurations, steps, and step order shown in the following embodiments are examples only and are not intended to limit this disclosure. Furthermore, any components in the following embodiments that are not described in an independent claim will be described as optional components.

[0028] Furthermore, each figure is a schematic diagram and not necessarily a strictly accurate representation. Therefore, for example, the scale may not necessarily match in each figure. Also, in each figure, substantially identical components are given the same reference numerals, and redundant explanations are omitted or simplified.

[0029] Furthermore, in this specification, terms indicating relationships between elements such as "same," as well as numerical values ​​and numerical ranges, are not expressions that represent only strict meanings, but also expressions that include substantially equivalent ranges, for example, differences of a few percent (or about 10%).

[0030] (Embodiment) The image transfer system according to this embodiment will be described below with reference to Figures 1 to 9.

[0031] [1. Configuration of the Image Transfer System] First, the configuration of the image transfer system according to this embodiment will be described with reference to Figures 1 to 3. Figure 1 is a diagram showing the configuration of the image transfer system 1 according to this embodiment.

[0032] As shown in Figure 1, the image transfer system 1 is a system that outputs images assigned to endpoints using USB. In this embodiment, it is a system that transfers images captured by multiple imaging devices 30 to the image acquisition device 20 using UVC, which is standardized by USB. However, it is not limited to using UVC.

[0033] The multiple imaging devices 30 are webcams, video cameras, digital cameras, etc., and are connected to the image output device 10 in a communicative manner via an interface other than USB. The interface other than USB may use a camera-specific standard or a general-purpose standard other than USB.

[0034] The image output device 10 is an information processing device that outputs images assigned to endpoints using USB. In this embodiment, it transfers images captured by the imaging device 30 to the image acquisition device 20 using UVC, which is standardized by USB.

[0035] The image acquisition device 20 is an information processing device that acquires images assigned to endpoints using USB. In this embodiment, it acquires images output using the USB-standardized UVC, performs predetermined processing on the images, and outputs the processed images to a subsequent device. The subsequent device is not particularly limited, but may be a display device, a storage device, a PC (personal computer), a mobile terminal such as a smartphone, or another device that processes the transferred images.

[0036] The image output device 10 and the image acquisition device 20 are connected, for example, by a USB cable.

[0037] Figure 2 is a block diagram showing the functional configuration of the image output device 10 according to this embodiment. In Figure 2, for identification purposes, the three imaging devices are denoted by the reference numerals 30, 31, and 32. The multiple imaging devices 30 to 32 are also simply referred to as imaging device 30, etc. Note that Figure 2 shows an exemplary functional configuration of the image output device 10, and the functional configuration of the image output device 10 is not limited to Figure 2.

[0038] As shown in Figure 2, the image output device 10 comprises an image processing unit 11, an output control unit 12, a transmission control unit 13, a storage unit 14, and a USB controller 15. The functions of the image processing unit 11, the output control unit 12, the transmission control unit 13, and the USB controller 15 are realized, for example, by a processor such as a CPU in the image output device 10 executing a program stored in the memory of the image output device 10. At least a part of the image processing unit 11, the output control unit 12, the transmission control unit 13, and the USB controller 15 may be configured as circuits.

[0039] The image processing unit 11 performs predetermined image processing on a plurality of images acquired from the imaging device 30 or the like, and outputs the plurality of image-processed images to the output control unit 12. Examples of the predetermined image processing include general image processing such as AWB (auto white balance) processing, noise removal processing, and edge enhancement processing, but the predetermined image processing is not limited thereto. The image processing unit 11 also functions as an acquisition unit that acquires images from each of the imaging device 30 and the like via an interface different from USB.

[0040] Note that the image in the present specification may be a still image, or may be a moving image (that is, a video).

[0041] The output control unit 12 performs processing such as selection of an image to be output to the transmission control unit 13 from among the plurality of images output from the image processing unit 11, and when combining two or more images, selection of images to be combined and determination of a combining method.

[0042] The output control unit 12 combines two or more images to be combined, and outputs the combined image to the transmission control unit 13. The output control unit 12 also stores, in the storage unit 14, information indicating a combining method by which two or more images are combined. The information indicating the combining method includes information indicating how the two or more images are combined. The information indicating the combining method may be stored directly in the storage unit 14 by the output control unit 12, or may be stored in the storage unit 14 via the transmission control unit 13.

[0043] The output control unit 12 determines whether the number of connected imaging devices is greater than the number of endpoints to be used, and when the number of imaging devices is greater than the number of endpoints, combines two or more images such that the number of types of images output to the image acquisition device 20 is equal to or less than the number of endpoints. The endpoint herein is an endpoint used in a USB interface.

[0044] The transmission control unit 13 stores the image acquired from the output control unit 12 in the storage unit 14.

[0045] The storage unit 14 is a storage device that stores an image acquired from the transmission control unit 13 and header information attached to the image. The storage unit 14 is implemented by, for example, a non-volatile storage device such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive), but is not limited thereto.

[0046] The USB controller 15 performs control to output images from each of the imaging devices 30 and the like to a plurality of endpoints. The USB controller 15 performs control such that it treats a combined image obtained by combining two or more images from two or more imaging devices among the imaging devices 30 and the like as one image, and outputs the combined image to one endpoint. The USB controller 15 is an example of a control unit.

[0047] For example, the USB controller 15 reads an image and header information corresponding to the image from the storage unit 14, and outputs the image and the header information to an endpoint corresponding to the image. The USB controller 15 specifies an endpoint corresponding to the image and outputs data.

[0048] The USB controller 15 includes a built-in memory 15a (the "memory" shown in FIG. 2). The built-in memory 15a is a built-in memory included in the USB controller 15, and stores transfer setting information for each endpoint.

[0049] FIG. 3 is a block diagram showing the functional configuration of the image acquisition device 20 according to the present embodiment. Note that FIG. 3 shows an exemplary functional configuration of the image acquisition device 20, and the functional configuration of the image acquisition device 20 is not limited to that shown in FIG. 3.

[0050] As shown in FIG. 3, the image acquisition device 20 includes a USB controller 21, a storage unit 22, and a processing unit 23. The functions of the USB controller 21 and the processing unit 23 are implemented, for example, by a processor such as a CPU included in the image acquisition device 20 executing a program stored in a memory included in the image acquisition device 20. Note that at least part of the USB controller 21 and the processing unit 23 may be configured by a circuit.

[0051] The USB controller 21 controls the storage of images acquired from the image output device 10 in the storage unit 22. The USB controller 21 acquires the combined image and the uncombined image via different endpoints. In the example in Figure 3, the USB controller 21 acquires the combined image v5 via endpoint EP0 and the uncombined image v2 via endpoint EP2. The USB controller 21 functions as an acquisition unit.

[0052] The storage unit 22 is a storage device that stores images acquired from the image output device 10 and header information attached to the images. The storage unit 22 is implemented by, for example, a non-volatile storage device or an HDD, but is not limited to these.

[0053] The processing unit 23 performs predetermined processing on the image acquired by the USB controller 21. For example, the processing unit 23 performs processing to split the combined image into two or more pre-combined images based on header information attached to the combined image acquired via one endpoint. The processing unit 23 outputs the two or more pre-combined images obtained by splitting the combined image, and the uncombined image, to a predetermined device. There may be one predetermined device or multiple predetermined devices.

[0054] Here, the number of images output by the processing unit 23 (in this case, two or more images before merging and an unmerged image) may be greater than the number of endpoints between the image output device 10 and the image acquisition device 20.

[0055] [2. Operation of the Image Transfer System] Next, the operation of the image transfer system 1 configured as described above will be explained with reference to Figures 4 to 9. Figure 4 is a flowchart showing the operation (image output method, image acquisition method) of the image transfer system 1 according to this embodiment. In the following, the case in which three imaging devices are connected will be explained mainly using Figures 2 and 3, etc.

[0056] As shown in Figure 4, the image processing unit 11 acquires multiple images (S11). The image processing unit 11 acquires image v0 from imaging device 30, image v1 from imaging device 31, and image v2 from imaging device 32. The images acquired from different imaging devices are of different types and may, for example, be unrelated images. The image processing unit 11 outputs the acquired multiple images to the output control unit 12. In the following explanation, it will be assumed that the data sizes of the acquired images v0, v1, and v2 are equal, but this is not limited to the case.

[0057] The timing of acquiring multiple images is not particularly limited. Multiple images may be acquired simultaneously, or they may be acquired at different times within a predetermined period.

[0058] Next, the output control unit 12 and the transmission control unit 13 combine at least two of the multiple images and store them in the storage unit 14 along with header information (S12). The output control unit 12 stores information indicating the combining method as header information in the storage unit 14. The transmission control unit 13 also stores images v5 and v2 acquired from the output control unit 12 in the storage unit 14. Image v5 is a combined image generated by combining images v0 and v1. Information indicating the combining method of image v5 may be included in the header information added to the combined image.

[0059] Here, the combination of two or more images by the output control unit 12 will be explained with reference to Figures 5A to 5F. Figures 5A to 5F are schematic diagrams showing various examples of image combination according to this embodiment. An example of generating a combined image in which two or more images are combined in a planar manner will be explained using Figures 5A to 5F. Planar combination means arranging two or more images side by side in the memory space (that is, their addresses in the memory space are consecutive). The positions of the images shown in Figures 5A to 5F are indicated, for example, by the addresses in the memory space of the storage unit 14.

[0060] The output control unit 12 may generate a combined image by combining two images v0 and v1 horizontally, as shown in Figure 5A; or by combining two images v0 and v1 vertically, as shown in Figure 5B; or by combining four images v0 to v3 vertically and horizontally, as shown in Figure 5C.

[0061] Thus, the combined image may be an image in which two or more images are arranged side by side in at least one of the vertical and horizontal directions. Note that the combined images shown in Figures 5A to 5C are images with a larger data size than the original images.

[0062] The number of images to be combined is not particularly limited; it may be three, five or more, or any number of images.

[0063] Furthermore, the output control unit 12 may generate a combined image by further compressing images arranged side by side in at least one of the vertical and horizontal directions. Any known technique may be used as the compression method. The compression method may be lossless compression, or it may be lossy compression. Lossless compression is an image compression method in which the original image can be completely restored from the compressed image, and examples include, but are not limited to, compression methods using entropy coding, Huffman coding, arithmetic coding, etc. Lossy compression is an image compression method in which the original image cannot be completely restored from the compressed image, and examples include, but are not limited to, methods of encoding in JPEG (Joint Photographic Experts Group) format. When compression is performed, the output control unit 12 may store information indicating the compression method (i.e., information for restoring to the original image) as header information in the storage unit 14.

[0064] The output control unit 12 compresses the combined image obtained by combining images v0 and v1 so that the data size of the combined image is equivalent to the data size of image v0 or v1 (in this embodiment, the same data size). The output control unit 12 may compress the images before combining them or after combining them.

[0065] The output control unit 12 may generate the combined image shown in Figure 5D by compressing images v0 and v1 horizontally and then combining them horizontally. Alternatively, the output control unit 12 may generate the combined image shown in Figure 5E by compressing images v0 and v1 vertically and then combining them vertically. Furthermore, the output control unit 12 may generate the combined image shown in Figure 5F by compressing images v0 to v3 horizontally and vertically and then combining them vertically and horizontally.

[0066] Referring again to Figure 4, the USB controller 15 reads the image (S13) based on the transfer setting information and outputs the combined image (e.g., image v5 shown in Figure 2) and the uncombined image (e.g., image v2 shown in Figure 2) to different endpoints (S14). The USB controller 15 may output at least one image (in the example of Figure 2, image v2) from among the images from two or more imaging devices other than the multiple imaging devices to another endpoint without combining them. In other words, the USB controller 15 outputs at least one image (in the example of Figure 2, image v2) from among the images from imaging devices that were not to be combined to another endpoint.

[0067] Now, the transfer settings information will be explained with reference to Figure 6. Figure 6 is a diagram showing the transfer settings information stored in the built-in memory 15a according to this embodiment.

[0068] As shown in Figure 6, the internal memory 15a stores transfer setting information for endpoint 0 (EP0) and transfer setting information for endpoint 2 (EP2). Transfer setting information is generated for each endpoint and indicates the relationship between the image and the endpoint.

[0069] The number of transfer setting information entries stored in the internal memory 15a is less than the maximum number of imaging devices that can be connected to the image output device 10. For example, the number of transfer setting information entries stored in the internal memory 15a is less than the number of imaging devices connected to the image output device 10. In the examples of Figures 2 and 6, there are three imaging devices connected to the image output device 10, while there are two transfer setting information entries stored in the internal memory 15a.

[0070] Next, the image reading process in the USB controller 15 will be explained with reference to Figures 7A to 7C. Figure 7A is a diagram illustrating the data reading order in a conventional example. Figures 7B and 7C are diagrams illustrating various examples of the data reading order in this embodiment. Figure 7A shows the reading order of four uncombined images, while Figures 7B and 7C show the reading order of a compressed and combined image. The images shown in Figures 7B and 7C represent an image of the image arrangement in the storage space of the storage unit 14. Furthermore, for convenience, Figures 7A to 7C will use four images v10 to v13 for explanation.

[0071] As shown in Figure 7A, conventionally, image data (pixel values) are read sequentially row by row, starting from the pixel row containing the top-left pixel in image v10. Subsequently, image data is read in the same manner for images v11, v12, and v13.

[0072] As shown in Figure 7B, when reading a combined image obtained by combining horizontally compressed images v10 and v11 in the left-right direction, the combined image and the image v10 shown in Figure 7A have the same data size. Therefore, the image data of the combined image can be read using the same reading method as the image v10 shown in Figure 7A. In other words, the USB controller 15 does not need to take any special measures for reading images between combined and uncombined images (for example, by using different reading methods).

[0073] Furthermore, as shown in Figures 7A and 7B, it can be seen that the readout time can be reduced by combining images. This contributes to improving the efficiency of image transfer.

[0074] As shown in Figure 7C, in the case of a combined image formed by combining four images v10 to v13, which are compressed horizontally and vertically, in the vertical and horizontal directions, the data size of the combined image is the same as that of image v10 shown in Figure 7A. Therefore, the image data of the combined image can be read using the same reading method as for image v10 shown in Figure 7A.

[0075] Next, the output of images v5 and v2 by the USB controller 15 will be explained with reference to Figure 8. Figure 8 is a diagram illustrating an example of image output according to this embodiment. Figure 8 shows an example in which a combined image v5 and an uncombined image v2 are output.

[0076] As shown in Figures 2 and 8, in step S14, the combined image v5 of images v0 and v1 can be output from a single endpoint, thus reducing the number of endpoint switches when transferring the three images v0 to v2 to just one. This reduces the transfer loss when switching endpoints (transmission data switching period shown in Figure 8) compared to using three endpoints, allowing for more efficient data transfer. Therefore, the data transfer speed is improved.

[0077] In step S12, the output control unit 12 is not limited to spatially combining two or more images, but may also combine two or more images in a time series. Figure 9 is a diagram illustrating another example of image output according to this embodiment.

[0078] As shown in Figure 9, the output control unit 12 may output images v0 and v1 as a single combined image by, for example, alternately outputting images v0 and v1 in the time axis direction. In this case, the USB controller 15 may set the frame rate when outputting the combined image to one endpoint to be higher than the frame rate when outputting the uncombined image to the other endpoint. In other words, the USB controller 15 may change the frame rate when transferring images depending on whether the image to be output is a combined image or not.

[0079] Furthermore, by combining two or more images, it becomes possible to connect more imaging devices to the image output device 10 than the number of endpoints. The image output device 10 may have, for example, more connectors for connecting to imaging devices than the number of endpoints being used (for example, the maximum number of endpoints). In this way, the image output device 10 can relax the limitation on the number of imaging devices that can be connected.

[0080] Referring again to Figure 4, the USB controller 21 of the image acquisition device 20 acquires the combined image and uncombined image output from the image output device 10 (S15). The USB controller 21 acquires the combined image and the uncombined image via different endpoints. For example, the USB controller 21 acquires the combined image (image v5) via endpoint EP0 and the uncombined image (image v2) via endpoint EP2.

[0081] When the USB controller 21 receives an image, it switches endpoints to acquire the image. When the USB controller 21 receives an image, it receives the transmitted data (in this case, the combined image and the uncombined image) as is, so the switching period during transmission is directly reflected in the switching period during reception. In other words, the number of endpoint switches can be reduced even during reception, and the transfer loss during reception that occurs due to endpoint switching can be reduced. Therefore, the image acquisition device 20 can improve the transfer efficiency when receiving images. Furthermore, similar to the built-in memory 15a of the USB controller 15, the number of transfer setting information entries stored in the built-in memory (not shown) of the USB controller 21 can be made less than the number of imaging devices connected to the image output device 10. In other words, the amount of built-in memory area used by the USB controller 21 can be reduced.

[0082] Next, the USB controller 21 stores the combined image and the uncombined image in the storage unit 22 (S16).

[0083] Next, the processing unit 23 divides the combined image based on the header information of the combined image stored in the storage unit 22 (S17). Since the header information includes information indicating the method of combining the images, the processing unit 23 divides the combined image based on this information and generates two or more images before combining.

[0084] For example, if the header information indicates that two images have been joined horizontally, the processing unit 23 generates images v0 and v1 before joining by splitting image v5 horizontally. Also, if the header information indicates that two images have been joined vertically and compressed vertically, the processing unit 23 splits image v5 vertically and generates images v0 and v1 before joining by decompressing the two split images vertically.

[0085] Furthermore, for example, if the header information indicates that two or more images are combined in chronological order, the processing unit 23 generates images v0 and v1 before the combination by alternately splitting the combined image into separate videos. For example, the processing unit 23 may split the chronologically combined image into even-numbered images and odd-numbered images, and generate two videos from each of these images.

[0086] In this way, the processing unit 23 handles the image acquired via one endpoint as multiple images (in this case, two images).

[0087] Next, the processing unit 23 performs predetermined processing on the multiple images (S18). For example, the processing unit 23 performs predetermined processing such as displaying images v0, v1, and v2 on a display device or storing them in a storage device. The predetermined processing may be set in advance or may be included in the header information.

[0088] (Other Embodiments) Although one or more embodiments of image output devices, etc. have been described above based on embodiments, this disclosure is not limited to these embodiments. Without departing from the spirit of this disclosure, various modifications that a person skilled in the art could conceive of may be applied to these embodiments, and forms constructed by combining components from different embodiments may also be included in this disclosure.

[0089] For example, although the image output device according to the above embodiment has been described as being provided separately from the imaging device, it is not limited to this and may be built into the imaging device. Similarly, although the image acquisition device has been described as being provided separately from subsequent devices such as display devices, it is not limited to this and may be built into subsequent devices. For example, the image output device and the subsequent device may be connected via a USB cable.

[0090] Furthermore, although the above embodiment describes an example where the image output device outputs each image to one image acquisition device, it is not limited to this, and different images may be output to two or more image acquisition devices.

[0091] Furthermore, although the image output device according to the above embodiment has been described as an example of outputting an image, it may also output information other than images, such as control information, by specifying an endpoint. In this case, two or more pieces of control information may be combined.

[0092] Furthermore, in the above embodiment, each component may be implemented by being composed of dedicated hardware or by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0093] Furthermore, the order in which each step in the flowchart is performed is illustrative for the purpose of specifically illustrating this disclosure, and may be in a different order. Also, some of the above steps may be performed simultaneously (in parallel) with other steps, and some of the above steps may not be performed.

[0094] Furthermore, the division of functional blocks in the block diagram is just one example; multiple functional blocks can be implemented as a single functional block, a single functional block can be divided into multiple parts, or some functions can be moved to other functional blocks. In addition, the functions of multiple functional blocks with similar functions can be processed in parallel or time-sharing by a single piece of hardware or software.

[0095] Furthermore, the image output device and image acquisition device according to the above embodiment may be implemented as a single device or as multiple devices. When the image output device and image acquisition device are implemented as multiple devices, the components of the image output device and image acquisition device may be distributed among the multiple devices in any manner.

[0096] Furthermore, each component described in the above embodiment may be implemented as software, or typically as an integrated circuit (LSI). For example, the image acquisition device and the image output device may each be implemented as an LSI. These may be individually integrated into a single chip, or some or all of them may be integrated into a single chip. Here, we refer to them as LSIs, but depending on the degree of integration, they may also be called ICs, system LSIs, super LSIs, or ultra LSIs. Moreover, the method of integrated circuit implementation is not limited to LSIs; it may also be implemented using dedicated circuits (general-purpose circuits that execute dedicated programs) or general-purpose processors. After LSI manufacturing, a programmable FPGA (Field Programmable Gate Array) or a reconfigurable processor that can reconfigure the connections or settings of circuit cells inside the LSI may be used. Furthermore, if an integrated circuit implementation technology that replaces LSIs emerges due to advances in semiconductor technology or other derived technologies, it is natural that the components may be integrated using that technology.

[0097] A system LSI is a highly functional LSI manufactured by integrating multiple processing units onto a single chip. Specifically, it is a computer system composed of a microprocessor, ROM (Read Only Memory), RAM (Random Access Memory), and other components. The ROM stores the computer program. The system LSI achieves its function by having the microprocessor operate according to the computer program.

[0098] Furthermore, one aspect of this disclosure may be a computer program that causes a computer to execute the characteristic steps included in the image output method and image acquisition method shown in Figure 4.

[0099] Furthermore, for example, the program may be a program to be executed by a computer. Also, in one aspect of this disclosure, such a program may be recorded on a computer-readable non-temporary recording medium. For example, such a program may be recorded on a recording medium and distributed or made available. For example, by installing the distributed program on a device having another processor and having that processor execute the program, it becomes possible to have that device perform the above-mentioned processes.

[0100] (Note) Based on the above description of embodiments, the following technologies are disclosed.

[0101] (Technology 1) An image output device that outputs images assigned to endpoints using USB (Universal Serial Bus), comprising: an acquisition unit that acquires images from each of a plurality of imaging devices via an interface different from USB; and a control unit that performs control to output the images from each of the plurality of imaging devices to a plurality of endpoints, wherein the control unit outputs a combined image obtained by combining two or more images from two or more of the plurality of imaging devices to one of the plurality of endpoints.

[0102] This allows two images to be output from a single endpoint, reducing the number of endpoint switches compared to outputting two images from two separate endpoints. In other words, it reduces the transfer loss that occurs when switching endpoints. Therefore, it is possible to realize an image output device that can improve the transfer efficiency when transferring images.

[0103] (Technical 2) The control unit is equipped with an internal memory that stores transfer setting information indicating the relationship between an image and an endpoint, and the number of transfer setting information stored in the internal memory is less than the maximum number of imaging devices that can be connected to the image output device, as described in Technical 1.

[0104] This reduces the number of transfer settings, thereby reducing the amount of internal memory used to store them. Consequently, images can be output with less internal memory usage, improving the transfer efficiency of the image output device.

[0105] (Technical 3) The control unit is an image output device according to Technical 1 or 2, which adds information indicating the method of combining images to the combined image and outputs it.

[0106] This allows the device that acquires the combined image to easily separate the combined image into the original image.

[0107] (Technology 4) The combined image is an image output device from any of Technologies 1 to 3, in which the two or more images are combined in a planar manner.

[0108] This makes it easy to create a combined image by combining two or more images in a planar manner.

[0109] (Technical 5) The combined image is an image output device of Technical 4 in which the two or more images are arranged side by side in at least one of the vertical and horizontal directions on a plane.

[0110] This makes it easy to create a combined image from two or more images.

[0111] (Technical 6) The combined image is an image output device of Technical 5, which is an image compressed so that the data size is the same as one of the two or more images.

[0112] This allows the control unit to read images using a common reading method, whether the image is a combined image or an uncombined image. In other words, the control unit does not need to change the image reading method depending on whether the image is combined or not. Therefore, the reading efficiency when reading images to be transferred can be improved, and thus the transfer efficiency when transferring images can be improved.

[0113] (Technical 7) The combined image is an image output device from any of Technical 1 to 6, in which the two or more images are combined in a time series.

[0114] This makes it easy to create a combined image by chronologically combining two or more images.

[0115] (Technical 8) The control unit is an image output device of Technical 7, which outputs images from imaging devices other than the two or more imaging devices among the plurality of imaging devices to one of the plurality of endpoints without combining them, and the frame rate when outputting the combined image to the one endpoint is higher than the frame rate when outputting the image to the other one endpoint.

[0116] This helps to suppress the increased transmission time of time-series combined images. Therefore, since the increase in transfer time can be suppressed, the transfer efficiency when transferring images is improved.

[0117] (Technical 9) An image acquisition device that acquires images assigned to endpoints using USB, comprising: an acquisition unit that acquires a combined image obtained by combining two or more images from at least two or more imaging devices among a plurality of imaging devices via one endpoint; and a processing unit that divides the combined image acquired via one endpoint into the two or more images before combining.

[0118] This allows two images to be acquired from a single endpoint, reducing the number of endpoint switches required when outputting images compared to acquiring two images from two separate endpoints. In other words, it reduces the transfer loss associated with endpoint switching. Therefore, it is possible to realize an image acquisition device that can improve the transfer efficiency when transferring images.

[0119] (Technical 10) An image output method executed by an image output device that outputs an image assigned to an endpoint using USB, wherein the device acquires an image from each of a plurality of imaging devices via an interface different from USB, executes control to output the image from each of the plurality of imaging devices to a plurality of endpoints, and in the execution of the control, outputs a combined image obtained by combining two or more images from two or more of the plurality of imaging devices to one of the plurality of endpoints.

[0120] This produces the same effect as the image output device described above.

[0121] (Technical 11) An image acquisition method performed by an image acquisition device that acquires images assigned to an endpoint using USB, wherein a combined image obtained by combining two or more images from at least two or more imaging devices among a plurality of imaging devices is acquired via one endpoint, and the combined image acquired via one endpoint is divided into the two or more images before combining.

[0122] This achieves the same effect as the image acquisition device described above.

[0123] This disclosure is useful for systems that transfer images using UVC, which is standardized by USB.

[0124] 1 Image transfer system 10 Image output device 11 Image processing unit 12 Output control unit 13 Transmission control unit 14, 22 Storage unit 15 USB controller (control unit) 15a Built-in memory 20 Image acquisition device 21 USB controller (acquisition unit) 23 Processing unit 30, 31, 32 Imaging device EP0, EP1, EP2 Endpoints v0, v1, v2, v3, v5, v10, v11, v12, v13 Image

Claims

1. An image output device that outputs an image assigned to an endpoint using USB (Universal Serial Bus), comprising: an acquisition unit that acquires an image from each of a plurality of imaging devices via an interface different from USB; and a control unit that performs control to output the images from each of the plurality of imaging devices to a plurality of endpoints, wherein the control unit outputs a combined image obtained by combining two or more images from two or more of the plurality of imaging devices to one of the plurality of endpoints.

2. The image output device according to claim 1, wherein the control unit includes an internal memory that stores transfer setting information indicating the relationship between an image and an endpoint, and the number of transfer setting information items stored in the internal memory is less than the maximum number of imaging devices that can be connected to the image output device.

3. The image output device according to claim 1 or 2, wherein the control unit adds information indicating the method of combining images to the combined image and outputs it.

4. The image output device according to claim 1 or 2, wherein the combined image is an image obtained by combining the two or more images in a planar manner.

5. The image output device according to claim 4, wherein the combined image is an image in which the two or more images are arranged side by side in at least one of the vertical and horizontal directions on a plane.

6. The image output device according to claim 5, wherein the combined image is an image compressed so that its data size is the same as that of one of the two or more images.

7. The image output device according to claim 1 or 2, wherein the combined image is an image obtained by combining the two or more images in chronological order.

8. The image output device according to claim 7, wherein the control unit outputs images from imaging devices other than the two or more imaging devices among the plurality of imaging devices to one of the plurality of endpoints without combining them, and the frame rate when outputting the combined image to the one endpoint is higher than the frame rate when outputting the image to the other one endpoint.

9. An image acquisition device that acquires images assigned to endpoints using USB, comprising: an acquisition unit that acquires a combined image obtained by combining two or more images from at least two or more imaging devices among a plurality of imaging devices via one endpoint; and a processing unit that divides the combined image acquired via one endpoint into the two or more images before combining.

10. An image output method performed by an image output device that outputs an image assigned to an endpoint using USB, the method comprising: acquiring an image from each of a plurality of imaging devices via an interface different from USB; executing control to output the images from each of the plurality of imaging devices to a plurality of endpoints; and in the execution of the control, outputting a combined image obtained by combining two or more images from two or more of the plurality of imaging devices to one of the plurality of endpoints.

11. An image acquisition method performed by an image acquisition device that acquires images assigned to an endpoint using USB, comprising: acquiring a combined image obtained by combining two or more images from at least two or more imaging devices among a plurality of imaging devices via one endpoint; and dividing the combined image acquired via the one endpoint into the two or more images before combining.

Images